Detect color with a Viam Rover

The Vision Service enables a robot to use its cameras to see and interpret the world around it. The service also allows you to create different types of detectors with which the robot can recognize objects, scan QR codes, perform optical quality inspections, sort different colored objects, take measurements, and more.

In this tutorial you will learn how to configure a color detector with the Vision Service and how to leverage that detector with a transform camera to detect the color red.

You can follow this tutorial with a rented Viam Rover or with your own Viam Rover.

Add the Vision Service to detect a color

This tutorial uses the hex color #7a4f5c (a reddish color).

Hex color #7a4f5c: A color swatch for the color that you will be detecting with your color detector. It's a reddish, maroon color.

Navigate to the robot page on the Viam app. Click on the robot you wish to add the Vision Service to. Select the Config tab, and click on Services.

Scroll to the Create Service section. To create a Vision Service:

  1. Select vision as the Type.
  2. Enter my_color_detector as the Name.
  3. Select color_detector as the Model.
  4. Click Create Service.

In your Vision Service’s panel, fill in the Attributes field.

{
      "segment_size_px": 100,
      "detect_color": "#7a4f5c",
      "hue_tolerance_pct": 0.06
}

Add the Vision Service object to the services array in your rover’s raw JSON configuration:

"services": [
  {
    "name": "my_color_detector",
    "type": "vision",
    "model": "color_detector",
    "attributes": {
      "segment_size_px": 100,
      "detect_color": "#7a4f5c",
      "hue_tolerance_pct": 0.06
    }
  },
  ... // Other services
]

The configuration adds a model of type color_detector with the color as a parameter. The color_detector is a heuristic-based detector that draws boxes around objects according to their hue.

Click Save config and head to the Components tab.

You cannot interact directly with the Vision Service. To be able to interact with the Vision Service you must configure a camera component.

Configure a transform camera to use the color detector

Viam camera components can be physical like the one already configured on the rover, or virtual. A virtual camera transforms the output from a physical camera.

To view output from the color detector overlaid on images from a physical camera, you need to configure a transform camera.

Navigate to the Components tab in the Viam app and scroll to the Create Component section. To create a transform camera:

  1. Enter a name for Name, for example detectionCam.
  2. Select camera as the Type.
  3. Select transform as the Model.
  4. Click Create Component.

The Viam app showing the Create Component panel populated with a camera component. The name is detectionCam, the type is camera, and the model is transform.

Viam generates an empty Attributes section for the detection camera’s component panel. The panel’s Attribute Guide section displays the available attributes for the selected component.

The Viam app showing the detectionCam component section. The Attributes section contains a skeleton configuration, including source, pipeline, type, and attributes. The Attributes Guide section lists the available camera component attributes. There are buttons labeled Data Capture Configuration, and Frame, and a drop-down labeled, Depends On. On the upper right there is a trash bin icon.

Copy the following JSON configuration into the Attributes section:

{
 "source": "cam",
 "pipeline": [
   {
     "attributes": {
       "detector_name": "my_color_detector",
       "confidence_threshold": 0.3
     },
     "type": "detections"
   }
 ]
}
FieldDefault valueDescription
sourcecamThe name of the physical camera on the rover, which provides the visual feed to get detections from.
attributesThe attributes of this detectionCam.
attributes.detector_namemy_color_detectorThe name of this detectionCam.
attributes.confidence_threshold0.3 (30%)The percentage of confidence in the color being present the detection service needs to detect a color.
typedetectionsThe type of the component.

The Viam app showing the detectionCam component section. It contains the Attributes section with a skeleton configuration, including source, pipeline, type, and attributes. The panel has an Attributes section populated with transform camera component attributes. The are buttons labeled Data Capture Configuration, and Frame, and a drop-down labeled, Depends On. On the upper right there is a trash bin icon.

After adding the component and its attributes, click Save config.

Test your transform camera in the CONTROL tab

In the Control tab, click on your base component and add the detection camera from the Select Cameras drop down.

Next, enable the keyboard and move your rover around until your camera detects the configured color. Each time the camera detects the color, you will see a red rectangle around the color labeled with the detection confidence level.

Base component panel displaying an example color detection.

If you scroll down in the Control tab, you can also click on the detectionCam’s own section to view its stream there.

Next Steps

If you’re ready for more, try making your rover detect other colors. You could also write some code with a Viam SDK to make your Viam Rover move in a square.

You can also ask questions in the Community Discord and we will be happy to help.



Have questions, or want to meet other people working on robots? Join our Community Discord.