Detect color with a Viam Rover

In this tutorial you will learn how to configure a color detector with the vision service and leverage that detector with a transform camera to detect the color red.

The vision service enables a robot to use its cameras to see and interpret the world around it. The service also allows you to create different types of detectors with which the robot can recognize objects, scan QR codes, perform optical quality inspections, sort different colored objects, take measurements, and more.

You can follow this tutorial with a rented Viam Rover or with your own Viam Rover.

Enable the cameras

Before configuring color detection, enable the rover’s camera to get a better sense of what it perceives.

  • If you are running this tutorial with a rented Viam Rover, enable both provided cameras: the front-facing camera and the overhead cam. In the viam_base component panel under the CONTROL tab, enable both the cam for the front-facing camera and the overhead-cam:overheadcam for an overhead view of your rover.

    The viam_base component panel showing both the ‘cam’ and ‘overheadcam’ camera feeds enabled.

    You can also view and control the camera streams from the individual camera component panels.

  • If you are running this tutorial on your own Viam Rover, enable the front facing camera. If you are using the ViamRover fragment with your rover, the front facing camera is named cam and can be enabled in the viam_base component panel under the CONTROL tab.

Add the vision service to detect a color

This tutorial uses the color #7a4f5c or rgb(122, 79, 92) (a reddish color).

Hex color #7a4f5c: A color swatch for the color that you will be detecting with your color detector. It's a reddish, maroon color.

Navigate to your machine’s CONFIGURE tab on the Viam app and configure your vision service color detector:

  1. Click the + (Create) icon next to your machine part in the left-hand menu and select Service.
  2. Select the vision type, then select the color detector model.
  3. Enter my_color_detector as the name for your detector and click Create.
  4. In the resulting vision service panel, click the color picker box to set the color to be detected. For this tutorial, set the color to rgb(122, 79, 92) or use hex code #7a4f5c.
  5. Then, set Hue Tolerance to 0.06 and Segment size px to 100.

Your configuration should look like the following:

The vision service configuration panel showing the color set to a reddish color, the hue tolerance set to 0.06, and the segment size set to 100.

Select JSON mode on the CONFIGURE tab. Add the vision service object to the services array in your rover’s JSON configuration:

"services": [
  {
    "name": "my_color_detector",
    "type": "vision",
    "model": "color_detector",
    "attributes": {
      "segment_size_px": 100,
      "detect_color": "#7a4f5c",
      "hue_tolerance_pct": 0.06
    }
  },
  ... // Other services
]

The color_detector is a heuristic-based detector that draws boxes around objects according to their hue.

Click Save to save your configuration.

You cannot interact directly with the vision service. To be able to interact with the vision service you must configure a camera component.

Configure a transform camera to use the color detector

Viam camera components can be physical like the one already configured on the rover, or virtual. A virtual transform camera transforms the output from a physical camera.

To view output from the color detector overlaid on images from a physical camera, configure a transform camera:

  1. Navigate to the CONFIGURE tab in the Viam app.
  2. Click the + (Create) icon next to your machine part in the left-hand menu and select Component.
  3. Select camera as the type.
  4. Select transform as the model.
  5. Enter a name, for example detectionCam, and click Create.

On the detection camera’s component panel, click {} (Switch to advanced) to switch to the advanced mode, where you can edit the attributes directly with JSON. Copy and paste the following JSON configuration into the attributes field:

{
  "source": "cam",
  "pipeline": [
    {
      "attributes": {
        "detector_name": "my_color_detector",
        "confidence_threshold": 0.3
      },
      "type": "detections"
    }
  ]
}

Explanations of each attribute are as follows:

  • source: The name of the physical camera on the rover, which provides the visual feed to get detections from. If you used a different name for your camera, edit this field.
  • pipeline: Contains the transformation objects to apply to the camera.
  • attributes: The attributes of this transform camera.
    • detector_name: The name of the detector.
    • confidence_threshold: The percentage of confidence needed by the detection service to identify a color. Since we set it to 0.3, this means that detections with less than 30% confidence won’t be recognized.
    • type: The type of transform camera.

The filled-in transform camera configuration panel will look like this:

The Viam app showing the detectionCam component section. The Attributes section contains a skeleton configuration, including source, pipeline, type, and attributes. On the upper right there is a trash bin icon.

After adding the component and its attributes, click Save in the top right corner of the page to save your config.

Test your transform camera in the CONTROL tab

In the CONTROL tab, click on your base component and enable the detection camera in the Live Feeds section.

Next, enable the keyboard and move your rover around until your camera detects the specified color.

Each time the camera detects the color, you will see a red rectangle around the color labeled with the detection confidence level.

Base component panel displaying an example color detection.

Scroll down in the CONTROL tab and select the dedicated section for detectionCam to access its live stream.

Next steps

If you’re ready for more, try making your rover detect other colors. You could also write some code with a Viam SDK to make your Viam Rover move in a square.

You can also ask questions in the Community Discord and we will be happy to help.