Try Viam: Detect a Color with your Viam Rover using the Vision Service

Introduction

This tutorial will guide you through using a Try Viam Rover rental to detect the color red by adding a color detector to Viam’s vision service. The only prerequisite is you need to have an active reservation for one of Viam’s rental rovers. We already added different colors around the rink on each wall.

But why should I use this service, one might ask? The vision service enables the robot to use its on-board cameras to see and interpret the world around it. The service also grants you the ease of creating different types of detectors, which allows you to then leverage them to program your robots to do interesting things. We can recognize objects, scan QR codes, perform optical quality inspections, sort different colored objects, take measurements, etc. The opportunities are endless.

Now let’s dive in.

Adding the vision service

Let’s start with adding the vision service to your rover. In Viam App, head to the CONFIG tab, and click on SERVICES. Scroll down to the Create Service section.

1. Enter “vision” in Type from the drop-down.
2. Enter “vision” in Name.
3. Click Create Service.

Clicking Create Service automatically creates a new service for your rover, but that does not set any attributes for the service.

Copy the following configuration into the attributes of your rover’s vision service.

{
 "register_models": [
   {
     "type": "color_detector",
     "parameters": {
       "detect_color": "#7a4f5c",
       "hue_tolerance_pct": 0.06,
       "segment_size_px": 100
     },
     "name": "my_color_detector"
   }
 ]
}

After adding the detector configuration, the Vision Service panel should look like this:

This configuration creates a detector that will find the hex color #7a4f5c (a reddish color) when you use it with images from your robot’s cameras. You can change the color to your liking and experiment with it. You can also play with the hue_tolerance_pct value to detect this color with more or less precision.

Hex color #7a4f5c:

The color_detector we use is a heuristic-based detector that draws boxes around objects according to their hue (note that the detector does not detect black, perfect greys¹, or white).

To learn more, head to our Vision Service topic (https://docs.viam.com/services/vision/) for additional information on how to configure and use Viam’s Vision Service. Now, click SAVE CONFIG and head to the COMPONENTS tab.

How to configure a transform camera to use the color detector

After adding the vision service and color detector, we’ll add a new camera to our Viam Rover. Viam camera components can be physical (like that already configured on the rover), or virtual (such as the transform camera, which transforms the output from the physical camera). To display the detections, we will use the transform camera.

To learn more about the camera component, head to the Camera component topic.

Scroll down to see the Create Component section in COMPONENTS tab of the Viam app.

1. Enter the desired name in Name. We named ours, “detectionCam”.
2. Enter “camera” in Type.
3. Enter “transform” in Model.
4. Click Create Component.

After creating the component, Viam automatically generates an empty Attributes section for the detectionCam’s component panel. The panel’s Attribute Guide section displays the available attributes for the selected component.

source is the name of the physical camera on the rover, which is the camera you want to get the detections from. The camera’s name in the config is “cam”, so we will use that. The Type will be “detections”. In Attributes, you must add a detector_name. We named ours “my_color_detector”. Next, you must add the confidence_threshold. We picked 0.3 which means if the detection service is 30% confident of the color, it will detect it.

JSON configuration is here if you wish to copy paste it into your Viam rover’s config:

{
 "source": "cam",
 "pipeline": [
   {
     "attributes": {
       "detector_name": "my_color_detector",
       "confidence_threshold": 0.3
     },
     "type": "detections"
   }
 ]
}

After adding the component and its attributes, click SAVE CONFIG and head to the CONTROL tab.

Testing your transform camera with the vision service in the control tab

In the CONTROL tab, toggle your base component and add the detection camera from Select Cameras drop down. detectionCam should be available if you have configured your component and service correctly. When you enable the keyboard, you will be able to move the rover and see how your detection works. Each time your camera sees the color, it will draw a red rectangle around it labeled with the detection confidence level.

We recommend that you test the functionality of Viam’s vision services and cameras by enabling your other camera, cam, to compare its view with the detection camera’s view.

If you scroll down in the CONTROL tab, you will see the detectionCam’s own section. You can also view its stream there.

Summary

Congratulations! If you followed along, you have successfully used Viam to make your rental rover detect the color red in the rental rink. You have learned how to configure a color detector with the vision service, and how to leverage that detector with a transform camera.

If you are ready to continue tinkering with your rental rover, head to our “How to use the Viam SDK to control your Viam Rover” tutorial. In that tutorial, we will introduce you to the Viam SDK (software development kit) so that you can write code in either Python or Golang to make your Viam Rover move in a square.

If you want to connect with other developers learning how to build robots or if you have any issues whatsoever during Try Viam experience, let us know on the Viam Community Slack, and we will be happy to help you get up and running.