Configure a color_detector

The color_detector vision service model is a heuristic detector that draws boxes around objects according to their hue. Color detectors do not detect black, perfect grays (grays where the red, green, and blue color component values are equal), or white. It only detects hues found on the color wheel.

"services": [
  {
    "name": "<service_name>",
    "type": "vision",
    "model": "color_detector",
    "attributes": {
      "segment_size_px": <integer>,
      "detect_color": "#ABCDEF",
      "hue_tolerance_pct": <number>,
      "saturation_cutoff_pct": <number>,
      "value_cutoff_pct": <number>
    }
  },
  ... // Other services
]

"services": [
  {
    "name": "blue_square",
    "type": "vision",
    "model": "color_detector",
    "attributes": {
      "segment_size_px": 100,
      "detect_color": "#1C4599",
      "hue_tolerance_pct": 0.07,
      "value_cutoff_pct": 0.15
    }
  },
  {
    "name": "green_triangle",
    "type": "vision",
    "model": "color_detector",
    "attributes": {
      "segment_size_px": 200,
      "detect_color": "#62963F",
      "hue_tolerance_pct": 0.05,
      "value_cutoff_pct": 0.20
    }
  }
]

The following parameters are available for a color_detector:

Click the Save button in the top right corner of the page. Proceed to test your detector.

Test your detector

You can test your detector with live camera footage or existing images.

Live camera footage

If you intend to use the detector with a camera that is part of your machine, you can test your detector from the Control tab or with code:

1. Configure a camera component.

   Tip

   This is the camera whose name you need to pass to vision service methods.

2. (Optional) If you would like to see detections from the Control tab, configure a transform camera with the following attributes:

   {
     "pipeline": [
       {
         "type": "detections",
         "attributes": {
           "confidence_threshold": 0.5,
           "detector_name": "<vision-service-name>",
           "valid_labels": ["<label>"]
         }
       }
     ],
     "source": "<camera-name>"
   }
   

3. After adding the components and their attributes, click the Save button in the top right corner of the page.

4. Navigate to the CONTROL tab, click on your transform camera and toggle it on. The transform camera will now show detections with bounding boxes around the object.

   

5. To access detections with code, use the Vision Service methods on the camera you configured in step 1. The following code gets the machine’s vision service and then runs a color detector vision model on output from the machine’s camera "cam1":

   Tip

   Pass the name of the camera you configured in step 1. Do not pass a transform camera that already has the “detections” or “classifications” transform applied to it.

   • Python
   • Go

   from viam.services.vision import VisionClient
   
   robot = await connect()
   camera_name = "cam1"
   
   # Grab camera from the machine
   cam1 = Camera.from_robot(robot, camera_name)
   # Grab Viam's vision service for the detector
   my_detector = VisionClient.from_robot(robot, "my_detector")
   
   detections = await my_detector.get_detections_from_camera(camera_name)
   
   # If you need to store the image, get the image first
   # and then run detections on it. This process is slower:
   img = await cam1.get_image()
   detections_from_image = await my_detector.get_detections(img)
   
   await robot.close()
   

   To learn more about how to use detection, see the Python SDK docs.

   import (
     "go.viam.com/rdk/config"
     "go.viam.com/rdk/services/vision"
     "go.viam.com/rdk/components/camera"
   )
   
   // Grab the camera from the machine
   cameraName := "cam1" // make sure to use the same component name that you have in your machine configuration
   myCam, err := camera.FromRobot(robot, cameraName)
   if err != nil {
     logger.Fatalf("cannot get camera: %v", err)
   }
   
   myDetector, err := vision.from_robot(robot, "my_detector")
   if err != nil {
       logger.Fatalf("Cannot get vision service: %v", err)
   }
   
   // Get detections from the camera output
   detections, err := myDetector.DetectionsFromCamera(context.Background(), myCam, nil)
   if err != nil {
       logger.Fatalf("Could not get detections: %v", err)
   }
   if len(directDetections) > 0 {
       logger.Info(detections[0])
   }
   
   // If you need to store the image, get the image first
   // and then run detections on it. This process is slower:
   
   // Get the stream from a camera
   camStream, err := myCam.Stream(context.Background())
   
   // Get an image from the camera stream
   img, release, err := camStream.Next(context.Background())
   defer release()
   
   // Apply the color classifier to the image from your camera (configured as "cam1")
   detectionsFromImage, err := myDetector.Detections(context.Background(), img, nil)
   if err != nil {
       logger.Fatalf("Could not get detections: %v", err)
   }
   if len(detectionsFromImage) > 0 {
       logger.Info(detectionsFromImage[0])
   }
   

   To learn more about how to use detection, see the Go SDK docs.

Existing images

If you would like to test your detector with existing images, load the images and pass them to the detector:

from viam.services.vision import VisionClient
from PIL import Image

robot = await connect()
# Grab Viam's vision service for the detector
my_detector = VisionClient.from_robot(robot, "my_detector")

# Load an image
img = Image.open('test-image.png')

# Apply the detector to the image
detections_from_image = await my_detector.get_detections(img)

await robot.close()

import (
  "go.viam.com/rdk/config"
  "go.viam.com/rdk/services/vision"
  "image/jpeg"
  "os"
)

myDetector, err := vision.from_robot(robot, "my_detector")
if err != nil {
    logger.Fatalf("Cannot get Vision Service: %v", err)
}

// Read image from existing file
file, err := os.Open("test-image.jpeg")
if err != nil {
    logger.Fatalf("Could not get image: %v", err)
}
defer file.Close()
img, err := jpeg.Decode(file)
if err != nil {
    logger.Fatalf("Could not decode image: %v", err)
}
defer img.Close()

// Apply the detector to the image
detectionsFromImage, err := myDetector.Detections(context.Background(), img, nil)
if err != nil {
    logger.Fatalf("Could not get detections: %v", err)
}
if len(detectionsFromImage) > 0 {
    logger.Info(detectionsFromImage[0])
}

Next steps

Detect Color

Use the vision service in the Viam app to detect a color with the Viam Rover.

Colored Object Follower

Instructions for detecting and following a colored object with a rover, like a SCUTTLE robot.

RGB Line Follower

Build a line-following robot that relies on a webcam and color detection.