Configure a Join Color Depth View

The join_color_depth model combines the streams of a color and depth camera already registered in your config to create a view that outputs the combined and aligned image.

This specific model is good if you don’t need to align the streams. If you need to adjust the alignment between the depth and color frames, use the align_color_depth_extrinsics model or thealign_color_depth_homography model.

Navigate to the CONFIGURE tab of your machine’s page in the Viam app. Click the + icon next to your machine part in the left-hand menu and select Component. Select the camera type, then select the join_color_depth model. Enter a name or use the suggested name for your camera and click Create.

Configuration of a join color depth view in the Viam app config builder.

Copy and paste the following attribute template into your camera’s attributes field. Then remove and fill in the attributes as applicable to your camera, according to the table below.

{
  "output_image_type": "<color|depth>",
  "color_camera_name": "<your-camera-name>",
  "depth_camera_name": "<your-camera-name>",
  "intrinsic_parameters": {
    "width_px": <int>, # the expected width of the aligned pic
    "height_px": <int>, # the expected height of the aligned pic
    "fx": <float>,
    "fy": <float>,
    "ppx": <float>,
    "ppy": <float>
  },
  "distortion_parameters": {
    "rk1": <float>,
    "rk2": <float>,
    "rk3": <float>,
    "tp1": <float>,
    "tp2": <float>
  },
  "debug": <boolean>
}

{
  "name": "<your-camera-name>",
  "model": "join_color_depth",
  "type": "camera",
  "namespace": "rdk",
  "attributes": {
    "output_image_type": "<color|depth>",
    "color_camera_name": "<your-camera-name>",
    "depth_camera_name": "<your-camera-name>",
    "intrinsic_parameters": {
      "width_px": <int>, # the expected width of the aligned pic
      "height_px": <int>, # the expected height of the aligned pic
      "fx": <float>,
      "fy": <float>,
      "ppx": <float>,
      "ppy": <float>
    },
    "distortion_parameters": {
      "rk1": <float>,
      "rk2": <float>,
      "rk3": <float>,
      "tp1": <float>,
      "tp2": <float>
    },
    "debug": <boolean>
  }
}

The following attributes are available for join_color_depth views:

NameTypeInclusionDescription
output_image_typestringRequiredSpecify color or depth for the output stream.
color_camera_namestringRequiredname of the color camera to pull images from. If your camera provides both color and depth image streams, you must set this to be the same as depth_camera_name.
depth_camera_namestringRequiredname of the depth camera to pull images from. If your camera provides both color and depth image streams, you must set this to be the same as color_camera_name.
intrinsic_parametersobjectRequiredThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • width_px: The expected width of the aligned image in pixels.
  • height_px: The expected height of the aligned image in pixels.
  • fx: The image center x point.
  • fy: The image center y point.
  • ppx: The image focal x.
  • ppy: The image focal y.
distortion_parametersobjectOptionalModified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens:
  • rk1: The radial distortion x.
  • rk2: The radial distortion y.
  • rk3: The radial distortion z.
  • tp1: The tangential distortion x.
  • tp2: The tangential distortion y.
debugbooleanOptionalEnables the debug outputs from the camera if true.
Default: false.

View the camera stream

Once your camera is configured and connected, go to the CONTROL tab, and click on the camera’s dropdown menu. Then toggle the camera or the Point Cloud Data view to ON. If everything is configured correctly, you will see the live video feed from your camera. You can change the refresh frequency as needed to change bandwidth.

Example Camera view inside Viam app

Next steps

Calibrate a Camera
Calibrate a Camera

Calibrate a camera and extract the intrinsic and distortion parameters.

transform
transform

Instructions for transforming a camera.

Computer Vision
Computer Vision

The vision service enables your machine to use its on-board cameras to intelligently see and interpret the world around it.



Have questions, or want to meet other people working on robots? Join our Community Discord.

If you notice any issues with the documentation, feel free to file an issue or edit this file.