Configure an Align Color Depth Homography View

When you have a depth image and you need it to overlay on top of a color image exactly, a homography matrix can apply the necessary distortions to the depth image to make it align with the color image. Use the align_color_depth_homography camera model to do this alignment.

Navigate to the Config tab of your machine’s page in the Viam app. Click on the Components subtab and click Create component. Select the camera type, then select the align_color_depth_homography model. Enter a name for your camera and click Create.

Configuration of an align color depth homography view in the Viam app config builder.

Copy and paste the following attribute template into your camera’s Attributes box. Then remove and fill in the attributes as applicable to your camera, according to the table below.

{
  "intrinsic_parameters": {
    "width_px": <int>,
    "height_px": <int>,
    "fx": <float>,
    "fy": <float>,
    "ppx": <float>,
    "ppy": <float>
  },
  "homography": {
    "transform": [ <float>, <float>, <float>,
                   <float>, <float>, <float>,
                   <float>, <float>, <float> ],
    "depth_to_color": <boolean>,
    "rotate_depth_degs": <int>
  },
  "color_camera_name": "<your-camera-name>",
  "depth_camera_name": "<your-camera-name>",
  "output_image_type": "<color|depth>",
  "distortion_parameters": {
    "rk1": <float>,
    "rk2": <float>,
    "rk3": <float>,
    "tp1": <float>,
    "tp2": <float>
  },
  "debug": <boolean>
}
{
  "name": "<your-camera-name>",
  "model": "align_color_depth_homography",
  "type": "camera",
  "namespace": "rdk",
  "attributes": {
    "intrinsic_parameters": {
      "width_px": <int>,
      "height_px": <int>,
      "fx": <float>,
      "fy": <float>,
      "ppx": <float>,
      "ppy": <float>
    },
    "homography": {
      "transform": [ <float>, <float>, <float>,
                     <float>, <float>, <float>,
                     <float>, <float>, <float> ],
      "depth_to_color": <boolean>,
      "rotate_depth_degs": <int>
    },
    "color_camera_name": "<your-camera-name>",
    "depth_camera_name": "<your-camera-name>",
    "output_image_type": "<color|depth>",
    "distortion_parameters": {
      "rk1": <float>,
      "rk2": <float>,
      "rk3": <float>,
      "tp1": <float>,
      "tp2": <float>
    },
    "debug": <boolean>
  }
}

The following attributes are available for align_color_depth_homography views:

NameTypeInclusionDescription
intrinsic_parametersobjectRequiredThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • width_px: The expected width of the aligned image in pixels.
  • height_px: The expected height of the aligned image in pixels.
  • fx: The image center x point.
  • fy: The image center y point.
  • ppx: The image focal x.
  • ppy: The image focal y.
homographyobjectRequiredParameters that morph the depth points to overlay the color points and align the images:
  • transform: 9 floats representing the 3x3 homography matrix of the depth to color, or color to depth camera.
  • depth_to_color: Whether to transform depth camera points to color camera points.
  • rotate_depth_degs: Degrees by which to rotate the depth camera image.
color_camera_namestringRequiredname of the color camera to pull images from.
depth_camera_namestringRequiredname of the depth camera to pull images from.
output_image_typestringRequiredSpecify color or depth for the output stream.
distortion_parametersobjectOptionalModified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens:
  • rk1: The radial distortion x.
  • rk2: The radial distortion y.
  • rk3: The radial distortion z.
  • tp1: The tangential distortion x.
  • tp2: The tangential distortion y.
debugbooleanOptionalEnables the debug outputs from the camera if true.
Default: false

View the camera stream

Once your camera is configured and connected, go to the Control tab, and click on the camera’s dropdown menu. Then toggle the camera or the Point Cloud Data view to ON. If everything is configured correctly, you will see the live video feed from your camera. You can change the refresh frequency as needed to change bandwidth.

Example Camera view inside Viam app

Next steps



Have questions, or want to meet other people working on robots? Join our Community Discord.

If you notice any issues with the documentation, feel free to file an issue or edit this file.