Configure an Align Color Depth Homography View

When you have a depth image and you need it to overlay on top of a color image exactly, a homography matrix can apply the necessary distortions to the depth image for it to overlap.

On the COMPONENTS subtab, navigate to the Create Component menu. Enter a name for your camera, select the type camera, and select the align_color_depth_homography model.

Creation of an align color depth homography view in the Viam app config builder.

Fill in the attributes for your align color depth homography view:

Configuration of an align color depth homography view in the Viam app config builder.
{
    "name": "<camera_name>",
    "type": "camera",
    "model" : "align_color_depth_homography",
    "attributes": {
        "intrinsic_parameters": {
            "width_px": <integer>,
            "height_px": <integer>,
            "fx": <float64>,
            "fy": <float64>,
            "ppx": <float64>,
            "ppy": <float64>
        },
        "homography": {
            "transform": [ <float64>, <float64>, <float64>,
                           <float64>, <float64>, <float64>,
                           <float64>, <float64>, <float64> ],
            "depth_to_color": <boolean>,
            "rotate_depth_degs": <integer>
        },
        "color_camera_name": "<camera_name>",
        "depth_camera_name": "<camera_name>",
        "output_image_type": "<color|depth>",
        "distortion_parameters": {
            "rk1": <float64>,
            "rk2": <float64>,
            "rk3": <float64>,
            "tp1": <float64>,
            "tp2": <float64>
        },
        "debug": <boolean>
    }
}

The following attributes are available for align color depth homography views:

NameInclusionDescription
intrinsic_parametersRequiredThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • width_px: The expected width of the aligned image in pixels.
  • height_px: The expected height of the aligned image in pixels.
  • fx: The image center x point.
  • fy: The image center y point.
  • ppx: The image focal x.
  • ppy: The image focal y.
homographyRequiredParameters that morph the depth points to overlay the color points and align the images:
  • transform: 9 floats representing the 3x3 homography matrix of the depth to color, or color to depth camera.
  • depth_to_color: Whether to transform depth camera points to color camera points.
  • rotate_depth_degs: Degrees by which to rotate the depth camera image.
color_camera_nameRequiredName of the color camera to pull images from.
depth_camera_nameRequiredName of the depth camera to pull images from.
output_image_typeRequiredSpecify color or depth for the output stream.
distortion_parametersOptionalModified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens:
  • rk1: The radial distortion x.
  • rk2: The radial distortion y.
  • rk3: The radial distortion z.
  • tp1: The tangential distortion x.
  • tp2: The tangential distortion y.
debugOptionalEnables the debug outputs from the camera if true. Defaults to false.

View the camera stream

Once your camera is connected, go to the CONTROL tab, and click on the camera’s dropdown menu. Toggle “View Camera”. If everything is configured correctly, you will see the live video feed from your camera.

Example Camera view inside Viam app

Next Steps