Configure an Align Color Depth Extrinsics View

Use the intrinsics of the color and depth camera, as well as the extrinsic pose between them, to align two images.

On the COMPONENTS subtab, navigate to the Create Component menu. Enter a name for your camera, select the type camera, and select the align_color_depth_extrinsics model.

Creation of an align color depth extrinsics view in the Viam app config builder.

Fill in the attributes for your align color depth extrinsics view:

Configuration of an align color depth extrinsics view in the Viam app config builder.
{
    "name": "<camera_name>",
    "type": "camera",
    "model" : "align_color_depth_extrinsics",
    "attributes": {
        "camera_system": {
            "color_intrinsic_parameters": {
                "width_px": <integer>,
                "height_px": <integer>,
                "fx": <float64>,
                "fy": <float64>,
                "ppx": <float64>,
                "ppy": <float64>
            },
            "depth_intrinsic_parameters": {
                "width_px": <integer>,
                "height_px": <integer>,
                "fx": <float64>,
                "fy": <float64>,
                "ppx": <float64>,
                "ppy": <float64>
            },
                "depth_to_color_extrinsic_parameters": {
                "translation_mm": [ <float64>, <float64>, <float64>],
                "rotation_rads": [ <float64>, <float64>, <float64>,
                                   <float64>, <float64>, <float64>,
                                   <float64>, <float64>, <float64> ],
            }
        },
        "intrinsic_parameters": {
            "width_px": <integer>,
            "height_px": <integer>,
            "fx": <float64>,
            "fy": <float64>,
            "ppx": <float64>,
            "ppy": <float64>
        },
        "output_image_type": "<color|depth>",
        "color_camera_name": "<camera_name>",
        "depth_camera_name": "<camera_name>",
        "debug": false,
    }
}

The following attributes are available for align color depth extrinsics views:

NameInclusionDescription
camera_systemRequiredThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • color_intrinsic_parameters: The model uses the color camera intrinsics to project the 3D points to a 2D depth map, but “as if” it was taken from the POV of the color camera.
  • depth_intrinsic_parameters: The model uses the depth camera intrinsics to de-project the 2D depth points to 3D points, from the point of view of the depth camera.
  • depth_to_color_extrinsic_parameters: The model uses the extrinsic parameters to shift the 3D depth points to be from the POV of the color camera.
intrinsic_parametersRequiredThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • width_px: The expected width of the aligned image in pixels.
  • height_px: The expected height of the aligned image in pixels.
  • fx: The image center x point.
  • fy: The image center y point.
  • ppx: The image focal x.
  • ppy: The image focal y.
output_image_typeRequiredSpecify color or depth for the output stream.
color_camera_nameRequiredName of the color camera to pull images from.
depth_camera_nameRequiredName of the depth camera to pull images from.
distortion_parametersOptionalModified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens:
  • rk1: The radial distortion x.
  • rk2: The radial distortion y.
  • rk3: The radial distortion z.
  • tp1: The tangential distortion x.
  • tp2: The tangential distortion y.
debugOptionalEnables the debug outputs from the camera if true. Defaults to false.

View the camera stream

Once your camera is connected, go to the CONTROL tab, and click on the camera’s dropdown menu. Toggle “View Camera”. If everything is configured correctly, you will see the live video feed from your camera.

Example Camera view inside Viam app

Next Steps