Configure an Align Color Depth Extrinsics View

Use the intrinsics of the color and depth camera, as well as the extrinsic pose between them, to align two images.

Config Builder
JSON Template

Navigate to the Config tab of your robot’s page in the Viam app. Click on the Components subtab and click Create component. Select the camera type, then select the align_color_depth_extrinsics model. Enter a name for your camera and click Create.

Configuration of an align color depth extrinsics view in the Viam app config builder.

Edit and fill in the attributes as applicable.

{
    "name": "<your-camera-name>",
    "type": "camera",
    "model" : "align_color_depth_extrinsics",
    "attributes": {
        "camera_system": {
            "color_intrinsic_parameters": {
                "width_px": <int>,
                "height_px": <int>,
                "fx": <float>,
                "fy": <float>,
                "ppx": <float>,
                "ppy": <float>
            },
            "depth_intrinsic_parameters": {
                "width_px": <int>,
                "height_px": <int>,
                "fx": <float>,
                "fy": <float>,
                "ppx": <float>,
                "ppy": <float>
            },
                "depth_to_color_extrinsic_parameters": {
                "translation_mm": [ <float>, <float>, <float>],
                "rotation_rads": [ <float>, <float>, <float>,
                                   <float>, <float>, <float>,
                                   <float>, <float>, <float> ],
            }
        },
        "intrinsic_parameters": {
            "width_px": <int>,
            "height_px": <int>,
            "fx": <float>,
            "fy": <float>,
            "ppx": <float>,
            "ppy": <float>
        },
        "output_image_type": "<color|depth>",
        "color_camera_name": "<your-color-camera-name>",
        "depth_camera_name": "<your-depth-camera-name>"
    }
}

The following attributes are available for align_color_depth_extrinsics views:

Name	Type	Inclusion	Description
`camera_system`	object	Required	The intrinsic parameters of the camera used to do 2D <-> 3D projections: `color_intrinsic_parameters`: The model uses the color camera intrinsics to project the 3D points to a 2D depth map, but “as if” it was taken from the POV of the color camera. `depth_intrinsic_parameters`: The model uses the depth camera intrinsics to de-project the 2D depth points to 3D points, from the point of view of the depth camera. `depth_to_color_extrinsic_parameters`: The model uses the extrinsic parameters to shift the 3D depth points to be from the POV of the color camera.
`intrinsic_parameters`	object	Required	The intrinsic parameters of the camera used to do 2D <-> 3D projections: `width_px`: The expected width of the aligned image in pixels. `height_px`: The expected height of the aligned image in pixels. `fx`: The image center x point. `fy`: The image center y point. `ppx`: The image focal x. `ppy`: The image focal y.
`output_image_type`	string	Required	Specify `color` or `depth` for the output stream.
`color_camera_name`	string	Required	`name` of the color camera to pull images from.
`depth_camera_name`	string	Required	`name` of the depth camera to pull images from.
`distortion_parameters`	object	Optional	Modified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens: `rk1`: The radial distortion x. `rk2`: The radial distortion y. `rk3`: The radial distortion z. `tp1`: The tangential distortion x. `tp2`: The tangential distortion y.
`debug`	boolean	Optional	Enables the debug outputs from the camera if `true`. Default: `false`.

View the camera stream

Once your camera is configured and connected, go to the Control tab, and click on the camera’s dropdown menu. Then toggle the camera or the Point Cloud Data view to ON. If everything is configured correctly, you will see the live video feed from your camera. You can change the refresh frequency as needed to change bandwidth.

Next Steps

Have questions, or want to meet other people working on robots? Join our Community Discord.

Was this page helpful?

Glad to hear it! If you have any other feedback please let us know:

We're sorry about that. To help us improve, please tell us what we can do better:

Thank you!