Camera Component

A camera component is a source of 2D and/or 3D images. You can use the component to configure a webcam, lidar, time-of-flight sensor, or another type of camera.

The API for camera components allows you to:

Request single images or a stream in 2D color, or display z-depth.
Request a point cloud. Each 3D point cloud image consists of a set of coordinates (x,y,z) representing depth in mm.

The configuration of your camera component depends on your camera model. You can use different models to:

Configure physical cameras that generate images or point clouds.
Combine streams from multiple cameras into one.
Transform and process images.

Data Management

Computer Vision

Frame System

SLAM

Deploy Model

Supported Models

To use your camera with Viam, check whether one of the following built-in models or modular resources supports your camera.

Add support for other models

If none of the existing models fit your use case, you can create a modular resource to add support for it.

Built-in models

For configuration information, click on the model name:

Model	Description
`ffmpeg`	Uses a camera, a video file, or a stream as a camera.
`image_file`	Gets color and depth images frames from a file path.
`velodyne`	Uses velodyne lidar.
`webcam`	A standard camera that streams camera data.
`rtsp`	A streaming camera with an MJPEG track.
`fake`	A camera model for testing.
`single_stream`	A HTTP client camera that streams image data from an HTTP endpoint.
`dual_stream`	A HTTP client camera that combines the streams of two camera servers to create colorful point clouds.
`join_color_depth`	Joins the outputs of a color and depth camera already registered in your config to create a third “camera” that outputs the combined and aligned image.
`align_color_depth_extrinsics`	Uses the intrinsics of the color and depth camera, as well as the extrinsic pose between them, to align two images.
`align_color_depth_homography`	Uses a homography matrix to align the color and depth images.
`join_pointclouds`	Combines the point clouds from multiple camera sources and projects them to be from the point of view of target_frame.
`transform`	A pipeline for applying transformations to an input image source.
`ultrasonic`	The HC-S204 ultrasonic distance sensor.

Modular Resources

Search for additional camera models that you can add from the Viam Registry:

For configuration information, click on the model name:

Model

Description

Control your camera with Viam’s client SDK libraries

To get started using Viam’s SDKs to connect to and control your robot, go to your robot’s page on the Viam app, navigate to the Code sample tab, select your preferred programming language, and copy the sample code generated.

API key and API key id

By default, the sample code does not include your robot API key and API key id. We strongly recommend that you add your API key and API key id as an environment variable and import this variable into your development environment as needed.

To show your robot’s API key and API key id in the sample code, toggle Include secret on the Code sample tab. You can also see your API key and API key id on your robot’s Security tab.

Caution

Do not share your API key or robot address publicly. Sharing this information could compromise your system security by allowing unauthorized access to your robot, or to the computer running your robot.

When executed, this sample code will create a connection to your robot as a client. Then control your robot programmatically by adding API method calls as shown in the following examples.

These examples assume you have a camera called "my_camera" configured as a component of your robot. If your camera has a different name, change the name in the code.

Be sure to import the camera package for the SDK you are using:

Python
Go

from viam.components.camera import Camera

import (
  "go.viam.com/rdk/components/camera"
)

API

The camera component supports the following methods:

Method Name	Description
`GetImage`	Return an image from the camera.
`GetImages`	Return multiple simultaneous images from different imagers, along with associated metadata.
`GetPointCloud`	Return a point cloud from the camera.
`GetProperties`	Return the camera intrinsic and camera distortion parameters, as well as whether the camera supports returning point clouds.
`GetGeometries`	Get all the geometries associated with the camera in its current configuration, in the frame of the camera.
`DoCommand`	Send or receive model-specific commands.
`Close`	Safely shut down the resource and prevent further use.

GetImage

Returns an image from the camera. You can request a specific MIME type but the returned MIME type is not guaranteed. If the server does not know how to return the specified MIME type, the server returns the image in another format instead.

Python
Go

Parameters:

mime_type (str): The requested MIME type of the image. The returned MIME type is not guaranteed to match the image output type: If the requested MIME type has +lazy appended to it or the requested MIME type is not supported for decoding/encoding, GetImage returns a RawImage instead.

Returns:

(PIL.Image or RawImage): The requested frame.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

frame = await my_camera.get_image(mime_type="image/jpeg")

If the mime_type of your image is image/vnd.viam.dep, pass the returned image data to the Viam Python SDK’s RawImage.bytes_to_depth_array() method to decode the raw image data to a standard 2D image representation.

For example:

# Assume "frame" has a mime_type of "image/vnd.viam.dep"
frame = await my_camera.get_image()

# Convert "frame" to a standard 2D image representation.
# Remove the 1st 3x8 bytes and reshape the raw bytes to List[List[Int]].
standard_frame = frame.bytes_to_depth_array()

Tip

Be sure to close the image when finished.

For more information, see the Python SDK Docs.

Info

Unlike most Viam component APIs, the methods of the Go camera client do not map exactly to the names of the other SDK’s camera methods. To get an image in the Go SDK, you first need to construct a Stream and then you can get the next image from that stream.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
errHandlers (ErrorHandler): A handler for errors allowing for logic based on consecutively retrieved errors).

Returns:

(gostream.VideoStream): A VideoStream that streams video until closed.
(error): An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

// gets the stream from a camera
stream, err := myCamera.Stream(context.Background())

// gets an image from the camera stream
img, release, err := stream.Next(context.Background())
defer release()

For more information, see the Go SDK Docs.

GetImages

Get simultaneous images from different imagers, along with associated metadata. The multiple images returned from GetImages do not represent a time series of images.

Python
Go

Parameters:

None.

Returns:

(List[NamedImage]): The list of images returned from the camera system.
(ResponseMetadata): The metadata timestamp with this response.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

images, metadata = await my_camera.get_images()
img0 = images[0].image
timestamp = metadata.captured_at

For more information, see the Python SDK Docs.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Returns:

([]camera.NamedImage): The list of images returned from the camera system, with the name of the imager associated with the image.
(resource.ResponseMetadata): The metadata, which holds the timestamp of when the data was captured.
(error): An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

images, metadata, err := myCamera.Images(context.Background())

For more information, see the Go SDK Docs.

GetPointCloud

Get a point cloud from the camera as bytes with a MIME type describing the structure of the data. The consumer of this call should decode the bytes into the format suggested by the MIME type.

Python
Go

Parameters:

None.

Returns:

(Tuple[bytes,str]): The pointcloud data as bytes paired with a string representing the MIME type of the pointcloud (for example, PCD).

To deserialize the returned information into a numpy array, use the Open3D library:

import numpy as np
import open3d as o3d

my_camera = Camera.from_robot(robot=robot, name="my_camera")

data, _ = await my_camera.get_point_cloud()

# write the point cloud into a temporary file
with open("/tmp/pointcloud_data.pcd", "wb") as f:
    f.write(data)
pcd = o3d.io.read_point_cloud("/tmp/pointcloud_data.pcd")
points = np.asarray(pcd.points)

For more information, see the Python SDK Docs.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Returns:

(pointcloud.PointCloud): A general purpose container of points. It does not dictate whether or not the cloud is sparse or dense.
(error): An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

pointCloud, err := myCamera.NextPointCloud(context.Background())

For more information, see the Go SDK Docs.

GetProperties

Get the camera intrinsic parameters and camera distortion, as well as whether the camera supports returning point clouds.

Python
Go

Parameters:

None.

Returns:

(Properties): The properties of the camera.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

properties = await my_camera.get_properties()

For more information, see the Python SDK Docs.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Returns:

(Properties): Properties of the particular implementation of a camera.
(error): An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

// gets the properties from a camera
properties, err := myCamera.Properties(context.Background())

For more information, see the Go SDK Docs.

GetGeometries

Get all the geometries associated with the camera in its current configuration, in the frame of the camera. The motion and navigation services use the relative position of inherent geometries to configured geometries representing obstacles for collision detection and obstacle avoidance while motion planning.

Python

Parameters:

extra (Optional[Dict[str, Any]]): Extra options to pass to the underlying RPC call.
timeout (Optional[float]): An option to set how long to wait (in seconds) before calling a time-out and closing the underlying RPC call.

Returns:

(List[Geometry]): The geometries associated with the camera, in any order.

For more information, see the Python SDK Docs.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

geometries = await my_camera.get_geometries()

if geometries:
    # Get the center of the first geometry
    print(f"Pose of the first geometry's centerpoint: {geometries[0].center}")

DoCommand

Execute model-specific commands that are not otherwise defined by the component API. For native models, model-specific commands are covered with each model’s documentation. If you are implementing your own camera and adding features that have no native API method, you can access them with DoCommand.

Python
Go

Parameters:

command (Dict[str, Any]): The command to execute.

Returns:

(Dict[str, Any]): Result of the executed command.

my_camera = Camera.from_robot(robot, "my_camera")

command = {"cmd": "test", "data1": 500}
result = my_camera.do(command)

For more information, see the Python SDK Docs.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
cmd (map[string]interface{}): The command to execute.

Returns:

(map[string]interface{}): Result of the executed command.
(error): An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

command := map[string]interface{}{"cmd": "test", "data1": 500}
result, err := myCamera.DoCommand(context.Background(), command)

For more information, see the Go SDK Code.

Close

Safely shut down the resource and prevent further use.

Python
Go

Parameters:

None

Returns:

None

my_camera = Camera.from_robot(robot, "my_camera")

await my_camera.close()

For more information, see the Python SDK Docs.

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.

Returns:

(error) : An error, if one occurred.

myCamera, err := camera.FromRobot(robot, "my_camera")

err := myCamera.Close(ctx)

For more information, see the Go SDK Docs.

Next Steps

You can also ask questions in the Community Discord and we will be happy to help.

Was this page helpful?

Glad to hear it! If you have any other feedback please let us know:

We're sorry about that. To help us improve, please tell us what we can do better:

Thank you!

Camera Component

Related Services

Supported Models

Add support for other models

Built-in models

Modular Resources

Control your camera with Viam’s client SDK libraries

API key and API key id

Caution

API

GetImage

Tip

Info

GetImages

GetPointCloud

GetProperties

GetGeometries

DoCommand

Close

Next Steps