Camera Component

A camera component is a source of 2D and/or 3D images. You can use the component to configure a webcam, lidar, time-of-flight sensor, or another type of camera.

The API for camera components allows you to:

• Request single images or a stream in 2D color, or display z-depth.

• Request a point cloud. Each 3D point cloud image consists of a set of coordinates (x,y,z) representing depth in mm.

The configuration of your camera component depends on your camera model. You can use different models to:

• Configure physical cameras that generate images or point clouds.
• Combine streams from multiple cameras into one.
• Transform and process images.

Related services

Data Management

Computer Vision

Frame System

SLAM

Machine Learning

Supported models

To use your camera with Viam, check whether one of the following built-in models or modular resources supports your camera.

Built-in models

For configuration information, click on the model name:

Modular resources

Search for additional camera models that you can add from the Viam Registry:

For configuration information, click on the model name:

Control your camera with Viam’s client SDK libraries

To get started using Viam’s SDKs to connect to and control your machine, go to your machine’s page on the Viam app, navigate to the CONNECT tab’s Code sample page, select your preferred programming language, and copy the sample code generated.

When executed, this sample code will create a connection to your machine as a client. Then control your machine programmatically by adding API method calls as shown in the following examples.

These examples assume you have a camera called "my_camera" configured as a component of your machine. If your camera has a different name, change the name in the code.

Be sure to import the camera package for the SDK you are using:

from viam.components.camera import Camera

import (
  "go.viam.com/rdk/components/camera"
)

API

The camera component supports the following methods:

GetImage

Returns an image from the camera. You can request a specific MIME type but the returned MIME type is not guaranteed. If the server does not know how to return the specified MIME type, the server returns the image in another format instead.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

frame = await my_camera.get_image(mime_type="image/jpeg")

# Assume "frame" has a mime_type of "image/vnd.viam.dep"
frame = await my_camera.get_image()

# Convert "frame" to a standard 2D image representation.
# Remove the 1st 3x8 bytes and reshape the raw bytes to List[List[Int]].
standard_frame = frame.bytes_to_depth_array()

myCamera, err := camera.FromRobot(robot, "my_camera")

// gets the stream from a camera
stream, err := myCamera.Stream(context.Background())

// gets an image from the camera stream
img, release, err := stream.Next(context.Background())
defer release()

GetImages

Get simultaneous images from different imagers, along with associated metadata. The multiple images returned from GetImages() do not represent a time series of images.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

images, metadata = await my_camera.get_images()
img0 = images[0].image
timestamp = metadata.captured_at

myCamera, err := camera.FromRobot(robot, "my_camera")

images, metadata, err := myCamera.Images(context.Background())

GetPointCloud

Get a point cloud from the camera as bytes with a MIME type describing the structure of the data. The consumer of this call should decode the bytes into the format suggested by the MIME type.

import numpy as np
import open3d as o3d

my_camera = Camera.from_robot(robot=robot, name="my_camera")

data, _ = await my_camera.get_point_cloud()

# write the point cloud into a temporary file
with open("/tmp/pointcloud_data.pcd", "wb") as f:
    f.write(data)
pcd = o3d.io.read_point_cloud("/tmp/pointcloud_data.pcd")
points = np.asarray(pcd.points)

myCamera, err := camera.FromRobot(robot, "my_camera")

pointCloud, err := myCamera.NextPointCloud(context.Background())

GetProperties

Get the camera intrinsic parameters and camera distortion, as well as whether the camera supports returning point clouds.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

properties = await my_camera.get_properties()

myCamera, err := camera.FromRobot(robot, "my_camera")

// gets the properties from a camera
properties, err := myCamera.Properties(context.Background())

GetGeometries

Get all the geometries associated with the camera in its current configuration, in the frame of the camera. The motion and navigation services use the relative position of inherent geometries to configured geometries representing obstacles for collision detection and obstacle avoidance while motion planning.

my_camera = Camera.from_robot(robot=robot, name="my_camera")

geometries = await my_camera.get_geometries()

if geometries:
    # Get the center of the first geometry
    print(f"Pose of the first geometry's centerpoint: {geometries[0].center}")

DoCommand

Execute model-specific commands that are not otherwise defined by the component API. For native models, model-specific commands are covered with each model’s documentation. If you are implementing your own camera and adding features that have no native API method, you can access them with DoCommand.

my_camera = Camera.from_robot(robot, "my_camera")

command = {"cmd": "test", "data1": 500}
result = my_camera.do(command)

myCamera, err := camera.FromRobot(robot, "my_camera")

command := map[string]interface{}{"cmd": "test", "data1": 500}
result, err := myCamera.DoCommand(context.Background(), command)

Close

Safely shut down the resource and prevent further use.

my_camera = Camera.from_robot(robot, "my_camera")

await my_camera.close()

myCamera, err := camera.FromRobot(robot, "my_camera")

err := myCamera.Close(ctx)

Next steps

Computer Vision

The vision service enables your machine to use its on-board cameras to intelligently see and interpret the world around it.

Detect Color

Use the vision service in the Viam app to detect a color with the Viam Rover.

Colored Object Follower

Instructions for detecting and following a colored object with a rover, like a SCUTTLE robot.

You can also ask questions in the Community Discord and we will be happy to help.