Configure an ffmpeg Camera

The ffmpeg camera model uses a camera device, a video file, or a stream as a camera.

When used with a streaming camera, the ffmpeg camera model supports any streaming camera format that is supported by the ffmpeg program, including MJPEG, H264, and MP4.

First, connect your camera to your machine’s computer and power both on. Then, configure your camera:

Navigate to the CONFIGURE tab of your machine’s page in the Viam app. Click the + icon next to your machine part in the left-hand menu and select Component. Select the camera type, then select the ffmpeg model. Enter a name or use the suggested name for your camera and click Create.

Configuration of a ffmpeg camera in the Viam app config builder.

Edit the attributes as applicable to your camera, according to the table below.

{
  "name": "<your-camera-name>",
  "model": "ffmpeg",
  "type": "camera",
  "namespace": "rdk",
  "attributes": {
    "intrinsic_parameters": {
      "width_px": <int>,
      "height_px": <int>,
      "fx": <float>,
      "fy": <float>,
      "ppx": <float>,
      "ppy": <float>
    },
    "distortion_parameters": {
      "rk1": <float>,
      "rk2": <float>,
      "rk3": <float>,
      "tp1": <float>,
      "tp2": <float>
    },
    "video_path": "<your-video-path>",
    "input_kw_args": { ... },
    "filters": [
      {
        "name": <string>,
        "args": [ "<first>", "<second>", ... ],
        "kw_args": { ... }
      }
    ],
    "output_kw_args": { ... },
  }
}

The following attributes are available for ffmpeg cameras:

NameTypeRequired?Description
video_pathstringRequiredThe file path to the camera device, color image file, or streaming camera. If you are using a camera with an RTSP stream, provide the RTSP address to this attribute.
intrinsic_parametersobjectOptionalThe intrinsic parameters of the camera used to do 2D <-> 3D projections:
  • width_px: The expected width of the aligned image in pixels. Value must be positive.
  • height_px: The expected height of the aligned image in pixels. Value must be positive.
  • fx: The image center x point.
  • fy: The image center y point.
  • ppx: The image focal x.
  • ppy: The image focal y.
distortion_parametersobjectOptionalModified Brown-Conrady parameters used to correct for distortions caused by the shape of the camera lens:
  • rk1: The radial distortion x.
  • rk2: The radial distortion y.
  • rk3: The radial distortion z.
  • tp1: The tangential distortion x.
  • tp2: The tangential distortion y.
input_kw_argsobjectOptionalThe input keyword arguments.
filtersarrayOptionalThe file path to the depth image. Array of filter objects that specify:
  • name: The name of the filter.
  • args: The arguments for the filter.
  • kw_args: Any keyword arguments for the filter.
output_kw_argsobjectOptionalThe output keyword arguments.

View the camera stream

Once your camera is configured and connected, expand the TEST panel on the CONFIGURE or CONTROL tabs. If everything is configured correctly, you will see the live feed from your camera.

Example Camera view inside Viam app

Troubleshooting

If your camera is not working as expected, follow these steps:

  1. Check your machine logs on the LOGS tab to check for errors.
  2. Review this camera model’s documentation to ensure you have configured all required attributes.
  3. Click on the TEST panel on the CONFIGURE or CONTROL tab and test if you can use the camera there.

If none of these steps work, reach out to us on the Community Discord and we will be happy to help.

Common errors

Failed to find the best driver that fits the constraints

When working with a camera component, depending on the camera, you may need to explicitly provide some camera-specific configuration parameters.

Solution: Check the specifications for your camera, and manually provide configuration parameters such as width and height to the camera component configuration page on the Viam app. On the CONFIGURE page, find your camera, then fill in your camera’s specific configuration either using the Show more button to show the relevant configuration options, or the {} (Switch to Advanced) button in the top right of the component panel to enter these attributes manually. Provide at least the width and height values to start.

Next steps

For more configuration and usage info, see:

Camera API

Give commands for getting images or point clouds.

Capture and sync image data

Capture images from a camera on your machine and selectively sync images to the cloud with filtering.

Detect objects or people

Detect people and their location in an image with any webcam and a vision service.

Have questions, or want to meet other people working on robots? Join our Community Discord.

If you notice any issues with the documentation, feel free to file an issue or edit this file.