Configure Visual Odometry

Viam provides a viam-visual-odometry modular resource which uses monocular visual odometry to enable any calibrated camera to function as a movement sensor. In this way, you can add basic movement sensing to your camera-equipped robot without needing a dedicated hardware movement sensor.

The viam-visual-odometry module implements the following two methods of the movement sensor API:

• GetLinearVelocity()
• GetAngularVelocity()

Note that GetLinearVelocity() returns an estimation of the instantaneous linear velocity without scale factor. Therefore, you should not consider returned unit measurements trustworthy: instead, GetLinearVelocity() should serve as a direction estimation only.

While viam-visual-odometry enables you to add movement sensing abilities to your robot without needing specialized hardware, a dedicated movement sensor will generally provide more accurate readings. If your robot requires precise awareness of its location and its movement, you should consider using a dedicated movement sensor in addition to the viam-visual-odometry module.

The viam-visual-odometry module is available from the Viam registry. See Modular resources for instructions on using a module from the Viam registry on your robot.

The source code for this module is available on the viam-visual-odometry GitHub repository.

Requirements

Follow the instructions below to download and set up the viam-visual-odometry module on your robot:

1. Clone the viam-visual-odometry to your system:

   git clone git@github.com:viamrobotics/viam-visual-odometry.git
   cd viam-visual-odometry
   

2. Install the necessary Python dependencies:

   pip install -r requirements.txt
   

3. If you haven’t already, install viam-server on your robot.

Configuration

To configure the viam-visual-odometry module on your robot, follow the instructions below:

{
  "modules": [
    {
      "name": "<your-visual-odometer-name>",
      "executable_path": "</path/to/run.sh>",
      "type": "local"
    }
  ],
  "components": [
    {
      "name": "<your-camera-name>",
      "type": "camera",
      "model": "webcam",
      "attributes": {
        "video_path": "<path-to-video-stream>",
        "height_px": <height>,
        "width_px": <width>,
        "intrinsic_parameters": {
          "ppx": <ppx>,
          "ppy": <ppy>,
          "fx": <fx>,
          "fy": <fy>
        },
        "distortion_parameters": {
          "rk3": <rk3>,
          "tp1": <tp1>,
          "tp2": <tp2>,
          "rk1": <rk1>,
          "rk2": <rk2>
        }
      },
      "depends_on": []
    },
    {
      "name": "<your_movement_sensor_name>",
      "type": "movement_sensor",
      "namespace": "rdk",
      "model": "viam:visual_odometry:opencv_orb",
      "attributes": {
        "camera_name": "<your-camera-name>",
        "time_between_frames_s": <time_seconds>,
        "lowe_ratio_threshold": <lowe_ratio_threshold>
      },
      "depends_on": []
    }
  ]
}

{
  "modules": [
    {
      "name": "my-odometry-module",
      "executable_path": "/path/to/run.sh",
      "type": "local"
    }
  ],
  "components": [
    {
      "name": "my-camera",
      "type": "camera",
      "model": "webcam",
      "attributes": {
        "video_path": "FDF90FEB-59E5-4FCF-AABD-DA03C4E19BFB",
        "height_px": 720,
        "width_px": 1280,
        "intrinsic_parameters": {
          "ppx": 446,
          "ppy": 585,
          "fx": 1055,
          "fy": 1209
        },
        "distortion_parameters": {
          "rk3": -0.03443,
          "tp1": 0.01364798,
          "tp2": -0.0107569,
          "rk1": -0.1621,
          "rk2": 0.13632
        }
      },
      "depends_on": []
    },
    {
      "name": "my_movement_sensor",
      "type": "movement_sensor",
      "namespace": "rdk",
      "model": "viam:visual_odometry:opencv_orb",
      "attributes": {
        "camera_name": "my-camera",
        "time_between_frames_s": 0.2,
        "lowe_ratio_threshold": 0.75
      },
      "depends_on": []
    }
  ]
}

Camera calibration

Once you have configured a camera component, you need to calibrate it. Because the viam-visual-odometry module performs visual odometry calculations, its visual data source (the camera) must be as well defined as possible. These calibration steps ensure that the video stream data that reaches the module is as uniform as possible when calculating measurements.

1. Follow the Calibrate a camera procedure to generate the required intrinsic parameters specific to your camera.
2. Copy the resulting intrinsics data into your robot configuration, either in the Config builder or in the Raw JSON. See the JSON Example tab above for an example intrinsics configuration.

Camera calibration results should look similar to the following example, with readings specific to your camera:

Example output:

"intrinsic_parameters": {
    "fy": 940.2928257873841,
    "height_px": 480,
    "ppx": 320.6075282958033,
    "ppy": 239.14408757087756,
    "width_px": 640,
    "fx": 939.2693584627577
},
"distortion_parameters": {
    "rk2": 0.8002516496932317,
    "rk3": -5.408034254951954,
    "tp1": -0.000008996658362365533,
    "tp2": -0.002828504714921335,
    "rk1": 0.046535971648456166
}

Attributes

The following attributes are available to configure the viam-visual-odometry module:

See the ORB openCV documentation for more details.

Test the movement sensor

After you configure your movement sensor, navigate to the Control tab and select the dedicated movement sensor dropdown panel. This panel presents the data collected by the movement sensor. The sections in the panel include the position, orientation, angular velocity, linear velocity, and linear acceleration.