Edit

The Robot Frame System

Any machine configured in Viam comes with the frame system service: an internally managed and mostly static system for storing the “reference frame” of each component of a machine within a coordinate system configured by the user.

Visualization of a wheeled base configured with motors and a mounted camera in the frame system tab of the Viam app UI

The frame system is the basis for some of Viam’s other services, like motion and vision. It stores the required contextual information to use the position and orientation readings returned by some components.

Used with

Arm

Base

Board

Camera

Gantry

Motor

Configuration

You can configure a reference frame within the frame system for each of your machine’s components on the Frame subtab of the CONFIGURE tab or in the raw JSON configuration.

Frame Editor
JSON Editor

Navigate to the CONFIGURE tab of the machine’s page in the Viam app and select the Frame mode.
From the left-hand menu, select your component. If you haven’t adjusted any parameters yet, the default reference frame will be shown for the component:
To adjust the frame from its default configuration, change the parameters as needed for your machine before saving. Select a Parent frame and fill in the coordinates for Translation (mm) and Orientation (deg, rad, or q), according to the position and orientation of your component in relation to the Parent frame.
Select Save in the top right corner of the page to save your config.

Navigate to the CONFIGURE tab of the machine’s page in the Viam app and select the JSON mode.
Edit the JSON inside your component object to add a "frame" configuration.

JSON Template
JSON Example

You can add a reference frame to your component with the following template:

{
  "components": [
    {
      "name": "<your_component_name_1>",
      "type": "<your_component_type_1>",
      "model": "<your_component_model_1>",
      "attributes": { ... },
      "depends_on": [],
      "frame": {
        "parent": "<world_or_parent_component_name>",
        "translation": {
          "y": <int>,
          "z": <int>,
          "x": <int>
        },
        "orientation": {
          "type": "<type>",
          "value": {
            "x": <int>,
            "y": <int>,
            "z": <int>,
            "th": <int> // "w": <int> if "type": "quaternion"
          }
        },
        "geometry": {
          "type": "<type>",
          "x": <int>,
          "y": <int>,
          "z": <int>
        }
      }
    }
  ]
}

{
  "components": [
    {
      "name": "cam",
      "namespace": "rdk",
      "type": "camera",
      "model": "webcam",
      "attributes": {
        "video_path": "FDF90FEC-59E5-4FCF-AABD-DA03C4E19BFB"
      },
      "frame": {
        "parent": "world",
        "translation": {
          "x": 0,
          "y": 0,
          "z": 0
        },
        "orientation": {
          "type": "quaternion",
          "value": {
            "x": 0,
            "y": 0,
            "z": 0,
            "w": 1
          }
        },
        "geometry": {
          "type": "box",
          "x": 100,
          "y": 100,
          "z": 100
        }
      }
    }
  ]
}

Configure the reference frame as follows:

Parameter	Inclusion	Required
`parent`	Required	Default: `world`. The name of the reference frame you want to act as the parent of this frame.
`translation`	Required	Default: `(0, 0, 0)`. The coordinates that the origin of this component’s reference frame has within its parent reference frame. Units: mm.
`orientation`	Required	Default: `(0, 0, 1), 0`. The orientation vector that yields the axes of the component’s reference frame when applied as a rotation to the axes of the parent reference frame. Types: Orientation Vector Degrees (`ov_degrees`), Orientation Vector Radians (`ov_radians`), Euler Angles (`euler_angles`), and Quaternion (`quaternion`).
`geometry`	Optional	Default: `none`. Collision geometries for defining bounds in the environment of the machine. Units: mm Types: Sphere (`sphere`), Box (`box`), and Capsule (`capsule`).

Info

The orientation parameter offers types for ease of configuration, but the frame system always stores and returns orientation vectors in Orientation Vector Radians. Other types will be converted to ov_radian.

Tip

For base components, Viam considers +X to be to the right, +Y to be forwards, and +Z to be up. You can use the right-hand rule to understand rotation about any of these axes.

For non base components, there is no inherent concept of “forward,” so it is up to the user to define frames that make sense in their application.

For more information about determining the appropriate values for these parameters, see these two examples:

A Reference Frame: A component attached to a static surface
Nested Reference Frames: A component attached to another, dynamic, component

Visualize the frame system

You can visualize how your machine is oriented in the frame system in the Viam app. Navigate to the CONFIGURE tab on your machine’s page and select the Frame mode.

The Viam app shows you a 3D visualization of the spatial configuration of the reference frames of components configured on your machine:

Default frame system configuration grid visualization for a single component, shown in the Frame System Editor

On this tab, you can simultaneously view and edit the position, orientation, and geometries of your machine’s components in the frame system.

For example:

Consider a machine configured with a jetson board, wired to a webcam camera and a wheeled base with two motors driving its wheels.

No reference frame configuration has been specified, so on the Frame subtab of the CONFIGURE tab, the components are shown to all be located on the default world origin point as follows:

Example machine’s default frame configuration shown in the Frame System Editor. All components are stuck on top of each other

The distance on the floor from the wheeled base to the board and camera setup is 200 millimeters.

Add this value to "x" in the base’s reference frame Translation attribute, and the frame system readjusts to show the base’s translation:

Base translated 200mm forwards shown in the Frame System Editor

The distance from the board to the camera mounted overhead is 50 millimeters.

Add this value to "z" in the camera’s reference frame Translation attribute, and the frame system readjusts to show the camera’s translation:

Camera translated 50 mm overhead shown in the Frame System Editor

Now the distance between these components is accurately reflected in the visualization. However, the camera doesn’t yet display as oriented towards the base.

Adjust the orientation vector to 0.5 degrees in "ox" in the camera’s reference frame Orientation attribute, and the frame system readjusts to show the camera’s orientation:

Camera oriented .5 degrees OX shown in the Frame System Editor

Now that the frame system is accurately configured with the machine’s spatial orientation, motion service methods that take in reference frame information can be utilized.

Display options

Click and drag on the Frame visualization to view the display from different angles, and pinch to zoom in and out:

There should have been a video here but your browser does not seem to support it.

Click the grid icons below and to the right of the Frame button or press the C key to switch beween the default perspective and the orthographic view:

Perspective
Orthographic

Default Perspective Camera view shown in the Frame System Editor

Orthographic Camera view shown in the Frame System Editor

Bounding geometries

To visualize a component’s spatial constraints, add geometry properties by selecting a component and selecting a Geometry type in the Frame subtab of the CONFIGURE tab of a machine’s page on the Viam app.

By default, a Geometry is shown surrounding the origin point of a component. You can adjust the parameters of a Geometry to change its size. Parameters vary between Geometry types, but units are in mm.

Demo robot with default box bounds added to the wheeled base, shown in the Frame System Editor

For example:

Demo robot with translated box bounds added to the wheeled base, shown in the Frame System Editor

Demo robot with default sphere bounds added to the wheeled base, shown in the Frame System Editor

Demo robot with default capsule bounds added to the wheeled base, shown in the Frame System Editor

How the frame system works

viam-server builds a tree of reference frames for your machine with the world as the root node and regenerates this tree following reconfiguration.

Access a topologically-sorted list of the generated reference frames in the machine’s logs at --debug level:

an example of a logged frame system

Consider the example of nested reference frame configuration where two dynamic components are attached: a robotic arm, A, attaches to a gantry, G, which in turn is fixed in place at a point on the World of a table.

The resulting tree of reference frames looks like:

reference frame tree

viam-server builds the connections in this tree by looking at the "frame" portion of each component in the machine’s configuration and defining two reference frames for each component:

One with the name of the component, representing the actuator or final link in the component’s kinematic chain: like "A" as the end of an arm.
Another representing the origin of the component, defined with the component’s name and the suffix "_origin".

Access the frame system

The Robot API supplies the following method to interact with the frame system:

Method Name	Description
`TransformPose`	Transform a pose measured in one reference frame to the same pose as it would have been measured in another.

TransformPose

Transform a given source pose from the reference frame to a new specified destination reference frame. For example, if a 3D camera observes a point in space you can use this method to calculate where that point is relative to another object.

Python
Go

Parameters:

query (PoseInFrame): The pose that should be transformed.
destination (str): The name of the reference frame to transform the given pose to.
additional_transforms (Optional[List[Transform]]): A list of additional transforms.

Returns:

PoseInFrame (PoseInFrame): Transformed pose in destination reference frame.

For more information, see the Python SDK Docs.

# Transform a pose into the frame of myArm
first_pose = Pose(x=0.0, y=0.0, z=0.0, o_x=0.0, o_y=0.0, o_z=1.0, theta=0.0)
first_pif = PoseInFrame(reference_frame="world", pose=first_pose)
transformed_pif = await robot.transform_pose(first_pif, "myArm")
print("Position: (x:", transformed_pif.pose.x,
      ", y:", transformed_pif.pose.y,
      ", z:", transformed_pif.pose.z, ")")
print("Orientation: (o_x:", transformed_pif.pose.o_x,
      ", o_y:", transformed_pif.pose.o_y,
      ", o_z:", transformed_pif.pose.o_z,
      ", theta:", transformed_pif.pose.theta, ")")

Parameters:

ctx (Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
pose ([PoseInFrame]): The pose that should be transformed.
dst (string): The name of the reference frame to transform the given pose to.
additionalTransforms (Optional[LinkInFrame]): A list of additional transforms.

Returns:

(error): An error, if one occurred.
PoseInFrame (referenceframe.PoseInFrame): Transformed pose in destination reference frame.

For more information, see the Go SDK Docs.

// Define a Pose coincident with the world reference frame
firstPose := spatialmath.NewPoseFromPoint(r3.Vector{X: 0.0, Y: 0.0, Z: 0.0})

// Establish the world as the reference for firstPose
firstPoseInFrame := referenceframe.NewPoseInFrame(referenceframe.World, firstPose)

// Calculate firstPoseInFrame from the perspective of the origin frame of myArm
transformedPoseInFrame, err := robot.TransformPose(ctx, firstPoseInFrame, "myArm", nil)
fmt.Println("Transformed Position:", transformedPoseInFrame.Pose().Point())
fmt.Println("Transformed Orientation:", transformedPoseInFrame.Pose().Orientation())

Additional transforms

Additional transforms exist to help the frame system determine the location of and relationships between objects not initially known to the machine.

For example:

In our example of nested dynamic attachment, the arm can be managed by the frame system without additional transforms because the base of the arm is fixed with respect to the gantry’s platform, and the gantry’s origin is fixed with respect to the world reference frame (centered at (0, 0, 0) in the machine’s coordinate system).
However, an arm with an attached camera might generate additional information about the poses of other objects with respect to references frames on the machine.
With the vision service, the camera might detect objects that do not have a relationship to a world reference frame.
If a camera is looking for an apple or an orange, the arm can be commanded to move to the detected fruit’s location by providing an additional transform that contains the detected pose with respect to the camera that performed the detection.
The detecting component (camera) would be fixed with respect to the world reference frame, and would supply the position and orientation of the detected object.
With this information, the frame system could perform the right calculations to express the pose of the object in the world reference frame.

Usage:

You can pass a detected object’s frame information to the supplemental_transforms parameter in your calls to Viam’s motion service’s GetPose method.
Functions of some services and components also take in a WorldState parameter, which includes a transforms property.
TransformPose has the option to take in these additional transforms.

Was this page helpful?

Glad to hear it! If you have any other feedback please let us know:

We're sorry about that. To help us improve, please tell us what we can do better:

Thank you!