An Introduction to Using Viam's Data Management Service
In this tutorial, we will cover how to use Data Management, including capturing camera data, syncing that data from your robot to the cloud, viewing and filtering synced data, and downloading it to your computer.
One key feature of Viam is Data Management, which helps you manage data on your robot every step of the way, from capturing component data on your robot, to managing your data securely in the cloud.
Viam’s Data Management Service has two distinct parts: data capture and cloud sync.
Data capture allows you to capture data from specific components on your robot running Viam. You can choose the components, corresponding methods, and the frequency of the data capture all within the Viam app.
Cloud sync runs in the background and uploads your robot’s captured data to Viam’s cloud at a defined frequency. Cloud sync is designed to be resilient and to preserve your data even during a network outage or if your robot has low network bandwidth. It also manages syncing data from your robot to the Viam app and deletes the captured data from your robot after a successful sync. Plus, your data is encrypted in transit and at rest in the cloud, so it is always protected.
Data capture is frequently used with data sync. However, if you want to manage your robot’s captured data yourself, it’s possible to enable data capture for components on your robot, but disable data sync.
A rented or owned robot
Viam’s Data Management features will work on any robot running Viam. To follow along with this tutorial, you can use one of your own robots, or you can rent a rover and use it to try out Viam’s Data Manager without having to configure your own hardware.
To learn more about the Try Viam experience, see Try Viam.
If you are using your own robot, be sure that you have
viam-serverinstalled on your robot. The data management setup process will be mostly the same, but you will need to substitute your robot’s components.
To use the data export feature, you need to have the Go binary installed on your local development computer. We suggest that you install this before your reservation starts to maximize your time with your rover.
Add the Data Management Service
First, you need to add and configure the Data Management Service to capture data and store it at a specified location. To enable the data capture on your robot, do the following:
On your robot’s Config page, navigate to the Services tab.
At the bottom of the page, create a service. Choose
Data Managementas the type and specify
viam-data-manageras the name for your instance of the Data Management Service. This service syncs data from your robot to the Viam app in the cloud.
Then click Create Service.
On the panel that appears, you can manage the capturing and syncing functions individually. The Data Management Service captures data every 0.1 minutes in the
~/.viam/capturedirectory by default.
You can leave the default settings as they are. Click Save Config at the bottom of the window.
For more detailed information see Add the Data Management Service.
Configure data capture for a component
With the Data Management service added, you can now configure data capture for specific components on your robot running Viam. You can choose the components, corresponding methods, and the frequency of the data capture all within the Viam app.
We’re enabling data capture from a camera for this tutorial, but you can enable data capture on most Viam components. This allows you to capture not just image data, but sensor data, robot state data, and so on.
To enable image data capture for a camera component, follow these steps:
Navigate to the Components tab on your robot’s Config page.
Scroll down to the camera component. If you are using a Viam Rover, the camera is named
cam. The camera component has a section labeled Data Capture Configuration.
Add Methodto enable data capture for this camera.
Set the Type to “ReadImage” and the Frequency to
0.333. This will capture an image from the camera roughly once every 3 seconds. Feel free to adjust the frequency if you want the camera to capture more or less image data.
Avoid configuring data capture to higher rates than your hardware can handle, as this leads to performance degradation.
Select the MIME type you want to capture. Select
image/jpegdata for this tutorial.
Click Save Config at the bottom of the window.
Now your image data will be saved locally on your robot in
If you are following along using your own robot, you can confirm that your data is being captured locally.
On your robot, navigate to the
For more detailed information see Configure Data Capture and Configure Cloud Sync.
View and filter captured data
Now that you have data capture enabled for a camera on your Viam Rover, you can see the images from your robot in the Viam app.
Head over to the DATA page.
Rover Rentalas the Location, and click SEARCH. If you have data capture set up correctly, you will see the captured images from your rover’s camera.
For more detailed information see View and Filter Data.
Export captured data
You’ve successfully saved data from your robot in the cloud. Now, let’s export that image data from the Viam app onto your local computer.
To export data from Viam:
First, install the Viam CLI and authenticate:
go install go.viam.com/rdk/cli/viam@latest viam auth
Head back to the DATA page in the Viam app.
Below the SEARCH button in the Filtering panel, click Copy Export Command to copy the export command to the clipboard.
Run the copied command in a terminal:
viam data export --org_ids=<org_id> --data_type=binary --mime_types=<mime_types> --destination=.
This command uses the Viam CLI to download the data locally onto your computer based on the search criteria you select in the Viam app.
Once the command has finished running and downloading the data, you can view and use the data locally in a new directory named
Since images are downloaded in parallel, some may be out of order. Sort your folder by filename to see them in chronological order.
For more detailed information see Export Data.
In this tutorial, you have learned how to use Data Management to capture camera data, sync that data from your robot to the cloud, and view and export the data.
You can now set up data management for your robots to track your robot’s usage over time, build dashboards to visualize the data from your robot, or train machine learning models to detect hand gestures or objects.
You can also ask questions in the Community Discord and we will be happy to help.
Have questions, or want to meet other people working on robots? Join our Community Discord.
Was this page helpful?
Glad to hear it! If there is anything we could be doing better, please create an issue.
We're sorry about that. If you'd like to talk to us for help, please join the Community Discord. To ensure we know what's wrong with this page, you can also open an issue.