Data Management
The data management service is a robust solution for smart machine data handling, data capture, and cloud synchronization. Using the data management service, you can collect data from different parts of a robot, IoT device, or any other machine. Once collected, you can configure which data to securely sync to the cloud, and view, sort, or manage it directly in the cloud without needing to manually gather data from each machine.
Viam’s data management features can be broken down into on-device and cloud data capabilities:
On-device data management
Manage data directly on your robot, with configurable data capture from various components, automated recovery after interruptions, and secure cloud synchronization for efficient data storage and management.
Cloud data management
Experience streamlined data handling with advanced querying, viewing, and filtering capabilities, along with efficient data labeling and exporting tools.
Get started
Collect your data and send to the Viam platform
You must configure data capture and cloud sync with the data management service to be able to view, label, and export data.
Used with
You can configure the frequency of data capture individually for each supported component:
Query your data
Once you have synced, you can query the data you’ve collected in multiple ways, including through the data client API or inside the Viam app. For tabular sensor data, you can run SQL or MQL queries against your synced data from the Query subtab of the Data tab in the Viam app.
Permissions
Data management permissions vary between owners and operators. For more information about who can do what with data, see Data Permissions.
API
The data management service supports the following methods:
| Method Name | Description | micro-RDK Support |
|---|---|---|
Sync | Sync data stored on the machine to the cloud. | |
Reconfigure | Reconfigure this resource. | |
DoCommand | Execute model-specific commands that are not otherwise defined by the service API. | |
Close | Safely shut down the resource and prevent further use. |
The data client API supports a separate set of methods that allow you to upload and export data to and from the Viam app. For information about that API, see Data Client API.
Tip
The following code examples assume that you have a machine configured with a data management service called "my_data_service", and that you add the required code to connect to your machine and import any required packages at the top of your code file.
Go to your machine’s CONNECT tab on the Viam app and select the Code sample page for boilerplate code to connect to your machine.
Sync
Important
This method is not yet available in the Viam Python SDK.
Sync data stored on the machine to the cloud.
Parameters:
ctx(Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.extra(map[string]interface{}): Extra options to pass to the underlying RPC call.
Returns:
- (error): An error, if one occurred.
Example:
// Sync data stored on the machine to the cloud.
err := data.Sync(context.Background(), nil)
For more information, see the Go SDK Docs.
Reconfigure
Reconfigure this resource. Reconfigure must reconfigure the resource atomically and in place.
Parameters:
ctx(Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.deps(Dependencies): The resource dependencies.conf(Config): The resource configuration.
Returns:
- (error): An error, if one occurred.
For more information, see the Go SDK Docs.
DoCommand
Execute model-specific commands that are not otherwise defined by the service API.
For built-in service models, any model-specific commands available are covered with each model’s documentation.
If you are implementing your own data manager service and add features that have no built-in API method, you can access them with DoCommand.
Parameters:
ctx(Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.cmd(map[string]interface{}): The command to execute.
Returns:
- (map[string]interface{}): The command response.
- (error): An error, if one occurred.
Example:
// This example shows using DoCommand with an arm component.
myArm, err := arm.FromRobot(machine, "my_arm")
command := map[string]interface{}{"cmd": "test", "data1": 500}
result, err := myArm.DoCommand(context.Background(), command)
For more information, see the Go SDK Docs.
Close
Safely shut down the resource and prevent further use.
Parameters:
ctx(Context): A Context carries a deadline, a cancellation signal, and other values across API boundaries.
Returns:
- (error): An error, if one occurred.
Example:
// This example shows using Close with an arm component.
myArm, err := arm.FromRobot(machine, "my_arm")
err = myArm.Close(ctx)
For more information, see the Go SDK Docs.
Next steps: train and deploy machine learning
You can use data synced to the cloud to train machine learning models and then deploy these models to your machines from the Viam app.
For a comprehensive tutorial on using data capture and synchronization together with the ML model service, see:
Was this page helpful?
Glad to hear it! If you have any other feedback please let us know:
We're sorry about that. To help us improve, please tell us what we can do better:
Thank you!