Configure Data Capture
The data management service captures data from one or more components locally on the machine’s storage.
The process runs in the background and, by default, stores data in the
If a machine restarts for any reason, capture automatically resumes.
The service can capture data from multiple components at the same or different frequencies. Viam does not impose a lower or upper limit on the frequency of data collection. However, in practice, your hardware may impose limits on the frequency of data collection.
You can change the frequency of data capture at any time for individual components. If you use fragments, you can change the frequency of data capture in real time for some or all machines in a fleet at the component or machine level.
For example, consider a tomato picking robot with a 3D camera and an arm. When you configure the robot, you may set the camera to capture point cloud data at a frequency of 30Hz. For the arm, you may want to capture joint positions at 1Hz. If your requirements change and you want to capture data from both components at 10Hz, you can change the configurations at any time by changing the number.
Data capture is frequently used with Cloud Sync. However, if you want to manage your machine’s captured data yourself, you can enable only data capture without cloud sync.
Add the data management service
To capture data from one or more machines, you must first add the data management service:
From your machine’s Config tab, navigate to the Services subtab.
Click Create service in the lower-left corner of the page. Choose
Data Managementas the type and specify a name for your data management service, for example
On the panel that appears, you can manage the capturing and syncing functions and specify the directory, the sync interval and any tags to apply to captured data.
If the sync interval or the directory is not specified, the data management service captures data at the default frequency every 0.1 minutes (after every 6 second interval) in the default
If you change the directory for data capture only new data is stored in the new directory. Existing data remains in the directory where it was stored.
Click Save Config.
Configure data capture for individual components
Once you have added the data capture service, you can specify the data you want to capture at a component level. The following components support data capture:
- Movement Sensor (includes GPS)
To add data capture for a component, navigate to the Config tab of your machine’s page in the Viam app.
For each component you can capture data for, there is a
Data Capture Configuration section in its panel.
Add Method and then select the method type and the capture frequency.
Avoid configuring data capture to higher rates than your hardware can handle, as this leads to performance degradation.
Click Save Config at the bottom of the window.
Now your data will be saved locally on your machine to the directory specified in the data management service.
For example, a camera has the options
NextPointCloud and a motor has the options
You may capture data from one or more component methods:
- To enable or disable data capture for a configured component or method, use the
- To change the frequency of data capture for a method, enter the number of measurements you wish to capture per second in the frequency field.
After adding configuration for the methods, click Save Config.
If you want to remove a capture method from the configuration, click the
Configure data capture for remote parts
Viam supports data capture from resources on remote parts. For example, if you use a part that does not have a Linux operating system or that does not have enough storage or processing power, you can still process and capture the data from that part’s components by adding it as a remote part.
Currently, you can only configure data capture from remote components in your raw JSON configuration.
To add them to your JSON configuration you must explicitly add the remote component’s
additional_params to the
data_manager service configuration in the
|The type tells your machine what the component is. For example, a board.
|The model is a colon-delimited-triplet that specifies the namespace, the type of the part, and the part itself.
|The name specifies the fully qualified name of the part.
|The additional parameters specify the data sources when you are using a board.
Images are dim on start up
If you are capturing camera data, it can happen that the camera captures and syncs miscolored or dark images upon start up.
To sync your captured data with the cloud, configure cloud sync.
If you have synced tabular data, such as sensor readings, you can query that data with SQL or MQL from the Viam app or a MQL-compatible client. If you have synced images, you can use those images to train machine learning models within the Viam app.
For a comprehensive tutorial on using data capture and synchronization together with the ML model service, see Capture Data and Train a Model.
Have questions, or want to meet other people working on robots? Join our Community Discord.
Was this page helpful?
Glad to hear it! If you have any other feedback please let us know:
We're sorry about that. To help us improve, please tell us what we can do better: