Capture, filter, and sync image data
You can use Viam’s built-in data management service to capture images from a camera on your machine and sync the images to the cloud.
If you want to control the volume of data your machine writes and syncs, you can use a “filtering camera” and only capture images that contain a person or object detected by an ML model. The filtering camera uses a computer vision model to detect objects in your camera feed and selectively stores and uploads only those images that are matched by your model.
With your images synced to the cloud, you can view images from all your machines in the Viam app interface. From there, you can use your image data to do things like train ML models.
In this page
Prerequisites
Collect image data and sync it to the cloud
Adding the data management service to your machine enables you to capture and sync data:
1. Enable the data management service
In the configuration pane for your configured camera component, find the Data capture section. Click Add method.
When the Create a data management service prompt appears, click it to add the service to your machine. You can leave the default data manager settings.
2. Capture data
With the data management service configured on your machine, configure how the camera component itself captures data:
In the Data capture panel of your camera’s configuration, select ReadImage from the method selector.
Set your desired capture frequency.
For example, set it to 0.05 to capture an image every 20 seconds.
Set the MIME type to your desired image format, for example image/jpeg.
3. Save to start capturing
Save the config. With cloud sync enabled, captured data is automatically uploaded to the Viam app after a short delay.
4. View data in the Viam app
Click on the … menu of the camera component and click on View captured data. This takes you to the data tab.
If you do not see images from your camera, try waiting a minute and refreshing the page to allow time for the images to be captured and then synced to the app at the interval you configured.
If no data appears after the sync interval, check the Logs.
Tip
To keep your data organized, you can configure a tag in your data management service config panel. This tag will be applied to all data synced from that machine in the future. If you apply the same tag to all data gathered from all machines that you want to use in your dataset, you can filter by that tag in the Viam app DATA tab, or when querying data.
This is not required, since you can use other filters like time or machine ID in the DATA tab to isolate your data.
You now know how to capture and sync image data. For many use cases, you may not want to capture data unless it meets certain conditions. Consider the example of a security camera that captures data every 5 seconds. This setup would result in a large number of images, of which most wouldn’t be interesting.
In the next part of this guide, you will learn how to filter the data before capturing and syncing it.
Stop data capture
If this is a test project or if you are continuing this how to set up filtering, make sure you stop data capture to avoid incurring fees for capturing large amounts of test data.
In the Data capture section of your camera’s configuration, toggle the switch to Off.
Click the Save button in the top right corner of the page to save your config.
Use filtering to collect and sync only certain images
You can use filtering to selectively capture images using a machine learning (ML) model, for example to only capture images with people or specific objects in them.
Contributors have written several filtering modules that you can use to filter image capture.
The following steps use the filtered_camera module:
1. Add an ML model service to your machine
Add an ML model service on your machine that is compatible with the ML model you want to use, for example TFLite CPU.
2. Select a suitable ML model
Click Select model on the ML model service configuration panel, then select an existing model you want to use, or click Upload a new model to upload your own.
If you’re not sure which model to use, you can use EfficientDet-COCO from the Registry, which can detect people and animals, among other things.
3. Add a vision service to use with the ML model
You can think of the vision service as the bridge between the ML model service and the output from your camera.
Add and configure the vision / ML model service on your machine.
From the Select model dropdown, select the name of your ML model service (for example, mlmodel-1).
4. Configure the filtered camera
The filtered-camera modular component pulls the stream of images from the camera you configured earlier, and applies the vision service to it.
Configure a filtered-camera component on your machine, following the attribute guide in the README.
Use the name of the camera you configured in the first part of this guide as the "camera" to pull images from, and select the name of the vision service you just configured as your "vision" service.
Then add all or some of the labels your ML model uses as classifications or detections in "classifications" or "objects".
For example, if you are using the EfficientDet-COCO model, you could use a configuration like the following to only capture images when a person is detected with more than 60% confidence in your camera stream.
{
"window_seconds": 0,
"objects": {
"Person": 0.8
},
"camera": "camera-1",
"vision": "vision-1"
}
Additionally, you can also add a buffer window with window_seconds which controls the duration of a buffer of images captured prior to a successful match.
If you were to set window_seconds to 3, the camera would also capture and sync images from the 3 seconds before a person appeared in the camera stream.
5. Configure data capture and sync on the filtered camera
Configure data capture and sync on the filtered camera just as you did before for the physical camera. The filtered camera will only capture image data that passes the filters you configured in the previous step.
Turn off data capture on your original camera if you haven’t already, so that you don’t capture duplicate or unfiltered images.
6. Save to start capturing
Save the config. With cloud sync enabled, captured data is automatically uploaded to the Viam app after a short delay.
7. View filtered data in the Viam app
Once you save your configuration, place something that is part of your trained ML model within view of your camera.
Images that pass your filter will be captured and will sync at the specified sync interval, which may mean you have to wait and then refresh the page for data to appear. Your images will begin to appear under the DATA tab.
If no data appears after the sync interval, check the Logs and ensure that the condition for filtering is met. You can test the vision service from the CONTROL tab to see its classifications and detections live.
7. (Optional) Trigger sync with custom logic
By default, the captured data syncs at the regular interval you specified in the data capture config. If you need to trigger sync in a different way, see Conditional cloud sync for a documented example of syncing data only at certain times of day.
Stop data capture on the filtered camera
If this is a test project, make sure you stop data capture to avoid incurring fees for capturing large amounts of test data.
In the Data capture section of your filtered camera’s configuration, toggle the switch to Off.
Click the Save button in the top right corner of the page to save your config.
Next steps
Now that you have collected image data, you can train new computer vision models or programmatically access your data:
To see image data filtering in action, check out these tutorials:
Have questions, or want to meet other people working on robots? Join our Community Discord.
If you notice any issues with the documentation, feel free to file an issue or edit this file.
Was this page helpful?
Glad to hear it! If you have any other feedback please let us know:
We're sorry about that. To help us improve, please tell us what we can do better:
Thank you!