Filter data before sync

1. Add an ML model service to your machine

Add an ML model service on your machine that is compatible with the ML model you want to use, for example TFLite CPU.

2. Select a suitable ML model

Click Select model on the ML model service configuration panel, then select an existing model you want to use, or click Upload a new model to upload your own. If you’re not sure which model to use, you can use EfficientDet-COCO from the Registry, which can detect people and animals, among other things.

3. Add a vision service to use with the ML model

You can think of the vision service as the bridge between the ML model service and the output from your camera.

Add and configure the vision / ML model service on your machine. From the Select model dropdown, select the name of your ML model service (for example, mlmodel-1).

4. Configure the filtered camera

The filtered-camera modular component pulls the stream of images from the camera you configured earlier, and applies the vision service to it.

Configure a filtered-camera component on your machine, following the attribute guide in the README. Use the name of the camera you configured in the first part of this guide as the "camera" to pull images from, and select the name of the vision service you just configured as your "vision" service. Then add all or some of the labels your ML model uses as classifications or detections in "classifications" or "objects".

For example, if you are using the EfficientDet-COCO model, you could use a configuration like the following to only capture images when a person is detected with more than 60% confidence in your camera stream.

{
  "window_seconds": 0,
  "objects": {
    "Person": 0.8
  },
  "camera": "camera-1",
  "vision": "vision-1"
}

Additionally, you can also add a buffer window with window_seconds which controls the duration of a buffer of images captured prior to a successful match. If you were to set window_seconds to 3, the camera would also capture and sync images from the 3 seconds before a person appeared in the camera stream.

5. Configure data capture and sync on the filtered camera

Configure data capture and sync on the filtered camera just as you did before for the physical camera. The filtered camera will only capture image data that passes the filters you configured in the previous step.

Turn off data capture on your original camera if you haven’t already, so that you don’t capture duplicate or unfiltered images.

6. Save to start capturing

Save the config. With cloud sync enabled, captured data is automatically uploaded to the Viam app after a short delay.

7. View filtered data in the Viam app

Once you save your configuration, place something that is part of your trained ML model within view of your camera.

Images that pass your filter will be captured and will sync at the specified sync interval, which may mean you have to wait and then refresh the page for data to appear. Your images will begin to appear under the DATA tab.

If no data appears after the sync interval, check the Logs and ensure that the condition for filtering is met. You can test the vision service from the CONTROL tab to see its classifications and detections live.

7. (Optional) Trigger sync with custom logic

By default, the captured data syncs at the regular interval you specified in the data capture config. If you need to trigger sync in a different way, see Conditional cloud sync for a documented example of syncing data only at certain times of day.