Part 1: Vision Pipeline

Goal: Get a computer vision pipeline working.

Skills: Connect a machine to Viam, configure components in the Viam UI, configure services in the Viam UI.

Time: ~15 min

Prerequisites

Before starting this tutorial, you need the can inspection simulation running. Follow the Gazebo Simulation Setup Guide to:

1. Build the Docker image with Gazebo Harmonic
2. Create a machine in Viam and get credentials
3. Start the container with your Viam credentials

Once you see “Can Inspection Simulation Running!” in the container logs and your machine shows Online in the Viam app, return here to continue.

1.1 Verify Your Machine is Online

If you followed the setup guide, your machine should already be online.

1. Open app.viam.com (the “Viam app”)
2. Navigate to your machine (for example, inspection-station-1)
3. Verify the status indicator shows Live
4. Click the Configure tab if not already selected

Ordinarily, after creating a machine in Viam, you would download and install viam-server together with the cloud credentials for your machine. For this tutorial, we’ve already installed viam-server and launched it in the simulation Docker container.

1.2 Locate Your Machine Part

Your machine is online but empty. To configure your machine, you will add components and services to your machine part in the Viam app. Your machine part is the compute hardware for your robot. This might be a PC, Mac, Raspberry Pi, or another computer.

In the case of this tutorial, your machine part is a virtual machine running Linux in the Docker container.

Find inspection-station-1-main in the Configuration tab.

1.3 Configure the Camera

You’ll now add the camera as a component.

Add a camera component

To add the camera component to your machine part:

1. Click the + button and select Component or service
2. Click Camera
3. Search for gz-camera
4. Select gz-camera:rgb-camera
5. Click Add module
6. Enter inspection-cam for the name

Configure the camera

To configure your camera component to work with the camera in the simulation, you need to specify the correct camera ID. Most components require a few configuration parameters.

1. In the Attributes section, add:

   {
     "id": "/inspection_camera"
   }
   

2. Click Save in the top right

1.4 Test the Camera

Verify the camera is working. Every component in Viam has a built-in test card right in the configuration view.

Open the test panel

1. You should still be on the Configure tab with your inspection-cam selected
2. Look for the Test section at the bottom of the camera’s configuration panel
3. Click Test to expand the camera’s test card

The camera component test card uses the camera API to add an image feed to the Viam app, enabling you to determine whether your camera is working. You should see a live video feed from the simulated camera. This is an overhead view of the conveyor/staging area.

1.5 Add an ML Model Service

Now you’ll add machine learning to run inference on your camera feed. You’ll configure two services:

1. ML model service—Loads a trained model for the inference task
2. Vision service—Connects the camera to the model and returns detections

Create the ML model service

1. Click the Configure tab
2. Click + next to your machine part
3. Select Component or service
4. Select ML model
5. Select TFLite CPU
6. Click Add module
7. Name it model-service
8. Click Create

Select a model from the registry

Configure the model-service ML model service you just included in your configuration.

1. In the model-service configuration panel, click Select model
2. Click the Registry tab
3. Search for can-defect-detection and select it from the list (a model that classifies cans as PASS or FAIL based on defect detection)
4. Click Choose to save the model selection
5. Click Save in the upper right corner to save your configuration

1.6 Add a Vision Service

Now add a vision service that connects your camera to the ML model service.

Create the vision service

1. Click + next to your machine part
2. Select Component or service
3. Search for vision
4. Select vision / ML Model
5. Name it vision-service
6. Click Create

Link the camera and model in the vision service

1. Select the vision-service service in your machine’s configuration
2. Find the ML Model dropdown and select model-service (the ML model service you just created)
3. Find the Default Camera dropdown and select inspection-cam
4. Find the Attributes section and set Minimum confidence threshold to 0.75
5. Click Save in the upper right corner

Test the vision service

1. Find the Test section at the bottom of the vision-service configuration panel
2. Expand the Test card
3. If not already selected, select inspection-cam as the camera source
4. Set Detections/Classifications to Live
5. Check that detection and labeling are working

Continue to Part 2: Data Capture →