Detect People with a Webcam in 4 minutes

No problem.

Use Try Viam to borrow a rover free of cost online. The rover already has viam-server installed and is configured with some components to test with, including a webcam. You may not be able to test using the supplied ML model, as your borrowed rover will generally not be able to see people. We recommend you follow the Detect color with a Webcam guide instead.

Once you have borrowed a rover, go to the CONFIGURE tab of the machine, find the cameras and click on the Test panel at the bottom of each camera’s configuration panel to test the camera stream. You should have a front-facing camera and an overhead view of your rover. Now you know what the rover can perceive.

If your rover is facing a wall, find the base configuration panel and click on its Test panel. Use the controls to drive your rover to a different location.

Now that you have seen that the cameras on your Try Viam rover work, continue with Step 4.

Add a new machine in the Viam app by providing a name in the New machine field and clicking Add machine.

Click the name of a machine to go to that machine’s page, where you’ll find a variety of tools for working with your machine.

Click the + icon next to your machine part in the left-hand menu and select Component. Select the camera type, then select the webcam model. Enter a name or use the suggested name for your camera and click Create.

Click the Save button in the top right corner of the page to save your configuration. Then click on the Test panel at the bottom of the camera’s configuration panel to test the camera stream.

If you don’t see an image stream, you need to configure the video_path attribute.

For more detailed information, including optional attribute configuration, see the webcam docs.

Now add an ML model service that can detect a person: Click +, click Service and select the ML model type, then select the TFLite CPU model. Create the service.

In the resulting ML model service configuration pane, ensure that Deploy model on machine is selected for the Deployment field.

Click on Select model and select the ml-models-scuttle:people model from the registry (the people model by ml-models-scuttle) to deploy an object detection TFLite model that has been trained to be able to detect a person.

For more detailed information, including optional attribute configuration, see the tflite_cpu docs.

To use the deployed person detection model to detect people on a camera stream, you need to configure a vision service. This service applies the ML model to the camera input stream.

Add a vision Service and select the ML model model.

Select the ML model service your person detector model is deployed on (which you created in step 4) from the ML Model dropdown. Click the Save button in the top right corner of the page to save your vision service configuration.

Now, test your person detection in the Test section of the computer vision service’s configuration panel or on the CONTROL tab.

You will see your camera stream and see detections as labeled boxes on the images along with labels and confidence data. Detections in class Person with a high confidence score show positive person detections, but the ML model can also detect other objects:

If you are seeing a lot of detections, you can set a minimum confidence threshold.

On the configuration page of the vision service in the top right corner, click {} (Switch to advanced). Add the following JSON to the JSON configuration to set the default_minimum_confidence of the detector:

"default_minimum_confidence": 0.82

The full configuration for the attributes of the vision service should resemble:

{
  "mlmodel_name": "mlmodel-1",
  "default_minimum_confidence": 0.82
}

This optional attribute reduces your detections output by filtering out detections below the threshold of 82% confidence. You can adjust this attribute as necessary.

Click the Save button in the top right corner of the page to save your configuration and close and reopen the Test panel of the vision service configuration panel. Now if you view detections, you will only see detections with a confidence value higher than the "default_minimum_confidence" attribute.

For more detailed information, including optional attribute configuration, see the mlmodel docs.