Machine Learning

Machine Learning (ML) provides your machines with the ability to adjust its behavior based on models that recognize patterns or make predictions.

Common use cases include:

• Object detection, which enables machines to detect people, animals, plants, or other objects with bounding boxes, and to perform actions when they are detected.
• Object classification, which enables machines to separate people, animals, plants, or other objects into predefined categories based on their characteristics, and to perform different actions based on the classes of objects.
• Speech recognition, natural language processing, and speech synthesis, which enable machines to verbally communicate with us.

For other use cases, consider creating custom functionality with a module.

Viam provides two services that enable machine learning capabilities: the ML model service and the Computer Vision service.

The ML model service deploys and runs a machine learning model, such as a TensorFlow or ONNX model, on your machine and makes its output accessible to other services. For example, the Computer Vision mlmodel service, which can detect or classify objects, is built to work with the inferences from an ML model service. As a detector, the service uses these inferences to interpret image data from images on your computer or a camera, drawing bounding boxes around objects. As a classifier, the service returns class labels and confidence score based off the inferences the underlying ML model makes from image data.

See Use machine learning with your machine for step by step instructions.

Example tutorials

AI Companion Robot

Add object detection, speech recognition, natural language processing, and speech synthesis capabilities to a machine.

Helmet Monitor

Get an email alert if people are not wearing hard hats.

Capture Data and Train a Model

Configure data capture and cloud sync, filter and tag captured data, and train an ML model.

Smart Pet Feeder

Use a Raspberry Pi, a motor, and machine learning to build a smart pet feeder.

Model support

You have four options when choosing a model to deploy onto an ML model deployment service. You can:

• train a model on the Viam app and deploy it
• deploy a pre-trained model another user has published from the registry
• upload a model trained outside the Viam platform to the registry privately or publicly and deploy it
• deploy a model trained outside the Viam platform that’s already available on your machine

The model you use must be supported on the Viam platform. Viam supports the following model frameworks:

• TensorFlow Lite (as long as your models adhere to the model requirements): with the tflite_cpu ML model service
• TensorFlow: with the triton ML model service
• PyTorch: with the triton ML model service
• ONNX: with the onnx_cpu ML model service

For more information, see Model framework support.

Use machine learning with your machine

1. Collect Start by collecting data from your cameras, sensors, or any other source on your machine with the data management service. You can view the data on the Data tab.
2. Create a dataset and label Once you have collected data, label your data and create a dataset in preparation for training machine learning models.
3. Train or upload an ML model Use your labeled data to train your own model for object detection or classification. If you don't want to train your own model, you can also use an ML model from the registry or upload an existing model.
4. Deploy your ML model To use ML models with your machine, you must first deploy the model using an ML model service. The ML model service will run the model and allow the vision service to use it.
5. Configure a vision service For object detection and classification, use the mlmodel detector or the mlmodel classifier from the vision service. The mlmodel vision service uses the ML model that you deployed with the ML model service in step 4. If you have another use case, you can use a modular resource to create a custom ML model service or a custom vision service for your machine.
6. Test your detector or classifier Follow the instructions to test your mlmodel detector or classifier.