Machine Learning

ML training

Machine learning (ML) provides your machines with the ability to adjust their behavior based on models that recognize patterns or make predictions.

Common use cases include:

  • Object detection, which enables machines to detect people, animals, plants, or other objects with bounding boxes, and to perform actions when they are detected.
  • Object classification, which enables machines to separate people, animals, plants, or other objects into predefined categories based on their characteristics, and to perform different actions based on the classes of objects.
  • Speech recognition, natural language processing, and speech synthesis, which enable machines to verbally communicate with us.

For other use cases, consider creating custom functionality with a module.

Viam provides two services that enable machine learning capabilities: the ML model service and the Computer Vision service.

The ML model service deploys and runs a machine learning model, such as a TensorFlow or ONNX model, on your machine and makes its output accessible to other services. For example, the Computer Vision mlmodel service, which can detect or classify objects, is built to work with the inferences from an ML model service. As a detector, the service uses these inferences to interpret image data from images on your computer or a camera, drawing bounding boxes around objects. As a classifier, the service returns class labels and confidence score based off the inferences the underlying ML model makes from image data.

See Use machine learning with your machine for step by step instructions.

Example tutorials

Model support

You have four options when choosing a model to deploy onto an ML model deployment service. You can:

  • train a model on the Viam app and deploy it
  • deploy a pre-trained model another user has published from the registry
  • upload a model trained outside the Viam platform to the registry privately or publicly and deploy it
  • deploy a model trained outside the Viam platform that’s already available on your machine

The model you use must be supported on the Viam platform. Viam supports the following model frameworks:

For more information, see Model framework support.

Use machine learning with your machine

Collect data 1. Collect

Start by collecting data from your cameras, sensors, or any other source on your machine with the data management service. You can view the data on the Data tab.

Label data 2. Create a dataset and label

Once you have collected data, label your data and create a dataset in preparation for training machine learning models.

Train models 3. Train or upload an ML model

Use your labeled data to train your own model for object detection or classification. If you don't want to train your own model, you can also use an ML model from the registry or upload an existing model.

4. Deploy your ML model

To use ML models with your machine, you must first deploy the model using an ML model service. The ML model service will run the model and allow the vision service to use it.

Configure a service 5. Configure a vision service

For object detection and classification, use the mlmodel detector or the mlmodel classifier from the vision service. The mlmodel vision service uses the ML model that you deployed with the ML model service in step 4.

If you have another use case, you can use a modular resource to create a custom ML model service or a custom vision service for your machine.

Deploy your model 6. Test your detector or classifier

Follow the instructions to test your mlmodel detector or classifier.