Edit

Deploy a model

After training or uploading a machine learning model, use a machine learning (ML) model service to deploy the ML model to your machine.

Deploy your ML model on an ML model service

Navigate to the CONFIGURE tab of one of your machines.
Add an ML model service that supports the ML model you want to use.
- For example, use the ML model / TFLite CPU service for TFlite ML models that you trained with Viam’s built-in training.
Click Select model and select a model from your organization or the registry.
Save your config.
Use the Test panel to test your model.

For configuration information, click on the model name:

Model

Description

Want more information about model framework and hardware support for each ML model service? Click here.

Viam currently supports the following frameworks:

Model Framework	ML Model Service	Hardware Support	Description
TensorFlow Lite	`tflite_cpu`	linux/amd64, linux/arm64, darwin/arm64, darwin/amd64	Quantized version of TensorFlow that has reduced compatibility for models but supports more hardware. Uploaded models must adhere to the model requirements.
ONNX	`onnx-cpu`, `triton`	Nvidia GPU, linux/amd64, linux/arm64, darwin/arm64	Universal format that is not optimized for hardware inference but runs on a wide variety of machines.
TensorFlow	`tensorflow-cpu`, `triton`	Nvidia GPU, linux/amd64, linux/arm64, darwin/arm64	A full framework that is made for more production-ready systems.
PyTorch	`torch-cpu`, `triton`	Nvidia GPU, linux/arm64, darwin/arm64	A full framework that was built primarily for research. Because of this, it is much faster to do iterative development with (model doesn’t have to be predefined) but it is not as “production ready” as TensorFlow. It is the most common framework for OSS models because it is the go-to framework for ML researchers.

Note

For some models of the ML model service, like the Triton ML model service for Jetson boards, you can configure the service to use either the available CPU or a dedicated GPU.

Models available to deploy on the ML Model service

You can also use these publicly available machine learning models with an ML model service:

Model

Type

Framework

Description

Deploy a specific version of an ML model

When you add a model to the ML model service in the app interface, it automatically uses the latest version. In the ML model service panel, you can change the version in the version dropdown. Save your config to use your specified version of the ML model.

How the ML model service works

The service works with models trained on Viam or elsewhere:

You can train TensorFlow or TensorFlow lite or other model frameworks on data from your machines.
You can use ML models from the registry.
You can upload externally trained models from a model file on the MODELS tab.
You can use a model trained outside the Viam platform whose files are on your machine. See the documentation of the model of ML model service you’re using (pick one that supports your model framework) for instructions on this.

On its own the ML model service only runs the model. After deploying your model, you need to configure an additional service to use the deployed model. For example, you can configure an mlmodel vision service to visualize the inferences your model makes. Follow our docs to run inference to add an mlmodel vision service and see inferences.

For other use cases, consider creating custom functionality with a module.

Add support for other models

ML models must be designed in particular shapes to work with the mlmodel classification or detection model of Viam’s vision service. See ML Model Design to design a modular ML model service with models that work with vision.

Was this page helpful?

Glad to hear it! If you have any other feedback please let us know:

We're sorry about that. To help us improve, please tell us what we can do better:

Thank you!