Configure a tflite_cpu

The tflite_cpu ML model service allows you to deploy TensorFlow Lite ML models as long as your models adhere to the model requirements. It is supported on any CPU and Linux, Raspbian, MacOS and Android machines.

To work with the tflite_cpu ML model service, an ML model is comprised of a .tflite model file which defines the model, and optionally a .txt labels file which provides the text labels for your model. With the tflite_cpu ML model service, you can deploy:

To configure a tflite_cpu ML model service:

Navigate to the CONFIGURE tab of your machine’s page in the Viam app. Click the + icon next to your machine part in the left-hand menu and select Service. Select the ML model type, then select the TFLite CPU model. Enter a name or use the suggested name for your service and click Create.

You can choose to configure your service with an existing model on the machine or deploy a model onto your machine:

  1. To configure your service and deploy a model onto your machine, select Deploy model on machine for the Deployment field in the resulting vision service configuration pane,.

  2. Click on Select models to open a dropdown with all of the ML models available to you privately, as well as all of the ML models available in the registry, which are shared by users. Models that your organization has trained that are not uploaded to the registry will appear first in the dropdown. You can select from any of these models to deploy on your robot. Only TensorFlow Lite models are shown.

Models dropdown menu with models from the registry.
  1. Also, optionally select the Number of threads.
Create a machine learning models service with a model to be deployed
  1. To configure your service with an existing model on the machine, select Path to existing model on robot for the Deployment field.
  2. Then specify the absolute Model path and any Optional settings such as the absolute Label path and the Number of threads.

Create a machine learning models service with an existing model

Add the tflite_cpu ML model object to the services array in your raw JSON configuration:

"services": [
  {
    "name": "<mlmodel_name>",
    "type": "mlmodel",
    "model": "tflite_cpu",
    "attributes": {
      "model_path": "${packages.<model_name>}/<model-name>.tflite",
      "label_path": "${packages.<model_name>}/labels.txt",
      "num_threads": <number>
    }
  },
  ... // Other services
]

{
"packages": [
  {
    "package": "39c34811-9999-4fff-bd91-26a0e4e90644/my_fruit_model",
    "version": "YYYY-MM-DDThh-mm-ss",
    "name": "my_fruit_model",
    "type": "ml_model"
  }
], ... // < Insert "components", "modules" etc. >
"services": [
  {
    "name": "fruit_classifier",
    "type": "mlmodel",
    "model": "tflite_cpu",
    "attributes": {
      "model_path": "${packages.my_fruit_model}/my_fruit_model.tflite",
      "label_path": "${packages.my_fruit_model}/labels.txt",
      "num_threads": 1
    }
  }
]
}

The "packages" array shown above is automatically created when you deploy the model. You do not need to edit the configuration yourself, expect if you wish to change the Versioning for deployed models.

The following parameters are available for a "tflite_cpu" model:

ParameterInclusionDescription
model_pathRequiredThe absolute path to the .tflite model file, as a string.
label_pathOptionalThe absolute path to a .txt file that holds class labels for your TFLite model, as a string. This text file should contain an ordered listing of class labels. Without this file, classes will read as “1”, “2”, and so on.
num_threadsOptionalAn integer that defines how many CPU threads to use to run inference. Default: 1.

Save the configuration.

Model requirements

We strongly recommend that you package your TensorFlow Lite model with metadata in the standard form.

In the absence of metadata, your tflite_cpu model must satisfy the following requirements:

  • A single input tensor representing the image of type UInt8 (expecting values from 0 to 255) or Float 32 (values from -1 to 1).
  • At least 3 output tensors (the rest won’t be read) containing the bounding boxes, class labels, and confidence scores (in that order).
  • Bounding box output tensor must be ordered [x x y y], where x is an x-boundary (xmin or xmax) of the bounding box and the same is true for y. Each value should be between 0 and 1, designating the percentage of the image at which the boundary can be found.

These requirements are satisfied by a few publicly available model architectures including EfficientDet, MobileNet, and SSD MobileNet V1. You can use one of these architectures or build your own.

Have questions, or want to meet other people working on robots? Join our Community Discord.

If you notice any issues with the documentation, feel free to file an issue or edit this file.