viam-micro-server Development Setup

viam-micro-server is the lightweight version of viam-server which can run on resource-limited embedded systems (ESP32) that cannot run the fully-featured viam-server.

This page provides instructions for configuring a development environment for working with viam-micro-server, outlines the steps for performing various development tasks, and provides troubleshooting and development tips to help organize and streamline work.

The instructions below are for configuring a development environment in order to:

  • Develop custom firmware which combines viam-micro-server with one or more modules.
  • Develop modules for viam-micro-server.
  • Develop viam-micro-server itself.

Required software

viam-micro-server is written in Rust. To be able to develop for viam-micro-server on macOS and Linux systems, you must install the following software on your development machine:

  1. Install dependencies:

    sudo apt-get install bison ccache cmake curl dfu-util flex git gperf libffi-dev libssl-dev libudev-dev libusb-1.0-0 ninja-build python3 python3-pip python3-venv wget

    If you haven’t already, install Homebrew:

    /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

    Then, install dependencies:

    brew install cmake dfu-util ninja

  2. If you do not yet have a Rust environment installed, install it with rustup.

    curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

    Once completed, start a new shell instance, or run . "$HOME/.cargo/env" in the current one.

    See the Rust Installation guide for more information and other installation methods.

  3. Install espup and ESP development utilities with cargo:

    cargo install cargo-espflash cargo-generate espflash espup ldproxy

  4. Use espup to download and install the ESP Rust toolchain:

    espup install -s -f /dev/null -v 1.83.0

Creating a project or module

With the dev setup for viam-micro-server, you can:

  • Create a project that combines viam-micro-server with one or more modules to produce a new firmware instance with expanded functionality.
  • Create modules that can integrate with viam-micro-server to deliver new opt-in functionality or device support.
  • Work on viam-micro-server itself, to add features and fix bugs, or to produce a build for a specific ESP-IDF version or platform for which Viam does not offer a pre-built solution.

  1. Create a new machine and obtain its credentials:

    Navigate to the Viam app and add a new machine. Click on the name of the machine to go to the machine’s page, then select the CONFIGURE tab.

    Then select the part status dropdown to the right of your machine’s name on the top of the page and copy the Machine cloud credentials:

    Restart button on the machine part info dropdown

    viam-micro-server needs these credentials, which contains your machine part secret key and cloud app address, to connect to the Viam app.

  1. Generate a new project skeleton from this template:

    cargo generate --git https://github.com/viamrobotics/micro-rdk.git

    Select templates/project when prompted. Give the project a name of your choice. Select esp32 for MCU. If you wish to configure an esp32-camera or a fake camera as a component of your machine, select true for include camera module and traits.

    You will be prompted to paste your machine’s viam-server robot JSON configuration into the terminal, which is the same thing as its machine cloud credentials. Paste in the credentials you obtained in step 1.

  2. Change directories into the generated project:

    cd <your-path-to/your-project-directory>

  3. If you wish to use version control for this project, this is the best time to initialize a git repository and commit all the generated files, but be sure to exclude the generated viam.json file, which includes secrets:

    git add .
git restore viam.json
git commit -m "initial commit"

  4. Compile the project:

    make build-esp32-bin

    Please note that the first build may be fairly time consuming, as ESP-IDF must be cloned and built, and all dependent Rust crates must be fetched and built as well. Subsequent builds will be faster.

  5. Upload the generated firmware to your ESP32:

    Connect the ESP32 board you wish to flash to a USB port on your development machine, and run:

    make flash-esp32-bin

    When prompted, select the serial port that your ESP32 is connected to through a data cable.

    If successful, you will retain a serial connection to the board until you press Ctrl-C. To manage this connection, consider running it within a dedicated terminal session, or under tmux or screen. While the serial connection is live, you can also restart the currently flashed image with Ctrl-R.

  1. Navigate to your new machine’s page on the Viam app. If successful, Live should be displayed underneath Last online.

  2. You may now add any desired modules to the project by including them in the dependencies section of the Cargo.toml for the generated project. After adding (or removing) a module or changing the version of a module, you must rerun steps 5-6 above in order to rebuild the firmware and reflash the device.

  1. If you have not previously developed a module for viam-micro-server, please review the module template README and the example module implementation walkthrough before continuing.

  2. Generate a new module skeleton from this template:

    cargo generate --git https://github.com/viamrobotics/micro-rdk.git

    Select templates/module when prompted, give the module a name of your choice, and answer any additional prompts.

  3. Change directories into the generated tree:

    cd <your-path-to/your-module-directory>

  4. If you wish to use version control for the module, this is the best time to initialize a git repository and commit all the generated files. There are no secrets in a newly generated module repository:

    git add .
git commit -m "initial commit"

  5. Develop the module by defining structs which implement the necessary traits and adding tests and registration hooks for them, per the walkthrough.

  6. To consume the module, follow the “Create a Project” workflow in a different directory, and register your module in the dependencies section of the project’s Cargo.toml file, then build and flash the project. The module will now be available for use by adding it to your machine configuration on the Viam app.

  1. Clone the viamrobotics/micro-rdk repository (optionally specifying a path for the new clone) and cd into the clone:

    git clone https://github.com/viamrobotics/micro-rdk [path-to-new-clone]
cd <your-path-to/your-viam-micro-server-clone>

  2. Make any necessary additions or fixes.

  3. Compile and run the project. The viam-micro-server can either be built for ESP32, or as a native version for testing purposes:

Compile the project for ESP32:

make build-esp32-bin

Please note that the first build may be fairly time consuming, as ESP-IDF must be cloned and built, and all dependent Rust crates must be fetched and built as well. Subsequent builds will be faster.

Next, flash the generated firmware to your ESP32.

Connect the ESP32 board you wish to flash to a USB port on your development machine, and run:

make flash-esp32-bin

When prompted, select the serial port that your ESP32 is connected to through a data cable.

The native server can be built and run with one command:

make native

For further details on viam-micro-server development, including credentials management and developer productivity suggestions, please see the development technical notes page on GitHub.

Troubleshooting

Error: xtensa-esp32-elf-gcc: error: unrecognized command line option '--target=xtensa-esp32-espidf' when building on macOS

This is caused by an upstream bug. To work around this issue, ensure that CRATE_CC_NO_DEFAULTS=1 is set in the environment when building.

Error: error: externally-managed-environment when building ESP-IDF on macOS

This is encountered when attempting to build ESP-IDF projects while using Python obtained from Homebrew. Homebrew’s Python infrastructure is not intended for end-user consumption but is instead made available to support Homebrew packages which require a python interpreter. The preferred workaround for this issue is to obtain a user-facing python installation not from Homebrew, but there are other, less safe, workarounds. Please see the macOS specific notes in the development technical notes on GitHub for more details.

Error: Failed to open serial port

If you run into the error Failed to open serial port when flashing your ESP32 with Linux, make sure the user is added to the group dialout with sudo gpasswd -a $USER dialout.

Error: espflash::timeout

If you get the following error while connecting to your ESP32:

`Error: espflash::timeout

  × Error while connecting to device
  ╰─▶ Timeout while running command

Run the following command:

espflash flash --erase-parts nvs --partition-table partitions.csv  target/xtensa-esp32-espidf/release/esp32-camera --baud 115200 && sleep 2 && espflash monitor

Try the connection command again. The baud rate on your device may not have been fast enough to connect. If successful, the Viam app will show that your machine part’s status is Live.

Error: viam.json not found

If you get the error viam.json not found try the following to manually add your machine cloud credentials as a file in your project:

  1. Navigate to your machine’s page on the Viam app and select the CONFIGURE tab.
  2. Select the part status dropdown to the right of your machine’s name on the top of the page: Restart button on the machine part info dropdown
  3. Click the copy icon underneath Machine cloud credentials. viam-micro-server needs this JSON, which contains your machine part secret key and cloud app address, to connect to the Viam app.
  4. Navigate to the directory of the project you just created.
  5. Create a new viam.json file and paste the viam-server machine cloud credentials in.
  6. Save the file.

Error: failed to run custom build command for esp32-explorer (/host)

This may occur for various reasons such as your machine cloud credentials, WiFi SSID, or password not being populated. Check that your machine cloud credentials are provided in your project directory as viam.json and ensure that your WiFi credentials are provided.

Error: invalid value 460800 for --before <BEFORE>

Change "-b" to "-B in the Makefile, as "-B" is the Baudrate config. Run the following commands to flash esp32-server.bin to your ESP32 microcontroller at a high baud rate, wait for 2 seconds, and observe the device’s output:

espflash write-bin 0x0 target/esp32-server.bin -B 460800  && sleep 2 && espflash monitor

You can find additional assistance in the Troubleshooting section.

You can also ask questions in the Community Discord and we will be happy to help.