Iterative Module Development

Once you have created the first version of your module, you can use the procedure outlined on this page to iterate on your design, and test changes quickly.

Generally, when developing a module, you have two options for iterative development, depending on the specific platform you want your module to support:

  • Test Locally: If you want your module to support the same platform as your development workstation, you can test your module locally. For example, if you are developing a module on a macOS computer with an M2 processor (the arm64 architecture), and want your module to support only macOS computers running the arm64 architecture, you can perform your module development and testing workflow entirely on your macOS workstation.
  • Sync Code and Test Remotely: If you want your module to support a different architecture than your development workstation, you can sync your module code to a machine running your desired target architecture and test remotely. For example, if you are developing a module on a macOS computer, but want your module to support a Raspberry Pi running Linux on the arm64 architecture, you can set up syncing for your module code to be able to continue development on your macOS workstation, but test on your remote Raspberry Pi.

Both of these options involve deploying your module to the target test system as a local module, without uploading it to the Viam registry. Even if you have already published a version of your module to the registry, you might still find it useful to follow the steps in this section to verify that changes you make as part of releasing a new version work as expected on your target platform.

Then, once you have tested it, you can upload your module to the Viam registry.

If you are developing a module for the same target architecture as your development workstation, you can test your module locally using the following procedure:

  1. Navigate to the Viam app and add a new machine to serve as your development machine. Be sure to follow the steps shown in the Viam app to install viam-server on your local machine.

  2. If you are using a programming language that requires that you build your module, such as Go or C++, follow the instructions for your language to compile or package your module. If you are using a programming language that does not require compilation, such as Python, you can skip this step.

  3. Navigate to the Viam app, select your machine, and add your module as a local module to your machine. Provide the Executable path in the configuration, pointing to the compiled or built binary, or the executable script, depending on your language. If you have previously added your module as a registry module, you will need to first remove the registry version of your module before then adding the local version.

  4. Click Save config.

  5. Check the Logs tab for your machine in the Viam app to ensure that viam-server properly started your module. For example, the following log message indicates that viam-server was able to find and start the local module named my-module successfully:

    1/16/24, 4:44:25.085 PM   info robot_server   modmanager/manager.go:862   registering component from module   module my-module   API rdk:component:base   model acme:demo:mybase  
    
  6. Make your desired code changes to your module using your favorite editor. Save your changes.

  7. If applicable (for example, using Go or C++), compile or package your module again. Otherwise (for example, using Python), skip this step.

  8. Restart viam-server on your machine in order for it to pick up the code changes to your module, using your system’s service management command. For example:

    • On macOS, using brew:

      brew services restart viam-server
      
    • On Linux, using systemctl:

      sudo systemctl restart viam-server
      
  9. Once viam-server has restarted, check the Logs tab for your machine in the Viam app again to ensure that viam-server properly started your module.

  10. Then test your module to verify that your code changes work as expected.

  11. Repeat steps 6 - 10 to continue developing your module, as needed. Remember to check the Logs tab each time to verify that the module registered successfully, and to troubleshoot any error or warning messages. If you haven’t already, you can add custom log messages to your code, which appear under the Logs tab to assist with troubleshooting.

If you are developing a module for a different target architecture than your development workstation, you can sync your module code and test your module remotely using the following procedure:

  1. Navigate to the Viam app and add a new machine to serve as your development machine. Be sure to follow the steps shown in the Viam app to install viam-server on the target machine you want to test and build on. For example, to test and build your module on your Raspberry Pi, be sure to install viam-server on the Pi itself, not your macOS workstation.

  2. Set up file sync between your development workstation and the machine running your target platform that you want to build and test on, using your favorite file sync utility. For example, you could use mutagen.io to sync your files using the following steps:

    1. Install Mutagen on your development workstation, following the instructions for your operating system:

      • To install Mutagen on macOS:

        brew install mutagen-io/mutagen/mutagen
        
      • To install Mutagen on Linux, download the latest release from GitHub. Mutagen is not currently available for Linux through package managers.

    2. If you haven’t already, ensure that your target test system is up and running, and accessible to ssh. Mutagen uses ssh to sync files between your systems in an unattended manner (that is, without user input), and so requires that you set up a valid ssh key for it to use to authenticate during syncs. If you already have an ssh keypair configured to your remote testing system, you can skip this step and proceed to step 3. Follow the steps below to create a new ssh key to use, and copy it to your remote testing system:

      1. Create a new ssh key on your development workstation, meeting your security requirements. For example, on a macOS machine, you can generate an ssh key using the ed25519 algorithm with a key size of 4096 bits using the following command:

        ssh-keygen -t ed25519 -b 4096 -C "developer@example.com"
        

        When prompted to select a file location, press enter to accept the default location.

        This command will generate a new key in your ~/.ssh/ directory. If you already have an ssh key in that directory with the same name, you will be prompted to overwrite your existing key: type n (for “no”) to cancel. You can use your existing key in the next steps.

      2. Verify that you have both parts of the ssh keypair present as follows:

        ls -l ~/.ssh/id_ed25519*
        

        For example, on macOS, you should see files similar to /Users/username/.ssh/id_ed25519 and /Users/username/.ssh/id_ed25519.pub.

      3. Copy the .pub file only to your target remote system using the ssh-copy-id command. For example, to transfer a id_ed25519.pub file from a macOS machine to a remote Linux system named my-pi.local for the user username, you could use the following commands:

        ssh-copy-id -i ~/.ssh/id_ed25519.pub username@my-pi.local
        

        Provide the ssh password for your user account when prompted. Do not copy the private key file, which does not have a file extension.

    3. Then, start a new ssh session to your remote system, and verify that you are able to connect without being prompted for a password. Mutagen requires a working, passwordless ssh configuration in order to be able to sync files. If you receive a connection refused error, or are still prompted for a password, see the Troubleshooting ssh section for further guidance.

  3. Return to your local development system, and navigate to your module’s directory. For example, if you are developing a module named my-module in the home directory:

    cd ~/my-module
    
  4. Set up a new Mutagen sync from this directory to sync to your remote system, providing the local path to your module and the target path on the remote system to sync your module files:

    mutagen sync create /path/to/local/module username@remote-hostname.local:/path/to/remote/sync-target
    

    For example, you could use the following to transfer the example module from earlier to the same location on the remote system, if desired:

    mutagen sync create ~/my-module username@my-pi.local:/home/username/my-module
    
  5. In an ssh session to your remote system, ensure that you now see the synced files appear as expected in the filesystem location you chose to sync to. If the files haven’t appeared, consult the Mutagen documentation for further troubleshooting.

  6. Return to your local development system, and navigate back to your module directory.

  7. If you are using a programming language that requires that you build your module, such as Go or C++, follow the instructions for your language to compile or package your module. If you are using a programming language that does not require compilation, such as Python, you can skip this step.

  8. Navigate to the Viam app, select your machine, and add your module as a local module to your machine. Provide the Executable path in the configuration, pointing to the compiled or built binary, or the executable script, depending on your language. Provide the remote system’s path to this file, for example: /home/username/my-module/run.sh. With the Mutagen sync in place, Mutagen transfers the binary or executable automatically when you created it in the previous step. If you have previously added your module as a registry module, you will need to first remove the registry version of your module before then adding the local version.

  9. Click Save config.

  10. Check the Logs tab for your machine in the Viam app to ensure that viam-server properly started your module. For example, the following log message indicates that viam-server was able to find and start the local module named my-module successfully:

    1/16/24, 4:44:25.085 PM   info robot_server   modmanager/manager.go:862   registering component from module   module my-module   API rdk:component:base   model acme:demo:mybase  
    
  11. Make your desired code changes to your module using your favorite editor. Save your changes.

  12. If applicable (for example, using Go or C++), compile or package your module again. Otherwise (for example, using Python), skip this step.

  13. Restart viam-server on your remote system in order for it to pick up the code changes to your module, using your system’s service management command. For example, on Linux using systemctl:

    sudo systemctl restart viam-server
    
  14. Once viam-server has restarted, check the Logs tab for your machine in the Viam app again to ensure that viam-server properly started your module.

  15. Then test your module to verify that your code changes work as expected.

  16. Repeat steps 11 - 15 to continue developing your module, as needed. Remember to check the Logs tab each time to verify that the module registered successfully, and to troubleshoot any error or warning messages. If you haven’t already, you can add custom log messages to your code, which appear under the Logs tab to assist with troubleshooting.

When you are satisfied that your module is ready for release, follow the steps to upload your module to the Viam registry, to facilitate streamlined deployment to other machines or to make it available to the Viam community.

Troubleshooting ssh

Connection refused

If you receive connection refused or similar messages when attempting to connect to the remote system using your ssh key, ensure the permissions of the key are correct:

  • Start an ssh session to your remote system using your password.

  • Make sure that the public key you created with ssh-keygen and copied with ssh-copy-id has the correct file permissions on the remote filesystem:

    ls -l ~/.ssh/id_ed25519.pub
    
  • If the permissions shown are not exactly -rw-r--r--, use the following command to set them appropriately:

    chmod 644 ~/.ssh/id_ed25519.pub
    

Then test your ssh connection once more to ensure that you are connected without being prompted for a password.

Prompted for ssh account password

If you are able to ssh to the remote system, but are still prompted for your password each time you attempt to connect with a message similar to username@amy-pi.local's password:, check your remote system’s sshd configuration:

  • Start an ssh session to your remote system using your password.

  • Make sure that the sshd configuration for your remote system includes the following settings:

    PubkeyAuthentication yes
    

    For example, on Ubuntu, you can check your sshd configuration with the following command:

    grep -i pubkey /etc/ssh/sshd_config
    
    • If this value is set to no, change this value to yes.
    • If this value is prepended by # character, remove it so that the line reads exactly: PubkeyAuthentication yes.
  • If you needed to change this value in either way listed, restart the sshd service from your ssh session to your remote system. For example, if your remote system is Linux, you would run the following on the remote system:

    sudo systemctl restart sshd
    

    Note that restarting the sshd service in this manner will disconnect your current ssh session to the remote system. For more information on the sshd service and related service management, please consult the documentation for your specific operating system.

Then test your ssh connection once more to ensure that you are connected without being prompted for a password.

Prompted for ssh key passphrase

If you are prompted with a message similar to Enter passphrase for key '/Users/username/.ssh/id_ed25519', add your ssh key passphrase to your local macOS keychain:

  • For macOS 12.0 or later, run the following on your local macOS system:

    ssh-add --apple-use-keychain ~/.ssh/id_ed25519
    
  • For macOS 11.0 or previous, run the following on your local macOS system:

    ssh-add -K ~/.ssh/id_ed25519
    

Then test your ssh connection once more to ensure that you are connected without being prompted for a password.



Have questions, or want to meet other people working on robots? Join our Community Discord.

If you notice any issues with the documentation, feel free to file an issue or edit this file.