Alert on machine telemetry

You can receive alerts for the following events involving machine performance telemetry:

For example, you can configure a trigger to send you a notification when your machine’s CPU usage reaches a certain threshold.

Data meets condition

The following steps let you configure the viam-telegraf-sensor to monitor the following metrics about the performance of individual machines or your entire fleet:

  • Wireless Signal Strength and Quality: Signal level, link quality, and noise level
  • Memory Usage: Memory statistics, including total available memory, used percentage, and specifics on various types of memory (cached, free, slab, etc.)
  • CPU Usage: CPU usage across different states (user, system, idle, etc.)
  • Disk I/O: Metrics on read and write operations, including bytes transferred and operation times
  • Network Traffic: Detailed network statistics, including bytes sent and received, packet information, and error counts, providing a deep dive into a device’s network performance

Prerequisites

A running machine connected to the Viam app. Click to see instructions.
Add a new machine in the Viam app. On the machine’s page, follow the setup instructions to install viam-server on the computer you’re using for your project. Wait until your machine has successfully connected to the Viam app.
Install telegraf. Click to see instructions.

On macOS, you must also install telegraf by running brew install telegraf in your terminal before using this module.

If you are on another operating system, telegraf will be installed automatically for you.

Add performance sensor

1. Add the performance metrics sensor

On your machine’s CONFIGURE page, click the + icon next to your machine part in the left-hand menu and select Component.

Search for and add the viam:viam-sensor:telegrafsensor model provided by the viam-telegraf-sensor module.

2. (Optional) Customize the sensor configuration

To enable or disable specific metrics, add them to the attributes configuration. You can find a list of configurable attributes in the module README. For example:

{
  "disable_kernel": true
}

3. Test the sensor

Save the configuration.

Now, click Test at the bottom of the sensor configuration card to view the readings. You can also see readings on the CONTROL tab.

Test panel with readings displayed.

Configure data management

To capture or alert on the data from your configured sensor, you must add the data management service and configure it to capture and sync the sensor data:

1. Add the data management service

On your machine’s CONFIGURE page, click the + icon next to your machine part in the left-hand menu and select Service.

Select the data management / RDK service and click Create. You can leave the default data sync interval of 0.1 minutes to sync every 6 seconds. Also leave both Capturing and Syncing toggles in the “on” position.

2. Configure data capture on the telegraf sensor

Return to your telegrafsensor’s configuration card.

In the Data capture section, click Add method.

From the Method dropdown select Readings. Set the Frequency to 0.05 Hz to capture readings once every 20 seconds.

Sensor readings capture configuration.

Save your config.

Click for instructions to view synced data

Click the menu in the upper-right corner of the sensor configuration card. Select View captured data. If you do not immediately see data, wait a minute for the data to be captured and synced at the intervals you specified, then refresh the page.

View of sensor data

Configure conditional trigger

  1. Go to the CONFIGURE tab of your machine on the Viam app. Click the + (Create) button in the left side menu and select Trigger.

    The Create menu with Trigger at the bottom of the list of options.
  2. Name the trigger and click Create.

  3. Select Conditional data ingestion as the trigger Type.

    Then select the component you want to capture data from and the method you want to capture data from. Next, add any conditions.

    These can include a key, a value, and a logical operator. For example, a trigger configured to fire when data is captured from the motor motor-1’s IsPowered method when is_on is equal to True:

    Example conditional data ingestion trigger with a condition.

    For more information, see Conditions.

  4. Add Webhooks or Emails and configure the time between notifications. For more information on webhooks, see Webhooks.

To configure your trigger by using JSON mode instead of Builder mode, paste one of the following JSON templates into your JSON config. "triggers" is a top-level section, similar to "components" or "services".

"triggers": [
    {
      "name": "<trigger name>",
      "event": {
        "type": "conditional_data_ingested",
        "conditional": {
          "data_capture_method": "<component>:<name-of-component>:<method>",
          "condition": {
            "evals": [
              {
                "operator": "<lt|gt|lte|gte|eq|neq>",
                "value": <object, string, or int>
              }
            ]
          }
        }
      },
      "notifications": [
        {
          "type": "email",
          "value": "<fill-in-email-here>",
          "seconds_between_notifications": <number of seconds>
        }
      ]
    }
]
{
  "components": [
    {
      "name": "local",
      "model": "pi",
      "type": "board",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": []
    },
    {
      "name": "my_temp_sensor",
      "model": "bme280",
      "type": "sensor",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": [],
      "service_configs": [
        {
          "type": "data_manager",
          "attributes": {
            "capture_methods": [
              {
                "method": "Readings",
                "additional_params": {},
                "capture_frequency_hz": 0.017
              }
            ]
          }
        }
      ]
    }
  ],
  "triggers": [
    {
      "name": "trigger-1",
      "event": {
        "type": "conditional_data_ingested",
        "conditional": {
          "data_capture_method": "sensor:my_temp_sensor:Readings",
          "condition": {
            "evals": [
              {
                "operator": "gt",
                "value": {
                  "co2": 1000
                }
              }
            ]
          }
        }
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": 10
        }
      ]
    }
  ]
}

The following attributes are available for triggers:

NameTypeRequired?Description
namestringRequiredThe name of the trigger
eventobjectRequiredThe trigger event object:
  • type: conditional_data_ingested.
  • conditional: Condition object. See Conditions for more information.
notificationsobjectRequiredThe notifications object:
  • type: The type of the notification. Options: webhook, email
  • value: The URL to send the request to or the email address to notify.
  • seconds_between_notifications: The interval between notifications in seconds.

Conditions

The conditional object for the conditional_data_ingested trigger includes the following options:

NameTypeRequired?Description
data_capture_methodstringRequiredThe method of data capture to trigger on.
Example: sensor:<name-of-component>:Readings.
conditionobjectOptionalAny additional conditions for the method to fire the trigger. Leave out this object for the trigger to fire any time there is data synced.
Options:
  • evals:
    • operator: Logical operator for the condition.
    • value: An object, string, or integer that specifies the value of the method of the condition, along with the key or nested keys of the measurements in data capture.

Options for operator:

NameDescription
ltLess than
gtGreater than
lteLess than or equal to
gteGreater than or equal to
eqEquals
neqDoes not equal

Examples:

"condition": {
  "evals": [
    {
      "operator": "lt",
      "value": {
        "Line-Neutral AC RMS Voltage": 130
      }
    }
  ]
}

This eval would trigger for the following sensor reading:

{
  "readings": {
    "Line-Neutral AC RMS Voltage": 100
  }
}
"condition": {
  "evals": [
    {
      "operator": "lt",
      "value": {
        "coordinate": {
          "latitude": 50
        }
      }
    }
  ]
}

This eval would trigger for the following sensor reading:

{
  "readings": {
    "coordinate": {
      "latitude": 40
    }
  }
}

Stop data capture

If this is a test project, make sure you stop data capture to avoid charges for a large amount of unwanted data.

In the Data capture section of your sensor’s configuration, toggle the switch to Off.

Click the Save button in the top right corner of the page to save your config.

Data synced

You must configure data capture for your machine to use this trigger.

  1. Go to the CONFIGURE tab of your machine on the Viam app. Click the + (Create) button in the left side menu and select Trigger.

    The Create menu with Trigger at the bottom of the list of options.
  2. Name the trigger and click Create.

  3. Select Data synced to cloud as the Type.

    Then, select the data types for which the Trigger should send requests. Whenever data of the specified data types is ingested, a POST request will be sent.

  4. Add Webhooks or Emails and configure the time between notifications. For more information on webhooks, see Webhooks.

To configure your trigger by using JSON mode instead of Builder mode, paste one of the following JSON templates into your JSON config. "triggers" is a top-level section, similar to "components" or "services".

  "triggers": [
    {
      "name": "<trigger name>",
      "event": {
        "type": "part_data_ingested",
        "data_ingested": {
          "data_types": ["binary", "tabular", "file"]
        }
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws",
          "seconds_between_notifications": <number of seconds>
        }
      ]
    }
  ]
{
  "components": [
    {
      "name": "local",
      "model": "pi",
      "type": "board",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": []
    },
    {
      "name": "my_temp_sensor",
      "model": "bme280",
      "type": "sensor",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": [],
      "service_configs": [
        {
          "type": "data_manager",
          "attributes": {
            "capture_methods": [
              {
                "method": "Readings",
                "additional_params": {},
                "capture_frequency_hz": 0.017
              }
            ]
          }
        }
      ]
    }
  ],
  "triggers": [
    {
      "name": "trigger-1",
      "event": {
        "type": "part_data_ingested",
        "data_ingested": {
          "data_types": ["binary", "tabular", "file"]
        }
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": 10
        }
      ]
    }
  ]
}

The following attributes are available for triggers:

NameTypeRequired?Description
namestringRequiredThe name of the trigger
eventobjectRequiredThe trigger event object:
  • type: part_data_ingested.
  • data_types: The data types that trigger the event. Options: binary, tabular, file, unspecified.
notificationsobjectRequiredThe notifications object:
  • type: The type of the notification. Options: webhook, email
  • value: The URL to send the request to or the email address to notify.
  • seconds_between_notifications: The interval between notifications in seconds.

Part is online

  1. Go to the CONFIGURE tab of your machine on the Viam app. Click the + (Create) button in the left side menu and select Trigger.

    The Create menu with Trigger at the bottom of the list of options.
  2. Name the trigger and click Create.

  3. Select Part is online as the trigger Type.

  4. Add Webhooks or Emails and configure the time between notifications. For more information on webhooks, see Webhooks.

To configure your trigger by using JSON mode instead of Builder mode, paste one of the following JSON templates into your JSON config. "triggers" is a top-level section, similar to "components" or "services".

  "triggers": [
    {
      "name": "<trigger name>",
      "event": {
        "type": "part_online"
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": <number of seconds>
        }
      ]
    }
  ]
{
  "components": [
    {
      "name": "local",
      "model": "pi",
      "type": "board",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": []
    }
  ],
  "triggers": [
    {
      "name": "trigger-1",
      "event": {
        "type": "part_online"
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": 10
        }
      ]
    }
  ]
}

The following attributes are available for triggers:

NameTypeRequired?Description
namestringRequiredThe name of the trigger
eventobjectRequiredThe trigger event object:
  • type: part_online.
notificationsobjectRequiredThe notifications object:
  • type: The type of the notification. Options: webhook, email
  • value: The URL to send the request to or the email address to notify.
  • seconds_between_notifications: The interval between notifications in seconds.

Part is offline

  1. Go to the CONFIGURE tab of your machine on the Viam app. Click the + (Create) button in the left side menu and select Trigger.

    The Create menu with Trigger at the bottom of the list of options.
  2. Name the trigger and click Create.

  3. Select Part is offline as the trigger Type.

  4. Add Webhooks or Emails and configure the time between notifications. For more information on webhooks, see Webhooks.

To configure your trigger by using JSON mode instead of Builder mode, paste one of the following JSON templates into your JSON config. "triggers" is a top-level section, similar to "components" or "services".

  "triggers": [
    {
      "name": "<trigger name>",
      "event": {
        "type": "part_offline"
      },
       "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": <number of seconds>
        }
       ]
    }
  ]
{
  "components": [
    {
      "name": "local",
      "model": "pi",
      "type": "board",
      "namespace": "rdk",
      "attributes": {},
      "depends_on": []
    }
  ],
  "triggers": [
    {
      "name": "trigger-1",
      "event": {
        "type": "part_offline"
      },
      "notifications": [
        {
          "type": "webhook",
          "value": "<https://1abcde2ab3cd4efg5abcdefgh10zyxwv.lambda-url.us-east-1.on.aws>",
          "seconds_between_notifications": 10
        }
      ]
    }
  ]
}

The following attributes are available for triggers:

NameTypeRequired?Description
namestringRequiredThe name of the trigger
eventobjectRequiredThe trigger event object:
  • type: part_offline.
notificationsobjectRequiredThe notifications object:
  • type: The type of the notification. Options: webhook, email
  • value: The URL to send the request to or the email address to notify.
  • seconds_between_notifications: The interval between notifications in seconds.

Webhooks

Example cloud function

If you are using a cloud function or lambda to process the request from viam-server, you can use this template.

The following example function prints the received headers:

from flask import Flask, request

app = Flask(__name__)


@app.route("/", methods=['GET', 'POST'])
def trigger():
    headers = request.headers
    data = {}
    if request.data:
        data = request.json
    payload = {
        "Org-Id": headers.get('org-id', 'no value'),
        "Organization-Name": headers.get('organization-name', '') or
        data.get('org_name', 'no value'),
        "Location-Id": headers.get('location-id', 'no value'),
        "Location-Name": headers.get('location-name', '') or
        data.get('location_name', 'no value'),
        "Part-Id": headers.get('part-id', 'no value'),
        "Part-Name": headers.get('part-name', 'no value'),
        "Robot-Id": headers.get('robot-id', 'no value'),
        "Machine-Name": headers.get('machine-name', '') or
        data.get('machine_name', 'no value'),
        "Component-Type": data.get('component_type', 'no value'),
        "Component-Name": data.get('component_name', 'no value'),
        "Method-Name": data.get('method_name', 'no value'),
        "Min-Time-Received": data.get('min_time_received', 'no value'),
        "Max-Time-Received": data.get('max_time_received', 'no value'),
        "Data-Type": data.get('data_type', 'no value'),
        "File-Id": data.get('file_id', 'no value'),
        "Trigger-Condition": data.get("trigger_condition", 'no value'),
        "Data": data.get('data', 'no value')
    }
    print(payload)

    return payload


if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8080)
import functions_framework
import requests
import time


@functions_framework.http
def hello_http(request):
    headers = request.headers
    data = {}
    if request.data:
        data = request.json
    payload = {
        "Org-Id": headers.get("org-id", "no value"),
        "Organization-Name": headers.get("organization-name", "")
        or data.get("org_name", "no value"),
        "Location-Id": headers.get("location-id", "no value"),
        "Location-Name": headers.get("location-name", "")
        or data.get("location_name", "no value"),
        "Part-Id": headers.get("part-id", "no value"),
        "Part-Name": headers.get("part-name", "no value"),
        "Robot-Id": headers.get("robot-id", "no value"),
        "Machine-Name": headers.get("machine-name", "")
        or data.get("machine_name", "no value"),
        "Component-Type": data.get("component_type", "no value"),
        "Component-Name": data.get("component_name", "no value"),
        "Method-Name": data.get("method_name", "no value"),
        "Min-Time-Received": data.get("min_time_received", "no value"),
        "Max-Time-Received": data.get("max_time_received", "no value"),
        "Data-Type": data.get("data_type", "no value"),
        "File-Id": data.get('file_id', "no value"),
        "Trigger-Condition": data.get("trigger_condition", "no value"),
        "Data": data.get('data', "no value")
    }
    print(payload)

    return 'Received headers: {}'.format(payload)

Returned headers

When an event occurs, Viam sends a HTTP request to the URL you specified for the trigger:

Trigger typeHTTP Method
part_data_ingestedPOST
conditional_data_ingestedPOST
part_onlineGET
part_offlineGET

The request includes the following headers:

Header KeyDescriptionTrigger types
Org-IdThe ID of the organization that triggered the request.all
Organization-NameThe name of the organization that triggered the request.part_online, part_offline
Location-IdThe location of the machine that triggered the request.all
Location-NameThe location of the machine that triggered the request.part_online, part_offline
Part-IdThe part of the machine that triggered the request.all
Machine-NameThe name of the machine that triggered the request.part_online, part_offline
Robot-IdThe ID of the machine that triggered the request.all

Returned data

The request body includes the following data:

Data KeyDescriptionTrigger types
component_nameThe name of the component for which data was ingested.part_data_ingested, conditional_data_ingested
component_typeThe type of component for which data was ingested.part_data_ingested, conditional_data_ingested
method_nameThe name of the method from which data was ingested.part_data_ingested, conditional_data_ingested
min_time_receivedIndicates the earliest time a piece of data was received.part_data_ingested
max_time_receivedIndicates the latest time a piece of data was received.part_data_ingested
method_nameThe name of the method that triggered the request.conditional_data_ingested
machine_nameThe name of the machine that triggered the request.part_data_ingested, conditional_data_ingested
location_nameThe location of the machine that triggered the request.part_data_ingested, conditional_data_ingested
org_nameThe name of the organization that triggered the request.part_data_ingested, conditional_data_ingested
file_idThe id of the file that was ingested.part_data_ingested
trigger_conditionThe condition that triggered the request.conditional_data_ingested
dataThe ingested sensor data. Includes metadata with received_at and requested_at timestamps and data in the form map[string]any.part_data_ingested, conditional_data_ingested (sensor data)

Example project

To see an example project that uses triggers to send email notification, see the Monitor Job Site Helmet Usage with Computer Vision tutorial.