Visualize Data With Grafana

Once you have configured data query for your organization’s data store, you can visualize your data from a variety of third-party tools, including Grafana. You can choose to visualize data from a component on one machine, from multiple components together, or from many components across a fleet of machines, all from a single pane of glass.

Only components that capture tabular data, such as sensor readings or other time-series data, support data visualization.

You can visualize both the captured tabular data itself as well as its metadata, including machine ID, organization ID, and tags.

Follow the steps in this tutorial to learn how to collect data from your machine, sync that data to the Viam app, enable third-party access to that data, and present that data visually and flexibly in Grafana.

Prerequisites

Before following this tutorial, ensure you have:

• A machine with connected to the Viam app.

• A configured sensor component, such as the ultrasonic sensor, that reports tabular data.

  • This tutorial uses a dataset of plant moisture measurements, originally captured for our Plant watering tutorial using an analog resistive soil moisture sensor connected to an analog-to-digital-converter (ADC). The ADC functionality was provided by the mcp300x-adc-sensor module from the Viam registry.

  • If you already have tabular data synced to the Viam app that you want to use, you can skip this requirement, and can skip directly to the Configure data query portion of this tutorial.

• A Grafana instance.

  • You can use the free trial version for this tutorial.
  • You can use either a local instance of Grafana, or Grafana Cloud. This tutorial will use Grafana Cloud.

The data management service

You can manage how your machine works with data by using the data management service.

The data management service has two parts: data capture and cloud sync.

• Data capture allows you to capture data locally from specific components on your machine running Viam. You can choose the components, corresponding methods, and the frequency of the data capture from the Viam app.

• Cloud sync runs in the background and uploads your machine’s captured data to the Viam app at a defined frequency. Cloud sync is designed to be resilient and to preserve your data even during a network outage or if your machine has low network bandwidth. With cloud sync enabled for a component, data captured locally to your machine is automatically deleted after a successful sync. Data synced between your machine and the Viam app is encrypted in transit (over the wire) and when stored in the cloud (at rest).

Data capture and data sync are frequently used together, and are both enabled by default when you add the data management service to your machine.

To capture data from your machine and sync to the Viam app, add the data management service and configure data capture for at least one component.

Add the data management service

First, add the data management service to your machine to be able capture and sync data:

1. Navigate to your machine’s CONFIGURE tab in the Viam app.

2. Click the + button in the left-hand menu and select Service from the dropdown.

3. Select data management.

4. Give the service a name, like viam-data-manager, then click Create.

5. On the panel that appears, you can manage the capturing and syncing functions individually. By default, the data management service captures data locally to the ~/.viam/capture directory, and syncs captured data files to the Viam app every 6 seconds (0.1 minutes in the configuration). Leave the default settings as they are, and click Save in the upper-right corner of the screen to save your changes.

   

For more information, see Add the data management service.

Configure data capture for a component

Once you have added the data management service, you can configure data capture for specific components on your machine. For this tutorial, you will configure data capture for a sensor component, gathering sensor readings to later visualize in Grafana. Only tabular data, such as sensor readings or other time-series data, can be visualized in this manner.

To enable data capture for a sensor component:

1. Navigate to your machine’s CONFIGURE tab in the Viam app.

2. In the configuration pane for your configured sensor component, find the Data capture section, and click the Add method button to enable data capture for this camera.

   • Set the Method to Readings and the Frequency to 0.333. This will capture readings from the sensor device roughly once every 3 seconds. You can adjust the capture frequency if you want the sensor to capture more or less data, but avoid configuring data capture to higher rates than your hardware can handle, as this could lead to performance degradation.

     

3. Click Save Config at the bottom of the window to save your changes.

After a short while, your sensor will begin capturing live readings, and syncing those readings to the Viam app. You can check that data is being captured and synced by clicking on the menu icon on the sensor configuration pane. and selecting View captured data.

For more information see Configure data capture and Configure cloud sync.

Configure data query

Next, enable the ability to query your synced data. When you sync captured data to Viam, that data is stored in the Viam organization’s MongoDB Atlas Data Federation instance. Configuring data query allows you to directly query your data using the Viam app or query your data from a compatible client (such as mongosh), but also allows Grafana to access that data and visualize it.

To enable data query:

1. Follow the steps to configure data query.

2. Note the username and hostname returned from these steps, in addition to the password you chose for that user. You will use this information in the next section.

Configure Grafana

With your machine capturing data and syncing it to Viam, and direct query of that data configured, you can now configure Grafana to access and visualize that data:

1. Navigate to your Grafana web UI, and add the Grafana MongoDB data source plugin to your Grafana instance.

   

   For more information, see Install Grafana Plugins.

2. Navigate to the Grafana data source management page, and select the Grafana MongoDB data source that you just added. For more information, see Grafana data source management.

3. Enter the following information in the configuration UI for that plugin:

   • Connection string: Enter the following connection string, substituting the hostname returned from the previous section for <MONGODB-ATLAS-DF-HOSTNAME> and the desired database name to query for <DATABASE-NAME>:

     mongodb://<MONGODB-ATLAS-DF-HOSTNAME>/<DATABASE-NAME>?directConnection=true&authSource=admin&tls=true
     

     For example, to use the sensorData database, the default name for uploaded tabular data, your connection string would resemble:

     mongodb://data-federation-abcdef12-abcd-abcd-abcd-abcdef123456-e4irv.a.query.mongodb.net/sensorData?directConnection=true&authSource=admin&tls=true
     

     The connection string is specific to your organization ID and configured user. You must have followed the steps under configure data query previously in order for this URL to be valid.

   • Credentials: User: Enter the following username, substituting your organization ID as determined earlier, for <YOUR-ORG-ID>:

     db-user-<YOUR-ORG-ID>
     

     For example, using the organization ID from the previous example, your connection string would resemble:

     db-user-abcdef12-abcd-abcd-abcd-abcdef123456
     

   • Credentials: Password: Enter the password you provided when you configured data query previously.

   

4. Click the Save & test button to save your settings. Grafana will perform a health check on your configuration settings to verify that everything looks good.

This connection allows Grafana to access all synced tabular data under your organization, from any machine.

Visualize your data

With the data connection configured, Grafana now has access to your data and you are ready to create a new dashboard to visualize that data.

• If you are using a local Grafana instance, see Build your first dashboard
• If you are using Grafana Cloud, see Create a dashboard in Grafana Cloud

When prompted to select a data source, select the MongoDB data source that you added earlier.

Select the visualization type that matches your data, which is most likely Time series.

When done, click Save to save your dashboard.

The example below displays readings from a fleet of deployed machines, each equipped with a temperature sensor, reporting the ambient temperature of the office over the course of the day.

Tip

If you have a lot of data, or are exploring an existing data set that you are not familiar with, you may find it useful to browse your data first, before creating a visualization dashboard based on it.

You can use MongoDB Compass to connect to your Viam data store using the same connection string and credentials as above, and browse your data using Compass’ Schema Analyzer, which offers fast insights into what types of data you have available.

Query your data in Grafana

You can also use query language directly in Grafana using the MongoDB Query Editor, which enables data query functionality similar to that of the MongoDB shell, mongosh.

For example, to limit the visualization of the plant watering data set to just the moisture-sensor component, within a certain time range, and limiting returned results to 1000 records, you could use the following query in the query editor:

sensorData.readings.aggregate([
            {$match: {
              component_name: "moisture-sensor",
              time_received: {$gte: ISODate(${__from})}
              }},
            {$limit: 1000}
            ]
          )

This query uses the Grafana global variable $__from, which is populated by the value set from the From dropdown menu on your dashboard, allowing for visualizations based on this query to be dynamically updated when you change your desired time range from that dropdown menu. See Grafana’s Global variables documentation for more information.

Next steps

In this tutorial, you learned:

• how to use the data management service to capture data from your machine and sync it to the Viam app
• how to enable data query access to your synced data
• how to connect Grafana to your data
• how to build a dashboard visualizing that data
• how to use query language to dynamically update the visualization based on UI selections

From here you could:

• include data from additional sensors, or from more machines in your fleet, to be able to view everything together from a single pane of glass
• experiment with your query language syntax to give more flexibility to the operator, by allowing for more UI-based customization of the visualization using Grafana global variables

For more ideas, check out our other tutorials.

You can also ask questions in the Community Discord and we will be happy to help.