Previous
Visualize data
You can query all sensor data in Viam using SQL and MQL. For example, you may sync temperature data from several sensors on one machine or across multiple machines. Once synced, you can run queries against that data to search for outliers or edge cases and analyze how the ambient temperature affects your machines’ operation.
You can query data using the Web UI or with code:
Query with SQL or MQL
Navigate to the Query page. Then, select either SQL or MQL.
Optionally, you can change the data source to query data from a configured hot data store.
Run your query
This example query returns the last 5 readings from any component named sensor-1
in your organization:
SELECT * FROM readings
WHERE component_name = 'sensor-1' LIMIT 5
Select the Stages or Text mode:
Select the $match
stage for Stage 1 from the dropdown.
Add the expression to match against:
{ "component_name": "sensor-1" }
Click + Add query stage to add another aggregation stage.
Select the $limit
stage for Stage 2 from the dropdown.
Add the number of documents to limit to:
5
Add the query in the text field:
[
{ "$match": { "component_name": "sensor-1" } },
{ "$limit": 5 }
]
Review results
Click Run query when ready to perform your query and get matching results. You can view your query results as a JSON array below your query. Click the table icon to switch to table view.
import asyncio
from viam.rpc.dial import DialOptions, Credentials
from viam.app.viam_client import ViamClient
# Configuration constants – replace with your actual values
API_KEY = "" # API key, find or create in your organization settings
API_KEY_ID = "" # API key ID, find or create in your organization settings
ORG_ID = "" # Organization ID, find or create in your organization settings
async def connect() -> ViamClient:
"""Establish a connection to the Viam client using API credentials."""
dial_options = DialOptions(
credentials=Credentials(
type="api-key",
payload=API_KEY,
),
auth_entity=API_KEY_ID
)
return await ViamClient.create_from_dial_options(dial_options)
async def main() -> int:
viam_client = await connect()
data_client = viam_client.data_client
tabular_data_mql = await data_client.tabular_data_by_mql(
organization_id=ORG_ID,
query=[
{
"$match": {
"component_name": "sensor-1"
},
}, {
"$limit": 5
}
]
)
print(f"Tabular Data: {tabular_data_mql}")
tabular_data_sql = await data_client.tabular_data_by_sql(
organization_id=ORG_ID,
sql_query="SELECT * FROM readings WHERE component_name = 'sensor-1' LIMIT 5"
)
print(f"Tabular Data: {tabular_data_sql}")
viam_client.close()
return 0
if __name__ == "__main__":
asyncio.run(main())
package main
import (
"context"
"fmt"
"go.viam.com/rdk/app"
"go.viam.com/rdk/logging"
)
func main() {
apiKey := ""
apiKeyID := ""
orgID := ""
logger := logging.NewDebugLogger("client")
ctx := context.Background()
viamClient, err := app.CreateViamClientWithAPIKey(
ctx, app.Options{}, apiKey, apiKeyID, logger)
if err != nil {
logger.Fatal(err)
}
defer viamClient.Close()
dataClient := viamClient.DataClient()
// Create MQL stages as map slices
mqlStages := []map[string]interface{}{
{"$match": map[string]interface{}{"component_name": "sensor-1"}},
{"$limit": 5},
}
tabularDataMQL, err := dataClient.TabularDataByMQL(ctx, orgID, mqlStages, &app.TabularDataByMQLOptions{})
if err != nil {
logger.Fatal(err)
}
fmt.Printf("Tabular Data: %v\n", tabularDataMQL)
tabularDataSQL, err := dataClient.TabularDataBySQL(ctx, orgID, "SELECT * FROM readings WHERE component_name = 'sensor-1' LIMIT 5")
if err != nil {
logger.Fatal(err)
}
fmt.Printf("Tabular Data: %v\n", tabularDataSQL)
}
import { createViamClient } from "@viamrobotics/sdk";
// Configuration constants – replace with your actual values
let API_KEY = ""; // API key, find or create in your organization settings
let API_KEY_ID = ""; // API key ID, find or create in your organization settings
let ORG_ID = ""; // Organization ID, find or create in your organization settings
async function main(): Promise<void> {
// Create Viam client
const client = await createViamClient({
credentials: {
type: "api-key",
authEntity: API_KEY_ID,
payload: API_KEY,
},
});
const tabularDataMQL = await client.dataClient.tabularDataByMQL(
ORG_ID,
[
{ "$match": { "component_name": "sensor-1" } },
{ "$limit": 5 }
],
false
);
console.log(tabularDataMQL);
const tabularDataSQL = await client.dataClient.tabularDataBySQL(
ORG_ID,
"SELECT * FROM readings WHERE component_name = 'sensor-1' LIMIT 5"
);
console.log(tabularDataSQL);
}
// Run the script
main().catch((error) => {
console.error("Script failed:", error);
process.exit(1);
});
If you want to query data from third-party tools, you have to configure data query to obtain the credentials you need to connect to the third-party service.
Authenticate with the CLI
Authenticate using a personal access token:
viam login
For alternative authentication methods, see Authenticate.
Find your organization ID
The following steps require your organization ID. To find, it use the following command:
viam organizations list
Configure a new database user
Configure a new database user. The database user will be able to connect to your data, which is stored in a MongoDB Atlas Data Federation instance.
The command will create a user with your organization ID as the username. If you or someone else in your organization have already created this user, the following steps update the password for that user instead. Dashboards or other integrations relying on this password will then need to be updated.
Provide your organization’s org-id
from step 2, and a password for your database user.
Your password must be at least 8 characters long with 1 uppercase, and 1 numeric character.
viam data database configure --org-id=<YOUR-ORGANIZATION-ID> --password=<NEW-DBUSER-PASSWORD>
Determine the connection URI
Determine the connection URI (also known as a connection string) for your organization’s MongoDB Atlas Data Federation instance by running the following command with the organization’s org-id
from step 2:
viam data database hostname --org-id=abcd1e2f-a1b2-3c45-de6f-ab123456c123
MongoDB Atlas Data Federation instance hostname: data-federation-abcd1e2f-a1b2-3c45-de6f-ab123456c123-0z9yx.a.query.mongodb.net
MongoDB Atlas Data Federation instance connection URI: mongodb://db-user-abcd1e2f-a1b2-3c45-de6f-ab123456c123:YOUR-PASSWORD-HERE@data-federation-abcd1e2f-a1b2-3c45-de6f-ab123456c123-0z9yx.a.query.mongodb.net/?ssl=true&authSource=admin
This command returns the:
hostname: the MongoDB Atlas Data Federation instance hostname
connection URI: the MongoDB Atlas Data Federation instance connection uniform resource indicator. This is the connection URI to your organization’s MongoDB Atlas Data Federation instance, which is of the form:
mongodb://<USERNAME>:<YOUR-PASSWORD>@<HOSTNAME>/?ssl=true&authSource=admin
Most MQL-compatible database clients require the connection URI, along with your user credentials, to connect to this server.
Some MQL-compatible database client instead require a hostname and database name, along with your user credentials, to connect to this server.
For sensor data, this database name will be sensorData
.
To query captured data, you can use any third-party tools that support querying from MongoDB, such as the mongosh
shell or MongoDB Compass.
Connect to your Viam organization’s data
Run the following command to connect to your Viam organization’s MongoDB instance from mongosh
using the connection URI you obtained during query configuration:
mongosh "mongodb://db-user-abcd1e2f-a1b2-3c45-de6f-ab123456c123:YOUR-PASSWORD-HERE@data-federation-abcd1e2f-a1b2-3c45-de6f-ab123456c123-0z9yx.a.query.mongodb.net/?ssl=true&authSource=admin"
For information on connecting to your Atlas Data Federation instance from other MQL clients, see the Connect to your Cluster Tutorial.
Query data from a compatible client
Once connected, you can run SQL or MQL statements to query captured data directly.
The following query searches the readings
collection in the sensorData
database and gets sensor readings from an ultrasonic sensor on a specific robot_id
where the recorded distance
measurement is greater than 0.2
meters.
use sensorData
db.readings.aggregate(
[
{ $match: {
'robot_id': 'abcdef12-abcd-abcd-abcd-abcdef123456',
'component_name': 'my-ultrasonic-sensor',
'data.readings.distance': { $gt: 0.2 } } },
{ $count: 'numStanding' }
] )
[ { numStanding: 215 } ]
See the MQL documentation for more information.
use sensorData
db.aggregate(
[
{ $sql: {
statement: "select count(*) as numStanding from readings \
where robot_id = 'abcdef12-abcd-abcd-abcd-abcdef123456' and \
component_name = 'my-ultrasonic-sensor' and (CAST (data.readings.distance AS DOUBLE)) > 0.2",
format: "jdbc"
}}
] )
[ { '': { numStanding: 215 } } ]
See the $sql
aggregation pipeline stage documentation for more information.
When querying large datasets, whether from default storage or a hot data store, you can improve the query’s efficiency by specifying the following parameters in the query:
organization_id
location_id
machine_id
part_id
component_type
component_name
method_name
capture_date
Viam stores data in blob storage using the pattern /organization_id/location_id/robot_id/part_id/component_type/component_name/method_name/capture_date/*
.
The more specific you can be, starting with the beginning of the path, the faster your query.
Filter and reduce the amount of data that needs to be processed early, especially when your query expands the data it works with using operators like $limit
and $unwind
.
If you don’t need all fields, use $project
early to reduce the fields in the processing dataset.
If you only need a certain number of results, use $limit
early in the pipeline to reduce data processing.
If you are frequently querying recent data, use the hot data store which provides faster data access.
If you frequently perform the same types of queries, for example for dashboards, use data pipelines. Data pipelines allow you to pre-compute a materialized view of your data at specified intervals.
Viam supports the MongoDB Query Language for querying captured data from MQL-compatible clients such as mongosh
or MongoDB Compass.
Viam supports the following MongoDB aggregation operators:
$addFields
$bucket
$bucketAuto
$count
$densify
$fill
$geoNear
$group
$limit
$match
$project
$redact
$replaceRoot
$replaceWith
$sample
$set
$setWindowFields
$skip
$sort
$sortByCount
$unset
$unwind
You can query data with SQL queries using the MongoDB Atlas SQL dialect, which supports standard SQL query syntax in addition to Atlas-specific capabilities such as FLATTEN
and UNWIND
.
SQL queries are subject to the following limitations:
`
) or double quotes ("
):1
)%
)select
)o''clock
to represent the literal o'clock
).date
data type is not supported. Use timestamp
instead.For a full list of limitations, see the MongoDB Atlas SQL Interface Language Reference.
Was this page helpful?
Glad to hear it! If you have any other feedback please let us know:
We're sorry about that. To help us improve, please tell us what we can do better:
Thank you!