Use Object Detection to Turn Your Lights On
Caution
There are breaking changes in the Vision Service. This tutorial has not yet been updated.
This tutorial uses the Viam Vision Service with your computer’s built-in webcam to detect the presence of a person and turn on a lamp when you sit down at your desk.
You can turn it into a night light for reading books, a security robot that alerts you when a person is close by, or a bathroom light that only activates when people enter; the opportunities are endless.
This project is a great place to start if you are new to building robots, because the only hardware it requires in addition to your computer is a smart plug or smart bulb.
Hardware requirements
You need the following hardware for this tutorial:
- Computer with a webcam
- This tutorial uses a MacBook Pro but any computer running macOS or 64-bit Linux will work
- Mobile phone (to download the Kasa Smart app)
- Either a smart plug or bulb:
- Kasa Smart Wi-Fi Plug Mini
- (This is what we used for this tutorial.)
- Kasa Smart Light Bulb
- Kasa Smart Wi-Fi Plug Mini
- Table Lamp Base or similar
Software requirements
You will use the following software in this tutorial:
- Python 3.8 or newer
viam-server
- Viam Python SDK
- The Viam Python SDK (software development kit) lets you control your Viam-powered robot by writing custom scripts in the Python programming language. Install the Viam Python SDK by following these instructions.
- Project repo on GitHub
Install viam-server
and connect to your robot
Go to the Viam app and create a new robot.
Go to the Setup tab of your new robot’s page and follow the steps to install viam-server
on your computer.
Configure the camera component
On your new robot’s page, go to the Config tab.
On the Config tab, create a new component:
- Name:
my-camera
- Type:
camera
- Model:
webcam
Click Create Component to add the camera.
Click the Video Path field to reveal a drop-down populated with camera paths that have been identified on your machine.
Select the path to the camera you want to use.
Click Save Config in the bottom left corner of the screen.
Navigate to the Control tab where you can see your camera working.
Set up the Kasa Smart Plug
Plug your smart plug into any power outlet and turn it on by pressing the white button on the smart plug. To connect the plug to your wifi, download the Kasa Smart app from the App Store or Google Play to your mobile phone. When you first open the app, you will be prompted to create an account. As you do this, you will receive an email with subject line “TP-Link ID: Activation Required” to complete your account registration.
Follow the steps in Kasa’s setup guide to add your device and connect it to your wifi. Once it is connected, you will no longer need to use the mobile app.
Open a terminal on your computer and run the following command to install the smart plug Python API:
pip3 install python-kasa
Run the following command to return information about your smart device:
kasa discover
You should see this command output something like this:
Write down or save the host address (for example,
10.1.11.221
). You will need to include it in your Python code in a later step.
Write Python code to control your object detection robot
Now that you have your robot configured and your Kasa plug set up, you are ready to set up the code for the logic of the robot. The files used in this section can all be found in the GitHub repo for this project.
Create the main script file
On your computer, navigate to the directory where you want to put the code for this project.
Create a file there called
Connect the code to the robot
You need to tell the code how to access your specific robot (which in this case represents your computer and its webcam).
- Navigate to the Code Sample tab on the Viam app. Make sure Python is selected in the Language selector.
- In the code sample, find the
payload
, a long string of numbers and letters. Copy it and paste it into line 13 oflightupbot.py in place ofROBOT_SECRET
. - Find the robot address, of the form
robot-name-main.abc1ab123a1.viam.cloud
, and paste it into line 14 oflightupbot.py in place ofROBOT_ADDRESS
.
You also need to tell the code how to access your smart plug.
- Add the host address (for example,
10.1.11.221
) of your smart plug that you found in thekasa discover
step to line 55 oflightupbot.py .
Set the model path and label path
You will use the Vision Service to interpret what your camera sees. You will configure the Vision Service to use a TFLite model to detect specific objects, and a corresponding text file that holds class labels for your TFLite model.
- Take a look at lines 42-48 in
lightupbot.py . These lines configure a Vision Service object detector to use the TFLite model and the list of labels:
vision = VisionClient.from_robot(robot)
params = {"model_path": "./effdet0.tflite", "label_path": "./labels.txt", "num_threads": 1}
personDet = VisModelConfig(name="person_detector", type=VisModelType("tflite_detector"), parameters=params)
await vision.add_detector(personDet)
names = await vision.get_detector_names()
print(names)
- Download
effdet0.tflite andlabels.txt to your project directory. - Line 44 of your
lightupbot.py is where you specify the paths to these files. If you put them in the same directory aslightupbot.py , you don’t need to edit this line. - Save the file.
Run the code
Now you are ready to test your robot!
From a command line on your computer, navigate to the project directory and run the code with this command:
python3 lightupbot.py
If the camera detects a person, it will print to the terminal “This is a person!” and turn on the smart plug. If it does not find a person, it will write “There’s nobody here” and will turn off the plug.
Try moving in and out of your webcam’s field of view. You will see your light turn on and off as the robot detects you!
Your terminal output should look like this as your project runs:
python3 lightupbot.py
This is a person!
turning on
There's nobody here
turning off
Info
You can actually detect any object that is listed in the
To detect something else with the camera, just change the string “person” on line 69 of
if d.class_name.lower() == "person":
print("This is a person!")
found = True
Next Steps
In this tutorial, you learned how to build an object detection robot that turns your lights on using Viam. You could use this same concept to build a smart fan that only turns on if you are sitting at your desk working, turn on the lights in your bathroom mirror only when you are in front of the sink, or activate a pet feeder every time your cat looks at the camera.
To turn this robot into a security alert system, try the other tutorial in this series: Build a Person Detection Security Robot That Sends You a Photo of the Person Stealing Your Chocolates.
For more robotics projects, check out our other tutorials.
You can also ask questions in the Community Discord and we will be happy to help.
Have questions, or want to meet other people working on robots? Join our Community Discord.
Was this page helpful?
Glad to hear it! If there is anything we could be doing better, please create an issue.
We're sorry about that. If you'd like to talk to us for help, please join the Community Discord. To ensure we know what's wrong with this page, you can also open an issue.