A security system based on face identification

Security systems often include a human component, like a supervisor who monitors camera streams for suspicious activity. In this tutorial, you will learn how to build a security system that detects people and intelligently decides whether to raise an alarm. The system decides whether to raise an alarm based on whether it recognizes the face of the detected person using a machine learning model.

Architecture

• The system uses a camera to obtain a live video stream.
• To detect people on the video stream, the system uses a people detection model, which can identify whether an object detected in your video stream is a person or not.
• You will need to obtain images of all the people whom the security system should know about. The face identification model uses the DeepFace library in conjunction with these photos to recognize people.
• The verification-system module provides a vision service that applies the two models to the camera stream. It also implements logic that starts a countdown when it detects a person in the camera stream and then decides whether to trigger an alarm based on whether it recognizes the person’s face or not.

Prerequisites

Configure the camera

Navigate to the CONFIGURE tab of your machine’s page.

1

Configure the camera you want to use for your security system.

Click the + icon next to your machine part in the left-hand menu and select Component or service. Select the camera type, then select the webcam model or another model if you are using a different camera. Enter the name my_webcam for your camera and click Create.

2

Position your camera somewhere where it can detect people.

3

Test the camera stream.

Click on the camera’s TEST panel to see the camera stream.

Configure the people detector

In order for your machine’s camera to detect the presence of a person in its field of vision, we recommend using the EfficientDet-COCO model from the Viam Registry. The model can detect a variety of objects, which you can see in the labels.txt file, including people with the person label.

To run the machine learning model on your machine, use the ML model service:

1. On your machine’s CONFIGURE tab, click the + icon next to your machine part in the left-hand menu and select Component or service.
2. Select type ML model, then select model TFLite CPU.
3. Enter persondetect as the name for your ML model service, then click Create.
4. On the ML model service panel, select Deploy model on machine for the Deployment field.
5. Click Select model, then select the EfficientDet-COCO model by viam-labs from the Registry tab of the modal that appears.
6. Add the vision / ML model vision service to your machine and name it person-detector.
7. On the vision service panel, select persondetect as the ML Model and my_webcam as the Default Camera.

Now you are ready to configure the face identification model.

Configure face identification

You now have a machine capable of detecting people in its camera feed. To make it an intelligent security system, it also needs to identify specific people in order to decide whether to trigger an alarm.

To do this, you can use the face-identification module, which uses Facebook’s DeepFace library to perform face identification:

path/
└── to/
    └── known_faces/
        └── john_doe/
        |   ├── john_1.jpeg
        |   └── john_2.jpeg
        └── jane_doe/
        |   ├── jane_1.png
        |   └── jane_2.jpeg
        └── admin_team/
            └── group_photo.png

scp -r /path/to/known_faces username@my-machine.local:/home/known_faces

{
  "camera_name": "my_webcam",
  "picture_directory": "/path/to/known_faces"
}

Configure the security system

Now that you have configured both the people detector and the face identification service, you are ready to add the alarm logic that combines both.

The verification-system module contains the required logic. It uses the camera stream, the people detector, and the face identification service to decide when to raise an alarm.

1. On your machine’s CONFIGURE tab, add the vision / verification-system vision service and name it verification-system.

2. On the panel that appears, enter the following configuration into the attributes field:

   {
     "camera_name": "my_webcam",
     "trigger_1_detector": "person-detector",
     "trigger_1_labels": ["Person"],
     "trigger_1_confidence": 0.35,
     "trigger_2_detector": "person-detector",
     "trigger_2_labels": ["Person"],
     "trigger_2_confidence": 0.5,
     "verification_detector": "face-detect",
     "verification_labels": ["john_doe", "jane_doe"],
     "disable_alarm": false,
     "disarmed_time_s": 10,
     "countdown_time_s": 10
   }
   

   The verification-system module uses distinct states. Its default state is called TRIGGER_1, which the system is in when there are low-confidence detections or no detections.

   If a person is detected in the camera stream with high confidence, the system will go into the COUNTDOWN state. During the countdown, it waits for a verified face to be identified by the configured verification_detector, which is the face-detect vision service. If the face identification service returns a label that is in the verification_labels array within the 10-second countdown, no alarm will be raised. Otherwise the system raises an alarm.

   For a more detailed overview on the states, see the verification-system module documentation.

3. If you used different names for your camera, model, or vision service, update them.

4. Also update the verification_labels to use the names of the people your system can detect - these are the folder names inside the known_faces folder.

5. For more configuration information, see the verification-system module documentation.

Test the system

At this point, your machine is fully capable of detecting people in its camera feed and identifying whether a specific detected person is “approved” (as specified under "verification_labels") or not. To see this in action, click on the verification-system’s TEST panel to see the camera stream.

The current state of the verification-system is visible as an overlay on the camera stream. It should be TRIGGER_1 if no people are visible. Have one or more people walk in front of the camera and look directly at it. Watch the state change to COUNTDOWN and then DISARMED when an approved person is detected, or to ALARM if no approved person appears within 10 seconds.

Using the alarm state

The alarm state currently does not cause anything to happen besides appearing as an overlay on the camera stream.

To trigger an audio alarm or have your machine take an action based on the reported state, you can fork the verification-system module to add logic when the system enters the ALARM state.

Next steps

Now that you have the verification aspect of your system working, you can use this as a launch point for customizing your own DIY home security system. To dive deeper, we recommend looking at the verification-system module logic.