Build Your Own AI Personal Assistant and Smart Security Camera in Python

In this instructable, I will show you how you can create two easy and practical projects using Face Recognition in Python.

• The first project will be Using Face Recognition as a Security Camera
• The second project will be A personal Assistant using Face Recognition

This tutorial assumes you have some experience with python and installing python modules using PIP

Hardware Requirements:

• A laptop/desktop with a webcam
• Andriod Phone to receive a push notification (only for Step 2)

Software Requirements:

• Python 3
• External Python Libraries
• face_recongition
• google tts
• playsound
• opencv
• pushbullet (only for Step 2)
• todoist-api (only for Step 3)

In the first step of the project, we will write the code to teach a face to the program and then see if it can correctly identify/recognize the face that we taught it. To do this we will use the face_recogniztion library/module in python

Install Python Library

pip3 install face_recognition
pip3 install opencv-python

Teach our face

We need to first teach a face to our program. We do this by loading an image ( make sure to have this image in the same folder as the script/program) of a face that would like to recognize. Once we load the image we use the "face encoding" function to get all the features of the face and save it a variable for later use.

import face_recogition
abbas_image = face_recognition.load_image_file("abbas.jpg")
abbas_face_encoding = face_recognition.face_encodings(abbas_image)[0]
known_face_encodings = [abbas_face_encoding,]
known_face_names = ["Abbas",]

Recognize our face

Once we have the encoding of a face that we want to recognize using the output of our webcam we can look at each frame of the video and see if we can detect a face. Once we have detected a face we get the encoding of that face and compare it to the encoding of the face that we want to recognize. If the encodings are similar then it is a match else the face will be categorized as "Unknown". The code to do this look like this:

import cv2
video_capture = cv2.VideoCapture(0)
while True:
	    # Grab a single frame of video
	    ret, frame = video_capture.read()

	    # Convert the image from BGR color (which OpenCV uses) to RGB color (which face_recognition uses)
	    rgb_frame = frame[:, :, ::-1]
	
	    # Find all the faces and face enqcodings in the frame of video
	    face_locations = face_recognition.face_locations(rgb_frame)
	    face_encodings = face_recognition.face_encodings(rgb_frame, face_locations)
	
	    # Loop through each face in this frame of video
	    for (top, right, bottom, left), face_encoding in zip(face_locations, face_encodings):
	        # See if the face is a match for the known face(s)
	        matches = face_recognition.compare_faces(known_face_encodings, face_encoding)
	
	        name = "Unknown"
	
	        # If a match was found in known_face_encodings, just use the first one.
	        if True in matches:
	         first_match_index = matches.index(True)
	         name = known_face_names[first_match_index]

	        # Draw a box around the face
	        cv2.rectangle(frame, (left, top), (right, bottom), (0, 0, 255), 2)
	
	        # Draw a label with a name below the face
	        cv2.rectangle(frame, (left, bottom - 35), (right, bottom), (0, 0, 255), cv2.FILLED)
	        font = cv2.FONT_HERSHEY_DUPLEX
	        cv2.putText(frame, name, (left + 6, bottom - 6), font, 1.0, (255, 255, 255), 1)
	
	    # Display the resulting image
	    cv2.imshow('Video', frame)
	
	    # Hit 'q' on the keyboard to quit!
	    if cv2.waitKey(1) & 0xFF == ord('q'):
	        break
	
	# Release handle to the webcam
	video_capture.release()
	cv2.destroyAllWindows()

Now when you run your program your a window will pop up and you should see your face is detected and also correctly identified. You can also exit the program by pressing 'q' on your keyboard

Now that we are able to recognize a face using python let's use this to create a face-recognizing security camera that can alert you when an unknown person is detected. We will reuse the code from step 1 for the face recognition part and for the alarm/announcement, we will use Google Text To Speech (gTTs) and playsound library. We will also send an alert notification on our smartphone using the Push Bullet API (only on Android)

Let's start by installing the necessary libraries using pip

pip3 install gTTs
pip3 install pushbullet.py
pip3 install playsound
pip3 install PyObjc

Adding an announcement

We will add this code in an else clause after the compare operation. Basically what we do is "if a match is not found then we create a custom speech mp3 using google text to speech and play it on our computer's speaker using the playsound library. The message will be played every 2s.

from gtts import gTTs
import datetime
import playsound

# Outside the while loop
last_time = datetime.datetime.now()

        if True in matches:
            first_match_index = matches.index(True)
            name = known_face_names[first_match_index]
        else:
            current_time = datetime.datetime.now()
            if current_time <= last_time + datetime.timedelta(seconds=2):
                tts = gTTS('Intruder Alert ! Intruder Alert.', lang='en')
                tts.save('intruder.mp3')
                playsound.playsound('intruder.mp3')
                last_time = current_time

Sending a notification ( Only on Android )

We will use the Push Bullet service to send notifications to our phones. To do this we need to first do the following:

1. Go to Push Bullet Website and Sign Up
2. Once signed in go to the settings tab and click on the button "Create an Access Token". We will use the token in our python script ( see image)
3. Download Push Bullet App on your Android and Sign in
4. Add the following code to send a notification on your smartphone

from pushbullet import Pushbullet
# Outside the while loop
api_key = "YOUR_ACCESS_TOKEN"
pb = Pusbullet(api_key)

# In the else clause we added in the last step
pb.push_note("Intruder Detected ", "An unknown intruder was detected")

Now run the script and when an "Unknown" person is detected an alarm should be played on the your computer and you should receive a push notification on your smartphone.

Another use of Face Recognition could be to use it as a personal assistant. The inspiration for this use case came from my laziness in checking my to-do list when I came to my home office desk and as a reason missing important deadlines. Furthermore, I also wanted it to only tell me the list and not when someone else ( e.g wife or kids) walked in, and therefore face recognition was a perfect fit

The features of this assistant are the following:

1. Greets you
2. Tells you the latest weather
3. Reads you your incomplete task from your Todoist Todo list

The face recognition code will be the same as Step 1

Adding the greeting

This is easy. We just create a new function that will take the name of the user that was detected and create greeting audio with gTTS and play in using the playsound library. Then we call the function from the "True in matches" section in the while loop

def greet_user(name):
    tts = gTTS(f'Hello {name}.', lang='en')
    tts.save('greet.mp3')
    playsound.playsound('greet.mp3')

Announcing the latest weather

For getting the latest weather information we will utilize the OpenWeatherMap API. To use the API we need to first generate an API key. To do this go to their website and signup. Once you have signed up and login click on your profile and go to the "My API keys" section ( See image ). Over here you should be able to generate a new API key to use. Once you have the API key let's query the data. We will use the "requests" library to query the data and the "json" library to convert it to python parseable format

import json
import requests

weather_api_key = "YOUR_API_KEY"
base_url = "http://api.openweathermap.org/data/2.5/weather?"
city_name = "YOUR_CITY_NAME"

def get_latest_weather():
    raw_response = requests.get(base_url + "appid=" + weather_api_key + "&q=" + city_name)
    data = raw_response.json()
    current_temp = round(int(data["main"]["temp"]) - 273.15)
    current_hum = round(int(data["main"]["humidity"]))
    current_cond = data["weather"][0]["description"]
    tts = gTTS(f'The current condition is {current_cond} with the temperature of {current_temp} degree celcius and current humidity at {current_hum} percent')
    tts.save("current_weather.mp3")
    playsound.playsound("current_weather.mp3")

Don't forget to call the "get_latest_weather" function from the "True in matches" section in the while loop

Reading you incomplete tasks from your todo list

For my to-do list, I didn't want to create a to-do list application but wanted to use an already available solution. That's where I found Todoist. It's a complete to-do list management application with a nice web UI + ios and android integration. It also has an open API so it integrates well with python also. To get data from the API as usual we need an API key, which you can get by logging into the Todoist click on your profile picture in the top-right hand corner, and then selecting ‘Integrations’. You should find your API Key there (see image). Next, we will create a new project from the web app and add a few tasks to them. Now let's get this information into our python application

Start by installing the Todoist python package

pip3 install todoist-python

We will also need to get the "id" of the new project that we created. We can do this by running these commands from the python repl

from todoist.api import TodoistAPI
api = TodoistAPI("YOUR_API_KEY")
api.sync()
print(api.state['projects'])

When you run this you might have a big list. Look for the 'name' of your project and once you have found it copy the "id" to your python program (see image). Once you have all this you can query and get all the uncompleted tasks running this code:

api_key = "YOUR API_KEY"
api = TodoistAPI(api_key)

def get_latest_todo_list():
    api.sync()
    all_data = api.projects.get_data(YOUR_PROJECT_ID)
    todo_list = [items["content"] for items in all_data["items"]]
    audio = f'''
    You have {len(todo_list)} items in your to do list
    '''
    for item in todo_list:
        audio = audio + item + ', '
    tts = gTTS(audio)
    tts.save("todo_list.mp3")
    playsound.playsound("todo_list.mp3")

Don't forget to call the "get_latest_todo_list" function from the "True in matches" section in the while loop