Use the Raspberry Pi Camera to Detect Faces and Read Emotions in Pictures

Give Your Raspberry Pi Smart Vision

Use Google Cloud Vision on the Raspberry Pi to take a picture with the Raspberry Pi Camera and classify it with the Google Cloud Vision API. First, we’ll walk you through setting up the Google Cloud Platform. Next, we will use the Raspberry Pi Camera to take a picture of an object, and then use the Raspberry Pi to upload the picture taken to Google Cloud. Finally, we can analyze the picture in the cloud, and the cloud will try to analyze faces in the picture (tell us everything from emotions on the face to where the face is on the picture). Skynet here we come!

Tools

• Raspberry Pi
• Raspbian for Robots SD Card
• Power Supply
• Raspberry Pi Camera
• Google Cloud Account

The Google Cloud Account is free for 60 days. After 60 days, you will be charged based on usage. You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.

These directions assume you’re using Raspbian for Robots on your SD Card. You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here.

In this picture we’ve asked Google Cloud Vision to analyze the GoPiGo on the left, with the returned text response on the right. It returns “labels” (the things it sees in the picture) such as “vehicle”, “wheel”, “toy”, “product”, and “model car”. Pretty good job Google!

Setting up a Google Cloud Account

The first step is to set up a Google Cloud Account. You can create an account here. You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.
After you’ve created your account, you can set up a project and enable billing. First, click here to go to the Google Cloud Projects Page.

Create a new project.

In our example, we named it “vision1”

You will need to enable billing for your account (you won’t be charged). You can do this by clicking here. Go to the Cloud Platform Console here. Select the project “vision1” and click the hamburger in the upper left hand corner of the page.
Select Billing on the left hand side. From here, you can enable billing on your Google Cloud account. Finally, we’ll enable the API. Click here to enable the API. Select the “vision1” account we created. Click “Continue”. You should get a message that API is enabled.

Optional: Test the Account

Now that the account is setup, why not upload an image from your computer and test it here? You can follow this tutorial to try out a few images and see what Google thinks they are!

Use our picture above. Is it an animal? A reptile? Google will tell you!

Now we want to get a JSON key to put on our Raspberry Pi. This JSON key will handle all the authentication to use our Google Cloud Account. Instead of a password, we can use the file to authenticate our account on the Raspberry Pi.

Caution: You should be careful where you store this key. Be sure to change the password on your Raspberry Pi before putting the JSON key on it, leaving your JSON key unprotected could expose you to something malicious! In the command line type:

sudo passwd<br>

And follow the prompts to change the password on the Raspberry Pi.

• Head back to the Console in Google Cloud. Find the box titled “Use Google APIs” and click “Enable and manage APIs”

• Click “Credentials” and Create Credentials. Credentials is on the left hand side, with a picture of a key next to it.

• Select “Create an Service Account Key”. See more about what we’re doing here.

• Under “Service Account” Select “New Service Account”. We’ll give it a name, “vision”

• Finally, create a role. We’ll give it full access, so select “Project” and “Owner” to give the Pi Full access to all resources.

• A popup should come up telling you you have created a new key, and an automatic download of the JSON key should begin. Keep track of this file!

That should be it, you should have a service account key!

You should now use an FTP program (such as FileZilla) or Samba (see our tutorial here!) to move the JSON file over to your Raspberry Pi. Place the JSON file in the home directory.

/home/pi

The hardware is pretty simple! We’ll be using a Raspberry Pi Camera with the Raspberry Pi, here are some directions on setting up the camera.

These Raspberry Pi Camera setup directions assume you’re using Raspbian for Robots on your SD Card. You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here. If you’re not using Raspbian for Robots, you will need to take additional steps to enable and setup your camera; here are some directions on setting up the camera.

In this project, we’re assuming you’re using our image, Raspbian for Robots. If you’re not, you can download it here!

Next, we’ll upgrade Pip. Pip is a package manager for python language installations. Note, you should have Pip installed (it comes installed on Raspbian for Robots); if you don’t upgrade, you will get an error on installation!

 sudo pip install --upgrade pip <br> sudo apt-get install libjpeg8-dev

Next, install Google API Python Client. Again in the command line, run:

sudo pip install --upgrade google-api-python-client

Next, install Python Imaging Library. Again in the command line, run:

sudo pip install --upgrade Pillow

Install Python Picamera:

sudo apt-get install python-picamera

Turn on Super User. In the command line, type the command “su”:

su

You’ll be prompted for your password; this is the password you used to login to your Raspberry Pi.

In the home directory, we will make the JSON file available to any application we’re running. Run the command:

export GOOGLE_APPLICATION_CREDENTIALS=filename.json

Be sure to substitute your JSON filename in this command with the name of the file you have on your Raspberry Pi.

Get the Example Code

Get the example code: we have three python examples on Github here.

sudo wget https://github.com/DexterInd/GoogleVisionTutorials

Here’s the scary part! Let’s detect a face and see if Google can detect some emotions! With Google Cloud Vision you can detect a range of emotions, headwear, some head coordinates, and other fun data. For this part, I made a special selfie-Pi using the GrovePi Case.

I ran the program in the command line:

python camera-vision-face.py

Here’s the picture I took:

Clearly I’m trying to smile a little for the camera! The response Google gives for facial analysis is pretty long, the JSON is very detailed, so I’m pulling the highlights below:

According to Google Vision, I’m not likely to be Angry (VERY_UNLIKELY).

I’m not likely to be wearing headwear, however, it is possible that I’ve got joy (joyLikelihood is “POSSIBLE”).

Finally, I’m not likely to be sorrowful, and I’m not likely to be surprised. These are just some of the emotions that Google Cloud Vision will scan faces for and return information on.

Google Cloud Vision is a really fun service to just play around with! Google Cloud Vision on the Raspberry Pi is perfect for projects with the Raspberry Pi and Raspberry Pi Camera, and adds in-context vision for the Raspberry Pi.

Learn more about our projects here

Questions? Refer to our tutorial here