FACE TRACKING USING ARDUINO !!!

In a previous instructable I shared how you can communicate between Arduino and Python using 'pyserial' module and control a LED. If you haven't seen it check it out here: COMMUNICATION BETWEEN ARDUINO & PYTHON !

And how you can detect colour of an object and track it on screen, Check that out here : COLOUR DETECTION USING OPENCV AND PYTHON.

In this Instructable I will be showing you how to track faces using Arduino & Python and make the Camera follow the face. This may sound difficult but trust me it isn't, All you need is basic knowledge of Arduino and Python.

So lets get started...

The requirements are minimum . Here I have provided part list of everything you need:

Hardware Requirement :

• Arduino UNO (Amazon US / Amazon EU)
• Web Cam (Amazon US / Amazon EU)
• Servos x 2 (Amazon US / Amazon EU)
• Breadboard (Amazon US / Amazon EU)
• Servo Pan Tilt Kit (Amazon US / Amazon EU)

Software Requirement :

1. Python 2.7 (Should be installed, Linux OS usually have it pre-installed)
2. OpenCV (You can download it separately or install using 'pip install' Explained further)
3. pyserial (Can be installed with pip)
4. numpy.
5. Haarcascade.

After every thing is gathered we can move on to the Installation Step...

Installing Python:

So first we need Python 2.7 up and running. To do this first download and Install python 2.7.14. To check if it is installed correctly Goto : Windows Search >> Type "IDLE" >> Hit Enter. A Python Shell should pop up.

OR

In search type 'CMD' and hit enter to open Command Prompt. In CMD type >> python and hit enter, Python interface should display .

If you see an error in CMD, Do not panic you probably need to set environment variable. You can follow this tutorial Here to set up Environment Variable.

Installing 'pyserial' , 'OpenCV" and "numpy" in python :

To install these modules we will use use pip install ,

First open CMD and type the following codes:-

>pip install serial
>pip install opencv-python
>pip install numpy

these commands will install the necessary modules. Now we can move to the coding part...

Before starting to write code first thing to do is make a new folder as all of the code needs to be stored in same folder. So create a new folder, name it anything you want. and download the 'Haarcascade' from below and paste it in the folder.

Now open notepad and write the script given below , Save it as 'face.py' in the same folder as haarcascade. (You can download the code I have provided the file below) :

#import all the required modules

import numpy as np
import serial
import time
import sys
import cv2

#Setup Communication path for arduino (In place of 'COM5' put the port to which your arduino is connected)
arduino = serial.Serial('COM5', 9600) 
time.sleep(2)
print("Connected to arduino...")

#importing the Haarcascade for face detection
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')

#To capture the video stream from webcam.
cap = cv2.VideoCapture(0)

#Read the captured image, convert it to Gray image and find faces
while 1:
    ret, img = cap.read()
    cv2.resizeWindow('img', 500,500)
    cv2.line(img,(500,250),(0,250),(0,255,0),1)
    cv2.line(img,(250,0),(250,500),(0,255,0),1)
    cv2.circle(img, (250, 250), 5, (255, 255, 255), -1)
    gray  = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    faces = face_cascade.detectMultiScale(gray, 1.3)

#detect the face and make a rectangle around it.
    for (x,y,w,h) in faces:
        cv2.rectangle(img,(x,y),(x+w,y+h),(0,255,0),5)
        roi_gray  = gray[y:y+h, x:x+w]
        roi_color = img[y:y+h, x:x+w]

        arr = {y:y+h, x:x+w}
        print (arr)
        
        print ('X :' +str(x))
        print ('Y :'+str(y))
        print ('x+w :' +str(x+w))
        print ('y+h :' +str(y+h))

# Center of roi (Rectangle)
        xx = int(x+(x+h))/2
        yy = int(y+(y+w))/2
        print (xx)
        print (yy)
        center = (xx,yy)

# sending data to arduino
        print("Center of Rectangle is :", center)
        data = "X{0:d}Y{1:d}Z".format(xx, yy)
        print ("output = '" +data+ "'")
        arduino.write(data)

#Display the stream.
    cv2.imshow('img',img)

#Hit 'Esc' to terminate execution 
    k = cv2.waitKey(30) & 0xff
    if k == 27:
       break

Once this is done , move on to write the code for arduino...

After the python script is ready we need arduino sketch to control the servo. Refer the code below , paste it in Arduino IDE and save it as 'servo.ino' in the same folder as face.py and haarcascade . upload the code and move on to the next step to make the connections.

(Downloadable file given below)

#include<servo.h>

Servo servoVer; //Vertical Servo
Servo servoHor; //Horizontal Servo

int x;
int y;

int prevX;
int prevY;

void setup()
{
  Serial.begin(9600);
  servoVer.attach(5); //Attach Vertical Servo to Pin 5
  servoHor.attach(6); //Attach Horizontal Servo to Pin 6
  servoVer.write(90);
  servoHor.write(90);
}

void Pos()
{
  if(prevX != x || prevY != y)
  {
    int servoX = map(x, 600, 0, 70, 179);
    int servoY = map(y, 450, 0, 179, 95);

    servoX = min(servoX, 179);
    servoX = max(servoX, 70);
    servoY = min(servoY, 179);
    servoY = max(servoY, 95);
    
    servoHor.write(servoX);
    servoVer.write(servoY);
  }
}

void loop()
{
  if(Serial.available() > 0)
  {
    if(Serial.read() == 'X')
    {
      x = Serial.parseInt();
      if(Serial.read() == 'Y')
      {
        y = Serial.parseInt();
       Pos();
      }
    }
    while(Serial.available() > 0)
    {
      Serial.read();
    }
  }
}

I have used a readily available kit for the Pan-Tilt. If you want you can make one yourself using wood/Plastic or even 3D print one.

The one I used is pretty cheap, and very easy to assemble. Yet if you want instructions on how to do that, you can find it here.

The Circuit is pretty simple. Just attach two servos to arduino.

1. Vertical to Pin 5
2. Horizontal to Pin 6
3. Power to +5V
4. Ground to GND

Check the circuit diagram for reference.

• After everything is done last thing to do is test if it works. To test first make sure that servos are properly connected to arduino and sketch is uploaded.
• After sketch is uploaded make sure to close the IDE so the port is free to connect to python.
• Now open 'face.py' with Python IDLE and press 'F5' to run the code. It will take a few seconds to connect to arduino and then you should be able to see a window streaming the web cam. Now the code will detect your face and the servos will track it track it.
• The Servo should move as you move the object . Now just attach the camera to the servos so it will move along with servos.

Thank you.