Autonomous Ball Chasing Rover Using Computer Vision

There's been a lot of talk of computer vision and artificial intelligence recently with the developing world becoming more digital day by day. This got me interested in computer vision especially, inspiring me to create a ball chasing rover car that identifies, and chases a red ball, and once in range, attempts to propel the ball using the servos at the front.

This project relies mostly on the Raspberry Pi, which processes the incoming video, identifies the ball, and controls the speed of the motors in relation to the position of the ball to chase it. The Arduino has a completely independent connection of electronics, that measures the distance of the ball from the rover, displays that distance using the Neopixel ring, and when too close, activates the servos.

This project was a lot of fun to do and i hope you like it.

Costs

• If you don't have anything, this may cost you around £150, but these are good value items which can be easily reused for other projects

Hardware

• Raspberry Pi 3+ or higher
• Raspberry Pi Motor HAT
• Robot Car Chassis
• Any 5V power supply (to power the Pi/Arduino)
• Lipo battery or a AA battery pack (to power the motors)
• Lipo battery charger
• Male/Female DC Connectors
• 5V Voltage Regulator
• Raspberry Pi Camera / USB WebCamera
• Spacers or standoffs
• Any Arduino (Nano preferred)
• Mini breadboard
• Ultrasonic rangefinder
• 2x 9g SG90 Servos
• Neopixel 8 ring (or any other Neopixel ring) - Optional
• Male-Male and Male-Female Dupont Wires

Tools

• Soldering iron and solder
• Definitely zipties
• Probably Blutack

So, computer vision. There's a load of complicated programs and algorithms for computer vision to detect cats and dogs, but for this project, seen as we need to identify an object as simple as a ball, OpenCV is a great and easy tool to use. However, as easy as it is to use OpenCV, it isnt as easy to install it on your Raspberry Pi. It can be quite complicated so I'm not going to confuse you with my own words, and instead link you two guides on how to do it. There's the hard and painful way, and the easy way. Trust me, it will save you a lot of time if you choose the easy way (I did)

Once you've got OpenCV installed and tested to make sure its working fine, it's now time to move onto the Motor and Camera software.

Motor Library

1) Enable IC2 by typing the following command in the Raspberry Pi terminal:

sudo raspi-config

• Go to interfacing options (or Advanced option on older models)
• Select IC2 and enable it
• Reboot your Pi

2) Check to see if your python version is 3 or better. If not, upgrade your python version to 3 or better.

3) To install the library, type the following command into the terminal:

sudo pip3 install adafruit-circuitpython-motorkit

Pi Camera Library

1) Similar to the Motor library, you need to enable the Pi Camera:

sudo raspi-config

• Go to interfacing options
• Select Camera and enable it
• Reboot your Pi

2) To connect the Camera to the Pi, firstly make sure your Pi is turned off

3) Locate the camera module port on the Pi and gently pull up on the edges of the port's plastic clip

4) Insert the Camera Module ribbon cable; make sure the connectors at the bottom of the ribbon cable are facing the contacts in the port shown in the photo

4) Push the plastic clip back into place

5V Power Supply

1) Connect the battery to the power board

2) Position the power board under the Pi to save space

3) Use provided or extra standoffs or spacers between power board and Pi

4) Connect the Pi to the power board using a micro-USB cable

Motor HAT

1) Solder the header pins and terminal blocks into the motor HAT given, making sure the header pins are pointing downwards and the terminal pins are on top of the motor HAT

2) Insert the header pins into the terminal pins of the Raspberry Pi, making sure that the Pi Camera cable goes through the slit gap in the motor HAT for the Pi Camera cable

3) Install spacers or standoffs between the Motor HAT and the Raspberry Pi, placing them on all 4 corners and securing them with glue/nuts and bolts (This is necessary to stop the Motor HAT coming into contact with the Pi metal HDMI casing, potentially risking a short circuit)

4) From top to bottom - Motor HAT, Raspberry Pi, Power board

For the rover’s function to *destroy* the ball, it will be controlled by an Arduino. Any Arduino will work but a Arduino Nano is preferred due to its small size. It's connected to an Ultrasound Detector to detect the distance to the ball, and transfers that information to the Arduino, which controls the servos and Neopixel ring. Start the simulation below to see whats happening:

1. Upload the Arduino Code in the Tinkercad circuit (press "code" tab) below to the Arduino
2. Place the Arduino on a mini breadboard or alternatively, solder the wires to the Arduino pins
3. Wire all electronics correctly shown below to the Arduino through the breadboard
4. Power the Arduino by connecting it to the same power board supplying power to the Pi using a USB connection (There are two 5V output USB ports on the power board)

To power the motors, a different power source is being used. In this case, it's a LiPo battery, either a 2 cell or 3 cell battery. As the voltage of the battery itself is too high for the motors being used, a voltage regulator is needed to reduce the voltage that the motors are able to run on (5-6V). Male and female power connectors are used so that power to the motors can be stopped at any time by unplugging the male connector from the female connector.

1. Charge up the battery
2. Using the JST connection on the battery, connect positive and negative wires to the right connections shown in the image. Use tape to secure the wires into the JST connection.
3. Fasten the other side of the wires to the terminals in the male DC power connector using a screwdriver
4. Fasten a separate pair of positive and negative wires to a female DC power connector
5. Connect the other side of the wires to the terminals on the voltage regulator, making sure input voltage wires are connected to the input terminals on the regulator and are the right polarity (You can check polarity and input/output terminals on the back of the regulator)
6. Connect another pair of positive and negative wires to the output terminals on the voltage regulator
7. Connect the other side of the wires to the input voltage terminal block on the motor HAT, making sure the polarity of wires are correct throughout
8. To power the motor HAT, connect the male and female DC power connectors together
9. To view output voltage of the regualator, press the button on the regulator until a reading of output voltage appears, indicated by a output voltage LED onboard
10. To get a output voltage of 5-6V from the regulator, using a screwdriver, rotate the potentiometer anti-clockwise until voltage is reduced to 5-6V

1. Solder the given wires to the motor connections
2. Using the side supports and given screws, fix the motor in place
3. Use given nuts to fix screws into place
4. Slide the wheels onto the motor shaft
5. Repeat for the other 3 motors

1. Using zip ties, secure the ultrasonic rangefinder onto the front on the lower floor
2. Secure two servos on either side of the ultrasonic finder but not too close so that the servos clash with the rangefinder
3. Behind the rangefinder and servos, secure the Arduino Nano using a ziptie and the holes in the floor between the front two motors
4. Behind the Arduino, secure the voltage regulator in the same way in between the two back motors
5. Behind the voltage regulator, secure the Lipo battery in a horizontal orientation

1. Using the given spacers and screws, connect the top and bottom floors together
2. Using a ziptie, secure the Pi/Arduino Battery at the back of the top floor above the Lipo Battery
3. Place the stack of the power board, Pi and Motor HAT in the middle of the top floor
4. Power the Pi using a micro-USB cable from the USB port of the power board to the micro-USB port of the Pi
5. Using the side holes in the top floor to feed the motor wires through, screw the motor wires into the motor terminal pins on the motor HAT
6. Connect the front left motor to M1, front right to M3, back left to M2, back right to M4

1. The Pi camera wire should go through the slit in the motor HAT to the camera at the front of the top floor
2. Secure the camera in place using Blu-tack or similar
3. Secure the Neopixel ring in front of the camera on the same Blu-tack, making sure the ring doesn't obstruct the view of the camera
4. The Neopixel ring wires should feed down into the bottom floor where the Arduino is
5. Using a USB 2.0 cable, power the Arduino by connecting the USB port of the same power board to the port of the Arduino, feeding the cable through the central hole in the floor.
6. Feed this wire through the large circular hole behind the Pi Camera to the Arduino underneath
7. If the wire is long and there is excess length, use a ziptie to bundle the excess wire so that it doesnt get in the way of the wheels.

Once you've sorted out the electronics and assembled the rover, the only thing left to do is to program the rover:

1) Using Thonny or another python IDE, copy the code below into a file and save it with a name like "BallChasingRover".

from picamera.array import PiRGBArrayfrom picamera import PiCameraimport timeimport cv2import numpy as npfrom adafruit_motorkit import MotorKitkit = MotorKit()motors = [kit.motor3, kit.motor4, kit.motor1, kit.motor2]motorSpeed = [0,0,0,0]speedDef = 0.5leftSpeed = speedDefrightSpeed = speedDefdefTime = 0.01driveTime = defTimemaxDiff = 0.2kp = 1.0ki = 1.0kd = 1.0ballX = 0.0ballY = 0.0x = {'axis':'X',     'lastTime':int(round(time.time()*1000)),     'lastError': 0.0,     'error': 0.0,     'duration': 0.0,     'sumError': 0.0,     'dError': 0.0,     'PID': 0.0}y = {'axis':'Y',     'lastTime':int(round(time.time()*1000)),     'lastError': 0.0,     'error': 0.0,     'duration': 0.0,     'sumError': 0.0,     'dError': 0.0,     'PID': 0.0}def driveMotors(leftChnl = speedDef, rightChnl = speedDef,                duration = defTime):        motorSpeed[0] = leftChnl    motorSpeed[1] = leftChnl     motorSpeed[2] = rightChnl     motorSpeed[3] = rightChnl        motors[0].throttle = motorSpeed[0]    motors[1].throttle = motorSpeed[1]    motors[2].throttle = motorSpeed[2]    motors[3].throttle = motorSpeed[3]                def PID(axis):    lastTime = axis['lastTime']    lastError = axis['lastError']        now = int(round(time.time()*1000))    duration = now - lastTime        axis['sumError'] += axis['error'] * duration    axis['dError'] = (axis['error'] - lastError)/duration        if axis['sumError'] > 1:        axis['sumError'] = 1            if axis['sumError'] < -1:        axis['sumError'] = -1            axis['PID'] = kp * axis['error'] + ki * axis['sumError'] + kd * axis['dError']        axis['lastError'] = axis['error']    axis['lastTime'] = now        return axisdef killMotors():    motors[0].throttle = 0.0    motors[1].throttle = 0.0    motors[2].throttle = 0.0    motors[3].throttle = 0.0params = cv2.SimpleBlobDetector_Params()params.filterByColor = Falseparams.filterByArea = Trueparams.minArea = 2500params.maxArea = 30000params.filterByInertia = Falseparams.filterByConvexity = Falseparams.filterByCircularity = Trueparams.minCircularity = 0.4params.maxCircularity = 1    det = cv2.SimpleBlobDetector_create(params)  #set red filter rangelower_red = np.array([160, 60, 20]) #80 for blueupper_red = np.array([180, 255, 255]) #130 for bluecamera = PiCamera()camera.resolution = (640, 480)camera.framerate = 32rawCapture = PiRGBArray(camera, size=(640, 480))time.sleep(0.1)for frame in camera.capture_continuous(rawCapture, format="bgr", use_video_port=True):    image = frame.array    height, width, chan = np.shape(image)    xMid = width/2 * 1.0    yMid = height/2 * 1.0    imgHSV = cv2.cvtColor(image, cv2.COLOR_BGR2HSV)    redMask = cv2.inRange(imgHSV, lower_red, upper_red)    blur = cv2.blur(redMask, (10,10))    res = cv2.bitwise_and(image, image, mask=blur)    keypoints = det.detect(blur)    try:        ballX = int(keypoints[0].pt[0])        ballY = int(keypoints[0].pt[1])    except:        pass        #not necessary if you dont need to see camera input via VNC    cv2.drawKeypoints(image, keypoints, image, (0,255,0), cv2.DRAW_MATCHES_FLAGS_DRAW_RICH_KEYPOINTS)    xVariance = (ballX - xMid) / xMid    yVariance = (ballY - yMid) / yMid    x['error'] = xVariance/xMid    y['error'] = yVariance/yMid    x = PID(x)    y = PID(y)        leftSpeed = (speedDef * y['PID']) + (maxDiff * x['PID'])    rightSpeed = (speedDef * y['PID']) - (maxDiff * x['PID'])    if leftSpeed > (speedDef + maxDiff):        leftSpeed = (speedDef + maxDiff)    if leftSpeed < -(speedDef + maxDiff):        leftSpeed = -(speedDef + maxDiff)    if rightSpeed > (speedDef + maxDiff):        rightSpeed = (speedDef + maxDiff)    if rightSpeed < -(speedDef + maxDiff):        rightSpeed = -(speedDef + maxDiff)            driveMotors(leftSpeed, rightSpeed, driveTime)    #also not necessary    cv2.imshow("Frame", image)    #cv2.imshow("Mask", blur)   <- shows the mask frame    key = cv2.waitKey(1) & 0xFF    rawCapture.truncate(0)    if key == ord("q"):        breakkillMotors()cv2.destroyAllWindows()

2) OpenCV doesn't work well running it straight from the IDE, so to run the code, open a terminal

3) If you installed OpenCV into a virtual environment (I did), type this into the terminal (If not, just ignore):

workon cv

4) Navigate to where your file was saved (Mine was saved in the folder "Documents"):

cd Documents/

5) Once you're in the folder where the file has been saved, type the following command to run the program:

python 'YourFileName'.py     

Once you press enter, your robot should start moving. If there is a red ball within its view, it should turn and follow it. Once it gets close, the servos should activate. You will have to alter some settings in the python code relative to what the lighting is in your area and how red and big the ball is to make sure the code can correctly identify it. To check if your code is identifying the ball correctly, include the lines of code beginning with "cv2.imshow" to get a live feed of what the camera sees and what it thinks is the ball.

If you have any questions or need any help - as im sure there will be, I am always happy to answer them.