BackUp Camera With OpenCV and YoloV5

My 2004 Toyota Camry needs a little help. After turning twenty years old it was time to add a backup camera. The features of the backup camera include object detection and a Hud. Depending on your vehicle and your access to its motor commands you can implement a real Hud. Unfortunately I could not access my motor commands, so I simulated the hud with the keys 'A' and 'D.'

There are two ways to download the source code: you can follow the steps down below for a clean install, you are doing all the steps I did, or you can skip all of that by cloning my github repo and running my installation script. This is something new to me, and something I have wanted to learn how to do for a while. Here are those steps:

git clone https://github.com/wyattcolburn/EE542.git
cd EE542
chmod +x install.sh
./install.sh
cd yolov5
python3 backupCamera.py

Demo Links:

https://youtu.be/3SsPh4-YQB4

https://youtu.be/kHEE_9Vd66E

ESP 32-Camera Module (breakout board is optional)

Raspberry Pi

A Display (make sure it can handle streaming, check the refresh rate)

Computer to program

Arduino

WSL (I use terminal with Ubuntu)

Our first step is to make sure the ESP32 camera works. First we need make sure we have Arduino setup. If you do not have Arduino, you can download it here: https://www.arduino.cc/en/software

Open Arduino:

Go to Tools->Board Manager and install ESP32 by Espressif Systems

Next Select the correct board, in my case it was the ESP32-Wrover kit. Change the upload options to match the image above.

Then download the Arduino Sketch attached and press upload. The terminal should show the uploading progress, once the upload is done hit the reset button to start the stream.

Next connect to the WiFi network created by the ESP and go to 192.168.4.1/stream to see your real time stream from your ESP camera.

Simple OpenCV implementation:

We want to be able to access the stream on a different device. OpenCV is perfect because it has built in support for video captures through a URL. OpenCV will handle the capturing the stream and opening a display for us. OpenCV also allows us to interact with our stream by drawing things on the frame (like our parking HUD) and using keypresses.

First we need to download OpenCV. We will be using sudo apt install because we are on raspbian OS (ubuntu based)

sudo apt update
sudo apt install python3-opencv

Now we want to use stream_capture.py to get the stream on our raspberry pi. I recommend creating a directory first for this project.

mkdir backupCamera //create the directory
cd backupCamera //enter the directory
vim stream_capture.py //create and edit the file
python3 stream_capture.py //run the file

Then we need to make sure the raspberry pi is connected to the wifi network created by the ESP32. Make sure everything is well.

Now that we have the stream accessed by the raspberry pi we can now add the HUD. Some modification should be added to account for your vehicle. Create another file called hud_dev.py

vim hud_dev.py
python3 hud_dev.py

Yolo stands for you only look once. The name stems from being able to detect multiple objects within an image in a single pass through the neural network. It was trained on CoCo data set, but also can train with your own data set. The YoloV5 library will allow us to seamlessly integrate object detection into our system without having to train our own data set. We are looking for the easiest plug and play solution. Additionally, YOLOv5 benefits from an active development community, providing extensive documentation and pre-trained models to accelerate development and improve performance.

I highly recommend checking out the github: https://github.com/ultralytics/yolov5

To install using git

1) Make sure we have git installed, the github says to use pip to install the dependencies so make sure to also have pip

sudo apt-get install git
sudo apt install python3-pip

2) Use git clone

 git clone https://github.com/ultralytics/yolov5
cd yolov5
pip install -r requirements.txt //this might take a while so sit back and relax

3) Before we can run detect.py, but first we must declare our source

vim detect.py 
parser.add_argument('--source', type=str, default='http://192.168.4.1/stream', help='file/dir/URL/glob, 0 for webcam') //edit the parse_opt funciton to include this line
python3 detect.py 

4) You have successfully implemented yolov5 to work on your own stream. Look into the github repo to change weights or train your own model

Now we are ready to combine our two python files. We still want to use opencv to open the stream and want to use yolo for obstacle detection. However, we do not want our displays to be constantly feeling up so we are only going to detect vehicles and cars (line 70 to add more classes to detect). Create one last file backupCamera.py and copy code provided. To run make sure you are in the yolov5 directory and do python3 backupCamera.py!