HackerBox 0024: Vision Quest

Vision Quest - This month, HackerBox Hackers are experimenting with Computer Vision and Servo Motion Tracking. This Instructable contains information for working with HackerBox #0024, which you can pick up here while supplies last. Also, if you would like to receive a HackerBox like this right in your mailbox each month, please subscribe at HackerBoxes.com and join the revolution!

Topics and Learning Objectives for HackerBox 0024:

• Experimenting with Computer Vision
• Setting up OpenCV (Computer Vision)
• Programming the Arduino Nano from the Arduino IDE
• Controlling Servo Motors with the Arduino Nano
• Assembling a Mechanical Pan and Tilt Assembly
• Controlling Pan and Tilt Motion with a Microcontroller
• Performing Face Tracking using OpenCV

HackerBoxes is the monthly subscription box service for DIY electronics and computer technology. We are hobbyists, makers, and experimenters. We are the dreamers of dreams. HACK THE PLANET!

• HackerBoxes #0024 Collectable Reference Card
• Three Bracket Pan and Tilt Assembly
• Two MG996R Servos with Accessories
• Two Aluminum Circular Servo Couplers
• Arduino Nano V3 - 5V, 16MHz, MicroUSB
• Digital Camera Assembly with USB Cable
• Three Lenses with Universal Clip Mount
• Medical Inspection Pen Light
• Dupont Male/Female Jumpers
• MicroUSB Cable
• Exclusive OpenCV Decal
• Exclusive Dia de Muertos Decal

Some other things that will be helpful:

• Small wooden board scrap for camera base
• Soldering iron, solder, and basic soldering tools
• Computer for running software tools

Most importantly, you will need a sense of adventure, DIY spirit, and hacker curiosity. Hardcore DIY electronics is not a trivial pursuit, and we are not watering it down for you. The goal is progress, not perfection. When you persist and enjoy the adventure, a great deal of satisfaction can be derived from learning new technology and hopefully getting some projects working. We suggest taking each step slowly, minding the details, and never hesitating to ask for help.

FREQUENTLY ASKED QUESTIONS: We like to ask all HackerBox members a really big favor. Please take a few minutes to review the FAQ on the HackerBoxes website prior to contacting support. While we obviously want to help all members as much as necessary, most of our support emails involve simple questions that are very clearly addressed in the FAQ. Thank you for understanding!

Computer vision is an interdisciplinary field that deals with how computers obtain high-level understanding from digital images or videos. From the perspective of engineering, computer vision seeks to automate tasks that the human visual system can do. As a scientific discipline, computer vision is concerned with the theory behind artificial systems that extract information from images. The image data can take many forms, such as video sequences, views from multiple cameras, or multi-dimensional data from a medical scanner. As a technological discipline, computer vision seeks to apply its theories and models for the construction of computer vision systems. Sub-domains of computer vision include scene reconstruction, event detection, video tracking, object recognition, 3D pose estimation, learning, indexing, motion estimation, and image restoration.

It is interesting to note that computer vision may be considered the inverse of computer graphics.

Processing is a flexible software sketchbook and a language for learning how to code within the context of the visual arts. Processing has promoted software literacy within the visual arts and visual literacy within technology. There are tens of thousands of students, artists, designers, researchers, and hobbyists who use Processing for learning and prototyping.

OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products. The OpenCV library has more than 2500 optimized algorithms, which include a comprehensive set of both classic and state-of-the-art computer vision and machine learning algorithms. These algorithms can be used to detect and recognize faces, identify objects, classify human actions in videos, track camera movements, track moving objects, and so forth.

Install OpenCV within Processing from the File > Examples menu by selecting “Add Examples” and then under the Libraries tab installing both the Video and the OpenCV libraries. Open the LiveCamTest example for basic face tracking. Check out some other OpenCV for Processing examples here.

More Resources:

Getting Started with Computer Vision is a book project providing an easy entry point for creative experimentation with computer vision. It introduces the code and concepts necessary to build computer vision projects.

Programming Computer Vision with Python is an O'Reilly book on PCV, an open source Python module for computer vision.

Learning OpenCV

Computer Vision: Algorithms and Applications

Mastering OpenCV

Stanford Course CS231n Convolutional Neural Networks for Visual Recognition (16 Videos)

Chris Urmson TED Talk How a driverless car sees the road

We can use any common microntroller platform for controlling servos in our pan and tilt camera mount. The Arduino Nano is a surface-mount, breadboard-friendly, miniaturized Arduino board with integrated USB. It is amazingly full-featured and easy to hack.

Features:

• Microcontroller: Atmel ATmega328P
• Voltage: 5V
• Digital I/O Pins: 14 (6 PWM)
• Analog Input Pins: 8
• DC Current per I/O Pin: 40 mA
• Flash Memory: 32 KB (2KB for bootloader)
• SRAM: 2 KB
• EEPROM: 1 KB
• Clock Speed: 16 MHz
• Dimensions: 17mm x 43mm

This particular variant of the Arduino Nano is the black Robotdyn design. The interface is by an on-board MicroUSB port that is compatible with the same MicroUSB cables used with many mobile phones and tablets.

Arduino Nanos feature a built-in USB/Serial bridge chip. On this particular variant, the bridge chip is the CH340G. Note that there are various other types of USB/Serial bridge chips used on the various types of Arduino boards. These chips allow you computer's USB port to communicate with the serial interface on the Arduino's processor chip.

A computer's operating system requires a Device Driver to communicate with the USB/Serial chip. The driver allows the IDE to communicate with the Arduino board. The specific device driver that is needed depends upon both the OS version and also the type of USB/Serial chip. For the CH340 USB/Serial chips, there are drivers available for many operating systems (UNIX, Mac OS X, or Windows). The maker of the CH340 supplies those drivers here.

When you first plug the Arduino Nano into a USB port of your computer, the green power light should come on and shortly after the blue LED should start to blink slowly. This happens because the Nano is pre-loaded with the BLINK program, which is running on the brand new Arduino Nano.

If you do not yet have the Arduino IDE installed, you can download it from Arduino.cc

If you would like additional introductory information for working in the Arduino ecosystem, we suggest checking out the instructions for the HackerBoxes Starter Workshop.

Plug the Nano into the MicroUSB cable and the other end of the cable into a USB port on the computer, launch the Arduino IDE software, select the appropriate USB port in the IDE under tools>port (likely a name with "wchusb" in it). Also select "Arduino Nano" in the IDE under tools>board.

Finally, load up a piece of example code:

File->Examples->Basics->Blink

This is actually the code that was preloaded onto the Nano and should be running right now to slowly blink the blue LED. Accordingly, if we load this example code, nothing will change. Instead, let's modify the code a little bit.

Looking closely, you can see that the program turns the LED on, waits 1000 milliseconds (one second), turns the LED off, waits another second, and then does it all again - forever.

Modify the code by changing both of the "delay(1000)" statements to "delay(100)". This modification will cause the LED to blink ten times faster, right?

Let's load the modified code into the Nano by clicking the UPLOAD button (the arrow icon) just above your modified code. Watch below the code for the status info: "compiling" and then "uploading". Eventually, the IDE should indicate "Uploading Complete" and your LED should be blinking faster.

If so, congratulations! You have just hacked your first piece of embedded code.

Once your fast-blink version is loaded and running, why not see if you can you change the code again to cause the LED to blink fast twice and then wait a couple of seconds before repeating? Give it a try! How about some other patterns? Once you succeed at visualizing a desired outcome, coding it, and observing it to work as planned, you have taken an enormous step toward becoming a competent hardware hacker.

Servo motors are generally controlled by a series of repeating electrical pulses where the width of the pulses indicates the position of the servo. The pulse width modulated (PWM) control signal is often generated by a common microcontroller such as an Arduino.

Small hobby servos, such as the MG996R, are connected through a standard three-wire connection: two wires for a DC power supply and one wire for carrying the control pulses. MG996R servos have an operating voltage rage of 4.8-7.2 VDC.

1. Pull both MG996R servos from their bags and set aside the included accessories for now.
2. Attach an aluminum, circular servo coupler to each servo. Note that the couplers come in separate bags from the servos. The coupler is a very tight fit. Start by pressing the coupler onto the end of the servo output and then thread a screw into the center hole. Tighten the thread to draw the coupler onto the servo output.
3. Note that there are three brackets for the pan-tilt assembly - two box-brackets and one U-bracket.
4. Mount one of the box-brackets onto the aluminum circle for one of the servos. We will call this servo the pan servo. Orient the box-bracket with its center wall against the aluminum circle such that the other two walls of the box bracket face away from the pan servo. Use the center holes on the middle wall of the box bracket. This arrangement should allow the pan servo to spin the attached box-bracket around once it is actuated.
5. Position the other servo (tilt servo) into the box-bracket that is attached to the aluminum circle of the pan servo. Use at least two nuts and bolts to affix the tilt servo - one on each side.
6. Holding the U-bracket, insert the brass "bearing" from the inside of the U through one of the large pivot mounting holes.
7. Place the U-bracket with the bearing onto the tilt servo that is inside the box bracket such that the other large pivot mounting hole (the one without the bearing) aligns with the aluminum circle on the tilt servo.
8. Use screws to affix the U-bracket onto the aluminum circle on one side of the U-bracket.
9. On the other side of the U-bracket, tighten a single screw through the bearing and into the small hole in the box-bracket within. This should allow the U-bracket to rotate around the box-bracket later when the tilt servo is actuated.

The remaining box-bracket can be screwed down to a small wooden board scrap to serve as a camera base as shown in the image. Finally, the pan servo is mounted within that remaining box-bracket using at least two nuts and bolts to affix the servo to the bracket - one on each side.

To wire the servos according to the schematic, it is quickest to just cut the original female connectors from the servos and then use some female DuPont jumper ends to get the signal and ground lines attached to the Nano pins.

The Nano does not have enough current on the 5V supply to power the servos from USB, so an additional supply is recommended. This can be anything in the 4.8-7.2 Volt range. For example, four AA batteries (in series) will work nicely. A bench supply or wall-wart is also a good choice.

The simple example Arduino code attached here as PanTiltTest.ino can be used to test control of the two servos from the serial monitor on the Arduino IDE. Set the monitor baud rate to match the 9600bps set in the example code. Entering angle values between 0 and 180 degrees will position the servos accordingly.

Finally, the USB Camera Module (or other sensor) can be mounted to the U-Bracket of the Pan-Tilt Assembly for use in tracking applications.

A machine vision face-tracking system can be implemented by combining subsystems as shown in the block diagram. The SerialServoControl sketch for Arduino can be found in the following Sparkfun tutorial along with a related demonstration using OpenCV, Processing, an Arduino, a USB Camera, and a Pan/Tilt Assembly to track a human face. The demo uses two servos to reposition the camera in order to keep the face centered in the video frame even as the user moves about the room. For example code in C#, check out the GitHub repository for the CamBot video.

If you have enjoyed this Instrucable and would like to have a box of electronics and computer tech projects like this delivered right to your mailbox each month, please join us by SUBSCRIBING HERE.

Reach out and share your success in the comments below or on the HackerBoxes Facebook Page. Certainly let us know if you have any questions or need some help with anything. Thank you for being part of HackerBoxes. Please keep your suggestions and feedback coming. HackerBoxes are YOUR boxes. Let's make something great!