AI Robotic Arm Which Plays Hand Cricket

Do you play hand cricket and have you ever wondered how cool will it be to play with a robotic arm?Well,you are in the right place.This instructable will guide you and at the end of this instructable you will be able to build your own robotic arm that plays hand cricket with you.

• 4 x micro servo motors
• Arduino Uno
• Wire insulation tubes
• Nylon threads
• CD cover(for the palm)
• mica sheet
• Open cv
• Python
• HD camera
• Fevikwik and double side sticky tape
• Some nuts and bolts

The picture gives you the overall idea.

HD camera: to get the input

Image processing:Process the image to tell the arm what number the human has played

Algorithms:To decide which number the arm has to play

Arduino:It controls the servo motors which in turn rotate the fingers.

The design of the robotic arm has a structure similar to that of a human hand with 5 fingers. Each finger is made with an elastic wire insulation tube and cut accordingly to represent the robot's arm.These tubes are used as they are cheap and reliable. Hexagonal Cuts are made on the tubes for the flexible movement .Nylon threads are used for the connection between the finger and the shaft of the motor. Nylon is used for its strength.The thread is tied to the top most part of the tube and the thread is passed inward, into the hollow tube.Then the threads are connected to the motor shaft which are fixed in a mica sheet.The fingers are sticked to a base(here a CD plate) for the palm and placed slightly above the servo motors base in the mica sheet to ease the pulling of the motors.

In order to power up the servos,voltage regulator circuit using 7805 is used.The operating voltage range of the micro servo motor is 4.8-6V and the current needed is about 500-600 milli amperes.So each servo is powered by an individual 7805 voltage regulator circuit.

The components needed for the circuit:

4 x 7805 ICs

4 x 100uF capacitors

4 x 10uF capacitors

4 x LEDs

4 x IN4007 diode

4 x 220 ohm resistors

A switch

A power socket

Male connector pins

The voltage regulator circuit using 7805 is soldered as per the block diagram and four such circuits are made.The input supply(12V battery) is given to the power socket and a switch and then the positive and ground are given to the ICs.

The servo motor has three wires.Red-5V supply,Brown-Ground and White/Yellow-signal.So the output from the 7805 IC is given to the motor and the ground wire is grounded and the wire from arduino is given as signal.The same thing is done for all the four servos.And make sure to ground the arduino ground as well.

Processing the image:

The HD camera recieves the input and a simple algorithm is used to find the number of fingers shown.The input frame is in RGB format which is Red,Green,Blue.It basically converts the input RGB video frame into YCrCbformat.Y is the luma and Cb and Cr are the blue,red chroma difference components.Each individual frame is processed and decoded.This process is repeated for the other successive frames. It is a way of encoding RGB information. The frame is now blurred by a Gaussian function.Next we find region with skin tone in YCrCb image. The method includes separating image data for each frame in the video signal into sets of data, averaging the image data in each data set to generate mean values for each color component in the data set, comparing the mean values to a stored color profile correlating to human skin tone, and identifying data sets falling within the stored color profile.

Finding the area by Contour tracing:

The contour is found after extracting the skin tone region. The boundary points of the contour is stored in the array. The contours is mostly used for object recognition. Contour tracing is one of many pre-processing techniques performed on digital images in order to extract the border of a particular image. Therefore, correct extraction of the contour will produce more accurate features which will increase the chances of correctly classifying a given object.All the contours present in the frame are stored from which the contour of the hand is extracted based on the area.The convex hull is drawn using the inbuilt function cv2. convexHull() function checks a curve for convexity defects and corrects it. Convex curves are the curves which are always bulged out, or at-least flat. And if it is bulged inside, it is called convexity defects. First we start from the leftmost point (or point with minimum x coordinate value) and we keep wrapping points in counter clockwise direction. The convex hull of the set is the smallest convex polygon that contains all the points of it.

Convexity Defects:

The red line(as in figure 3) bounding the hand is convex hull .Basically it’s a convex set.If we take any two points inside the red region and join them to form a line then the line entirely lies inside the set.ConvexityDefects is a feature that find the defects between a convex hull and a contour,those defects are useful to find features in a hand such as the number of fingers.The depth points are the space between the fingers.The defects() function returns the co-ordinates of the convex points and we draw lines joining these convex points. Two lines are drawn from the two convex points of the extreme fingers to outside of the image and there are n-1 lines joining n fingers. Hence the number of fingers shown is detected.

To differentiate showing the thumb which also indicates two convex points and indicates two, a circle is drawn with a radius in such a way that one convex point lies within the circle. Whereas, for the number two, both the convex points lie outside the circle. Hence, two is differentiated from the number six.The number shown is sent to another algorithm designed to play hand cricket.To avoid any discrepancies, the number is detected in 50 consecutive frames and the mode of these values is passed as the number to algorithm.

Algorithm:

The arm has to bat and bowl.Both then batting and bowling algorithms should generate a random
number with special cases of pattern deciphering included and must be unpredictable.So,the concept of random number with biased weights is used here.

Python of version 2.7 is used.

The number decided by the algorithms is given to the arduino.A simple code is written to drive the servos to represent the corresponding numbers.