Control Robot Hand With Scratch and Micro:bit

I am the author of 5 books about micro:bit and Scratch. I always enjoy building projects around micro:bit + Scratch and showing them to my audiences (teachers and students) to inspire them about amazing things they can build with block code and no need to write a program in complex syntax language.

During the time I wrote the latest book about Scratch and its extensions, I found many interesting extensions, which do not exist in the original Scratch at https://scratch.mit.edu/. For example, the Codelab version of Scratch has so many extensions including AI features and hardware connectivity.

From the list of interesting extensions, I found some extensions that can detect and track face, hand, and body, also some extensions to connect Scratch to micro:bit. So I come up with an idea to control my robot hand with Scratch and micro:bit. (I am a big fan of micro:bit.)

My idea is

• With a webcam on my computer and Handpose2Scratch extension, I can write the program in Scratch to track 5 fingers position.
• Convert each finger position into servo angle and send to micro:bit via USB port.
• On micro:bit side, I will connect 5 servos to micro:bit (each servo control 1 robot finger) and write the program in Makecode, so that micro:bit can receive data from Scratch and control the servo rotation angle.

First of all, I need one micro:bit. Both micro:bit versions 1.x and 2.x can be used in this project, but I preferred V2.x.

I bought a robot hand from AliExpress (https://th.aliexpress.com/item/32807694367.html) for about $60-$70 for one robot right-hand. The size of this robot's hand is about my hand size. It uses 5 small 180-degree servos to control each finger.

In order to connect the robot hand to micro:bit, I need an expansion board. I choose an inexpensive board from Keyestudio for about $9. It has SVG pin headers that can be easily connected to servos. (with 5V supply for servo)

I simplify circuit connection using a micro:bit expansion board.

1. 5 servos (from the robot hand) will be connected o pin12-16 of micro:bit through the expansion board.
2. To flash code to micro:bit, I need one USB cable to the computer. It will also be used to connect Scratch and micro:bit during runtime.
3. Another USB connection from the computer to the expansion board is required to supply 5V for servos.

On the expansion board, the pins are divided into 2 sides with supply voltages V1 and V2. V1 and V2 can be configured to 3V or 5V by connecting the jumper to the desired voltage.

Since I plan to connect 5 servos to pin 12-16 of micro:bit, which is on the V2 side. But by default configuration, the jumper connects V2 to 3V. So I have to switch the jumper to connect V2 to 5V.

To connect the robot arm to the expansion board, first I have to connect the yellow wire to the yellow header.

1. Servo on Thumb finger -> P12
2. Servo on Index finger -> P13
3. Servo on Middle finger -> P14
4. Servo on Ring finger -> P15
5. Servo on Little finger -> P16

Then connect the USB cable to micro:bit and the computer. This is to flash code to micro:bit and transfer command from Scratch to micro:bit

The last step is to connect the USB cable from the expansion board to the computer to the 5V supply. This is a power supply of 5 servos.

To be sure that the servos on the robot hand work well, I have to test them. I wrote a simple program on Makecode (https://makecode.microbit.org) to test it.

1. At first, the program set the variable "angle" to 90 degrees.
2. Press button A on micro:bit will decrease the angle by -2
3. Press button B on micro:bit will increase the angle by 2
4. Current angle will be shown on micro:bit LED

Not just testing servo, I have to record the angle which makes the finger in position to fully stretch or grab something.

In the example code, I test the servo connected to P12. After testing one servo, change to other pins (P13-P16).

In my case, setting the servo on P12 to 130 degrees will make the finger fully stretch, and also 20 degrees will be fully bent inside. 75 degrees move the finger somewhere in between.

For other fingers setting the servo to 45 degrees will make the finger fully stretch. I have recorded 4 steps of angles: 45, 85, 120, and 160 degrees.

I use these angle values to make the robot finger move.

I will not use the original Scratch on the MIT website, Instead, I'm using the Codelab version of Scratch at https://create.codelab.club/projects/editor

The main reason is Codelab Scratch has many more extensions than the original. especially, AI features and hardware connectivity.

2 extensions that I used are

• Handpose2Scratch - with this extension, Scratch can use a webcam to detect hand and fingers position
• micro:bit radio - use this extension to connect Scratch and micro:bit via USB port.

I will not go through the setup process in this article, please follow the instruction on the Codelab website to install the Codelab adapter, run and connect it to Scratch.

How program works

1. Use Handpose2Scratch to detect the position of the tips of 5 fingers, the base of the thumb, the base of the little finger, and the wrist
2. Move sprites to those 8 positions
3. Measure the distance from the base of the thumb to the base of the little finger as a reference. This is a palm size
4. Measure the distance from each fingertip to wrist then calculate the ratio to palm size
5. Use these ratios to determine the servo angle
6. Send servo number and angle value to micro:bit

These codes just move sprites to track position.

I separate code for the thumb and little finger. For the index, middle, and ring fingers, I just write one code and clone it to 3 fingers.

In the last part of each code, I use micro:bit broadcast and wait to send the servo number and angle to micro:bit. and let micro:bit control servo.

I modified this code from Codelab example.

1. When Scratch sends data to micro:bit via USB port. micro:bit receive these data using the serial read command
2. micro:bit extract servo number and angle
3. Use the servo write command to move the servo to desired angle.

To run the program, I recommend following sequences

1. Connect everything as per step 3 and power up. micro:bit should show a down arrow.
2. Run the Codelab adapter and enable 2 micro:bit extensions.
3. On Scratch, Scroll down to the micro:bit radio section.
4. Click an orange circle to start the connection.
5. Wait for Scratch to scan for micro:bit. Click connect.
6. Once micro:bit is connected, click Go to Editor
7. Finally, click the green flag to run the program on Scratch

Now, it is ready to track your hand and the robot hand follows your fingers' movement.