Robotic Hand Control With EMG

This project shows control of robotic hand (using opensource hand inMoov) with 3 opensource uECG devices used for measuring and processing muscle activity (electromyogram, EMG). Our team has a long story with hands and their control, and this is a good step in a right direction :)

3x uECG devices
1x Arduino (I'm using Nano but most others would work)
1x nRF24 module (any generic one would do)
1x PCA9685 or similar servo driver
1x inMoov hand
5x large servos (see inMoov instructions for compatible types)
1x 5V power supply capable of 5A or more current

Control is based on EMG - electrical activity of muscles. EMG signal is obtained by three uECG devices (I know, it is supposed to be an ECG monitor, but since it is based on a generic ADC, it can measure any biosignals - including EMG). For EMG processing, uECG has a special mode in which it sends out 32-bin spectrum data, and "muscle window" average (average spectral intensity between 75 and 440 Hz). Spectrum images look like blue-green patterns that change over time. Here frequency is on a vertical axis (on each of 3 plots, low frequency at the bottom, high at the top - from 0 to 488 Hz with ~15 Hz steps), time is on a horizontal (old data on the left overall here is about 10 seconds on the screen). Intensity is encoded with color: blue - low, green - medium, yellow - high, red - even higher.

For a reliable gesture recognition, a proper PC processing of these spectral images is required. But for simple activation of robotic hand fingers, it's enough to just use averaged value on 3 channels - uECG conveniently provides it at certain packet bytes so Arduino sketch can parse it. These values look much simpler - I've attached a chart of raw values from Arduino's Serial Plotter. Red, green, blue charts are raw values from 3 uECG devices on different muscle groups when I'm squeezing thumb, ring and middle fingers correspondingly. For our eye these cases clearly are different, but we need to turn those values into "finger score" somehow so a program can output values to hand servos. The problem is, signals from muscle groups are "mixed": in 1st and 3rd case blue signal intensity is about the same - but red and green are different. In 2nd and 3rd cases green signals are the same - but blue and red are different.

In order to "unmix" these signals, I've used a relatively simple formula:

S0=V0^2 / (( V1 *a0 +b0)( V2 * c0+d0)),
where S0 - score for channel 0, V0, V1, V2 - raw values for channels 0, 1, 2, and a, b, c, d - coefficients which I adjusted manually (a and c were from 0.3 to 2.0, b and d were 15 and 20, you would need to change them to adjust for your particular sensor placement anyway). The same score was calculated for channels 1 and 2. After this, charts became almost perfectly separated. For the same gestures (this time ring finger, middle, and then thumb) signals are clear and can be easily translated into servo movements just by comparing with threshold

Schematics is quite simple, you need only nRF24 module, PCA9685 or similar I2C PWM controller, and high amp 5V power supply that would be enough to move all these servos at once (so it requires at least 5A rated power for stable operation).

List of connections:
nRF24 pin 1 (GND) - Arduino's GND
nRF24 pin 2 (Vcc) - Arduino's 3.3v
nRF24 pin 3 (Chip Enable) - Arduino's D9
nRF24 pin 4 (SPI:CS) - Arduino's D8
nRF24 pin 5 (SPI:SCK) - Arduino's D13
nRF24 pin 6 (SPI:MOSI) - Arduino's D11
nRF24 pin 7 (SPI:MISO) - Arduino's D12
PCA9685 SDA - Arduino's A4
PCA9685 SCL - Arduino's A5
PCA9685 Vcc - Arduino's 5v
PCA9685 GND - Arduino's GND
PCA9685 V+ - high amp 5V
PCA9685 GND - high amp GND
Finger servos: to PCA channels 0-4, in my notation thumb - channel 0, index finger - channel 1 etc.

In order to get reasonable readings, it's important to place uECG devices, which are recording muscle activity, in right places. While many different options are possible here, each requires different signal processing approach - so with my code it's best to use placement similar to my photos.
It may be counter-intuitive, but thumb muscle signal is better visible on the opposite side of the arm, so one of sensors is placed there, and all of them are placed close to the elbow (muscles have most of their body in that area, but you want to check where exactly yours are located - there is quite a big individual difference)

Before running the main program, you will need to find out unit IDs of your particular uECG devices (it is done by uncommenting line 101 and turning devices on one by one, you will see ID of current device among other things) and fill them into unit_ids array (line 37). Other than this, you want to play with formula coefficients (lines 129-131) and check how it looks on serial plotter before attaching it to the robotic hand.

With some experiments that took about 2 hours, I was able to get quite reliable operation (video shows a typical case). It behaves not perfectly and with this processing can only recognize open and closed fingers (and not even each of the 5, it detects only 3 muscle groups: thumb, index and middle together, ring and little fingers together). But "AI" that analyzes signal takes 3 lines of code here and uses a single value from each channel. I believe way more could be done by analyzing 32-bin spectral images on PC or smartphone. Also, this version uses only 3 uECG devices (EMG channels). With more channels it should be possible to recognize really complex patterns - but well, that's the point of the project, to provide some starting point for anyone interested :) Hand control is definitely not the only application for such system.