Greek Sign Language Translation Glove

This project aims to build a sensors-glove system, which will translate the corresponding movement of the hand into letters of the Greek Sign Language and then will appear in the Bluetooth application of the mobile. It aims to help users of the Greek Sign Language to communicate with non-users. The focus of this work is the design and implementation of a lowcost glove using open source, hardware and software in the microcontroller, for the interpretation of basic gestures.

Components and Circuit

The way to connect all the components in our system is depicted above. Each input and output corresponds to the connection point shown , and is connected to the compatible pin on the board.

In the first stage, the precise operation of all the sensors and the correct wiring had to be checked. The first tests were done with a flex sensor, a 10k Ω resistor and an LED.

The point of these tests is to show that we do not have a technical problem with the sensors, as the LED will light up while the sensors are bend and will turn off when they come to their original position. If the above procedure works,then the flex sensors are ready to be used.

All the sensors, one by one, were placed and sewn on the glove. The wiring we followed for the flex sensors was to connect one terminal of each sensor with a 10k Ω resistor, one end of the resistor goes straight to + (5V) of the breadboard and the other to the analog input of the Arduino.

The second terminal pin of the sensor , is placed on the ground (GND) of the breadboard.

The gyroscope had to be tested also.

To be able to use the MPU6050 we had to solder it with an I2C protocol so that our cables could be connected there. Since the gyroscope uses I2C, the wiring I followed is written below :

The SDA pin of the MPU6050 is connected to the analog input A4 of the Arduino Nano, and the SCL pin to the analog input A5 respectively.

These pins are specially designed to handle I2C on the Arduino, while the Wire.h library is used. The VCC and GND pin are connected to the +3.3 V and GND of the Arduino.

To be able to program the code so it will run the test, the MPU6050.h library needs to be installed. One of the examples provided by the library , shows how to make sure the sensor is working properly.

Since we ascertain that all the components could work harmoniously without causing interference between one another, then they were sewn and glued onto the glove.

The MPU6050 sensor is sewn into the center of the glove and the breadboard is glued to the glove, at the area above the wrist.

With that being done, the system is complete and we move on the development of the necessary programming and the first tests for the translation of our movements into Greek Sign Language.

The way the algorithm works is as follows: By reading through the program the values ​​of the sensors, we set a minimum and maximum limit that the sensors can take during the formation of a letter.

In short, we use a combination of passive position measurements and estimated orientation.

Our programming begins by declaring the appropriate libraries for the operation of the entire system so that the Arduino recognizes the peripherals connected to it.

In order to record all the movements, the values ​​and variables of the MPU6050 sensors had to be correctly initialized, with the help of gyroscope and the accelerometer facilitates the accurate identification of the letter that is formed, and all five flex sensors. In these flex sensors, we declare the smallest value as 0 and the largest as 1023 and it is also indicated in the program to which analog input they are connected to.

Through the if and else if statements, preset values ​​were created on the six connected sensors, or else a database of values ​​we saved on the Arduino from previous experiments. If the indications are within these predetermined values, as a result we will get the letter we signed, which will appear on the screen of our mobile.

For example, we form the letter A and get five values ​​from the flex sensors as a result. If for the thumb finger the value we get is 85, the minimum and maximum default value declared in the code will be 82 and 87. This procedure requires the smartphone to be paired with the HC-05 (the bluetooth module).

Main conversion is changing analog x,y,z values ​​to digital values ​​and then changing from rad to degrees.

A downside to this system is that the rate of errors or deviations with this process has been calculated to be around 5%. That's because the sensors do not always give the same values ​​due to the way the glove is worn and how the movements are coded. The desired result may not be seen, as some letters of the Greek Sign Language Alphabet are similar enough in the signing maneuver to have almost the same values.

This is where the MPU6050 clears the way, the flex sensors may perceive the same range of values ​​in some letters, but the placement of the wrist on the symboling changes the result. Thus, the tilt angle is also differentiated and the final result will be what we expect.

We have reached in an era where the need for continuous development of aspects concerning the daily life of deaf people, reduces the communication gap between them and those who do not know sign language. By creating this project, a different approach was given to the way signers can communicate with non-signers.

Having reached into a very positive and desirable result, I managed to create a system, which has significantly high accuracy with the possibility of translating movements of the Greek alphabet. Bluetooth as a communication protocol improves the design part quite a bit compared to other wireless communication protocols.

However, there could be a few improvements and perhaps future extensions on this research. The response time of the movement programmed with the translation is not so short, it could become shorter in the future and with changes to basic parameters. 

A future proposal is to change the communication protocol and instead of Bluetooth, the data will be transferred via Wi-Fi or GSM to cover longer distances, simply by connecting a module to the Arduino. The Wi-Fi module should be connected to the same wireless network as the computer or mobile phone that will receive the data.

Unfortunately, with the use of a single glove, a wide range of meanings cannot be covered and the translation is limited only to basic gestures such as those of the alphabet.

Sign language is a language that uses the whole body to communicate and not just the hands but also facial expressions as well as body movements. So by programming another identical glvoe and using both at the same time, the range of words could have increased tremendously.

You can find the full code on my github link :

https://github.com/AndriannaTs/greek-sing-language-translator

Manon Kok, Jeroen D. Hol and Thomas B. Sch¨on (2017),” Using Inertial Sensors for Position and Orientation Estimation”, Foundations and Trends in Signal Processing: Vol. 11: No. 1-2, pages 1-153.

http://dx.doi.org/10.1561/2000000094

https://www.arduino.cc/reference/en/