Haptic Piano Teaching Device for Vissualy Impared People

Problem statement

When people are visually impaired, sounds become even more important. But even though music is very important for these people, it is very hard to teach a musical instrument when you have impaired vision. For this reason, a haptic device is designed to teach people to play piano and learn to play music all by themselves.

State of Art

In the past, different approaches were used to teach visually impaired people to play the piano. In many cases, it is done by a teacher who teaches the student which sound corresponds to which key. Many visually impaired people have excellent hearing and can distinguish very subtle differences between sounds. In the past, teachers used to have the students learn the sheet music in braille, study it before playing, and find a key with a specified note to start playing from there. Other parameters, like the placement of the piano and the influence of lights, were studied and showed that visually impaired students get easily distracted by the use of bright lights. Now there are more solutions to teach a visually impaired person to play the piano even without a special teacher. Technologies like a haptic baton or haptic gloves indicate how the person should play. These make use of different actuators to create muscle memory to play the piano pieces, and some systems allow passive learning. There are also systems using haptic sleeves and infrared camera goggles to make you “see” with your sleeves. These methods are quite expensive and require many different components. We are opting to make a more simplified and cheaper solution so people can build and use it by themselves. The student gets notified when their right thumb reaches the middle C of the piano. From there, they position the rest of their fingers to cover the white keys corresponding to each finger. The motor will tap their hands a certain number of times corresponding to which note of the C major scale they need to play. After memorizing this, the student can learn melodies by heart.

An Introduction to Piano Lessons for The Blind | Expert Articles. (z.d.). https://www.lessonrating.com/students/expert-articles/piano/an-introduction-to-piano-lessons-for-the-blind/

Anja. (2022, 7 oktober). Is there a special piano for visually impaired people? Klaviano Blog. https://www.klaviano.com/piano-blog/is-there-a-special-piano-for-visually-impaired-people.html

Evans, J. (2013, 25 juni). Autodesk Simulation Mechanical Modal Analysis and Gravity. Design & Motion. https://designandmotion.net/autodesk/simulation-mechanical-natural-frequency-modes-and-gravity/

Kadir Pala, F. (2019, februari). Developing a haptic glove for basic piano education. researchGate. https://www.researchgate.net/publication/331031976_Developing_a_haptic_glove_for_basic_piano_education

Learning to Play a Musical Instrument: Resources for People who are Blind or Visually Impaired. (z.d.). The American Foundation For The Blind. https://www.afb.org/aw/13/6/15804

Passive Haptic Learning: Learn to Type or Play Piano Without. (2018, 21 september). GVU Center. https://gvu.gatech.edu/research/projects/passive-haptic-learning-learn-type-or-play-piano-without-attention-using-wearables

Passive Learning via Wearable Haptic Stimuli System | Office of Technology Licensing. (z.d.). https://licensing.research.gatech.edu/technology/passive-learning-wearable-haptic-stimuli-system

Waywood, V. A. P. B. (2013, 7 mei). How Can You Play the Piano When You’re Blind? Stuart Wood’s Weblog. https://waywood.wordpress.com/2013/05/07/how-can-you-play-the-piano-when-youre-blind/

Williams, R. (2024, 1 mei). ‘Smart Gloves’ Teach Piano Playing through Touch. Scientific American. https://www.scientificamerican.com/article/smart-gloves-teach-piano-playing-through-touch/

Yahoo is part of the Yahoo family of brands. (z.d.). https://www.yahoo.com/news/haptic-baton-offers-hope-blind-135824143.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAMe9i8YEIHb-_ld-tC-KgcixtFQhA-_kAQHpCKRvuYi20xqDNOS60nAxY3f4tKW4jBFKI79k7QFa1sP7H_on5S5P0pkVjkuonp530uGRhFYNFGekh9IIuk_X9ksGen626eCHz5d82LLLMvdvfIiSPkmUhUZxSb3XGkO9Sp9rUHPV

Yirka, B. (2022, 24 januari). Haptic feedback sleeve and goggles allow blind people to “see” with their arm. Techxplore. https://techxplore.com/news/2022-01-haptic-feedback-sleeve-goggles-people.html

Arduino micro board

breadboard

USB A to USB micro cable

2 TacHamer impact motors model Drake LFi

1 arduino multiplexer

2 arduino controllers

jumper wires

1 HC-Sr04 ultrasonic distance sensor

First, build the Arduino circuit consisting of the two motors and the distance sensor.

Place the distance sensor at the end of the breadboard and make sure no wire jumpers are in front of it.

Connect the Arduino to the PC.

Place the sensor at the exact distance from the octave you want to play.

Place the motors on the hands of the person playing the piano. This can be done with some tape or by gluing them to some gloves.

Run the code.

The motor will vibrate more intensely when the hands get closer to the correct position. When the position is reached, a feedback signal is given. After reaching the correct position, the vibration motors will indicate which note to play by vibrating the number of times corresponding to the finger that has to be used. So, one vibration is for the leftmost finger of the hand, two vibrations are for the second finger, and so on up to the fifth finger of the hand.