SensEar: Alerting the Deaf With Feeling!

It is often said that when a person loses one sense, another one is enhanced. This project aims to support that compensation through the use of haptic technology. Around 466 million people in the world suffer from disabling hearing loss, representing around 6.1% of the world population [1]. Despite these significant numbers, society is predominantly designed for those with full hearing ability. This leaves the deaf and hard-of-hearing people at a disadvantage in terms of safety and situational awareness, especially in traffic or emergency situations.

Alerting deaf people can be done in different ways, for example by other people, visual cues or alerts on their phone [2]. For example a doorbell could work by connecting it to the lights [3]. Another option involves the use of hearing dogs [4]. However, these solutions have limitations. In situations like an ambulance approaching from behind, visual cues may be insufficient. While modern technology offers advanced options like cochlear implants, these also have some disadvantages [5]. It is costly, needs surgery and might not be practical for everyone. An easier and cheaper product is needed to help the deaf and hard-of-hearing. Studies have shown that haptic feedback is a good way to enhance the auditory perception for these people [6]. On top of that, haptic feedback has the advantage to visual feedback that it is relatively discrete because it leaves the field of vision unobstructed [7].

A haptic device is designed to help deaf people perceive crucial environmental sounds through vibrations. The person will wear two strategically placed microphones and haptic motors on their body. Studies show that the upper body, like the ears and the arms, work better for placing the haptic motors than positions on the lower body [5]. The lower arm is the most used position [7]. Our device allows users to choose exactly where on their arm they want to wear it. Only using two motors, for example on their left and right, has some disadvantages, like not being able to differentiate all directions [6]. Some studies shown though that using four motors do not improve the system significantly [5]. Because of that and to limit the complexity, only two microphones and haptic motors are used.

Two microphones detect loud noises in the environment. When such sounds are identified, haptic motors activate and vibrate to alert the deaf user. With this device, a deaf or hard-of-hearing person can be alarmed just like hearing people are when loud noises are made. Since there is always a lot background noise, the device will only respond when a certain threshold is reached. With this filtering, unnecessary vibrations are prevented, to improve user comfort.

More than a quarter of research into haptic devices is done to alert users [7]. Safety of deaf or hard-of-hearing people is something that is considered very important and therefore research into it occurs more often. However, haptic feedback also comes with some limitations. Users may need some time to learn how to interpret the signals effectively [8]. Moreover, long-term use of such a vibrating device may lead to discomfort.

To create this device, the following components are needed:

1. 1x Breadboard:
2. Function: to make a circuit without soldering.
3. Type: BusBoard BB830
4. Jumper Wires:
5. Function: to make connections between different components.
6. Type: Standard 7”
7. 1x Microcontroller:
8. Function: to process sound input and control the output.
9. Type: Arduino Micro
10. 1x USB Cable:
11. Function: to connect the Arduino Micro to a laptop for power and programming.
12. Type: USB A/Male to Micro USB/Male
13. 1x I2C-Multiplexer:
14. Function: to use multiple actuators.
15. Type: TCA9548A
16. 2x Haptic Motors:
17. Function: to provide vibrational feedback.
18. Type: Drake Haptic Actuators: Different models possible
19. 2x Haptic Motor Controller:
20. Function: to activate and control the motors.
21. Type: DRV2605L
22. 2x Microphones:
23. Function: to detect sounds.
24. Type: Electret Microphone Amplifier: MAX9814
25. 1x Battery:
26. Function: to power the system without connection to the laptop.
27. Type: 9V
28. 1x Battery Clip:
29. Function: to connect the battery to the system.
30. Type: 9V battery clip
31. 1x Prototyping Board:
32. Function: to make a circuit with soldering.
33. Type: D4MG-PCB-A

Reasoning behind choice of components:

The choice was made to use two microphones of the type MAX9814 [9]. The microphones in this circuit function as follows: you connect them to power and ground, and link the output to an analog pin on the Arduino. The microphone sends the detected sound to the Arduino as a value between 0 and 1023, where 0 represents no sound and 1023 represents the maximum sound level. There is also an option to adjust the gain to amplify the signal, although this feature is not used in this setup. The microphone operates within a frequency range of 20 Hz to 20 kHz, which is more than sufficient for this application.

The other components in the circuit were chosen primarily because they were already available, making them convenient and cost-effective for this project.

The initial prototype is constructed on a solderless breadboard to allow for flexible testing and easy correction of wiring errors. This approach is especially useful in the early development phase, where changes to wiring or component layout are often required. The schematic for the breadboard setup is shown. When testing on a laptop via USB, the pin on the Arduino Micro does not need to be connected, as the board receives power through the USB port. This changes once the device becomes autonomous using a battery.

To verify the basic functionality, the system is first tested with one microphone and one haptic motor. A simple test program is uploaded to the Arduino Micro that translatsd audio input into vibrational feedback [10]. This allows debugging of the signal flow and ensures that the motor drivers and microphones are working correctly.

After successful testing with a single channel, the system is expanded to include both microphones and both haptic motors. This could allow the prototype to detect sound from different directions. If a loud sound is detected in either direction, the motors are activated. The updated Arduino code is included in the attachment.

Sound is sampled in 50 millisecond intervals, during which the peak-to-peak amplitude is calculated. For easier processing, raw analog values (ranging from 0 to 1023) are scaled down to a 0 to 50 range. When the amplitude from either microphone exceeds a threshold of 15, both haptic motors are activated, providing immediate tactile feedback to the user. This threshold can be modified as needed to adapt the system’s sensitivity to different environments or user preferences.

Once the breadboard prototype is tested and functional, the design can be transferred to a permanent circuit. A Printed Circuit Board (PCB) is advised, but we did not have the time and knowledge to make this, so a permanent board was used for a more compact and reliable version. A scheme of this soldered circuit is shown in the schematic. If one is unfamiliar with soldering, it is recommended to seek assistance. The soldering instructions are the following:

1. Solder the small components first. Start with the Arduino Micro, multiplexer and motor drivers. This makes soldering easier and prevents access issues.
2. Create power and ground rails. Use stripped solid-core wires to make horizontal and vertical connections for Vcc and GND. Verify continuity with a multimeter.
3. Solder interconnect wires. Carefully connect all lines according to the scheme.
4. Attach the microphones and haptic motors. Ensure the cables are long enough to allow optimal placement on the user’s body.
5. Final power wiring. Connect the Vin and GND pins for external battery use. Do not connect the battery before the code has been uploaded.

Once the software was finalized and uploaded to the Arduino Micro, the permanent power solution was integrated. A battery clip was soldered to the Vin and GND pins to enable mobile operation.

In the final step, the two microphones and haptic motors are securely attached to an elastic band. This design choice allows the device to be comfortably worn around the user’s arm, which flexibility in positioning based on personal preferences. To carry the soldered circuit and power supply, a small bag can be used. This keeps the electronics safely enclosed while maintaining the mobility and wearability of the device. Our prototype is shown on the image.

Troubleshooting and Iterative Improvements

During the building process, we encountered a few challenges. One key issue involved powering the circuit correctly. When using a laptop, the Arduino should be connected to the 5V pin, whereas with a battery, the Vin pin must be used instead. This distinction was not immediately clear and led to some confusion at the start.

Another major challenge was soldering. Since none of us had soldered before, achieving strong and reliable connections was difficult. First, a poor solder joint on the power supply caused the circuit to stop working. It was difficult to trace this problem, as the Arduino code does not give an error message related to the power issue. Additionally, some soldered connections later brake off, indicating that our soldering technique needed improvement. These setback taught us the importance of being very careful during these steps.

Prototype’s Performance and Functionality

We observed that when a loud sound was detected by the microphones, both motors could be activated simultaneously. They both respond strongly to high sound levels, making it difficult to isolate input from one side. Despite this limitation, the solution still addresses the medical challenge of providing an auditory alert system for people with hearing impairments. The system reacts reliably to sound and provides clear haptic feedback through vibration motors, which can help alert users to alarms in their environment. It offers a discreet and immediate form of alerting without obstructing the user’s field of view.

Overall, the prototype shows that low-cost and accessible solutions to increase awareness of surrounding alarms and sounds are possible. This makes it possible and accessible for other people to replicate this project. It offers a device for increasing situational awareness without needing expensive components or invasive surgery. Enhancing directional detection by using more advance sensors could further improve this.

The SensEar project demonstrates the feasibility of creating a low-cost, wearable alert system that translates loud sounds into haptic feedback. This provides a form of environmental awareness for deaf and hard-of-hearing people. The prototype effectively alerts the user to loud noises through vibration, addressing a critical need in safety and inclusion. It also shows how even basic electronics can lead to impactful solutions.

Incorporating more advanced sensing and processing technologies could further enhance the functionality and user experience. When transitioning from a breadboard to a permanent circuit, it is important to approach soldering with care, as mistakes can affect the reliability of the device. Most importantly, this prototype emphasizes that inclusive technology does not always have to be complex or expensive. With some creativity and engineering, impactful devices can be developed to make the world more accessible.

[1] Golden Steps ABA. “79 Hearing Loss Statistics: How Many Deaf People In The U.S.?” Golden Steps ABA. https://www.goldenstepsaba.com/resources/hearing-loss-statistics (accessed March 27, 2025).

[2] S. Jones. “Alerting devices.” Healthy Hearing. https://www.healthyhearing.com/help/assistive-listening-devices/alerting-devices (accessed May 12, 2025).

[3] M.P. Saba, D. Filippo, F.R. Pereira, and P.L.P. de Souza, “Hey yaa: A haptic warning wearable to support deaf people communication,” Lect. Notes Comput. Sci., vol. 6969, pp. 215–223, 2011, doi: 10.1007/978-3-642-23215-8_24.

[4] Hearing Dogs. “Hearing Dogs for Deaf People.”Hearing Dogs. https://www.hearingdogs.org.uk/ (accessed May 12, 2025).

[5] M. Mirzaei, P. Kán, and H. Kaufmann, “Effects of using vibrotactile feedback on sound localization by deaf and hard-of-hearing people in virtual environments,” Electronics, vol. 10, no. 22, pp. 2794, Nov. 2021, doi: 10.3390/electronics10222794.

[6] P.K. Chelladurai, Z. Li, M. Weber, T. Oh, and R.L. Peiris, “SoundHapticVR: Head-based spatial haptic feedback for accessible sounds in virtual reality for deaf and hard of hearing users,” Proc. ACM SIGACCESS Conf. Comput. Access. (ASSETS ’24), vol. 2024, Oct. 2024, doi: 10.1145/3663548.3675639.

[7] A. Flores Ramones and M.S. del-Rio-Guerra, “Recent developments in haptic devices designed for hearing-impaired people: A literature review,” Sensors, vol. 23, no. 6, pp. 2968, Mar. 2023, doi: 10.3390/s23062968.

[8] Flatirons. “The Disadvantages of Haptic Feedback Technology.” Flatirons development. https://flatirons.com/blog/disadvantages-of-haptic-technology/ (accessed May 15, 2025).

[9] Analog Devices, “MAX9814 Microphone Amplifier with AGC and Low-Noise Microphone Bias,” Datasheet, rev. 6, pp. 1–20, Mar. 2015. [Online]. Available: https://www.analog.com/media/en/technical-documentation/data-sheets/max9814.pdf

[10] T.K. Hareendran. “Arduino & MAX9814 – Getting Started!” EDN. https://www.edn.com/arduino-max9814-getting-started/ (accessed May 8, 2025).

A short video was created to demonstrate the functionality and usefulness of our prototype, specifically how haptic feedback addresses the identified medical challenge. However, due to the file size (80 MB), it could not be uploaded directly to Instructables, which has a 25 MB limit.