Haptic Feedback As an Innovative Navigation System

1.Introduction

1.1 Problem statement

In the bustling chaos of traffic, vigilance is paramount. Picture this: you're on foot or cycling, your eyes glued on your phone for GPS guidance, oblivious to the world around you... But fear not! We've engineered a solution: an Arduino setup delivering route cues through haptics, keeping you connected to your environment while ensuring safety remains paramount.

The problem highlighted is the distraction caused by using smartphones for navigation while walking or cycling, leading individuals to focus more on their device screens than on their surroundings, potentially endangering themselves and others in traffic.

Stakeholders involved include pedestrians, cyclists, drivers, technology developers, emergency services, and GPS services. Pedestrians and cyclists are directly affected as they are at risk of accidents due to distracted navigation. Drivers are also impacted as distracted pedestrians and cyclists can pose hazards on the road. Technology developers and GPS services are stakeholders as they have an interest in creating and providing navigation solutions that enhance user experience and safety. Emergency services are stakeholders as they respond to accidents and incidents on the road, and reducing the occurrence of such incidents is in their interest.

1.2 Literature

1.2.1 Dangers of Mobile Phone Usage for Pedestrians and Cyclists

The rise of mobile phone usage has brought significant safety concerns for road users, particularly pedestrians and cyclists. A study published in Injury Prevention highlights that pedestrians using their phones, especially for texting while crossing streets, are at a heightened risk of accidents (gezondheid.be, 2013). Observations of 1,000 pedestrians at busy intersections in Seattle revealed that one-third used their phones for activities such as listening to music, making calls, or texting. These individuals took longer to cross streets, were less likely to use crosswalks, and paid less attention to traffic signals. The study concluded that texting pedestrians were particularly negligent of traffic lights, thereby increasing their vulnerability to accidents.

While specific statistics on distraction-related pedestrian accidents are not readily available, data from the United States provides a glimpse into the potential risks (Fischer, 2015). The percentage of pedestrians involved in traffic accidents while using their phones has risen significantly, from less than 1% in 2004 to 3.6% in 2010. This trend underscores the growing concern over mobile phone distractions among pedestrians and its impact on traffic safety.

A detailed survey on pedestrian phone usage at intersections found that approximately 11% of pedestrians used their phones while crossing (Moreau et al., 2022). The most common activities were manual interaction with the screen, such as typing or scrolling (40.3%), reading information or messages (35.1%), and talking on the phone (31.5%). Hands-free phone use was less common (11.6%). The survey also noted demographic variations: younger pedestrians (18-24 years old) were the most frequent phone users (16.8%), and phone use was higher among pedestrians walking alone (13.6%) compared to those in groups (5.0%). Additionally, phone use was slightly more common among women (11.8%) than men (10.3%).

Similarly, the distraction caused by mobile phones is a growing problem for cyclists. According to traffic regulations, all road users, including cyclists, are prohibited from using handheld electronic devices while moving (Van Cant, 2021). The European estimates suggest that distraction plays a role in 5 to 25% of traffic accidents (Baert, 2024). While the direct impact on cyclists is harder to quantify, it is clear that the use of mobile phones while cycling poses a significant risk. Cyclists distracted by their phones not only endanger themselves but also create hazards for other road users, including drivers and fellow cyclists. Surveys conducted by Fietsberaad Vlaanderen indicate that mobile phone use among cyclists is a notable source of irritation and concern among the cycling community.

The study of Moreau et al. (2022) observed that 2.9% of cyclists used their mobile phones while at intersections. The most common activities included typing or scrolling (38.0%), followed by hands-free calling (21.1%) and reading messages (18.7%). Talking on the phone or holding it while talking were less frequent. Interestingly, cyclists were three times more likely to use their phones when cycling alone compared to cycling with others (10.8% vs. 3.0%).

1.2.2 Relevance

Apps for location and route planning are essential tools in peoples' daily routines. A major player in this field is Google Maps, with over 1 billion people worldwide actively engaging with the app each month, and 41% of smartphone owners accessing it at least once weekly, highlighting its frequent usage among smartphone users (Lindner, 2023). A key aspect contributing to its widespread popularity is its turn-by-turn navigation feature, utilized regularly by 63% of all users, especially beneficial for navigating unfamiliar territories or optimizing routes efficiently. Google Maps operates in over 220 countries and territories worldwide. Its extensive coverage spans across the globe, providing users with detailed maps, real-time traffic information, and street views in over 80 countries.

This literature study highlights the importance of providing swift and precise travel guidance, especially for pedestrians or cyclists. However, it emphasizes the need for safer communication methods that enable users to maintain focus on surrounding traffic instead of constantly glancing at their phone screens for directions, a common practice in today's society. One potential solution to address this issue is through the integration of haptic feedback.

Following Xu (2023), haptics is a multidisciplinary field encompassing technologies designed to engage users' sense of touch. It involves the use of physical stimuli, such as vibrations, pressure, or temperature changes, to simulate tactile experiences. Haptic feedback, a subset of haptics, refers specifically to the tactile sensations provided to users through devices like smartphones, video game controllers, or wearables. These sensations enhance user experiences by providing tactile cues that correspond with visual or auditory stimuli, increasing immersion and accessibility. Various types of haptic feedback exist, including vibrotactile feedback, force feedback, electro tactile feedback, ultrasonic tactile feedback, and thermal feedback, each offering unique ways to simulate tactile sensations for users.

1.3 State of the art

1.3.1 Navigation systems

The research of Ertan et al. (1998) presents a wearable navigation system featuring a haptic directional display integrated into a vest. It consists of infrared transmitters and receivers for position sensing, a wearable computer for data processing and route planning, and a haptic display for delivering directional cues. The haptic display consists of a 4-by-4 array of micromotors positioned on the back of the vest. When activated, these micromotors produce vibrations that the user can feel against their back. To convey directions, the system generates specific patterns of vibrations corresponding to different movements, such as move forward, turn left, turn right and stop. During testing, users found the vibrations comfortable and easy to understand. However, a challenge arose due to the lack of feedback on when signals were correctly received, potentially causing timing errors in interpreting directions.

A similar system was designed by Yan et al. (2013). The navigation system relied on the accurate placement of 60 actuators across the user’s torso, which generated vibrations to convey directional cues. However, the motors were not automated through GPS signals. Instead, they were manually controlled by the researchers in real-time using Arduino and processing software. This manual control allowed the researchers to adjust the directional cues based on the participants' current positions and the route they needed to follow. This enabled testing the different settings of the vibration, such as the duration and intensity, and allowed the researchers to send corrective vibrations to guide the patients back on track if they deviated from the predefined route. 

Instead of using external components, the built-in sensors of Android-based smartphones, including vibration, accelerometer, compass and GPS, can be used in a haptic feedback system, as researched by Jacob et al. (2011). They control the sensors using the Android Smart Phone API, allowing for the generation of haptic feedback based on the user’s movement and orientation. The pedestrian navigation application begins with users selecting their destination from a map or a list of popular locations. The Cloudmade API then computes the shortest pedestrian route, with the route data stored in a spatial database after parsing XML. During navigation, users hold the device and scan the area, with the phone's compass determining orientation. When aligned correctly, the phone vibrates for two seconds, signaling the user to proceed. Additionally, as users approach waypoints, another distinct vibration alerts them. Then, the orientation determination starts again, guiding them along the route until they reach their destination. This system still necessitates users to hold onto their phones and move, which may not be practical for cyclists or in crowded traffic scenarios.

1.3.2 Information exchange

To automate the haptic feedback and provide accurate directions, route data must be available for the Arduino. One option is to use a GPX file, or GPS Exchange Format file. This is a text file containing geographic information like waypoints, tracks, and routes, crucial for transferring data between GPS units and computers (What Is a GPX File?, 2020). It uses XML format for organization, with tags indicating different types of data. Waypoints represent specific locations with latitude and longitude coordinates, while tracks are a chronological series of waypoints, forming a breadcrumb trail. Routes, on the other hand, are simply a series of points with the GPS device or software determining the path between them. GPX files use WGS 84 format for latitude and longitude coordinates and can include additional data like elevation, time, and descriptions.

To implement a specific route in the haptic system, the google maps route has to be converted into a GPX file. This can be done using an online maps to GPX converter.

This file can then be stored on a micro-SD card (In-Depth Tutorial to Interface Micro SD Card Module with Arduino, 2018). The microSD card module serves as the intermediary between the Arduino and the SD card, managing voltage regulation and logic level shifting to guarantee compatibility. To wire the microSD card module to the Arduino, connect the VCC pin to the 5V pin on the Arduino, the GND pin to ground, and the SPI communication pins (MISO, MOSI, SCK, CS) to their corresponding pins on the Arduino board. It's important to consult the pinout diagrams for the used Arduino board to ensure accurate connections, as the SPI communication pins may differ between different models.

Once the hardware setup is complete, the Arduino code to interact with the SD card can be written. The Arduino Integrated Development Environment (IDE) includes a useful library called SD, which simplifies tasks such as reading from and writing to SD cards (SD - Arduino Reference, n.d.). This library provides functions and tools to facilitate SD card communication, making it easier to incorporate SD card functionality into Arduino projects.

1.3.3 Location of the Arduino

The NEO-7M GPS module is a high-performance GNSS receiver supporting GPS, GLONASS, QZSS, and SBAS (Gps-module u-blox neo-7m voor arduino, n.d.). It offers high sensitivity for accurate tracking and positioning, making it suitable for various applications. The module includes a ceramic antenna, EEPROM memory for parameter storage, and versatile interfaces like SMA and TTL, allowing easy integration with microcontrollers such as Arduino. The NEO-7M works by receiving data from GNSS satellites and processing it to provide precise geographical coordinates (latitude and longitude), altitude, velocity, time, and date. It outputs this data via UART, which can be read by microcontrollers.

The accuracy of latitude and longitude coordinates depends on the number of decimal places used, with more decimals indicating greater precision (Accuracy of Decimal Places in Latitude and Longitude Degrees, n.d.). Dropping excess digits while still maintaining sufficient accuracy for most outdoor activities is common practice. However, it's essential to understand the relationship between decimal places and accuracy for effective navigation, as shown in the attached table.

2.Your Solution

This Instructable develops a haptic navigation system designed for use in traffic by pedestrians and cyclists, including those who are visually impaired, to help them safely reach their destinations. The process begins by configuring the desired route on a computer using a route planning application, such as Google Maps. This route is then converted to GPX format using an online converter and saved on a micro SD card. After simplifying the data, by alternating lines with latitude (line 0, 2, 4…) and longitude (line 1, 3, 5…) values, as you can see attached.

When the micro SD card is placed in the SD card module connected to an Arduino, the file is read to extract latitude and longitude values. These values are then compared to the current location data received from a GPS module also connected to the Arduino. As the user approaches a waypoint on the route, the system provides precise haptic feedback through motors attached to the left and right hands.

The haptic signals indicate various directions: turn left, turn right, go forward, and stop. Additionally, the Arduino continuously compares the actual location with the previous one and the next one to determine the correct direction of travel, and can indicate if the user moves in the wrong direction.

Summary of the project

1. Connect micro SD card
2. Insert route data
3. Code via example program SD library to read values ​​
4. Save value in array
5. Connect GPS module
6. With example code determine GPS current location
7. Add current location to code
8. Take current target from created array
9. Define direction vectors (vector between current and previous position and vector between current and next target )
10. Based on the angle between the vectors, the direction is determined and a specific haptic feedback will be given

Bill Of Materials ( BOM )

The following hardware and software components are needed for this application:

• 2x TITAN Haptics TacHammer Drake - LF/MF/HF (https://titanhaptics.com/shop/)
• 1x breadbord (https://www.mouser.be/ProductDetail/Mikroe/MIKROE-1097?qs=yR1Mpqbr%2FWJvGBmYu%252BI%2F7A%3D%3D)
• Arduino Micro with USB connection (https://store.arduino.cc/products/arduino-micro)
• Jumper wires (https://www.mouser.be/ProductDetail/Mikroe/MIKROE-513?qs=7CTBMF0jTsHujVymHwDfAg%3D%3D)
• Micro-SD card (https://www.mouser.be/ProductDetail/Kingston/SDCS2-64GBCP?qs=HFfMDpzxxd2vXv7JOsUwKA%3D%3D)
• Velleman WPI430 - GPS module U-BLOX NEO-7M for Arduino, high sensitivity, SMA and TTL interfaces, EEPROM, Micro-USB (https://www.velleman.eu/products/view/?id=460512)
• Velleman WPI304N - Data logging shield for Arduino®, set of 2, easy installation, compatible with MicroSD and MicroSDHC cards (https://www.velleman.eu/products/view?id=468530)
• 2x Adafruit DRV2605L Haptic Controller Breakout (https://learn.adafruit.com/adafruit-drv2605-haptic-controller-breakout/overview)
• Adafruit TCA9548A 1-to-8 I2C Multiplexer Breakout (https://learn.adafruit.com/adafruit-tca9548a-1-to-8-i2c-multiplexer-breakout/overview)
• A computer with the Arduino IDE

The extra budget for this assignment is between €50 and €60. Some components (the jumper cables, breadboard, arduino micro, multiplexer, haptic controllers, TacHammers) have already been provided by the laboratory assistant, so their original price is unknown.

Libraries such as SD (for the Micro-SD card reader), the GPS module and the Serial connection have to be included in the program. The required libraries are attached.

First of all, the SD card reader had to be connected to the arduino on the breadboard. For this, the specific terminals ( MISO, MOSI) have to be used, as shown in the attached figure.

For this connection, we first of all looked at the manual provided by the manufacturer in order to format the SD card properly. The file can be made using a Google Maps to GPX converter, like the one we used (https://mapstogpx.com/). The file then needs to be simplified so that you onlu get longitude and latitude coordinates. Afterwards, we treat the GPX file as a text file, which can be placed on the micro SD card. Afterwards, the card is inserted in the card reader. After this, the first step is to work with an example programme from the SD library (SD - CardInfo), where there is already code that can be run. This can be used to check whether the card is properly connected, properly read, properly initialised and to see what files are on the card. You can also use the example programme from the SD library (SD - ReadWrite), which shows how to use the library to read a file from a card. After all this is done, the data can be used in the programme. In this code, the data is read from the file and placed in an array.

The code for the initialisation and the setup is included in the Arduino IDE file.

Now the GPS module is added to the system. From the manufacturer's site, the library for the Arduino is downloaded. Again, an example program is available that retrieves the location and time and this is used to determine the current location in the Arduino code ( WPI430-VMA430 GPS - Show_time_location).

The code for the setup is included in the Arduino IDE file.

Lastly, the multiplexer and the 2 motor controllers are connected. The TacHammer motors can then be placed at the terminals.

The code for the setup is included in the Arduino IDE file.

The code indicates the current position obtained via the GPS. Then 2 positions have to be taken from the array: 

• the coordinates of the location you want to go to = the temporary target position
• the coordinates of the location you are coming from 

With these coordinates, 2 vectors are also drawn up: 

• the vector between the current location and the previous target = where we came from 
• the vector from the current location to the next target

The angle between these 2 vectors is calculated with the dot product and, depending on this angle, it will be indicated how you have to walk. The cross product is calculated, where the sign indicates movement in the left or the right direction.

Thresholds:

• > 10° = turn right or left
• < 10° = go straight
• > 170° = turn around

What is also included in the code is that if you are within 10 m of the target (0.0001°), this is approved and you can go to the next target. 

Each direction/signal has a different haptic feedback associated with it. 

• Turn right: Right motor vibrates at a certain frequency in a specific pattern
• Turn left: Left motor vibrates at a certain frequency in a specific pattern
• Turn around: Vibrate left and right motor alternately
• Straight ahead: Right motor vibrates continuously (5s)
• Stop: Left motor vibrates continuously (5s)

The .ino file can be opened using the Arduino IDE.

https://youtu.be/_rAqJPP5etU

Accuracy of Decimal Places in Latitude and Longitude Degrees. (n.d.). Retrieved 26 May 2024, from https://support.garmin.com/en-US/?faq=hRMBoCTy5a7HqVkxukhHd8

Baert, W. (2024, February 21). Hoe gevaarlijk of ergerlijk is gsm-gebruik door fietsers? Fietsberaad Vlaanderen. https://fietsberaad.be/nieuws/hoe-gevaarlijk-of-ergerlijk-is-gsm-gebruik-door-fietsers/

Ertan, S., Lee, C., Willets, A., Tan, H., & Pentland, A. (1998). A wearable haptic navigation guidance system. Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215), 164–165. https://doi.org/10.1109/ISWC.1998.729547

Fischer, P. (2015). Voetgangers—Hoe gevaarlijk is het gebruik van de mobiele telefoon tijdens het lopen? https://swov.nl/nl/fact/voetgangers-hoe-gevaarlijk-het-gebruik-van-de-mobiele-telefoon-tijdens-het-lopen

gezondheid.be, R. M. G. (2013, January). Mobiele telefoon ook gevaarlijk voor voetgangers | gezondheid.be. https://www.gezondheid.be/artikel/veiligheid/mobiele-telefoon-ook-gevaarlijk-voor-voetgangers-13052

Gps-module u-blox neo-7m voor arduino. (n.d.). Retrieved 23 May 2024, from https://www.velleman.eu/products/view/?id=460512&country=nl&lang=nl

In-Depth Tutorial to Interface Micro SD Card Module with Arduino. (2018, July 2). Last Minute Engineers. https://lastminuteengineers.com/arduino-micro-sd-card-module-tutorial/

Jacob, R., Mooney, P., Corcoran, P., & Winstanley, A. C. (2011). Integrating Haptic Feedback to Pedestrian Navigation Applications.

Lindner, J. (2023). Must-Know Google Maps Usage Statistics [Current Data] • Gitnux. https://gitnux.org/google-maps-usage-statistics/

Moreau, N., Boets, S., Wardenier, N., & Silverans, P. (2022). Meting van afleiding bij voetgangers en fietsers.

SD - Arduino Reference. (n.d.). Retrieved 23 May 2024, from https://www.arduino.cc/reference/en/libraries/sd/

Van Cant, W. (2021, February 26). Afleiding door de smartphone, moordend in het verkeer! Ethias. https://www.ethias.be/pro/nl/blog/afleiding-door-smartphone-in-het-verkeer.html

What is a GPX File? (2020, May 21). HikingGuy.Com. https://hikingguy.com/how-to-hike/what-is-a-gpx-file/

Xu, T. (2023, June 26). What Is Haptic Feedback? | Built In. https://builtin.com/hardware/haptic-technology

Yan, L., Yuki, O., Miyuki, K., Marina, I., Moeki, O., Natsuki, F., & Kiyoshi, T. (2013). A Design Study for the Haptic Vest as a Navigation System.