[Wearable Mouse] Bluetooth-based Wearable Mouse Controller for Windows 10 and Linux

I made a Bluetooth-based mouse controller that can be used to control the mouse pointer and perform PC-mouse related operations on the fly, without touching any surfaces. The electronic circuitry, which is embedded on a glove, can be used to track hand gestures through an Accelerometer and that can be translated into the movement of the mouse pointer. This device is also interfaced with a button that replicates the left-button click. The device can be attached serially to the PC (via USB) or wirelessly through a Bluetooth connection. Bluetooth provides a robust and universal wireless communication between the host device and this wearable mouse. As Bluetooth is widely available and comes embedded with almost all personal laptops, the use-case of such a wearable device is broad. Using Raspberry Pi, which is a commonly used development platform for various projects, the interfacing of different sensors, and the development of such a device is easy and scalable. The glove can be replaced with any other wearable to make its application more wide.

As a precaution against the COVID-19, it is advisable to avoid touching surfaces that can be shared among different people, and a touch-screen laptop or a mouse can be among those common surfaces. Using such a wearable device helps in maintaining hygiene and keep the commonly used surfaces sanitized :)

• Raspberry Pi 3 Model B V1.2
• SparkFun Triple Axis Accelerometer Breakout - MMA8452Q
• Male to Female Jumper Wire
• A glove
• Duct Tape
• Scissors
• Micro-USB cable
• HDMI Cable (for debugging through Raspberry Pi)

I used a MMA8542Q Triple-axis accelerometer from Sparkfun which uses the I2C communication protocol to talk to the Raspberry Pi GPIO pins and send the axes data. This sensor provides various modes of operation with the configurable data rate, sleep modes, acceleration range, filter mode, etc. I found the code from Pibits to be very helpful in my initial configuration of the sensor and testing it out with my hand gestures. It is better to first place the sensor on a flat surface and make deterministic tilts while observing the raw sensor values. This is particularly useful in understanding how this sensor reacts with various hand gestures and how we can set up thresholds for our application. Once the accelerometer is succesfully interfaced, you can see the raw axes data coming on the Pi's terminal screen.

In this wearable device, I interfaced a button that can work as a left-mouse button so that I can click on icons on the screen. The 2 ends of the button are then connected to 2 GPIO pins of the Pi. One of the pins outputs a logical high and the other pin reads that value. When the button is pressed, the circuit becomes closed and the input pin is able to read a logic-high value, which is then processed by the script I wrote to emulate left-mouse clicking. Because of the lack of soldering iron, I used duct tape to connect the jumpers with the button.

I used the Pyautogui Python library to control the mouse pointer. The reason to use this library was that it works on both Linux as well as the Windows platform. In order to control the mouse pointer on my Raspberry Pi, first I connected my Pi to a display. Then, I used the following APIs provided by the library to control my mouse pointer:

1. pyautogui.move(0, 200, 2)   # moves mouse down 200 pixels over 2 seconds

2. pyautogui.click()  # click the mouse

In order to filter out error data coming from Accelerometer, I used averaging and other filtering methods that can be easily understood through the attached code. The API pyautogui.move(0,y) was used in a way such that the mouse pointer can either go up-down or left-right at a time. This is because the accelerometer reports axes in X, Y, and Z direction, but the API takes only 2 arguments, X and Y axes. Hence, this approach was much suitable for my accelerometer and to map the gestures on the screen.

This part is an advanced application wherein any laptop having Bluetooth capabilities can communicate with Raspberry Pi in a server-client communication model and transmit mouse coordinates data wirelessly. In order to set up a Windows 10 64-bit laptop to allow Bluetooth Communication, we need to follow the steps below:

Windows 10:

1. Create an incoming Bluetooth COM port.
2. Pair the Pi's Bluetooth with the laptop's Bluetooth by making Pi discoverable.
3. Install Python on Windows.
4. Install pip on Windows. Pip is used to install libraries on a Linux or Windows machine.
5. Install pyautogui on Windows using: pip install pyautogui
6. Once pyautogui is installed on the device, install Pybluez on Windows using the following command on the Windows terminal using: pip install PyBluez-win10. PyBluez enable Bluetooth communication on both Windows and Linux PCs.
7. In order to develop an application on a Windows 10 laptop, we need to install Microsoft Visual Studio (15-20 GB of space required) and its build tools. Therefore, along with PyBluez, we need to follow the below instructions,

   1. Download and run "Visual Studio Installer": https://www.visualstudio.com/pl/thank-you-downloa...

   2. Install "Visual Studio Build Tools 2017", check "Visual C++ build tools" and "Universal Windows Platform build tools"

   3. git clone https://github.com/pybluez/pyblue

   4. cd pybluez

   5. python setup.py install

8. If the above instructions are correctly followed, running Python on windows terminal, and importing pyautogui and Bluetooth module should work without errors, as per the image above.

9. In the pybluez library installed on the Windows machine, navigate to: pybluez-master\examples\simple\rfcomm-server.py and execute using python rfcomm-server.py. If the terminal goes into a waiting state without errors, go to the below section for setting up Bluetooth on Pi. If there are errors in installing pybluez, refer to GitHub Issues for debugging.

Raspbian on Raspberry Pi:

1. Install PyBluez on Pi
2. Run the server example on Windows. Then, on Pi, navigate to pybluez-master\examples\simple\rfcomm-client.py and execute. If the two devices have started communicating, Bluetooth is now setup on both the devices. To understand more on how socket communication works with Python, refer to this link from MIT.

There will be some additional data parsing required to send axes data from Pi to PC, as the data is sent in bytes. Refer to the attached code for more info on the client and server data communication.

Once the accelerometer is well interfaced, the skeleton system looks something the first image on this step.

As the surface of the glove is not flat, I used a dummy credit card that comes to my mailbox every now and then. As per the second image on this step, I attached the dummy credit card on the top surface of my glove with duct tape. Over the card, I attached my accelerometer. This setup was robust enough to keep my accelerometer stable and able to track my gestures accurately.