Assistive Tongue Operated Mouse (ATOM)

This project initially started as a class assignment for an introductory course that I took as a first year engineering student. After completing the course, I gathered a small team comprised of myself and two art/design students and we continued development of the device to enter it in the Student Design Competition held annually by the Rehabilitation Engineering Society of North America.

Several months later we were notified that we were finalists, and flew to Toronto to present to the judges at RehabWeek 2019. We were fortunate enough to win first place in the international student design competition, along with a people's choice award in the professional developers' showcase, and a research award from the Rehabilitation Engineering Research Center at Penn State.

*******************************************************************************

This Instructable was made as an entry for the Assistive Tech Contest! If you found it useful, please vote!

*******************************************************************************

Project Background:

In the modern age, being able to interface efficiently with a computer is necessary in order to complete many everyday tasks. For individuals lacking manual dexterity, the task of using standard computer peripherals can prove to be nearly impossible. As technology becomes more integrated into different aspects of our society, being able to effectively use the internet is no longer just a matter of luxury or convenience but is instead closer to a basic right as a human being. Developing technology that can enable individuals to access the internet not only makes them more independent but can also aid in the advancement of our society.

With a vast majority of modern jobs requiring workers to possess basic computer skills, there exists a market to produce assistive technology for individuals who are unable to effectively utilize conventional peripheral devices. Several current products fulfill this need, yet are often costly, restrictive, and undoubtedly have room for improvement. One of the oldest input mechanisms developed for individuals with quadriplegia, or ALS, is the “Sip-and-puff” device which was originally used to control a typewriter by allowing the user to sip or blow into a small tube.[1] First prototyped by Reg Maling in 1960, it was the beginning of a long line of similar devices, some of which are still in use today. Other, simpler solutions involve passive pointing devices that are held within the mouth or mounted to the head that can be used to type characters on a keyboard, commonly referred to as a “mouth stick” or a “head pointer”. Although these tools are rudimentary, they remain to be some of the most popular assistive devices due to their simplicity and low price. However, even with practice these tools cannot offer the high levels of productivity that is required in many jobs. [2] Modern technologies such as gaze interaction and eye-tracking software can provide greater efficiency when it comes to accomplishing tasks on a PC, but as a consequence the price is dramatically increased. This presents an even greater issue when considering the lifetime costs of living with such a condition can easily exceed 1.35 million dollars. [3]

All of these existing tools grant their user greater capability, yet each device brings along with its various negative side effects. Examples of such effects for a toggle and sip-and-puff device might be restrictive equipment fixed around the users’ head or mouth that is required for a fixed toggle to function; the fact that the user’s posture has to conform to the device, causing fatigue in extended use; or the inability to talk while the device is operated. Alongside the costs, eye tracking equipment requires lots of calibration to be used effectively and avoiding subconscious eye movement requires constant focus from the user. This project aims to provide a cheaper solution that allows for greater comfort and can unlock a wider range of uses for the user.

In a broad sense, the ATOM is a computer mouse that could be used by someone living with a major spinal cord injury or ALS, who has little to capability of fine motor control on their extremities. The design we decided upon involves a plastic retainer that fits to the users maxillary teeth and holds several electronic components that can be interacted with using the teeth and jaw muscles. The retainer is connected via a thin tether to a base that houses the microcontroller and acts as a hands-free docking station for the retainer when not in use.

In order to control the computer mouse, the user gently lifts their tongue onto the downward-facing joystick and pushes in the direction they desire. Pushing your tongue out causes the cursor to move up vertically, and retracting your tongue back towards the throat moves the cursor down. Left and right movements control the cursor accordingly. In order to right or left click, the user can bite down lightly isolating pressure to one side of the mouth or the other to actuate the button switches. A flat cable made from 10 individual wires carries the signals from the retainer to the base, which is plugged into any computer via standard USB Type-A.

List of Materials:

• Pointing stick module: ebay ($12.99)
• Arduino Pro Micro: amazon ($20.89)
• Food safe solder: amazon ($10.99)
• Polycaprolactone beads: amazon ($11.99)
• Flat ribbon cable: amazon ($7.99)
• Tactile switches: amazon ($6.59)

Total cost: $71.44

This is a rough estimate that neglects the cost of the base. In our case it was made from recycled materials that were already on hand.

We tested several different materials to determine which would provide a light friction fit without excessive work to mold it to a patient. One of our earlier ideas was to use commercially available mouthguards intended for athletic use, though we learned that they were unable to provide a rigid enough substrate to mount electronic components to. Eventually we decided to use polycaprolactone beads that could be softened in warm water and formed into a crude, yet tight fitting retainer. The molding process involved making a “U” shaped sheet of plastic that was pressed up over the maxillary teeth, then quenched in cool water to lock in the form.

After the rough form becomes rigid, any areas with excess material are trimmed using high grit sand paper and buffed smooth. The goal is to keep the retainer rigid but thin as possible as the additional components will add bulk later on.

The main component within this device is the pointing stick module. Unfortunately the datasheet for the specific model we used in this prototype is unknown. The pinout was determined manually, using the method described in this video. The easiest way to purchase many of these devices is in laptop keyboard replacement kits, as the standalone modules can only be purchased wholesale.

Here are links to some existing datasheets:

Flexpoint SK8702

Flexpoint SK7102

Thick Film Po

CTS Series 105 & 106

Most of these operate under the PS/2 mouse protocol, which makes it easy to interface with the arduino. Adam Chapweske's article from 1999 is a great resource for understanding this protocol.

The mouse buttons are simply momentary tack switches wired in series with a 10kΩ resistor. Two of the leads from each switch are removed leaving the remaining two facing inwards on the retainer. The retainer plastic is softened near the molars and the switches are pressed into place. After all the components are secured and confirmed to be working, the entire mouthpiece is coated in a food-grade epoxy to waterproof and make sanitation easier. Care should be taken to ensure that the switches can still move and the epoxy doesn't cover the sensitive toggle on the pointing stick.

Because the pointing stick operates as a normal PS/2 mouse, emulating a USB mouse using an Arduino Pro Micro is quite easy. There are many online tutorials (and official references) that show how to use the Arduino as a HID using several common libraries. I used a variation of the code below, from this instructable.

#include "PS2Mouse.h"

PS2Mouse::PS2Mouse(int clk, int data){
  _ps2clk=clk;
  _ps2data=data;
  gohi(_ps2clk);
  gohi(_ps2data);  
}
void PS2Mouse::write(uint8_t data){
  uint8_t parity=1;

  gohi(_ps2data);
  gohi(_ps2clk);
  delayMicroseconds(300);
  golo(_ps2clk);
  delayMicroseconds(300);
  golo(_ps2data);
  delayMicroseconds(10);  
  gohi(_ps2clk);

  while(digitalRead(_ps2clk)==HIGH);

  for(int i=0; i<8; i++){
    if(data&0x01) gohi(_ps2data);
    else golo(_ps2data);
    while(digitalRead(_ps2clk)==LOW);
    while(digitalRead(_ps2clk)==HIGH);
    parity^=(data&0x01);
    data=data>>1;
  }

  if(parity) gohi(_ps2data);
  else golo(_ps2data);

  while(digitalRead(_ps2clk)==LOW);
  while(digitalRead(_ps2clk)==HIGH);

  gohi(_ps2data);
  delayMicroseconds(50);

  while(digitalRead(_ps2clk)==HIGH);
  while((digitalRead(_ps2clk)==LOW)||(digitalRead(_ps2data)==LOW));

  golo(_ps2clk);
}

uint8_t PS2Mouse::read(void){
  uint8_t data=0, bit=1;

  gohi(_ps2clk);
  gohi(_ps2data);
  delayMicroseconds(50);
  while(digitalRead(_ps2clk)==HIGH);

    delayMicroseconds(5);

  while(digitalRead(_ps2clk)==LOW);

  for(int i=0; i<8; i++){
    while(digitalRead(_ps2clk)==HIGH);
    bit=digitalRead(_ps2data);
    while(digitalRead(_ps2clk)==LOW);
    bitWrite(data,i,bit);
  }

  while(digitalRead(_ps2clk)==HIGH);
  while(digitalRead(_ps2clk)==LOW);
  while(digitalRead(_ps2clk)==HIGH);
  while(digitalRead(_ps2clk)==LOW);

  golo(_ps2clk);

  return data;
}


void PS2Mouse::begin(void){
  write(0xFF);
  for(int i=0; i<3; i++) read();
  write(0xF0);
  read();

  delayMicroseconds(100);
}


void PS2Mouse::getPosition(uint8_t &stat, int &x, int &y){
  write(0xEB);
  read();
  stat=read();
  uint8_t _x=read();
  uint8_t _y=read();
  
  bool negx=bitRead(stat,4);
  bool negy=bitRead(stat,5);
  x=twos(_x, negx);
  y=twos(_y, negy);
}


void PS2Mouse::golo(int pin){
  pinMode(pin, OUTPUT);
  digitalWrite(pin, LOW);
}


void PS2Mouse::gohi(int pin){
  pinMode(pin, INPUT);
  digitalWrite(pin, HIGH);
}


const int m=0x100;
int PS2Mouse::twos(uint8_t value, bool sign){
  int v=(int)value;
  if(sign) v|=0xFF00;
  return v;
}

The mechanical design for the retainer "stand" is fairly simple. We intended this prototype to be a tabletop version but some minor adjustments could be made to attach it to a wheelchair or some other fixed point.

The the most critical component is a small neodymium magnet which is recessed within the protruding lip such that the user can bite down on the retainer to release it from the docked position. In this way, the user can mount and dismount the device entirely without the use of hands. A 1/4" stainless steel plug is glued into the front of the retainer, which was determined through trial and error to be the perfect size to hold the device in place but be removed easily using the mouth.

A secondary feature of the stand is to guide the tether that connects the retainer to the microcontroller. In our designs, a roller or even small rectangular slot is effective enough to contain the cable when using the device.

After testing our functioning prototype, the following project goals were determined to have been met:

- Ability for fine XY control of a cursor by use of tongue

- Ability for right and left mouse clicking by use of jaw muscles

- Ability to attach and detach device without use of hands

- Capability to easily sanitize device without damaging electronics

- Low cost of production

- Incorporate reusable and reused materials

- Improve upon existing devices (e.g. cost, functionality, ease of use, etc.)

According to a study conducted in 2017 by the Capital One Financial Corp., an estimated 82% of middle-skill jobs require workers to possess digital skills. [4] This figure is increasing year to year with no end in sight. With more than 250,000 people in the United States suffering from spinal cord injuries on any given year, there exists an ever-present need for improved assistive technology that can empower individuals with motor function disabilities to compete in the modern job market. First year costs alone after injuries can easily exceed $500,000 so developing affordable assistive technology can have a significant impact on those affected, as well as their family and friends. Furthermore, individuals with this condition undergo major life changes, many of which can cause a significant impact on their social environment. Enabling those affected to efficiently and cheaply interface with computers can grant more opportunities to reconstruct and maintain social relationships to better fit their own needs. We believe that the ATOM can help individuals do precisely that by bridging a gap within the existing technologies by offering a high efficiency coupled with a low price.