DIY Hands-free Computer Interface for Under $200: Eyetracker+EMG+Arduino

Overall, this is a awesome, cool, and actually kind of practical project! (Tri-force of power!!!) But for real, at the end of the day, you'll learn how to do so many things and have some cutting edge tech on your desk =)

Alright! So you wanna build one of these? Well this is what you'll need:

- $99 An eyetracker: I'm using the Eyetribe, because when I was deciding which one to buy they actually listed the accuracy of the sensor. Also the included libraries and code are for this specific eyetracker. Eyetribe.com

- $49 An EMG kit: I'm using the Advancer Technologies board for muscle sensing this kit also comes with the electrodes. advancertechnologies.com

- $30 An Arduino to gather data from the EMG board: I'm using the UNO. Arduino on amazon (Out of stock at website)

- $5 Wires to connect everything

- $5 Two 9V batteries for the EMG board

- Computer that satisfies the requirements for the eyetracker

Here's what you'll need to install:

- Eyetracking software: http://dev.theeyetribe.com/general/

- Arduino software: https://www.arduino.cc/en/Main/Software

- python (the programming language that glues everything together) https://www.python.org/downloads/

- pyserial (used to communicate with the arduino with python) https://pypi.python.org/pypi/pyserial

- pyautogui (used to control the mouse with python) https://pypi.python.org/pypi/PyAutoGUI/0.9.5

Okay the eyetracking, Arduino, and python software is pretty straightforward to install. It's pretty much just like your standard double-click installation. Pyserial and pyautogui are installed a little bit differently. You'll need to open a Command-line prompt in Windows or a Terminal in Mac, and use the pip command (this command is included in the latest version of python installation 2.7 or 3.4). You'll need to type:

pip install pyserial

and then hit enter. You might need to confirm some stuff like "Do you want to install this?", but otherwise it'll display a bunch of stuff and at the end it should say something about successfully installing. Repeat the same for:

pip install pyautogui

In some cases, you might need to precede this with the sudo command. Sudo is kind of like the executive order, and tells it to do it regardless of what permissions have been setup. So: sudo pip install pyserialORpyautogui But if it successfully installs without the command, don't worry about using it.

(For pyautogui on Windows, you just need to install this guy and you're all ready to go. On Mac, you'll need to install some extra stuff to get pyautogui working. I had trouble getting it to work on Mac, but check out their website if you wanna have a crack at it yourself.)

Alright time to glue this stuff together (not literally!)

1. Here's the link to my github so that you can download everything. And a special Thank you to these individuals for bits of python code to that made setting up this project much easier:

- Edwin Dalmaijer for writing the python library to access the Eyetribe eyetracker that is bundled

- Al Sweigart for writing the python library to control the mouse commands

- Chris Liechti for writing the python library to access the serial port

2. Connect everything. So plug in the Eyetracker into the USB port, plug in the Arduino into another USB port, and according to the EMG manual connect the EMG board to the batteries/electrodes/Arduino(in my code it's connected to Digital Pin 7).

3. Start the Eyetracker UI and server, which all happens automatically when you open the Eyetracker software. Then calibrate the eyetracker to your eyes. Overall, performance is best when you're eyes are centered and close to the Eyetracker, and you're indoors. Indoors is better because bright light from one angle (the sun) makes it more difficult for the tracker to see your face/eyes.

4. Open up the Arduino file called "runArduino_EMGserver_digital.ino" and upload it to the Arduino board. Take note of the communications port in the bottom right hand corner of the Arduino window. It should say "Arduino Uno on blahblahblah." That blahblahblah part is the port the Arduino is connected to. You will edit the included python code so that it knows which port to find the Arduino.

5. In the folder 'PyTribe-master' in the runHandsfree.py file on line 13, this is where you'll edit the port. Currently it's '/COM3', but you need to change it to whatever you're using based on what was in the corner of your Arduino window as stated in Step 4 on this page. Line 12 is an example of what it'd look like if you were using a Mac.

(Optional) 6. The x_factor and y_factor variables in runHandsfree2.py is for scaling the values you get from the eyetracker. I found that for my retina macbook pro, when I was in Windows, the eyetracker software didn't return the right X,Y coordinates due to the retina scaling (thought it was 1920 by 1200 rather than 2880 by 1800), so I had to compensate. I left the code in here in case other people run into the same problem, you'll need to tweak the x_factor and y_factor to work with your particular screen. Otherwise, using the hastag # symbol, you can comment out the lines that say: X = X * x_factor and Y = Y * y_factor

7. Save those edits on the runHandsfree.py file.

8. Open up another Command-line prompt or Terminal. Type:

python

Then hit the spacebar so there's a space after the word python. Then click and drag the edited runHandsfree2.py onto the Command-line prompt or Terminal window. This should automatically write the File path so it looks like:

python directory/Path/To/File.py

9. Hit enter! And let it fire up!

One of the really great things about this kind of interface is that it's fairly accurate and it's pretty customizable. Python is really easy to learn, and you have access to all the different functions from the eyetracker to the mouse actions. For example, you could program it so that when the EMG picks up muscle activity, it unlocks the blink to mouse click command. Eyetracking programs commonly register blinks as a click, but blinking is also a natural unconsciousness action, so it's nice that you can blink without accidentally clicking on something. And that's just the tip of the iceberg. You can also play around with using different muscles and see what kind of user interface design works best.

Overall, I want this to be an open-source project, but not everything is exactly open-source. In addition, when I first conceived this idea, I did a search to see if anyone else had come up with this before and apparently Samsung has a patent on eye-tracker/eeg/emg computer interfaces although maybe they might open-source parts of it (I only found the patent when I googled something along the lines of 'mind controlled google glass' it was a very accidental discovery, and an important lesson for any would-be inventor :)

But anyways, anything that I created for this project is open-source! Use it to learn from, use it to help people, use it to do whatever you want!

Alright, well I hope you enjoyed this instructable. This is kind of a living document, so if I find people are having trouble with a particular step, I'll try to answer it and also edit the main document so that it's clearer as soon as I can. And if you like this kind of stuff check out our website at catalystframe.com, where you can check out what we've been up to and keep up-to-date with our newest concoctions!

Have fun!