Brainwaves and Eye-blinks Drive a Truck

11947 Views, 148 Favorites, 0 Comments

Brainwaves and Eye-blinks Drive a Truck

Would you like to control things with your mind? This instructable describes how to drive a remote-control truck using brainwaves. The headset senses my brainwaves and transmits them to a small computer. When I increase my attention level or blink my eyes, the computer converts the signals and passes them to the truck’s remote control, which I have connected to the computer. When I concentrate, the wheels spin. 0ne strong eye-blink does a left turn. A strong double-blink does a right turn.

The headset is a NeuroSky MindWave Mobile 2, the computer is an Arduino Uno microcontroller, and the Bluetooth module is an HC-05.

Practical Applications

Some developers have applied these ideas to help people who are paralyzed; for example, driving a wheelchair, home automation, or selecting letters and words on a screen in order to communicate on the screen or by voice. Their projects may be found on YouTube.

Main Steps in the Project

There are 5 main steps in this project:

Wire the HC-05 to the Arduino, configure it, and pair it with the MindWave.
Connect the servo to the remote control.
Wire the Arduino.
The Arduino sketch (the code).
Upload the Arduino sketch and use it.

If you’re not familiar with these devices, the following links may be useful:

Getting started with Arduino: https://www.arduino.cc/en/Guide

https://www.instructables.com/A-Beginners-Guide-to-Arduino/

Online Arduino class: https://www.instructables.com/Arduino-Class/

NeuroSky’s website: https://store.neurosky.com/pages/mindwave

http://developer.neurosky.com/docs/doku.php?id=start

Differences between the 4 models of MindWave: http://support.neurosky.com/kb/mindwave-mobile-2/whats-the-difference-between-mwm2-mwm-and-original-mwm

NeuroSky’s Arduino project (it uses a BlueSMiRF Blluetooth module instead of the HC-05 I used. The sample code (which runs on an Arduino) should work okay regardless of the type of Bluetooth module.):

http://developer.neurosky.com/docs/doku.php?id=mindwave_mobile_and_arduino

Ways of Controlling a Robot Vehicle

Before starting my project, I searched the Internet for NeuroSky projects in order to find out which brainwave signals the developers chose to use. Many projects used only the Attention and Meditation levels, and a few also used eye-blinking. For my project I decided to use the Attention level to control driving forward and stopping, and single blinks to go left or double blinks to go right.

There was one project with a very different method: When the user blinks, the software begins cycling thru forward, reverse, left, right for 2 seconds each. If the user does a double blink, the vehicle goes in whatever direction the cycle was in, at that instant. This method provides a lot of flexibility, but could be slow. Foe example, if the user turned right, they would not be able to turn left for about 6 seconds,

Methods of Connecting the NeuroSky MindWave Mobile 2 to an Arduino

Some developers connected their NeuroSky MindWave Mobile directly to an Arduino board via Bluetooth, which is what I decided to do. This method is easier than the next one (described below), but the eye-blinks take a little longer to be detected (slightly more than 1 second). This is okay for my project.

Other developers went thru a computer or smartphone for real-time pre-processing of the signals before sending data to an Arduino. This allows for more complex analysis or processing, and the eye-blinks can be detected faster (about half a second). NeuroSky has tutorials about this method for PC Windows, Apple Mac OSX, Android, and IOS at this website: http://mindwavemobileplus.neurosky.com/tutorial/ I decided to not use this method because it would involve having to learn how to program the software on the computer or smartphone.

Maximizing the Attention Level

Ways to maximize the Attention level vary for different people. Here are some suggestions:

Concentrate on an object.
Carefully read the list of ingredients on a food container.
Memorize some nutrition facts on a food container.
Count backwards from 100 by sixes or sevens.
Do arithmetic operations in your head.
Listen to a rap song and try to understand the words.
Read something in a language you’re not fluent in.

Eye-Blinking

Eye-blinks need to be strong in order to be detected. Normal eye-blinks are ignored.

The head should be held fairly still, to avoid false eye-blink detection. The method of detecting eye-blinks in this project involves checking the quality of the signal being received from the headset. A strong blink moves the skin on the forehead, which reduces the quality of the signal, and this is interpreted as a blink. If the head moves, the sensor of the headset could move slightly and thus reduce the quality of the signal, and this would be identified as a blink.

This method of detecting blinks does not use the brain’s nerve signals. My understanding is that NeuroSky’s computer/smartphone method does.

Parts for the electronics

NeuroSky MindWave Mobile 2 headset and 1 AAA battery
A computer (I used an Apple iMac)
Arduino Uno or another model and its USB cable
HC-05 Bluetooth module
Two servo motors (I used Solarbotics HXT900 Micro Servos)
Breadboard and breadboard wires
Green, yellow and red LEDs
3 x 330 ohm resistors
2 x 1K ohm resistors
2 x 2K ohm resistors
Power source, about 5 volts, such as 3 AA batteries in a holder

Parts for the servo connections to the remote control

Small plastic or cardboard box, or some other way to hold the servos in place
Glue or nuts and bolts to attach pieces of the box together
Some way to attach this to the remote control. (I used 2 screws.)
Tiny nuts and bolts to attach the servos. (I used some solid-core wire and also size 2-56 bolts, and had to slightly enlarge the holes in the servo for the bolts.)

Wire the HC-05 to the Arduino, Configure It, and Pair It

Please look at Steps 1 and 2 in one of my previous instructables for the details:

https://www.instructables.com/Brainwaves-Fly-a-Drone/

https://www.instructables.com/Brainwaves-Bend-a-Fork/

Connect the Servo to the Remote Control

There are two ways of connecting the Arduino to the truck’s remote control:

The remote control could be hacked as follows: Remove or bypass the switches that are pushed by the controls, then connect the Arduino to the controller’s circuit board and program the Arduino to apply a voltage the the proper wires. The switches in my remote are so tiny that I decided it was too tricky to try to solder new wires to the proper places on the circuit board. If I made a slight mistake, the board could be ruined.
I used servos to move the controls. Attaching the servos to the remote turned out to be an “interesting” challenge, but it all works well now.

Fortunately, there was a small plastic box of the proper size to attach the servos to the truck’s remote control, after some cutting and fitting. Nuts and bolts of size 2-56 hold the pieces of the box together. Two screws attach the box to the remote control.

The servo arm for the Left/Right control has a small notch at the end to give it some wiggle room. The centre of the servo arm is unlikely to line up exactly with the centre of the control. The screw holding the end of that arm needs to be slightly loose.

To attach the Left/Right servo I used size 2-56 nuts and bolts, and had to slightly enlarge the holes in the servo for the bolts. Solid-core wire holds the Go/Stop servo in place.

Wire the Arduino

The circuit is shown in the photo and described below:

Arduino’s +5V goes to + on the breadboard
Arduino’s GND goes to - on the breadboard
330 ohm resistor from pin 2 to the long wire of yellow LED and the other LED wire to - on the breadboard
330 ohm resistor from pin 4 to the long wire of green LED and the other LED wire to - on the breadboard
330 ohm resistor from pin 7 to the long wire of red LED and the other LED wire to - on the breadboard
Pin 9 goes to the signal wire of the Go/Stop servo. It’s the yellow one.
Pin 10 goes to the signal wire of the Left/Right servo.
Orange wires of the servos go to plus of the separate 4.5 volt power source.
Brown wires of the servos go to - on the breadboard.
The minus of the separate power source goes to - on the breadboard.
HC-05 TXD goes to Arduino’s RX (pin 0)
HC-05 + goes to + on the breadboard
HC-05 GND goes to - on the breadboard

The Arduino Sketch (The Code)

The code has the following main functions:

Read data from the MindWave headset.
Turn on the green LED when Attention is more than 70 and move the Go/Stop servo arm to push the Go/Stop trigger.
Turn off the green LED when Attention is less than 70 and move the Go/Stop servo arm back to its midpoint.
Turn on the yellow LED when a blink is detected and set a blink signal to 1.
Check for a second blink 1.2 seconds later.
If it was a single blink, move the Left/Right servo arm to “left” for 2 seconds and then turn it back, and turn off the yellow LED.
If it was a double blink, turn on the red LED, turn off the yellow LED, move the Left/Right servo arm to “right” for 2 seconds and then turn it back, and turn off the red LED.
At each main step, display data in the Serial Monitor on the computer.

The Arduino sketch (the code) may be downloaded from the .ino file located in this step, or the following code could be copied and pasted into a new Arduino sketch. (When writing these instructions, the instructables system sometimes dropped part of the #include statements located at the beginning of the code. The first one should be #include followed by a left arrow < followed by Servo.h followed by a right arrow > and the second one should be #include followed by a left arrow < followed by Mindwave.h followed by a right arrow >).

The sketch uses a special library that must be installed before running the sketch. It is available from this website: https://github.com/orgicus/Mindwave This website also contains instructions to install the library, and an example.

This project could be done without using the library, but the code would be more complicated. Code for reading the data from the headset, extracting the Attention level, and checking the quality of the signal is in the sample code included on this website:

http://developer.neurosky.com/docs/doku.php?id=mindwave_mobile_and_arduino

The sketch should compile successfully, but there will probably be a warning message shown for MindWave.cpp Library. This may be ignored.

//
// The sketch uses the brain’s Attention Level for Go/Stop and

// Eye-blinks for turns.

// Hardware: Arduino Uno, HC-05 Bluetooth, NeuroSky MindWave Mobile 2.

// High Attention = green LED on (go); Low Att = green LED off (stop).

// First blink & left turn= yellow LED; Double-blink= red LED (right).

// Right and left turns are for 2 seconds.

// Info is displayed on the Serial Monitor.

#include <Servo.h>

#include <Mindwave.h> // Mindwave library from GitHub>orgicus

Mindwave mindwave;

#define YLED 2 // define the LED pins

#define GLED 4

#define RLED 7

Servo myservoG; // Go/Stop servo

Servo myservoLR; // Left/Right servo

int gosignal = 0; // indicator for go or stop

int blinksignal = 0; // counter for signals after first blink

int blinks = 0; // number of blinks in a group

unsigned long time; // variable for elapsed time

unsigned long time0; // variable for time of first blink

unsigned long time1; // variable for time from 1st to 2nd blink

void setup() {

Serial.begin(MINDWAVE_BAUDRATE);

pinMode(13, OUTPUT); // initialize the LED pins as output

pinMode(GLED, OUTPUT);

pinMode(YLED, OUTPUT);

pinMode(RLED, OUTPUT);

myservoG.attach(9); // attach the Go/Stop servo on pin 9

myservoG.write(90); // move the servo to midpoint

myservoLR.attach(10); // attach the Left/Right servo on pin 10

myservoLR.write(90); // move the servo to midpoint

delay(15); // give the servo time to move there

}

void onMindwaveData() {

if (mindwave.quality() < 90) { // onboard LED on if poor quality

digitalWrite(13, HIGH);

} else {

digitalWrite(13, LOW);

}

// Check Attention Level

if ((mindwave.attention() > 70) && (gosignal == 0)) {

Serial.print("Go, ");

Serial.print("\tattention: ");

Serial.print(mindwave.attention());

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.println();

digitalWrite(GLED, HIGH); // green LED on

gosignal = 1; // indicates going

myservoG.write(130); // servo for robot to go forward

delay(4000); // go for 4 seconds before checking

// if attention has dropped

}

if ((mindwave.attention() < 70) && (gosignal == 1)) {

Serial.print("Stop, ");

Serial.print("\tattention: ");

Serial.print(mindwave.attention());

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.println();

digitalWrite(GLED, LOW); // green LED off

gosignal = 0; // indicates stopped going

myservoG.write(90); // servo for robot to stop

delay(15);

}

// First blink

if ((mindwave.quality() < 95) && (blinksignal == 0)) {

Serial.print("First blink ");

Serial.print("\tquality: ");

Serial.print(mindwave.quality());

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.print("\tgosignal: ");

Serial.print(gosignal);

Serial.print("\tblinksignal: ");

Serial.print(blinksignal);

Serial.println();

time0 = millis();

blinks = 1;

blinksignal = 1; // first blink

digitalWrite(YLED, HIGH); // yellow LED on

}

// Display quality right after the blink, & increase the counter

// for signals from the headset

if ((blinksignal > 0) && (blinksignal < 4)) {

Serial.print("\tquality: ");

Serial.print(mindwave.quality());

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.print("\tgosignal: ");

Serial.print(gosignal);

Serial.print("\tblinksignal: ");

Serial.print(blinksignal);

Serial.println();

if (blinksignal > 0) {

blinksignal = 1 + blinksignal; // increase counter for signals

}

time1 = millis() - time0;

}

// Double-blink?

if ((mindwave.quality()<95) && (time1 > 1200)){

Serial.print("\tquality: ");

Serial.print(mindwave.quality());

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.print("\tgosignal: ");

Serial.print(gosignal);

Serial.print("\tblinksignal: ");

Serial.print(blinksignal);

Serial.print("\ttime1: ");

Serial.print(time1);

Serial.print("\tblinks now = 2");

Serial.println();

blinks = 2;

}

// Single blink to turn left

if ((blinks == 1) && (blinksignal > 3)) {

Serial.print("Left, ");

Serial.print("\tSingle blink ");

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.print("\tgosignal: ");

Serial.print(gosignal);

Serial.print("\tblinksignal: ");

Serial.print(blinksignal);

Serial.println();

blinksignal = 0; // reset for next blink(s)

// code for robot to turn wheels left for 2 seconds

myservoLR.write(55);

delay(2000);

digitalWrite(YLED, LOW); // yellow LED off

// code for robot to turn wheels from left to straight

myservoLR.write(90);

delay(15);

}

// Double blink to turn right

if ((blinks == 2) && (blinksignal > 3)) {

Serial.print("Right, ");

Serial.print("\tDouble blink ");

time = millis();

Serial.print("\ttime: ");

Serial.print(time);

Serial.print("\tgosignal: ");

Serial.print(gosignal);

Serial.print("\tblinksignal: ");

Serial.print(blinksignal);

Serial.println();

digitalWrite(RLED, HIGH); // red LED on if right turn

digitalWrite(YLED, LOW); // yellow LED off

blinksignal = 0; // reset for next blink(s)

// code for robot to turn wheels right for 2 seconds

myservoLR.write(120);

delay(2000);

digitalWrite(RLED, LOW); // red LED off

// code for robot to turn wheels from right to straight

myservoLR.write(90);

delay(15);

}

void loop() {

mindwave.update(Serial, onMindwaveData);

}

Downloads

Truck.ino

Upload the Arduino Sketch and Use It

Start the Arduino app on the computer.
Connect the servos to the separate power source. The servo arms move a few degrees, but they will move back to the midpoint when the sketch runs.
Connect the Arduino’s USB cable to the computer. The HC-05 should make rapid blinks.
Disconnect the HC-05’s TXD wire from the Arduino’s RX (pin 0), upload the sketch, and reconnect this RX wire.
Open the Serial Monitor. Check that the baud rate is 57600.
Power-on the RC truck.
Power-on the MindWave. After a few seconds the HC-05 should make 2 quick blinks every 3-4 seconds, indicating the devices are paired.
Put the MindWave on your head and attach the ear-clip. It should now send data to the Arduino.
When finished, close the Serial Monitor.
Power-off the MindWave.
Disconnect the USB cable from the Arduino or from the computer.
Quit the Arduino app.
Disconnect the servo’s power wires from the separate power source.
Power-off the truck.

After the sketch has been uploaded to the Arduino, it could be used with separate power instead of the USB connection to the computer. The only difference would be that there is no Serial Monitor to display data. The steps would be as follows:

Connect the servos to the separate power source. The servo arms move a few degrees, but they will move back to the midpoint when the sketch runs.
Connect the Arduino to a power source.
Power-on the RC truck.
Power-on the MindWave. After a few seconds the HC-05 should make 2 quick blinks every 3-4 seconds, indicating the devices are paired.
Put the MindWave on your head and attach the ear-clip. It should now send data to the Arduino.
When finished, power-off the MindWave.
Disconnect the Arduino from its power source.
Disconnect the servo’s power wires from the separate power source.
Power-off the truck.

__________________________________________________________________

IMPORTANT NOTES

If your HC-05’s firmware is version 2 or 3, my understanding is that the command AT+INIT is needed before the AT+INQ command. My HC-05’s firmware is version 4.0-20190815, and the AT+INIT command was not recognized. To check your HC-05’s version, type this command: AT+VERSION
If you press Arduino’s Reset button, this restores the HC-05‘s settings to the factory defaults. All the pairing steps will have to be repeated.
If the HC-05’s TXD is not disconnected from the Arduino’s RX before a sketch is uploaded, an error message will appear at the bottom of the Arduino app’s window on the computer screen. The following error message is repeated 10 times: avrdude: stk500_recv(): programmer is not respondingavrdude: stk500_getsync() attempt 1 of 10: not in sync: resp=0x00
When using a servo, it needs a separate power source such as 3 1.5 volt batteries. The GND of the Arduino must be connected to the minus of the external power source. Without external power, there seems to be insufficient power from the USB source and the HC-05 loses its pairing when a servo activates. It pairs after a few seconds and then unpairs next time a servo activates.