Xbox Head-Tracking

Being an avid gamer myself and spending a lot of my free time racing on Circuit de la Sarthe (AKA LeMans) or roaming around in Los Santos causing mayhem, I noticed that a key aspect of PC gaming was lacking in the console world. Being able to look at the apex of a corner while driving at high speeds allow you to move your car precisely through a corner without upsetting the car's balance and therefore reducing lap times. A fixed cockpit camera hinders your ability to do this, since the camera is always pointing forward, moving with the car, as opposed to looking on the sides and into the corners, like when driving naturally. This was the main reason I set out to resolve the issue of a lack of head tracking for consoles. However, throughout the development of the project, I noticed such a device could be appreciated by an even wider group of gamers. Head tracking is a perfect solution for disabled gamers as it allows even more control over characters with the use of only one hand and head movement.

Unlike most PC head tracking systems, we will not be using IR tracking. This removes the need for external sensors, a dedicated computer for IR-tracking software and allows us to use a stock controller. In order to understand how our system will work we first need to understand how a stock controller works.

Majority of gaming controllers use two Joysticks as inputs: one to move a character or vehicle and a second to control the point of view.

These joysticks consist of two potentiometers (commonly known as volume knobs) joined perpendicular to each other and work by measuring the resistance values when you move the joystick along the X-axis and Y-axis. These types of potentiometers are known as Analog Potentiometers and require you to physically interact with them in order produce an outcome. However, if we are able to bypass this physical interaction and can some how control the potentiometer with the use of an electronic circuit then we could effectively program a desired outcome without touching the joystick. Luckily, we could produce such an effect by using a digital potentiometer! Now we can control the movement of our character or the point of view with the use of just a couple lines of code.

However, we can't stop there just yet if we want to build a head tracking system. We need to find a way to determine in which direction we are turning our heads and even the degrees of rotation. To do this we can use a gyroscopic sensor (AKA MPU) to track our head rotation and use those values to map onto our digital potentiometer to control the in game camera.

If you'd like to learn more about how to use an MPU 6050 and how they work, you can check out this video:

* These are all non-affiliated links, I only post these links as they are exactly what I used for this project. However, you may use different components.*

Essentials:

Xbox One Controller x1

Breadboard x1

Arduino Nano x1

MPU-6050 x1

Digital Potentiometer x2

Battery Charging Circuit x1

LiPo Battery x1

Mini Power Switch x1

Push Button x1

Other Materials:

Break-away male header

Hook-up Wire Spool Set

Jumper Cables

3-Pin Extension Cables

Servo Extension Cable

Tools:

Soldering Iron Kit

Wire Stripper

In order for our head tracking system to work, we must make some slight modifications to our controller to bypass the anolog potentiometers. This modification will not damage the controller. It will still be a fully functional and useable regular controller.

Begin by unscrewing and removing the casing.

Now that we have access to the circuit board we must locate the pins for the potentiometers. Be aware that we want to make the modifications to the right joystick (which controls the POV) therefore, since we are looking at the board from the back, we will be looking for the joystick connections on the left side of the circuit board. The joystick contains 6 pins which we need, and they can be found by using the pictures above. Once you have located these pins, we will need to solder a wire to each of them ensuring that we group them by 3. This will prevent us from confusing wires that do not belong together. (Use the Servo Extension Cables to do this as they are already in groups of 3 and solder 3 male headers at the end.)

Create small cutouts at the top of the controller so that the casing may fit back on without crushing the wires.

Put everything back together and now we have our own gaming controller with head tracking capabilities.

Now that we have a controller capable of head tracking, we must create a device that actually does the tracking and communicates with the controller.

It is best to do all wiring of the components using a breadboard to prevent soldering so many wires and to allow easier troubleshooting. Take note that the numbering of pins for the digital potentiometers begin in the upper left hand corner looking at the chip with the indentation facing upwards. You may also want to use male to female jumper wires for the Wire Out of the potentiometers.

MPU->Arduino

Vcc - 5v

GND - GND

SCL - A5

SDA - A4

XDA, XCL, ADO - Unused

INT - D2

Digital Potentiometer 1->Arduino

Pin 1(CS) - D4

Pin 2(SCK) - D13

Pin 3(SDI/SDO) - D11

Pin 4(Vss) - GND

Pin 5(P0A) - Wire Out

Pin 6(P0W) - Wire Out

Pin 7(P0B) - Wire Out

Pin 8(VDD) - 5v

Digital Potentiometer 2->Arduino
Pin 1(CS) - D5

Pin 2(SCK) - D13

Pin 3(SDI/SDO) - D11

Pin 4(Vss) - GND

Pin 5(P0A) - Wire Out

Pin 6(P0W) - Wire Out

Pin 7(P0B) - Wire Out

Pin 8(VDD) - 5v

Push Button->Arduino

Pin 1 - D12

Pin 3 - GND

Pin 2, Pin 4 - Unused

Charging Circuit->Arduino

BAT - VIN

G - GND

5V - 5V

In order to successfully program your Arduino, it is important the you download the MPU6050 Library. This will allow us to communicate with the sensor and read its values.

The Xbox_HeadTracker_V1 program consists mainly of the stock MPU Library example. I have removed unnecessary lines of code and made modifications to the code in order to map the MPU values to the Digital Potentiometers. You can upload this exact program without needing any more modifications.

Be aware, that the MPU might require a couple seconds of being laid flat in the orientation of use in order to calibrate itself.

The pushbutton can also be used as an instant (mid-game) recalibration, in case the drifting of the MPU's values are out of alignment with your head's orientation.

Use double sided tape to stick the breadboard, battery, and wires to a flat-brimmed hat. Then use the 3-Pin Extension cables to extend the wires coming out of the digital potentiometers and into the wires coming out of your controller.

Plug the controller in your Xbox while keeping the head-tracker off. When you're ready to use the head tracking, put on the hat and keep your head in a comfortable orientation (as this will be the deadzone of rotation) and turn the head tracker on. Keep your head still for about 5 seconds for it to calibrate.

Now that everything is ready, you can finally roam around in Los Santos looking around with head tracking or improving your lap times in online races on your console. This can even work on PC games using the controller as the input.

Tip #1

Be sure to disconnect or turn off your controller before opening or soldering wires to the circuit board. This will help prevent accidental shorting of the board from touching it with the soldering iron. This also applies to the wiring of the Arduino. It is best to always keep your devices off when making physical changes to its wiring as it will save you from destroying sensitive parts.

Tip #2

Although it is unnecessary to use Servo Extension cables for the wires going from the Digital Potentiometer to the controller, it is helpful to be able to easily plug and unplug these extension cables to lengthen or shorten them without having to change the entire wire.

Tip #3

It is best when wiring the Arduino, to test each individual component separately before building the entire circuit. This will be helpful to ensure that all the components are working as you continue building the circuit.

Tip #3

Ensure that when soldering wires to the controller, they do not touch each other or that the solder does not create a bridge between the potentiometer pins. If this happens you will notice when playing that the camera keeps rotating uncontrollably.