Happy Feet: Interactive Performances!

This instructable was made as part of the CS graduate course "Tangible Interactive Computing" at the University of Maryland, College Park taught by Professor Jon Froehlich. The course focused on exploring the materiality of interactive computing and, in the words of MIT Professor Hiroshii Ishii, sought to "seamlessly couple the dual worlds of bits and atoms." Please see http://cmsc838f-s14.wikispaces.com/ for more details.

During performance, ambient sounds usually come from sources other than the performer. This project’s goal was to use synesthetic design between movement and sound to enhance walking during performance in order to create a more immersive and realistic experience. As performers walk, their legs would produce sounds that mimic their movement through the imagined environment (e.g. if the performance takes place on a hiking trail during the fall, the audience would hear the performer crunch through leaves as they walked). In improv, performers often react to audience input, so we wanted an audience member to be able to directly manipulate the performer's footsteps as a way of controlling the direction of the story.

We decided to show our interaction using muppets with the setup on the performer's arms instead of legs.

Things you will need:

1. Arduino Yun with microSD
2. Flexible Speakers (so that you can make it wearable, we wanted to wrap it around the performer's feet to make it feel like sound is coming out of the feet)
3. USB audio adapter (you will connect the speakers to this)
4. Male-to-male Stereo Audio Cable
5. Button (you can also create a fabric button but we are using a simple button
6. Jumper Wires

You will need to connect the button as shown in the figure. One end of the button is connected to Digital Pin 2. The Arduino Sketch reads the value on pin 2. If the value is high (button is pushed), only then it plays the sound clip.

Instead of using a push button you can also create a fabric button. We tried to play around with pressure sensors as well as accelerometers to detect when a step is taken versus when a person is standing. There is some literature about using accelerometers to measure steps (Libby et al.). We also found that the pressure sensors did not work well on the foot because users put pressure on their feet differently during different steps, and often changed pressure while shifting their feet. So, for this project we decided to use a simple solution - push button.

To play audio from the USB on the Yun, you need to install certain audio drivers and softwares.

Make sure your Arduino Yun and your computer are connected on the same WiFi network. Also, insert your microSD card into the board. SSH into the Yun and install the following packages:

1. opkg update
2. opkg install kmod-usb-audio
3. opkg install madplay

After installing these, you will be able to play any audio file using the command 'madplay [path of the audio clip on SD]'

Connect the USB audio adapter to the USB port on the board. Insert one end of the stereo cable to this adapter and the other end to a jumper wire.

The flexible speaker comes with an amplifier which has Vcc, Ground and Input as shown in the figure. Connect the other end of the jumper wire to the input on this board.

The speakers also need to be powered by connecting Vcc and Ground to the positive and negative ends of 3-5V batteries.

You can now add mp3 sound clips to root directory of the SD card or create a directory called Sounds and then add all these files into this folder.

We downloaded sound clips from Sound Bible.

In the arduino sketch (which we will upload in the next step), you will have to update the path of where all the sound files are.

Download the sketch from GitHub and upload it to the board. Make sure you are using Arduino IDE 1.5.4 and later.

Also, you will have to modify the sketch depending on where your sound files are.

In this Arduino Sketch, we are reading from the Console and hence to say which file you want to play you will have to open the terminal window and type: ssh root@[your Yun's IP] 'telnet localhost 6571'

You can now type the song name you want to play when you push the button.

The above figure shows the entire setup of the hardware on foot too.

So once the button is pushed, a default sound clip will play. If you type the name of another sound clip that one will play at the push of the button.

Future Work:
To further enhance the audience and performer experience, we thought of several add-ons: an android app that would let audience members control what sound they want to be played when the performers are performing on stage, we also thought of letting audience members record short sound clips themselves, and directly sending the sound to the wearable to play. Additionally, we thought about including a mike in the wearable itself, and giving performers the opportunity to record clips of themselves walking, and then playing back these clips at a later time.