Audiovisual Speaker

Project Summary

Collaborators: Nikhil Madhu, Grant Young, Ian Driscoll

What - For our project, the system we designed was an audiovisual speaker. This speaker has two components: A keyboard, and an LED display. The keyboard acts as the user input and consists of 8 buttons. When a user presses any of these buttons, they play a range of different notes through a miniature speaker that resides within the keyboard enclosure. The keyboard can play a full octave of notes, from C4 to C5. This keyboard is connected to an external LED display stand, and when the user presses the button and plays a noise, the LED’s light up in response. The color of the light emitted from the LED is dependent upon the button that is pressed, so different notes will correlate to different lights. Additionally, the duration of the note being played is dependent on how long the user presses the button, and by extension the duration for which the LED’s display light is dependent on the length of the note being played.   

How - This system was made possible through the use of Arduino and coding. Each of the buttons in the system are arcade buttons with metal endpoints which were soldered with the metal ends of a wire. These wires are then connected to the mini-speaker and then to the Arduino. The Arduino is a piece of hardware that allows us to connect the keyboard to the computer. That way, we can upload code via the Arduino to program the mini-speaker to play notes when a button is pressed. The buttons and the speaker create a circuit and when the button is pressed, the circuit is “closed” and electricity runs through it. The code we wrote allows us to check if the circuit has electricity running through it, and if it does, programs the speaker to play a note. Since each button forms its own individual circuit, the code just checks which button has electricity running through its circuit while all other circuits are “open” (there is no electricity running through it) in order to single out that button and assign a specific note to play from it when it is pressed. That is why pressing one button may play a C note while another button may play a B. A similar process is used to turn on the LED’s. A second wire is soldered to the same buttons and connects to the LED’s to form a second set of circuits and when the code checks for current running through each button's circuit, it programs the LED’s to display a specific RGB color value depending on which button is pressed.

Why - The purpose of our project was to create a unique sensory experience for users by combining both visual and auditory stimulus into one mechanism. Humans have a unique ability to associate their sensory experiences with other experiences i.e. the kind of smell we receive from foods can impact how it tastes. In a similar way, we hoped to create an association between the music that users would hear and the visual display that it would create. Synesthesia is a unique condition in which some people are able to see colors when hearing different kinds of music. What if this could be implemented into a physical mechanism? When pressing the buttons on the audiovisual speaker’s keyboards, the resulting LED display would cause users to associate that note with a certain color. As such, through pattern recognition, they would be able to associate a certain color pattern with a certain arrangement of notes, giving entire songs or recognizable musical schemes a specific color sequence. We hoped that this project would give users insight into the ways in which our sense operate and interact with the real world.

• 8 Arcade buttons
• LED strips
• protoboards
• mini speaker
• 3D printed keyboard box
• 3D printed LED strip housing
• Arduino

a digital schematic of the circuitry in this mechanism

The keyboard acts as the user input and the interactive mechanism while the LED display acts as the output that displays light in response to user interaction.

These are the buttons used in the keyboard. The metal section at the bottom is where the wires were soldered

The keyboard enclosure has 8 equally spaced circular sockets to comfortable place the buttons in a position where they won't come loose. There is a little opening at the bottom of the enclosure so that wires can pass through and connect to the LED display

To test the functionality of our code and circuitry, we first created a rough prototype with a plastic breadboard and a buzzer. This prototype helped us confirm that our methods were effective and we could move on to creating the final version

To create the keyboard, we soldered wires to the buttons by connecting the ends of the wires to the metal sections of the buttons via liquid metal.

Due to loose wiring and unstable soldering, certain buttons did not work properly at certain angles. This was an issue that would need to be resolved by the final product

We created a cone-like structure within which the LEDs would wrap around the inner walls, which would allow the lights to be concentrated and shine through the opening in the middle and make it more noticeable. Additionally, we created a stand to hold the cone so that the light could be shined in any direction that the user wishes

link for the code that we used in our project:

https://gist.github.com/Grantwilliam1/d41e7ffccb279062d842ae3198faf511

A demonstration of how the final product works