Multitouch Music Controller
by amandaghassaei in Circuits > Arduino
119023 Views, 1054 Favorites, 0 Comments
Multitouch Music Controller
This project is an Arduino-powered infrared touchscreen / coffee-table interface that I've been using to control various music and graphics applications on my computer. This is an old project that I've recently had time to go back and document/fix up; this project is a little more special to me than most because it was my first Arduino/electronics project, and while I was doing research for it I ended up on Instructables for the very first time. I've been using this controller primarily to drive music and graphics applications running in MaxMSP. Here's a short demo:
The touchscreen uses infrared (IR) sensing to detect fingers and other objects on the screen. An IR laser at each corner shines IR light across the surface of the screen. When a finger or other object touches the surface, it causes the IR light to scatter in many directions. Some of this light is directed down into the screen, towards an array of 64 IR sensors. By scanning through the sensors, you can determine the x and y position of the touch event(s) and use this to control a variety of apps.
An array of 64 LEDs underneath the display provides visual feedback for the interface via rear projection. The LED array operates completely independently from the sensor array - this means the LEDs may be used in more ways that just displaying the current touch positions.
This interface is a controller, it does not run apps that are stored in its memory (though that is possible). Instead, it connects to a computer via USB and sends a series of touch messages to control applications running on the computer (similar to how a computer keyboard sends keystrokes to a computer). The computer processes the input data and determines the configuration of the output display, then it sends a series of LED messages back to the interface (similar to how the computer drives an LCD display). This way, the controller is very simple and does not need to handle any processing outside of the basic tasks of getting the states of its inputs (IR sensors) and setting the states of its outputs (LEDs); the computer is doing all the heavy lifting in this scenario.
As I said before, this device relies on IR sensing as input information, I found out by accident that it does some cool stuff when you put it outside while the intensity of sunlight is changing rapidly (sunrise/sunset). I had some fun trying to find interesting ways of transforming seemingly random noise from the inputs into sound/lights:
Although the project was done a while ago, I've taken it apart recently to refinish the wood and fix a few things that were bugging me. I've still written this Instructable as if I was building it from scratch, but it will be obvious at times that this is, in fact, a finished project.
Hot tip: the schematic and firmware used in this project is a modified version of the Arduinome project, which is a modified version of the Monome project. If you run into problems during any stage of this project, you might find what you are looking for in one of those forums.
Safety note: this project uses laser diodes strong enough to permanently damage your eyes (or the eyes of those around you), do not use lasers if you don't know how to handle them properly.
PARTS LIST:
Various Sources
(x4) 25 milliwatt 780nm laser diodes with 89 degree line lens Aixis AIX-780-25-8 - really important note here, even though these lasers are labelled "3.2V," they are actually 5V, this took me a really long time to figure out.
(x1) rosco black projection screen (a ~2' by 2' piece at least) Rose Brand
(x1) Arduino Uno (I used a Duemilanova, but an Uno should work too) Sparkfun DEV-11021
(x1) 20"x20" phenolic sheet ~1/16" (for installing LEDs and IR sensors) Amazon
(x1) wood 2x4 (for enclosure- I used some redwood that was laying around
(x1) 1mm aluminum sheet (for light-tight partitions underneath screens) enough to make 14 20" x 2.5" pieces - I used black anodized aluminum, but regular un-anodized aluminum should work fine too Amazon
(x1) 20"x20"x¼” glass pane (acrylic works too Amazon)
Digikey
(x64) 800nm IR phototransistors Digikey 511-1357-ND
(x64) white wide angle LEDs Digikey C535A-WJN-CS0V0231-ND - I found that the quality control on these LEDs is not so great, and they each turned out to be slightly different shades of white. I thought the effect actually looked pretty cool, but if that's not what you're into, I'd try to find some other wide angle LEDs.
(x1) MAX7219 LED driver Digikey MAX7219CNG -ND
(x1) 24 pin socket Digikey 3M5466-ND
(x1) 10uf capacitor Digikey P828-ND
(x1) 0.1uf capacitor Digikey 490-5401-ND
(x1) 74HC595 shift register Digikey 296-1600-5-ND
(x1) 16 pin socket Digikey A100206-ND
(x10) 10kOhm 1/4 watt resistors Digikey CF14JT10K0CT-ND
(x1) 1/4 watt 7219 resistor (value determined here depending on your LEDs)
Amazon
(x2) usb cable male type A to male type b Amazon
(x1) usb adapter female type a to female type b Amazon
(x1) perfboard with copper Amazon
(x1) polycrylic clear coat Amazon
(x1) silicon adhesive Amazon
(x1) black electrical tape Amazon
Jameco
(x1) 16 pin ribbon cable Jameco 643532
(x1) 16 pin crimp socket Jameco 1578111 (I didn't actually use this but I wish I had)
(x5) male header pins Jameco 103393
(x1) 22 gauge solid core wire Jameco 36792
Tools:
screwdriver
hot glue gun
drill
table saw
wood mill/router
aluminum mill
aluminum shear
bandsaw
soldering iron
infrared/nightvision camera (optional, but useful)
LLP (Laser Light Plane)
Infrared (IR) multitouch is an inexpensive alternative to the capacitive multitouch found in smartphones and tablets. There are a few popular techniques for building an IR multitouch system, all of which are outlined very nicely at the nuigroup wiki.
I used a technique called Laser Light Plane (LLP) for my touchscreen; the diagram above gives an overview of how it works. Several lasers positioned on top of the screen create a very thin layer or infrared (IR) light that completely covers the screen's surface. When a finger touches the screen, it breaks this plane of light and scatters some of the IR light into the screen. I partitioned the underside of the screen with pieces of aluminum sheet and placed one IR sensor in each partition. By measuring each of the sensors, you can determine the x and y coordinates of the touch event. I also added a white LED to each partition, so that the screen could be used as a projection surface. The diffuser material on the bottom of the screen (light grey) helps the LED more evenly distribute light across the surface of the screen.
This is what LLP looks like with red lasers, IR will work the same, but it is not visible to the human eye:
Here's what it looks like to a camera that's been modified to see IR:
This is what my final project looks like with night vision:
I won't get into the specifics of every IR multitouch technique here (again, check nuigroup for that info), but I'll say a few things about why I chose LLP:
zero force - no downward pressure on the screen is necessary to register a touch.
glass - I wanted to use glass as my screen surface (looks/feels nicer, less scratch prone than acrylic), but some IR multitouch methods (total internal reflection) require acrylic.
thin - many IR multitouch techniques us a camera to do the finger tracking, this means that the camera must be positioned so that it has an unobstructed view of the entire screen. Usually this means making the enclosure of the multitouch surface very deep so that the camera is sufficiently far away. By contrast, my method of using an array with LLP can be scaled down to thicknesses of less than an inch.
scalable - the unit piece of this touchscreen - the cell containing one LED and one IR sensor - can be repeated many times to make larger or denser arrays.
Some downsides to LLP:
- anything that breaks the surface of the screen will trigger the sensors - sleeves, elbows... sometimes that is annoying. It could be a good thing though, you can use regular objects to trigger the touch.
- the downfall of all IR touchscreens is that they are sensitive to IR light. This screen only works at night or in a room with no windows, and the light in the room can only be fluorescent - no incandescent bulbs.
Another good resource for LLP information is at the nuigroup llp page.
Image source: parts of the above image are taken from nuigroup
I used a technique called Laser Light Plane (LLP) for my touchscreen; the diagram above gives an overview of how it works. Several lasers positioned on top of the screen create a very thin layer or infrared (IR) light that completely covers the screen's surface. When a finger touches the screen, it breaks this plane of light and scatters some of the IR light into the screen. I partitioned the underside of the screen with pieces of aluminum sheet and placed one IR sensor in each partition. By measuring each of the sensors, you can determine the x and y coordinates of the touch event. I also added a white LED to each partition, so that the screen could be used as a projection surface. The diffuser material on the bottom of the screen (light grey) helps the LED more evenly distribute light across the surface of the screen.
This is what LLP looks like with red lasers, IR will work the same, but it is not visible to the human eye:
Here's what it looks like to a camera that's been modified to see IR:
This is what my final project looks like with night vision:
I won't get into the specifics of every IR multitouch technique here (again, check nuigroup for that info), but I'll say a few things about why I chose LLP:
zero force - no downward pressure on the screen is necessary to register a touch.
glass - I wanted to use glass as my screen surface (looks/feels nicer, less scratch prone than acrylic), but some IR multitouch methods (total internal reflection) require acrylic.
thin - many IR multitouch techniques us a camera to do the finger tracking, this means that the camera must be positioned so that it has an unobstructed view of the entire screen. Usually this means making the enclosure of the multitouch surface very deep so that the camera is sufficiently far away. By contrast, my method of using an array with LLP can be scaled down to thicknesses of less than an inch.
scalable - the unit piece of this touchscreen - the cell containing one LED and one IR sensor - can be repeated many times to make larger or denser arrays.
Some downsides to LLP:
- anything that breaks the surface of the screen will trigger the sensors - sleeves, elbows... sometimes that is annoying. It could be a good thing though, you can use regular objects to trigger the touch.
- the downfall of all IR touchscreens is that they are sensitive to IR light. This screen only works at night or in a room with no windows, and the light in the room can only be fluorescent - no incandescent bulbs.
Another good resource for LLP information is at the nuigroup llp page.
Image source: parts of the above image are taken from nuigroup
Schematic
My schematic is a modified version of the Arduinome schematic found here. A little info about why I based this off Arduinome... When I was first researching online for ideas I knew I wanted to make some kind of LED array / coffee table / control surface. I found some stuff like this that I liked on youtube:
Searching for more Arduino / grid controllers, I found monome and Arduinome. It was (and still is) the best solution for this type of project because once you build the interface, you have access to all the MaxMSP applications that people post on the monome site - and there's some really creative and amazing apps there - so you're getting a lot of value out of the work you put in. Because these apps are written in MaxMSP, they're open source, so if you want to, you can edit them and learn more about how they work. Aaand if you ever end up writing your own applications, you can post them and see what other people do with them, which is always kind of fun.
The most notable modification I've made form the original Arduinome schematic is that I've replaced all the buttons, and their corresponding diodes, with IR phototransistors (since the transistors have an inherent polarity, diodes are not necessary).
I've also replaced the 74HC164 with a 74HC595. On paper these chips are not too different, but I found that one of the pins on the 164 was dropping in voltage slightly in the Arduinome schematic. This is not a huge deal when you're controlling buttons because they are 2-state devices, but it was causing one row of my sensors to be significantly less sensitive than the rest. I tried to troubleshoot this and could not find a way, so I replaced it with the 595 and had much better results. I also simplified the circuit by removing the 74hc165 and using 8 pins on the Arduino instead. I had to change the Arduinome firmware to account for these changes, you'll find that in a later step.
Searching for more Arduino / grid controllers, I found monome and Arduinome. It was (and still is) the best solution for this type of project because once you build the interface, you have access to all the MaxMSP applications that people post on the monome site - and there's some really creative and amazing apps there - so you're getting a lot of value out of the work you put in. Because these apps are written in MaxMSP, they're open source, so if you want to, you can edit them and learn more about how they work. Aaand if you ever end up writing your own applications, you can post them and see what other people do with them, which is always kind of fun.
The most notable modification I've made form the original Arduinome schematic is that I've replaced all the buttons, and their corresponding diodes, with IR phototransistors (since the transistors have an inherent polarity, diodes are not necessary).
I've also replaced the 74HC164 with a 74HC595. On paper these chips are not too different, but I found that one of the pins on the 164 was dropping in voltage slightly in the Arduinome schematic. This is not a huge deal when you're controlling buttons because they are 2-state devices, but it was causing one row of my sensors to be significantly less sensitive than the rest. I tried to troubleshoot this and could not find a way, so I replaced it with the 595 and had much better results. I also simplified the circuit by removing the 74hc165 and using 8 pins on the Arduino instead. I had to change the Arduinome firmware to account for these changes, you'll find that in a later step.
Enclosure
This is a basic mockup of the enclosure. The main structural component is a square wood frame with a 1/4" recess cut out from the top (fig 2).
I cut 1-2mm thick aluminum sheet into strips and used notch cut-outs to join the strips together to form an 8x8 grid partition for the inside of the enclosure (fig 3 and 4, a 16x16 grid is shown in the diagram, but this was changed during fabrication, also note that the dimensions indicated in this mockup are not exactly correct). This grid fits snugly in the wood frame (fig 5).
Below the grid is a large piece of phenolic, where all the leds and phototransistors are mounted. Above the grid is the rear projection surface (the diffuser) and the 1/4" glass sheet (fig 6), these fit inside the top recess of the wood frame. Above the glass are four wood panels that form a frame; this frame holds/hides the 4 lasers at each corners and prevents laser light from leaving the enclosure (fig 7). A panel in the back closes the enclosure and provides a place to mount the remaining circuitry - Arduino and other electronic components.
I cut 1-2mm thick aluminum sheet into strips and used notch cut-outs to join the strips together to form an 8x8 grid partition for the inside of the enclosure (fig 3 and 4, a 16x16 grid is shown in the diagram, but this was changed during fabrication, also note that the dimensions indicated in this mockup are not exactly correct). This grid fits snugly in the wood frame (fig 5).
Below the grid is a large piece of phenolic, where all the leds and phototransistors are mounted. Above the grid is the rear projection surface (the diffuser) and the 1/4" glass sheet (fig 6), these fit inside the top recess of the wood frame. Above the glass are four wood panels that form a frame; this frame holds/hides the 4 lasers at each corners and prevents laser light from leaving the enclosure (fig 7). A panel in the back closes the enclosure and provides a place to mount the remaining circuitry - Arduino and other electronic components.
Lasers
I attached an 89 degree line lens to each of my lasers. These lenses turn a point/beam of light into an 89 degree fan/planar shape. This way four lasers (one at each corner of the screen) will get excellent coverage of the screen's surface.
At this point you will want to focus the laser. I think the best way to do this is to power the laser on and turn the lights off. Shine the laser on a white wall - you should see a faint red line appear. Twist the lens until you see the laser focus, you want the line to be as sharp as possible. It's a good idea to wrap the lasers in electrical tape to prevent them from getting bumped out of focus.
At this point you will want to focus the laser. I think the best way to do this is to power the laser on and turn the lights off. Shine the laser on a white wall - you should see a faint red line appear. Twist the lens until you see the laser focus, you want the line to be as sharp as possible. It's a good idea to wrap the lasers in electrical tape to prevent them from getting bumped out of focus.
Laser Mounts
I machined four laser mounts from aluminum. Each has a set screw for holding the lasers in place and another wide hole at the base for attaching to the wood enclosure (and adjusting vertical position). For now I loosely positioned the lasers (without losing focus) in the mounts, I will talk about aligning these lasers in more detail in another step.
Notice how I used hot glue to stabilize the wires coming out of the laser; I learned the hard way that the connections between the wires and the laser pcb are very weak and can easily get broken if you accidentally pull on the leads. It's a good idea to use hot glue as a buffer between the lasers and any pulling that might happen by accident on the other side of the enclosure.
Notice how I used hot glue to stabilize the wires coming out of the laser; I learned the hard way that the connections between the wires and the laser pcb are very weak and can easily get broken if you accidentally pull on the leads. It's a good idea to use hot glue as a buffer between the lasers and any pulling that might happen by accident on the other side of the enclosure.
Projection Screen (Diffuser)
I stretched a 17.5x17.5" piece of the rosco black projection screen across the top of the wood enclosure and secured it with some small nails. Be careful to distribute tension evenly across the screen so you don't end up with wrinkles.
Glass
Using the score and snap method, I cut my plate glass down to size (17.5"x17.5") and cut off small triangles at each of the corners to make room for the lasers. This piece of glass should fit inside the recess of the wood frame and lie flush with the top edge of the wood.
Secure Glass
I secured the glass to the wood enclosure with silicon adhesive all along the outer perimeter.
Wire Lasers
Originally I used an external 5V supply to power the lasers, but I've recently started powering them with the Arduino's 5V supply. the wiring instructions are the same for both cases:
Wire up 5V from the power jack (or Arduino) to the right pin of the the power switch, then wire the power switch center pin to all four lasers' red leads. Wire up the 0V from the power jack (or Arduino) to the remaining pin of the power switch and the black leads of all four lasers. This way the switch will control power to the lasers in parallel.
Wire up 5V from the power jack (or Arduino) to the right pin of the the power switch, then wire the power switch center pin to all four lasers' red leads. Wire up the 0V from the power jack (or Arduino) to the remaining pin of the power switch and the black leads of all four lasers. This way the switch will control power to the lasers in parallel.
Align Lasers
The IR lasers are somewhat difficult to see with human eyes, you may find that this step is more easily accomplished with the use of a nightvision camera or a modified webcam. If you don't have either of these, do this step in a dark room so you can see the IR.
Cover three of the IR lasers with black electrical tape to block them. Hold a piece of white cardstock vertically on the glass in front of the un-blocked laser diode. Your should see a line appear on the card, this is the beam from the laser. If you haven't already, focus the beam by tightening and loosening the lens attachment on the laser. You want find the position that makes the line on the cardstock appear as thin as possible. This is much easier to do when the cardstock is held farther away, you may even try shining the laser on a white wall a few feet away. Once the laser is in focus use a piece of electrical tape to secure the position of the lens.
Now loosen the laser mount and adjust the laser and the mount until the laser light shines parallel to the surface of the glass, about 1mm about the surface. You will need to rotate the laser in the mount and adjust the vertical position of the mount to get it in position. One you've found the optimal position, tighten your set screws so the mounts stay put. Repeat this for the remaining lasers.
Here's a video that shows what this looks like:
Cover three of the IR lasers with black electrical tape to block them. Hold a piece of white cardstock vertically on the glass in front of the un-blocked laser diode. Your should see a line appear on the card, this is the beam from the laser. If you haven't already, focus the beam by tightening and loosening the lens attachment on the laser. You want find the position that makes the line on the cardstock appear as thin as possible. This is much easier to do when the cardstock is held farther away, you may even try shining the laser on a white wall a few feet away. Once the laser is in focus use a piece of electrical tape to secure the position of the lens.
Now loosen the laser mount and adjust the laser and the mount until the laser light shines parallel to the surface of the glass, about 1mm about the surface. You will need to rotate the laser in the mount and adjust the vertical position of the mount to get it in position. One you've found the optimal position, tighten your set screws so the mounts stay put. Repeat this for the remaining lasers.
Here's a video that shows what this looks like:
Front Panels
The front panels are shown in detail in the images above. I used a router to remove material along their length and at each of the corners. The corner material was removed to make room for the laser and laser mounts at each corner of the enclosure. The space along the length was added so that when the IR light from the lasers hits the front panels and scatters, it will not direct light towards the phototransistors in the edge cells of the display. Instead, the light will travel under the front panel, about an inch past the edge of the visible part of the display. Any light that bounces down from there can be blocked with electrical tape without disrupting the appearance of the display.
Tape Front Panel Edges
I found that some black electrical tape along the edges of the front panel pieces helped to prevent unwanted ir light from scattering into the edge cells of the display.
Mount Front Panels
Once you're happy with the position of the lasers, and have secured them with tape/glue, mount the protective front panels. I used some tiny nails to attach the panels to the front of my enclosure, you could also use wood glue.
Light Tight
I used electrical tape to make the underside of the laser mounts light tight - you don't want any scattered IR light to hit the phototransistor in the corners of the display. I also lined the edges of the enclosure with electrical tape. Be sure that none of these blocked parts are visible from the front of the enclosure when the front panels are on.
Turn the lasers on and use your nightvision camera to be sure that no extraneous light is passing through the screen.
Turn the lasers on and use your nightvision camera to be sure that no extraneous light is passing through the screen.
USB/Power Access
I used a router to cut a few holes for my USB and power connections (extra power connection is optional), and my power switch. I glued these components in with silicon adhesive and epoxy. Later when I installed the back panel, these components were flush against it.
Aluminum Partitions
Cut 14 aluminum partitions from 1mm aluminum sheet. Cut the strips so that they fit snugly inside the square wood enclosure. I used the bandsaw to cut seven notches halfway through each of the pieces so that they could fit together as shown in the images above.
Install Partitions
Carefully place the aluminum partitions into the back opening of the enclosure so that it presses against the diffuser and glass pane.
LED/Phototransistor Array
Cut a piece of 17"x17" phenolic so that it fits snugly inside the back of the enclosure. Mark the centers of each cell on the phenolic and drill four small holes to mount the one with LED and one phototransistor. This is the tedious part - wire up 64 white LEDs in and 8x8 array and 64 phototransistors in an 8x8 array as shown in the schematic. Be sure to keep track of the anodes and cathodes of the leds and transistors so that you know they are wired correctly.
Wire Up Array
Wire up the 595 and 7219 as shown in the schematics above. Make all necessary connections to the LED and phototransistor arrays and the Arduino. The phototransistor are represented as diodes in the schematic above, since they only have two leads.
You will need to select a value for RSet of the Max7219 circuit based on the specificaltions of your LEDs. Find information about how to do that here.
You will also need to choose resistors for the rows of the phototransistor array. You can increase the sensitivity of your array by increasing the resistance of these resistors. I used 10kOhm resistors, that's probably a good place to start.
You will need to select a value for RSet of the Max7219 circuit based on the specificaltions of your LEDs. Find information about how to do that here.
You will also need to choose resistors for the rows of the phototransistor array. You can increase the sensitivity of your array by increasing the resistance of these resistors. I used 10kOhm resistors, that's probably a good place to start.
Install LED/Sensor Array
Flip the phenolic board over so that the LEDS and phototransistors are facing down and install the board in the enclosure. Use screws to secure the board in place.
Back Panel
I needed a little extra depth to fit all the electronics in my project so I cut a 3/4" piece of ply with a 10"x10"x0.5" routed-out pocket to use as a back panel for the enclosure. The Shopbot files for the part are attached. I painted the ply black so that it looked a little nicer with the redwood.
Downloads
Firmware
Upload the following firmware to the board:
You should see your leds light up one column at a time and scroll across the display when the firmware is finished uploading:
If your leds are not wired up correctly, fixed them now before moving on to the next step.
/* * "ArduinomeFirmware" - Arduino Based Monome Clone by Owen Vallis & Jordan Hochenbaum 06/16/2008 * Revised 06/26/2008 * Revised 07/20/2008 by Ben Southall * Revised 03/21/2009 Ben Southall * Revised 01/21/2012 Jordan Hochenbaum v3.3 rev.a * Revised 06/2012 for 74HC595 by Amanda Ghassaei * -------------------------------------------------------------------------- * This program is free software; you can redistribute it and/or modify * it under the terms of the GNU General Public License as published by * the Free Software Foundation; either version 2 of the License, or * (at your option) any later version. * * This program is distributed in the hope that it will be useful, * but WITHOUT ANY WARRANTY; without even the implied warranty of * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the * GNU General Public License for more details. * -------------------------------------------------------------------------- * * This code is largely a direct translation of the 40h protocol designed by Brian Crabtree * & Joe Lake, and coded by Joe Lake. We would like to express our utmost thanks for making * the monome & its inner-workings available and open source. We would also like to extend * our thanks to all those who have helped us get this far in successfully making an arduino- * based monome clone, including but not limited to Brad Hill for his awesome arduino shield, * and Melka for showing how to pack and unpack the serial data. * * Additional comment and attribution 7/20/2008 : * The changes on 7/20/2008 were inspired by, and at least in some part, directly * copied from the 'Octinct' code written by Jonathan M Guberman (upwardnotnorthward.com). * In particular, the use of timer 2 to drive an interrupt routine to read the serial port was * picked up from the Octinct project, and also, setting the baud rate to 57600 seemed to be * critical for getting everything to work reliably. My thanks to Jonathan for sharing his * software, and for figuring out how to keep that serial buffer in check. * * Changes 03/21/2009 * * Added ADC support for arduino ADCs 0 - 3. Thanks to Josh Lory for the first cut of the code for * this. Added handling of the ADC enable/disable serial messages, and changed Josh's checkADC code * to loop through the 4 ADCs rather than read each one individually. Also, cleaned up a number of * unused variables, and moved all PORT/pin definitions to the top of the file; this should make it * easier for people not using the unsped shield to change the hardware connections. * * Please DO NOT email monome with technical questions and/or help regarding this code or clone. * They are in NO WAY responsible or affiliated with this project other than they were our inspiration * and we used many of their methods and pulled from their code. * * Additionally, while we are availble and willing to help as much as possible, we too CANNOT be held * responsible for anything you do with this code. Please feel free to report any bugs, suggestions * or improvements to us as they are all welcome. Again, we cannot be held responsible for any damages * or harm caused by the use or misuse of this code or our instructions. Thank you for understanding. * * * * * Changes 01/21/2012 & 3.3a * Arduino 1.0+ compatability (uses .ino extension and no longer specifies BYTE type in Serial.write) * -------------------------------------------------------------------------- * * Links: * www.monome.org - Our website - Click "Work/Arduinome" on the Navigation Menu on the top. * www.monome.org - the "original" monome and our inspiration * www.monome.org * www.monome.org * */ // PIN and PORT assignments // If you're using hardware other than the unsped shield, you'll want to go through this portion of the code, // and change the pin numbers and atmega 168 PORTs that you're using. We're using pin/PORT assignments here, // NOT the pin number scheme from the arduino board outputs (e.g. arduino digital pin 8 = PORTB pin 1) // However, in the comments below, I do give the equivalents between arduino digital pin numbers and the // atmega168's PORT/bit numbering. // IMPORTANT - you'll need to make sure that you set the data following two direction register variables to match your // pin assignments. byte PORTD_Data_Direction = 0x02;//all inputs except pin 1 (TX) byte PORTB_Data_Direction = 0xFF;//all outputs byte PORTC_Data_Direction = 0xFC;//A2-A5 are outputs, A0, A1 inputs //Connections to 595 shift register //dataPin = A4 = PORTC, bit 4 //clockPin = A2 = PORTC, bit 2 //latchPin = A3 = PORTC, bit 3 #define DATA_PORT_595 (PORTC) byte dataPin595 = 4; byte dataPin595MaskHigh = 1 << dataPin595; byte dataPin595MaskLow = ~dataPin595MaskHigh; #define CLOCK_PORT_595 (PORTC) byte clockPin595 = 2; byte clockPin595MaskHigh = 1 << clockPin595; byte clockPin595MaskLow = ~clockPin595MaskHigh; #define LATCH_PORT_595 (PORTC) byte latchPin595 = 3; byte latchPin595MaskHigh = 1 << latchPin595; byte latchPin595MaskLow = ~latchPin595MaskHigh; // Connections to the Max7219 (drives LEDS) // dataIn = arduino pin A5 = PORTC, bit 5 // load = arduino pin 11 = PORTB bit 4 // clock = arduino pin 12 = PORTB, bit 5 #define LED_DATA_PORT (PORTC) byte MaxDataPin = 5; byte DataMaskHigh = 1 << MaxDataPin; byte DataMaskLow = ~DataMaskHigh; #define LED_CLOCK_PORT (PORTB) byte MaxClockPin = 5; byte ClockMaskHigh = 1<<MaxClockPin; byte ClockMaskLow = ~ClockMaskHigh; #define LED_LOAD_PORT (PORTB) byte MaxLoadPin = 4; byte LoadMaskHigh = 1 << MaxLoadPin; byte LoadMaskLow = ~LoadMaskHigh; // end connections to the 165 shift register //use this to fix mirroring boolean mirrorXLEDs = true; boolean mirrorYLEDs = false; boolean mirrorXButtons = false; boolean mirrorYButtons = false; // END pin and PORT assignments // ---------------------------- // Global variables byte byte0, byte1; // storage for incoming serial data byte WaitingForAddress = 1; // 1 when we expect the next serial byte to be an address value, 0 otherwise byte address = 0x00; // garbage byte to hold address of function byte state = 0x00; // garbage byte to hold state value byte x = 0x00; // garbage byte to hold x position byte y = 0x00; // garbage byte to hold y position byte z = 0x00; // garbage byte to iterate over buttons // The following variables are used to store flags that indicate whether we have received a given message, // the value of the data in that message. e.g. IntensityChange = 0 == no intensity change message received, // IntensityChange = 1 == an intensity change message has been received, and it's value will be in IntensityVal // The code that eventually acts upon the incoming message will reset the 'Change' variable to 0 once the // message has been handled. byte IntensityChange = 0; byte IntensityVal = 0; byte DisplayTestChange = 0; byte DisplayTestVal = 0; byte ShutdownModeChange = 0; byte ShutdownModeVal= 0; // These variables are used to handle the ADC messages, and store which ports are currently enabled, // and which are not. byte ADCnum; byte ADCstate; byte ADCEnableState[4]; byte ledmem[8]; // memory for LED states - 64 bits. byte j = 0; // temporary variable for button looping int i = 0; // temporary variable for looping etc. int id = 0; // temporary storage for incoming message ID byte firstRun = 1; // 1 when we have yet to receive an LED command, 0 after that. // used to ensure that our led state memory is properly initialized when we receive the first LED message int kButtonUpDefaultDebounceCount = 12; // Used in button debouncing int kButtonNewEvent = 1; // Used to store whether the state of a button has changed or not. byte t = 0; // temporary variable used in button processing /* BUTTON ARRAY VARIABLES For 1/0 button states, use 8 bytes (64 bits) For button debounce counter, use 8x8 array */ byte button_current[8]; byte button_last[8]; byte button_state[8]; byte button_debounce_count[8][8]; byte button_event[8]; /* END OF BUTTON ARRAY VARIABLES */ /* MAX 7219 INSTRUCTION ADDRESSES */ byte max7219_reg_noop = 0x00; // define max7219 registers (read the tech doc for explaination) byte max7219_reg_digit0 = 0x01; byte max7219_reg_digit1 = 0x02; byte max7219_reg_digit2 = 0x03; byte max7219_reg_digit3 = 0x04; byte max7219_reg_digit4 = 0x05; byte max7219_reg_digit5 = 0x06; byte max7219_reg_digit6 = 0x07; byte max7219_reg_digit7 = 0x08; byte max7219_reg_decodeMode = 0x09; byte max7219_reg_intensity = 0x0a; byte max7219_reg_scanLimit = 0x0b; byte max7219_reg_shutdown = 0x0c; byte max7219_reg_displayTest = 0x0f; /* END OF MAX 7219 INSTRUCTION ADDRESSES */ // putByte - helper function that sends a single byte to the MAX chip void putByte(byte data) { byte i = 8; byte mask; while(i > 0) { mask = 0x01 << (i - 1); // get bitmask if (data & mask) // check and send value of this bit { LED_DATA_PORT |= DataMaskHigh; } else { LED_DATA_PORT &= DataMaskLow; } // Pulse the MAX clock LED_CLOCK_PORT &= ClockMaskLow; // digitalWrite( clock, LOW); // "tick" prepeare for bit input LED_CLOCK_PORT |= ClockMaskHigh;// digitalWrite(clock, HIGH); // "tock" input bit --i; // move to lesser bit } } //maxSingle is the "easy" function to use for a single max7219 //dig is the row call, and seg is the column call dig and seg refer to pin names from tech doc void maxSingle( byte dig, byte seg) { LED_LOAD_PORT &= LoadMaskLow; // digitalWrite(load, LOW); // begin putByte(dig); // specify register putByte(seg); //((data & 0x01) * 256) + data >> 1); // put data LED_LOAD_PORT |= LoadMaskHigh; // digitalWrite(load,HIGH); } // buttonInit - helper function that zeros the button states void buttonInit(void){ byte i; for (i = 0; i < 8; i++) { button_current[i] = 0x00; button_last[i] = 0x00; button_state[i] = 0x00; button_event[i] = 0x00; } } // buttonCheck - checks the state of a given button. void buttonCheck(byte row, byte index) { if (((button_current[row] ^ button_last[row]) & (1 << index)) && // if the current physical button state is different from the ((button_current[row] ^ button_state[row]) & (1 << index))) { // last physical button state AND the current debounced state if (button_current[row] & (1 << index)) { // if the current physical button state is depressed button_event[row] = kButtonNewEvent << index; // queue up a new button event immediately button_state[row] |= (1 << index); // and set the debounced state to down. } else{ button_debounce_count[row][index] = kButtonUpDefaultDebounceCount; } // otherwise the button was previously depressed and now // has been released so we set our debounce counter. } else if (((button_current[row] ^ button_last[row]) & (1 << index)) == 0 && // if the current physical button state is the same as (button_current[row] ^ button_state[row]) & (1 << index)) { // the last physical button state but the current physical // button state is different from the current debounce // state... if (button_debounce_count[row][index] > 0 && --button_debounce_count[row][index] == 0) { // if the the debounce counter has // been decremented to 0 (meaning the // the button has been up for // kButtonUpDefaultDebounceCount // iterations/// button_event[row] = kButtonNewEvent << index; // queue up a button state change event if (button_current[row] & (1 << index)){ // and toggle the buttons debounce state. button_state[row] |= (1 << index); } else{ button_state[row] &= ~(1 << index); } } } } void buttonpress () { for(i = 0; i < 8; i++){ LATCH_PORT_595 &= latchPin595MaskLow;// set latch pin low so the outputs don't change while sending in bits for (j=0;j<8;j++){ CLOCK_PORT_595 &= clockPin595MaskLow;//digitalWrite(clockPin,LOW); if (j==i){//if index equal current i value, set low DATA_PORT_595 &= dataPin595MaskLow;//digitalWrite(A4,LOW); } else{//set the rest of the pins high DATA_PORT_595 |= dataPin595MaskHigh;//digitalWrite(A4,HIGH); } CLOCK_PORT_595 |= clockPin595MaskHigh;//digitalWrite(clockPin,HIGH); } //set latch pin high- this sends data to outputs LATCH_PORT_595 |= latchPin595MaskHigh;//digitalWrite(latchPin, HIGH); // SlowDown is put in here to waste a little time while we wait for the state of the output // pins to settle. Without this time wasting loop, a single button press would show up as // two presses (the button and its neighbour) volatile int SlowDown = 0; while (SlowDown < 15) { SlowDown++; } button_last [i] = button_current [i]; for(id = 0; id < 8; id++) { switch (id){ case 0: t = digitalRead(A0); break; case 1: t = digitalRead(A1); break; case 2: t = digitalRead(2); break; case 3: t = digitalRead(3); break; case 4: t = digitalRead(4); break; case 5: t = digitalRead(5); break; case 6: t = digitalRead(6); break; case 7: t = digitalRead(7); break; } t = (t == 0);//invert t if(t){ button_current [i] |= (1 << id); } else{ button_current [i] &= ~(1 << id); } buttonCheck(i, id); if (button_event[i] & (1 << id)) { button_event[i] &= ~(1 << id);//zero button event if(button_state[i] & (1 << id)){ Serial.write(1); } else{ Serial.write(byte(0)); } if (mirrorXButtons){ id=7-id; } if (mirrorYButtons){ i=7-i; } Serial.write(((id) << 4) | (i)); } } } //set all pins of 595 high LATCH_PORT_595 &= latchPin595MaskLow;// set latch pin low so the outputs don't change while sending in bits for (j=0;j<8;j++){ CLOCK_PORT_595 &= clockPin595MaskLow;//digitalWrite(clockPin,LOW); DATA_PORT_595 |= dataPin595MaskHigh;//digitalWrite(A4,HIGH); CLOCK_PORT_595 |= clockPin595MaskHigh;//digitalWrite(clockPin,HIGH); } //set latch pin high- this sends data to outputs LATCH_PORT_595 |= latchPin595MaskHigh;//digitalWrite(latchPin, HIGH); } ISR(TIMER2_COMPA_vect) { do { if (Serial.available()) { if (WaitingForAddress == 1) { byte0 = Serial.read(); address = byte0 >> 4; WaitingForAddress = 0; } // end if (WaitingForAddress == 1); if (Serial.available()) { WaitingForAddress = 1; byte1 = Serial.read(); switch(address) { case 2: state = byte0 & 15; x = byte1 >> 4; y = byte1 & 15; if (state == 0){ ledmem[7-y] &= ~( 1 << x); } else { ledmem[7-y] |= ( 1 << x); } break; case 3: IntensityChange = 1; IntensityVal = byte1 & 15; break; case 4: DisplayTestChange = 1; DisplayTestVal = byte1 & 15; break; case 5: state = byte1 & 0x0F; ADCEnableState[(byte1 >> 4)&0x03] = state; break; case 6: ShutdownModeChange = 1; ShutdownModeVal= byte1 & 15; break; case 7: if (firstRun == 1) { for (x = 0; x < 8; x++) { ledmem[x] = 0; } firstRun = 0; } x = ((byte0 & 15) & 0x7); // mask this value so we don't write to an invalid address. y = byte1; if (mirrorYLEDs){ y=7-y; } if (mirrorXLEDs){ x=flipByte(x); } ledmem[x] = y; break; case 8: if (firstRun == 1) { for (x = 0; x < 8; x++) { ledmem[x] = 0; } firstRun = 0; } x = ((byte0 & 15) & 0x7); y = byte1; if (mirrorYLEDs){ x=7-x; } if (mirrorXLEDs){ y=flipByte(y); } for (z = 0; z < 8; z++) { if (y & (1 << z)) { ledmem[z] |= 1 << x; } else { ledmem[z] &= ~(1 << x); } } break; } // end switch(address) } // end if (Serial.available() } // end if (Serial.available(); } // end do while (Serial.available() > 16); } void whoosh(void) { // reactivate overflow interrupt for // timer 1 - it's needed by delay(...) TIMSK0 |= (1 << TOIE0); for (int j = 0; j < 9; j++) { for (int i = 0; i < 8; i++) { maxSingle(i+1,1<<j); } delay(125); } // and switch the interrupt off. TIMSK0 &= ~(1 << TOIE0); } void setup () { DDRD = PORTD_Data_Direction; DDRB = PORTB_Data_Direction; DDRC = PORTC_Data_Direction; Serial.begin(57600); buttonInit(); //initiation of the max 7219 maxSingle(max7219_reg_scanLimit, 0x07); maxSingle(max7219_reg_intensity,0x0F); maxSingle(max7219_reg_shutdown, 0x01); // not in shutdown mode maxSingle(max7219_reg_displayTest, 0x00); // empty registers, turn all LEDs off h for (i=1; i<=8; i++){ maxSingle(i,0); ledmem[i-1]=0; } cli();//stop interrupts //set timer2 interrupt every 128us TCCR2A = 0;// set entire TCCR2A register to 0 TCCR2B = 0;// same for TCCR2B TCNT2 = 0;//initialize counter value to 0 // set compare match register for 7.8khz increments OCR2A = 255;// = (16*10^6) / (7812.5*8) - 1 (must be <256) // turn on CTC mode TCCR2A |= (1 << WGM21); // Set CS11 bit for 8 prescaler TCCR2B |= (1 << CS11); // enable timer compare interrupt TIMSK2 |= (1 << OCIE2A); sei();//allow interrupts // // Set up 8-bit counter 2, output compare switched off, // // normal waveform generation (whatever that might mean) // TCCR2A = 0; // // set counter to be clocked at 16Mhz/8 = 2Mhz // TCCR2B = 1<<CS21; // // // set the interrupt mask so that we get an interrupt // // on timer 2 overflow, i.e. after 256 clock cycles. // // The timer 2 interrupt routine will execute every // // 128 uS. // TIMSK2 = 1<<TOIE2; // // // NOTE: In my efforts to try to get this // // code to work reliably at 115200 baud // // I went about disabling unused timers that // // are set up by the Arduino framework. // // The framework sets up both timer 2 and // // timer 1. We use timer 2 to drive the // // serial reading interrupt, so we can only // // disable timer1 and its interrupt. // // // REALLY IMPORTANT NOTE - IF YOU WANT TO USE // // ANALOGWRITE // // // Disabling timer 1 will switch off the timer // // used for PWM, which underlies analogWrite. // // If you want to use analogWrite, remove // // the lines below. // // // DISABLE PWM COUNTER // TIMSK0 &= ~(1 << TOIE0); // // TCCR1B &= ~(1 << CS12); // TCCR1B &= ~(1 << CS11); // TCCR1B &= ~(1 << CS10); // // END DISABLE PWM COUNTER // // // Also, disable the interrupts on timer 0 // // REALLY IMPORTANT NOTE IF YOU WANT TO USE // // DELAY // // remove this line, and also look at // // 'whoosh', which enables and then disables // // the intterupt on timer 0. You'll want // // to get rid of those lines too. // TIMSK0 &= ~(1 << TOIE0); // pretty pattern to assure me that the firmware's // uploaded properly. whoosh(); // make sure to turn off the lights. for (int i = 0; i < 8; i++) { maxSingle(i+1,0); } } //void sendADC(int port, int value) { // Serial.write((1 << 4) | ((port << 2) & 0x0C) | ((value >> 8) & 0x03)); // Serial.write(value & 0xFF); //} // //int current[4]; //int previous[4]; //int tolerance = 7; //void checkADCs() { // // for (int adcCounter = 0; adcCounter < 4; adcCounter++) // { // // if (ADCEnableState[adcCounter] != 0) // { // current[adcCounter] = analogRead(adcCounter); // // if (abs(previous[adcCounter]-current[adcCounter]) > tolerance) // { // previous[adcCounter] = current[adcCounter]; // sendADC(adcCounter,current[adcCounter]); // } // // } // // } //} byte flipByte(byte numToFlip){ byte mirror = 0; for (int j=0;j<8;j++){ byte copy = numToFlip; mirror |= (copy>>j)&1; if (j<7){ mirror = mirror << 1; } } return mirror; } void loop () { // send the LED states to the LED matrix. for (int i = 0; i < 8; i++) { byte ledmemMirror = 0; if (mirrorXLEDs){ ledmemMirror=flipByte(ledmem[i]); } else { ledmemMirror = ledmem[i]; } int y; if (mirrorYLEDs){ y =(7-i)+1; } else { y = i+1; } maxSingle(y,ledmemMirror); } if (IntensityChange == 1) { IntensityChange = 0; maxSingle(max7219_reg_intensity, IntensityVal & 15); } if (DisplayTestChange == 1) { DisplayTestChange = 0; maxSingle(max7219_reg_displayTest, DisplayTestVal & 15); } if (ShutdownModeChange == 1) { ShutdownModeChange = 0; maxSingle(max7219_reg_shutdown, ShutdownModeVal & 15); } // check for button presses buttonpress(); // // check the state of the ADCs // checkADCs(); }
You should see your leds light up one column at a time and scroll across the display when the firmware is finished uploading:
If your leds are not wired up correctly, fixed them now before moving on to the next step.
Downloads
Flash New Serial Number on Arduino
This is the trickiest step. First you need to give your Arduino a new identity, you can follow these instructions with an Arduino Uno (Mac only). Disconnect your Arduino from your circuit while you do this. Be sure you read through the directions carefully and understand all the steps before you attempt this. If done incorrectly, you can potentially render your Arduino useless - though I'm not exactly sure how - I've just read a lot of warnings about it.
In case this is useful, these are the instructions (Windows only) I followed to flash my duemilanova 4 years ago (taken from Julien Bayle's site):
1. Download an install the D2XX drivers and the MProg application (in that order)
2. Connect your arduino to the PC and run MProg.
3. Select Device => Scan from the Menu. You should get a message like the following in the box bellow: Number Of Blank Devices = 0 Number Of Programmed Devices = 1
4. Select from the menu "Tools" => "Read and Parse".
5. At this point I saved the configuration I did "File" => "save as". This is not really needed, but I did it just in case I happen to screw up later on, to be able to recover the original configuration.
6. Check the “use fixed serial number” box and change the serial number value. Your board should have a serial with this shape: a40h-xxx (I'm using : a40h-001)
7. Save the configuration ("File" => "save as")
8. Once saved, program the FTDI by clicking on the flash icon (or doing "device" => "program").
9. Unplug / plug back your board from the usb port.
10. Download and install Arduinome serial.
11. Run Arduinomeserial, you should see a device with the name you gave your arduinoe in step 6.
I can't remember if it is necessary to upload the firmware on the Arduino again after this step. If so, you can select your board using its new name and upload like normal using the Arduino IDE. If the Arduino doesn't show up, try unplugging it and plugging it back in.
In case this is useful, these are the instructions (Windows only) I followed to flash my duemilanova 4 years ago (taken from Julien Bayle's site):
1. Download an install the D2XX drivers and the MProg application (in that order)
2. Connect your arduino to the PC and run MProg.
3. Select Device => Scan from the Menu. You should get a message like the following in the box bellow: Number Of Blank Devices = 0 Number Of Programmed Devices = 1
4. Select from the menu "Tools" => "Read and Parse".
5. At this point I saved the configuration I did "File" => "save as". This is not really needed, but I did it just in case I happen to screw up later on, to be able to recover the original configuration.
6. Check the “use fixed serial number” box and change the serial number value. Your board should have a serial with this shape: a40h-xxx (I'm using : a40h-001)
7. Save the configuration ("File" => "save as")
8. Once saved, program the FTDI by clicking on the flash icon (or doing "device" => "program").
9. Unplug / plug back your board from the usb port.
10. Download and install Arduinome serial.
11. Run Arduinomeserial, you should see a device with the name you gave your arduinoe in step 6.
I can't remember if it is necessary to upload the firmware on the Arduino again after this step. If so, you can select your board using its new name and upload like normal using the Arduino IDE. If the Arduino doesn't show up, try unplugging it and plugging it back in.
Arduinome Serial
Arduinome Serial is an application that packages the messages coming from your touchscreen into OSC, so that can be sent into MaxMSP. It also receives messages from MaxMSP and packages them for Arduino.
Download Arduinome Serial and plug in your Arduino. You should see Arduinome Serial recognize your Arduino with the new name you've given it. If it doesn't connect right away, make sure you have the Arduino Software closed, this can interfere with the connection.
Download Arduinome Serial and plug in your Arduino. You should see Arduinome Serial recognize your Arduino with the new name you've given it. If it doesn't connect right away, make sure you have the Arduino Software closed, this can interfere with the connection.
MaxMSP
MaxMSP is a graphical programming environment for running and writing interactive audio/video/graphics applications; it's what you'll be using to run apps on your touchscreen. Max is really fun and easy to use, I wrote a series of Instructables about how to write your own MIDI and audio applications in Max here, I even have a special section about how to write applications for monome here.
Download MaxMSP (or at least MaxMSP Runtime) and grab some monome patches (applications in Max are called "patches") that seem interesting. Start with the "monome_test" patch from monome base to make sure everything is wired up correctly. If it is, try out boiingg (because it's easy to get started with) and then go from there.
In all the Max patches for monome, you'll find a grey button that says "sys/prefix /XXXX" (where XXXX is the name of the patch), be sure to click that button so that it sets the correct prefix in the "Address Pattern Prefix" field in Arduinome Serial. This lets Arduinome Serial know the correct prefix to put on all the button press events it's receiving from your touchscreen before it sends them into Max, each monome patch only responds to messages with the corresponding prefix. This seems inconvenient, but it lets you have many patches open on your comp at once, while having control over which one is receiving messages at a time.
Download MaxMSP (or at least MaxMSP Runtime) and grab some monome patches (applications in Max are called "patches") that seem interesting. Start with the "monome_test" patch from monome base to make sure everything is wired up correctly. If it is, try out boiingg (because it's easy to get started with) and then go from there.
In all the Max patches for monome, you'll find a grey button that says "sys/prefix /XXXX" (where XXXX is the name of the patch), be sure to click that button so that it sets the correct prefix in the "Address Pattern Prefix" field in Arduinome Serial. This lets Arduinome Serial know the correct prefix to put on all the button press events it's receiving from your touchscreen before it sends them into Max, each monome patch only responds to messages with the corresponding prefix. This seems inconvenient, but it lets you have many patches open on your comp at once, while having control over which one is receiving messages at a time.
Troubleshooting
You can rotate the orientation of your touchscreen in Arduinome Serial by selecting from the various options in the "cable orientation" dropdown.
One of the most common mistakes with projects like this is to wire your array backwards. If the touchscreen is working with the "monome_test" patch in monome base, but the position of the buttons/leds is flipped from what shows up in Max, make the edits below to your firmware and re-upload it on the Arduino. Remember - You will need to disconnect from Arduinome Serial (by quitting the application) each time you connect to the Arduino through the Arduino IDE. And then you will need to quit Arduino to connect to Arduinome Serial again. you may also have to disconnect the Arduino from your computer and plug it in again to reinitialize the connection. This is a pain, but once you get the firmware right, you won't have to do it again.
Firmware Edits - Find the following lines in the firmware and change all/none/some of these to true until everything is connected right:
boolean mirrorXLEDs = false;
boolean mirrorYLEDs = false;
boolean mirrorXButtons = false;
boolean mirrorYButtons = false;
One of the most common mistakes with projects like this is to wire your array backwards. If the touchscreen is working with the "monome_test" patch in monome base, but the position of the buttons/leds is flipped from what shows up in Max, make the edits below to your firmware and re-upload it on the Arduino. Remember - You will need to disconnect from Arduinome Serial (by quitting the application) each time you connect to the Arduino through the Arduino IDE. And then you will need to quit Arduino to connect to Arduinome Serial again. you may also have to disconnect the Arduino from your computer and plug it in again to reinitialize the connection. This is a pain, but once you get the firmware right, you won't have to do it again.
Firmware Edits - Find the following lines in the firmware and change all/none/some of these to true until everything is connected right:
boolean mirrorXLEDs = false;
boolean mirrorYLEDs = false;
boolean mirrorXButtons = false;
boolean mirrorYButtons = false;