Touch Screen Coffee Table DIY With 32" TV and Low Cost CCD Sensor

UPDATE 27/09/2017 : A new video is online. It explain everything you need to know to understand how to build your touch screen!!

UPDATE 04/04/2016 : Multipoint software + Emulating NES video/pictures + OPTION_7 Point and Click + speed test + OPTION video

UPDATE 03/24/2016 : CCD sensor reverse Engineering + Using 2 CCD sensors + Laser engraving vector files + Retropie playing

Hi and welcome to my third Instructable !

I have been reading about DIY touch screen table projects for a while now, and I find most of them very complicated. Often They imply using a videoprojector, reflection mirror and IR camera. The result gives you a complete multi touch screen solution, but you need some depth for the videoprojector to have a big screen. Not the best solution for a coffee table...

I made this tutorial to help people to build their own touch screen solution based on any flat screen TV. It uses light triangulation technology to read the coordinates. This method is quite cheap ( the priciest part is the TV), with a good quality image and you can make the table almost as small and as thin as your TV is.

Of course, nothing is perfect and with this solution it's not possible to build a 100% multi-touch screen and it's hard to cover the entire screen area. But depending on the way you make it you can have some interesting results. Plus, I am sure there are a lot of possibilities I haven't thought about to improve this.

This solution implies the use of light triangulation to read a very precise coordinate. Basically, it's like having a huge number of IR receptors with some IR LED. You turn on every LED one by one and read the sensors. When one receptor can't see any light, it means that an object is blocking the light. With enough LED and sensors you can get a much more precise solution.

But for this project, I was looking for something more suited for the reading. A CCD sensor, is in every scanner. In those devices, it's basically a black and white camera which reads only one line of pixels. To scan a color document, a RGB LED blink in the three colors and the CCD reads all of them for every line of the document. Then it can calculate the exact colors depending of the Red Green and Blue reaction.

Using a CCD sensor gives you access to 2700 light sensors on a 20cm line. Most people who take sensors and actuators from an old printer/scanners only take the RGB LED and the light conductor from this part, as we can see here. If you like recycling, you could examine the possibilities of CCD. Not having found many DIY projects on the internet using them, there are probably a lot of different ways with which you could use these sensors! In this tutorial I will also explain how to hack your own scanner.

For this touch table, I used one CCD sensor with 24 IR LEDs. The object is detected on a 740x380mm surface by a Teensy board (which works under Arduino environnement). This board turns the IR LED ON, reads the CCD and calculates the exact position of an object. Then it sends the coordinates as a mouse, a keyboard, a Python program or just as Serial on a Raspberry Pi which shows the interface through the HDMI on a 32" TV screen.

When I am working on a project, I like to build things that are the most cost efficient possible but still of good quality. So even those on a tight budget can afford to build their own. It's hard to give the exact amount of money spent, as many things are second hand or from the tip. Here is an approximation :

Electronic :

• Raspberry Pi (41$). I built my project with a Pi 2, but the B+ should also be fast enough.

• Teensy 3.X (20$). I love building projects with my Arduino UNO. But the specifications of this tiny Teensy board are really crazy! (32 bits cortex M4, 72MHz, 256K Flash, 64K RAM, 34 digital PIN...). I think it's very useful to keep one at home. If one of your projects needs more PIN, more memory, more speed or just needs to be smaller, you can easily switch as it's fully Arduino compatible! Just install the Arduino plug-in and keep developing! The only little thing you have to be careful of is the access to the button to reprogram. I had to wire an external button when the table was built and I still needed to calibrate and debug.

• The CCD sensor. You can easily get one from any old scanners which can be found everywhere :)

• LED IR (0.1$). I used some 950nm.

• The TV. It's hard to give a price, mine is a LG 32" that I bought second hand for 80$.

• Wire

• resistor

• Computer Fan. You can find some in most old computers, you don't need to buy one.

• Male and female headers

Some parts can be a bit difficult to solder, I recommend having a good soldering iron. At home I have been using a WS81 from Weller. A bit expensive to buy but it's precise and you can keep it for 20 years ;-)

The wooden box and table :

This part is gonna depend on your TV screen. This is for my 32" LG TV :

The TV box :

• 2 panels of MDF wood 10mm for the sides : 140 x 845mm
• 2 panels of MDF wood 10mm for the sides : 140 x 560mm
• 1 panel of MDF wood 10mm for the top : 950 x 650mm
• 1 panel of MDF wood 3mm above the top : 950 x 650mm
• 1 panel of Pine wood 18mm under the TV : 825 x 560mm
• 2 large Framing lumber supporting the TV : 37 x 54mm (300mm long)
• Small Framing lumber to ease the screw 15 x 20mm
• Screws

The Table under :

• 1 panel of MDF wood 18mm for the top of table : 670 x 350mm
• 1 panel of MDF wood 10mm for the layer under the table : 670 x 350mm
• A very large framing lumber for the table's feet 37 x 54mm, 280mm long(2 for each feet)
• Screws

Approximate cost of the wood 70$

I also used a Trotec laser cutting machine to engrave some characters on it and to cut several small parts of the wood. You can rent it for 25$ an hour from the Fablab next to my place. As I often help them on electronic projects I can sometime have it for free :-)

Touch detection :

• 1 Plexiglass 5mm for the support : 750 x 460mm
• 2 Plexiglass 2.5mm, 735 x 12mm
• 1 Plexiglass 2.5mm, 430 x 12mm (protecting the sensor)
• 1 Plexiglass 5mm, 430 x 12mm (with the LED on it)
• 8 mirrors 150 x 15mm (approximately as it's hand cut)
• To create a IR lighting paper filter I combined a 106 primary red paper filter with a 181 congo blue paper filter
• black masking tape

As Wikipedia is way better than me on explaining :

"A charge-coupled device (CCD) is a device for the movement of electrical charge, usually from within the device to an area where the charge can be manipulated, for example conversion into a digital value. This is achieved by "shifting" the signals between stages within the device one at a time. CCDs move charge between capacitive bins in the device, with the shift allowing for the transfer of charge between bins.

The CCD is a major piece of technology in digital imaging. In a CCD image sensor, pixels are represented by p-doped MOS capacitors. These capacitors are biased above the threshold for inversion when image acquisition begins, allowing the conversion of incoming photons into electron charges at the semiconductor-oxide interface; the CCD is then used to read out these charges. Although CCDs are not the only technology to allow for light detection, CCD image sensors are widely used in professional, medical, and scientific applications where high-quality image data is required. In applications with less exacting quality demands, such as consumer and professional digital cameras, active pixel sensors (CMOS) are generally used; the large quality advantage CCDs enjoyed early on has narrowed over time."

Basically, the CCD you find in a scanner doesn't read lights in two dimensions, but on one line. With the one I tried, there are 2700 different light detectors that we can use as 2700 pixels on a X position.

As I couldn't find any tutorials explaining how they work on the internet, I tried to understand the logic by myself. I started by taking a printer scanner from the trash and opening it. Please be very careful with these devices, they use 220V! Don't do it if you are not sure of what you are doing. Always manipulate them when it's not plugged in and isolate any high power part.

By visually checking all the lines going from the control board to the CCD you can try to identify the 4 lines controlling the RGB LED, they are generally located at the extremity of the CCD board. With a multimeter, you can check where those lines are going on the communication flex.

Then you can check if some lines are bigger than others (it's probably going to be the power and the ground, you can verify which one it is with your oscilloscope when the power is ON). Plug an oscilloscope between the other lines and the ground to understand what they are doing.

I tried two CCD sensors from different brands and they where both using the same communication protocol but not on the same "line order on the flex". When I turned the scanner ON, I could see some data passing. It means that the device was probably testing the sensor when it is turned ON.

- One line was giving a perfect square signal, I deduced that it was a signal clock.

- One signal was turning ON and OFF at the beginning of the clock transmission, it's the Latch signal

- one signal was sending a lot of analog value. Those values changed when my hand was next to the sensor. It is the values from the CCD.

So I made a test program the on a Arduino UNO board. You can download it at the end of the step

With this, you can read on the serial port 16 average values (I call them pixels) of 168 light sensor on the sensor, so almost the 2700 total. Just plug in the lines as explained at the beginning of the above file on any Arduino board and read the data on the serial monitor.

Last week, I was trying to find some solutions to improve the cover of the detection area. To do this, I wanted to have a second sensor to test new configurations and speed. So I went back to the tip and I took several printer/scanner from different brands.

I was a bit frustrating to discover that some scanner have a different kind of CCD sensor. No need to open the entire device, they are pretty easy to recognize, as the black plastic package is way bigger (picture 1). Inside, they have a way more complex mirror architecture, a lens and... a very tiny CCD sensor! I will try to hack those one to see the possibilities soon, but for now, I needed another "normal" 22cm CCD. So I moved on!

Second printer I opened was from EP*** brand (Picture 2). The flex connection had 12 lines on 5V :

• 1 Analog OUTPUT to 4 are big lines easy to identify for the LED
• 2 (large) VCC? 5 (large) seem to be VCC
• 3 GND 6 GND
• 4 (thin) has a square signal with a 100ns period, the clock!
• 5 (thin) has a short square signal every 20us.
• 6 (thin) has a short square signal every 20us, another latch?? the latch.
• 7 (GND)
• 8 VCC
• 9-12 RGB LED

After few hours of test, I couldn't find the good protocol to have the OUTPUT response. Even when the CCD was plugged in the scanner, the analog wasn't giving any clear response (on startup test as the printer was then reporting an error).

So I moved on to a third printer from H* brand. The flex had 14 lines on 3.3V :

• 1 The analog
• 2 VCC
• 3 GND but not linked directly to the other GND
• 4 VCC but not linked directly to the other VCC
• 5 A latch every 40us
• 6 GND
• 7 ?
• 8 GND
• 9 CLK (clocking every 500ns)
• 10 VCC
• 11 - 14 RGB LED

I could access some PAD from the different line (picture 3). After a bit of testing I succeed to have a clean answer on the analog line (pictures and explanations 4 and 5).

The CCD I was using since the beginning is from CA*** brand. Here are the lines :

• 1 Analog
• 2-3 GND
• 4 VCC
• 5 ?
• 6 Latch
• 7 Clock
• 8 ?
• 9 - 12 RGB LED

I had now to build a little board to be able to control the two CCD with the same latch and clock signal.

The connection for this board are :

1. The Latch (orange) (common for the 2 CCD)
2. The Clock (yellow) (common for the 2 CCD)
3. Analog (green) of the second CCD
4. Analog (blue) of the first CCD
5. GND (green)
6. VCC (blue) (3.3V)

So I send the Latch signal, then I read the two Analog at every clock period. Most of the pictures and video have only one sensor, it's because I made them before upgrading the table :s

Now we have a working CCD sensor with Arduino, we can start building our first touch screen:

You can take the glass from the scanner to use it as the surface. On one side you add the sensor and on the other side a IR LED. Power the LED (don't forget to add a resistor depending on the spec!) and start reading the sensor, you will see that a part of the serial data will raise. On a 10 bit ADC like the one on Arduino, the range value is 0 to 2^10=1024. On my test, my value is always between 550 to 980.

Two important things :

• Any other light can disturb this experience, especially sunlight! So close your curtains, don't point any light directly to the sensor and ideally, add some black tape around your sensor.

• Most LED have a very small emitting angle (around 20°). Don't be to surprised if only a part of your sensor sees the light, it just means that the LED is too close!

To visualize this I made a little processing program that you can download at the bottom of this step. This program shows 20 different cluster of the CCD sensor. If the average analog value of the zone is close to 1024, the cluster is white. If it is lower it gets darker. As Processing needs to communicate with "Serial.write" data and not "Serial.print" like the Arduino Serial port, you need to upload this new Arduino code on your board.

Now, if you put your finger between the LED and the sensor, you should see the cluster getting darker. Congratulations! You have made your first touch interface!! Now you can detect the area where your finger is, not very precise, but functional!

To improve the precision of the detection we are going to have to complexify the system by adding some LED on different digital ports of the board. As you can see on the pictures, I now have 15 LEDs (choose as many as you want) wired in a plastic brown sticks. The distance between each LED is 5cm so the total distance is 70cm.

Now, how are we going to detect a very precise position?

1. we need to check if the system is detecting something. The method is gonna be :

• Turn ON LED 1,

• Read the sensor and check if something is blocking the light

• If something is in the way store the pixels above calibration in a table

• Turn OFF LED1

• Turn ON LED 2

• Read the sensor and check if something is blocking the light

• if something is in the way store the pixels above calibration in a table

• Turn OFF LED2

• ...(do this for every LED)

Now we have stored all the areas detecting something.

2. calculate the function

I know school is often far away and mathematics didn't seem very useful at the time. But do you remember the basic function Y = AX + B ?

Here we are going to have a full application of this. We are going to calculate the function crossing the LED and the middle of the area the CCD sensor detected.

To simplify, we are going to set the XY origin of the graphic on the first LED called LED 1. We can then calculate B for every LED (in mm) :

LED 1 = 0, LED 2 = 50 (50mm distance), LED 3 = 100, LED 4 = 150...

The A is a bit more tricky, if Y = AX + B then A = (Y - B)/X

If we take the average of the pixels detected (in the program it's "(maxi + mini)/2") we can calculate the exact height (Y). We also need to transform the pixels in dimension.

(NB_PIXELS-(maxi+mini)/2)*CCD_HEIGHT)/NB_PIXELS+(SCREEN_HEIGHT/2)-(CCD_HEIGHT/2)

and as we have Y, B and X we calculate A :

A = (Y - B)/X

Now we have A and B of all the lines going from every LED to the middle of the area they can't see on the sensor. We have function1 : Y = A1 X + B1 and Y = A2 X + B2

We can then calculate the crossing of those functions :

The coordinate X and Y of the crossing will be :

X = (B2 - B1) / (A1 - A2)

Y = (((A1*X) + B1) + ((A2*X) + B2)) / 2

The X and Y are the crossing coordinates. If you calculate all the functions crossing point and you average them you should have a good approximation of the central position of the object.The multi-point can't work with only averaging, it need a more complex algorithm (in step 9).

As we can see on the picture, this method is not really perfect. With 15 LEDs on one side, you can see on picture 1 there are still many areas we cannot detect.

So how are we going to make the detection area bigger?

• The easiest solution would be to add some LED which point to the sensors. This solution, which you can see on picture 2, would cover the entire central area but not the one closest to the LED. It's not that good as It would need a lot of LEDs, which would make our system go slower.

• The non detected areas are on the sides. An interesting solution could be to improve the distance between the LED and the sensor from the screen. This would solve part of the problem in the non-detected areas as you can see on picture 3. But the further the LEDs are from the sensors, the more interference we are going to have.

• After a bit of testing, I decided to add some reflective mirrors on the two sides. With this solution, if you add just a few LEDs which point to the side through the mirror you can cover a wide area. As you can see on picture 4.

• We could imagine a mixed solution like on the picture 5

• Another solution could be a combination of the three previous points. The sensors and LEDs are a bit far away from the screen, some mirrors cover most of the UP and DOWN sides. It's close to the picture 6. You can probably add some LED to cover the corners like on picture 5. Still not perfect, but almost entirely covered!

Using 2 CCD sensors

My new solution was to add a second CCD sensor (picture 7). Now, the entire screen is almost covered! But as two sensors is slowing down the system, I can play with the number of LED I use, and with the number of pixels. This way I am quiet flexible on the cover, the precision and the speed I want!

For example, if I need to emulate a keyboard for a Nintendo Nes, I only need 8 buttons. I can split my screen in 8 parts and detect if something is on this zone. Only 30 pixels (15 par sensor) and 8 LEDs is way enough to do that and I can reach around 40Hz speed.

If I want to emulate a mouse I need access to the entire screen with some precision. So I ll use 100 pixels (or more) and 16 LEDs. The speed will drop down to 15Hz but I can click where I want!

On the video, I show the Processing program running on the Raspberry Pi. I only used one CCD as it was before the 2 CCD upgrade, but the position still takes some time to synchronize. This problem is only because Processing is a heavy program, so the Raspberry Pi is struggling to refresh and synchronize the screen and the serial port fast. I had to add a delay on the Teensy program to slow down the communication. If you try the Processing GUI on a more powerful computer, it will be perfectly synchronize.

As my LEDs don't have a big enough angle to cover the two sensors, I added a second LEDs in parallel pointing to another direction. I just had to adapt the resistor to have twice the power, and it's done!

As I was saying, the multi-point cannot be perfect using a light triangulation technology with a CCD sensor. It is pretty complex to work on this part... But it is also very fun to imagine many different solutions on how to have the best results without adding problems.

Here is how I made mine :

• First, I store all the value of the CCD in tab[][] for every LEDs

• Then I calculate all the factor A and B of affine functions Y = A * X + B going from every LED to the center of the "black zones" that the sensor don't see (in red on the picture).

• I selected the first function of the first LED (lets call it N) and calculated the crossing with the first function of the next LED (N+1). This is my first reference point. The very important step here is to remove those two functions by decrementing k in tab_position[LED][k][Black_zone] (the tab where the B and A are stored).

• Now I just check if the first function of every next LED (N+LED_LEFT) are going close to this point. If they are, I adapt the position of the reference point and remove them as well.

• When I checked all the other LEDs, I just start over for the second point. I can perfectly get the position of two LEDs or more (like on the second picture with 4 points) as long as several black zones detection for a LED aren't detecting two points in the same time. Their is still some improving work to do on how to use those "big black zone" detecting two points. In my code I choose to ignore them as I could.

This algorithm isn't perfect, but it allow me to use some multi-point for games, as I can choose where the detection points are (see the NES remote on step "Final program, let's play!"). I will keep improving it, and hope some people will advice and help me as well :-)

On the Fritzing picture, you can see how everything is connected.

A good thing about using a Teensy Board is you have 34 digital Input/Output. So we have enough free digital PIN to control 30 LEDs without the need of external LED drivers. As only one LED is on at a time, we don't need much power either.

In this project we are going to use the following Pin :

• RX/TX (digital Pin 0 - 1) for the serial communication

• CCD sensor control (digital Pin 2 - 3)

• CCD sensor read on analog (analog Pin 9)

• direct LED control (digital Pin 6 - 22)

I now have two LEDs instead of one for every PIN (to improve the angle for two sensors). The LEDs are powered on their Anode, which means you have to set your Pin Low to turn the LED ON. A 110ohm resistor is wired on the power line to avoid sending to much power on the LEDs.

I have already talked about the great hardware specifications of the Teensy board. But combined with the Arduino software environment, the Teensy also offers some great possibilities. But first let's install the plug-in

Now you should see a category "Teensy" when you choose your board. And if you select it, you have access to several other options like the "USB Serial Type" to instantly transform your board into a USB keyboard or mouse (some possibilities can be found on Arduino Leonardo). The USB communication type is a "Full Speed" which means it can communicate at 12Mb/s. You can also change the speed of the board (on this project I used the overclock at 92Mhz).

For now, all of my Arduino UNO codes have been successfully built on the Teensy which means the adaptation between the ARM Cortex M4 and the AVR Atmega328 have been very well made. I tried to write the code to be understandable. Here is the structure of the main loop :

• A for loop from 0 to the number of LED

- Turn an LED on (as the LEDs are wired with the Anode on VCC and Cathode on the PIN, you have to give a LOW state to turn it ON)

- ClearCCD(), The sensor has to be cleared once before reading the value... I don't know why. If anyone has an idea of how I could speed up this part, I would very much appreciate it.

- readCCD(), read the sensor, if a value is above the calibration it is stored and the function return 1. If something have been detected, the flag_detect is set to 1.

- Turn the LED off (HIGH state for OFF in this configuration)

• If something has been detected, then the flag_detect is set to 1 and you enter the IF function. You can then choose your calibration between the following :

- OPTION 1; send Serial monitor.

Basic debugging : This option allows you to read the 16 clusters directly on the serial monitor.

- OPTION 2; calculate position + send Serial monitor

Debugging position : send the coordinates X and Y to the serial monitor

- OPTION 3; send Processing interface

Debugging visual : look at the CCD results on a Processing GUI. The interface is going to have to be adapted to the number of LED (16 default).

- OPTION 4; calculate position + send Python Pi interface

Send the position to the PyQT interface

- OPTION 5; calculate position + send mouse (only on Teensy)

Send the position as a USB mouse interface. To use this option you will have to go to "Tools" -> "USB Type" -> "Keyboard + Mouse + Joystick". (See the Warning at the bottom of this Step)

- OPTION 6; calculate position + send Keyboard (only on Teensy)

Send the position as a USB keyboard interface. To use this option you will have to go to "Tools" -> "USB Type" -> "Keyboard + Mouse + Joystick".

- OPTION 7; Point and click. Same as mouse but click instead of following the position

• You can eventually add a delay at the end of the loop(). Depending on the speed you need

WARNING: OPTION 5 uses the command mouse.mouveTo(X, Y) which doesn't work on a Linux station. If you want to do the same configuration as me with a Linux Raspberry Pi you are going to need to use the UGLY HACK made by Paul Stoffregen who seems to be one of the developers of the Teensy Project. He is really good and he helps a lot with debugging if you have any questions. Thank you Paul!

For this hack, you need to modify a part of the file name usb_desc.c It should be installed in your Arduino folder. The new configuration given by Paul is slightly different but he makes it work! By then, the mouse.move() function is not working anymore.

On the Raspberry Pi side, there are many different ways to show some data with a GUI. I choose to write a basic interface with PyQT following a friend's advice. Now, the interface has a full screen picture, read the serial datas and write them on a little panel on the left of the screen and draws a cursor following the serial position received.

The program run on Python3.2 and Pyqt4 on the Raspberry Pi. You can easily find many tutorial to install them and run a program.

As Raspberry Pi had made a good release on installing and running Processing on the board. I also decided to adapt my debugging code to show it on the big screen, it's very slow but but great for debugging and improving the Teensy code. To do this, you will have to rotate the screen 90° (If your sensors and LEDs are on both sides like me).

• To do this, you will have to modify the startup file config.txt. Here is how you do it :

• Open the Terminal (or use the one on startup)

• Access to the file : sudo nano /boot/config.txt

• Then go at the bottom and add : display_rotate=1

• Then : ctrl X

• And y to say yes on saving the program.

• Restart you Raspberry Pi : sudo reboot

Now your screen is rotated, you can play with the number to have different rotation (2 is 180°, 3 is 270°, 0x10000 for horizontal flipn 0x20000 for vertical flip)

Another tricky part was building the detection area. I didn't know which material to choose so I started with glass panels. But it was too fragile to manipulate and cut so I moved to plexiglass.

• A plexiglass panel of 420x720, 5mm thick carried most of the detection parts (except the sensors and the Teensy because I wanted them to be lower).

• From this first panel I cut a line 12mm height. I made some regular holes to insert the LED.

• Then I bought another plexiglass panel 2.5mm thick and I cut it into a 12 mm line to make the other borders.

• I fixed some black tape around the CCD sensor to isolate it from external light and set it at the good height.

• For the reflective mirror, the shop next to my place couldn't cut smaller than 100mm. So I bought some 100x300 panel and a glass cutting knife. On internet there are a lot of videos to learn how to cut glass. Still it's not very easy and I broke many... it's easier to buy smaller mirrors (like 100x150), but don't forget to buy more than you need, as it's hard not to break any!

When everything is fixed and working. You can add some transparent glue above the surrounding plexiglass lines and around the LED to create an isolation if any liquid falls on it.

The wooden enclosure depends exclusively on the TV you have. I will only give the basic specification for mine, but I think the pictures are going to explain it better.

• I used some 18mm thick Pine Wood to support the weight of the TV. I made several holes around the ventilation area of the TV.
• Some large framing lumber to support the two sides of the TV so the back is not in direct contact with the wood panel behind.
• Some small framing lumber all around to fix the decorative MDF around it
• Some laser engravings on the surrounding 10mm MDF panels adding Pacman, Mario, Toad, the opening for the Raspberry Pi USB, the fan and the infrared hole of the TV.
• The box is closed with one 10mm panel and one 3mm panel of MDF with a big square hole the size of the touch screen.
• I also added some brown varnish to give a more natural wooden color to the MDF

The little table under is made with :

• A 18mm panel MDF on the top to support the structure.
• 4 feets, each of it made with two large framing lumber screwed together.
• A layer made with a 10mm panel of MDF to store the remote of the TV and a multi-socket powering TV and Raspberry Pi

In this step, I added some of the illustrator files I used for the engraving.

For the others, they are from this awesome website. It's a really good database of Adobe Illustrator Vector files of games (like Zelda, Mario, Mortal Combat or Street Fighter...), movies (Star Wars, Despicable Me), Comics (Superman, Batman, Deadpool...) and many other categories!

Now we have everything fitting together, let's try a few games :)

With the keyboard option of the Teensy program you can emulate any key based game. I made few test with the PyGame already on the Debian Pi like the game squirrel.py. Then, I installed Retropie on a second SD card and tried playing with few games.As the engraving of the table, I have the games Mario, Zelda, Star Wars and Pac-Man (and others!)

But my keyboard code wasn't adapted to emulate a gaming keyboard so I had to make few modifications on pressing and holding the keys. If you use the normal program, the frequency of sending the key is not high enough, so Mario is gonna move very slow, like if you where just pressing the button very quickly instead of holding it.

My new keyboard game code hold the position of the button while it's reading the next one. With this configuration the movement of Mario are smooth :-) . I also implemented the multi-point as you need to be able to move and jump in the same time.

As we can easily access the usb port, it's also possible to plug a controller in and to emulate some games without a touch screen.

Still a lot of thing to do, it's hard to work on everything in the same time :-)

• Now I made the multi-touch work, I can't stop thinking on how to improve the algorithm. But it's quiet a complex part for me :p

• With the second CCD sensor, the screen is almost entirely covered. I still want to add more LED to create a slow mode but with a 100% screen cover.

• I added a small button on the Teensy to be able to change mode. But we can't switch from a keyboard to a mouse or to serial. I can only change the different serial mode (like processing/python or debugging). Maybe Pymouse could be an interesting way to improve.

• Maybe adding several sensor to use it as a monitor of the apartment. We can find some very interesting project on this area and I could add a nice Python GUI as the link between Teensy and Raspberry Pi isn't very useful right now.

I hoped you all enjoyed this Instructable and that it will inspire many other projects using CCD sensors or coffee tables!

As you can see in the last step, there are still many ways to improve it... If you have any ideas or any questions, I would be happy to work with other people on making it more reliable, efficient and fun! ;-)