Creating Your First Multi-touch Table
by Jamaal_7lna in Circuits > Computers
538473 Views, 3591 Favorites, 0 Comments
Creating Your First Multi-touch Table
As with every project, here is a bit of back-story:
When I was in my freshman year of high school, I was invited to attend an engineering "field day" taking place in Alabama's Redstone Arsenal. Although most of what we were shown during this trip were projects that I am not able to speak about, one of the more relaxed projects was the Multi-touch table. I am not allowed to say what the table was being used for--I've forgotten either way--but I can speak about its features. The table shown to us was a 30" Microsoft Pixelsense (then known as the Surface, before the tablet took its name) and to me, it was glorious. As a connoisseur of Natural User Interfaces (NUI), I'd always been fascinated by 3D spatial projections ( what most people call holograms) and anything of the like. I had never seen of these before so it would be easy for anyone to imagine my excitement. I asked the Project manager about what it would take to work with one of these and how I could be involved. You could imagine my disappointment when he told me that although it was simple, it wasn't anything that I could ever make by myself. This very conversation sparked a 5 year personal vendetta against that one employee that led to the creation of this instructable--as a hobbyist. I enrolled in optic courses and searched through every crevice of the Web until I had enough information to build my own table. I had two very important personal objectives:
1) Educational: To gain experience in NUI technologies and help spread this knowledge to students willing to learn via independent projects. I planned to stimulate interest with the introduction of a hobbyist-made Multi-touch table in order to engage myself deeper into NUI technologies
2) To create a product that was better than the table I was shown in my early high school years: bigger, faster, cheaper, and more efficient.
Thus, I can now say that the goal has been met. Now I can share this knowledge with all of you.
Also, a great amount of credit shall go to these mentioned organizations as they facilitated my goal. Without them, this would have taken much longer!
- NUI group--this collective of hobbyists and professionals dealing with NUI technologies shall gain the bulk of my credit. They taught me everything I needed to theoretically build my project and assisted me through every setback I encountered. They are also responsible for teaching me--above all things--how to conduct my own research. I strongly advise anyone who reads this to visit their website and forums.
- Ideum-- who would have thought that an established company would be so eager to lend a hand to an enthusiastic student? I literally called the hardware specialist at the time and asked him a bunch of questions, and he replied to each with informative tips. This company also led me to discover the NUI group and pushed me toward making my project bigger. If you get a chance, check them out too. If anything, they are a great source for inspiration.
**Note: I advise you to finish reading the guide at least once before starting.
**Disclaimer: I am not to be held in any way responsible for the damage you may inflict upon your equipment nor to yourself.
Notes on reading this guide:
- This project was conducted with 3 other friends, and the table is currently housed in one of the member's college dorms. As I am writing this alone and since I conducted all of the planning and designing, I will refer to myself when I speak of things that are design related. I may also refer to the group as a collective in the article, hence my using of "we" on the occasion.
- This will be a very long instructable. My aim is to make the most detailed instructable in order to increase interest in NUI tech, but above all, for you to understand all that you are doing as you are doing it.
- You may PM me for clarifications if needed.
- I may use the word "warez" if referencing to the collective software as a plural noun.
- Each section is pre-labeled in the title, allowing for your easy navigation. "Information" refers to my explaining theories, plans, mathematical or physical concepts, tools, etc. "Hardware" refers to sections where I outline the steps to take in actually building the hardware for the table. Finally, "software" or "warez" refers to sections where I explain how to set up software.
- Kareem Garriga (UAHuntsville)- Project Leader, Designer, Planner, Writer, Researcher, Photographer.
- Joshua Deaton (UAHuntsville)- Software Guru, Hardware help/ construction engineer, room provider.
- Jarrett Turman (UAHuntsville)- Hardware engineer, co-designer, Web blog manager.
the Planning (Information)
- 1. Vizio E471VLE LCD TV................................................ ~$600
- Target Purchase
- 2. Wood, screws, tools, and brackets.............................. ~$150
- Home Depot Purchase
- 3. Extra screws from home depot.................................... $3.82
- 4. Sample 2-way mirror.................................................... $10.80 (not needed at all)
- Wanted to experiment with skipping the use of a diffuser.
- 5. LED power supplies..................................................... $134.80
- They will come recommended once you choose your appropriate LED reel
- 6. Acrylic.......................................................................... $492.98
- i. Evonik Industries ACRYLITE LED (Endlighten), sheet, Colorless 0E012 XL
- ii. Evonik Industries ACRYLITE Abrasion Resistant, sheet, Colorless 0A000 MR2
- iii. Evonik Industries ACRYLITE FF (extruded), sheet, Colorless 0A000 GT
- 7. LEDs............................................................................ $216.95
- LED Lights World Flexible IR Tri-Chip LED Strip (We purchased the 300 LEDs in a 5 meter reel)
- LED Lights World Flexible White (we got the reel with 150 LEDs)
- 8. Cameras...................................................................... $239.25
- (2) Peau Productions Developer Camera (m12) w/ 850 nm Bandpass Filter and 2.5 mm lens.
- USB extension cable
- 9. Computer..................................................................... $0.00 (used my friend's)
We are going to be building a 47" Planar Infrared-based Multi-touch table using the Diffused Surface Illumination (DSI) Technique. This technique is not recommended if you aim to read objects (that are not fingers)--also known as fiducials--unless you plan on not using the diffusive panel of the LCD. It is also not the best to implement when using an LCD, but it is the most convenient due to requiring less materials and work. I chose this one because of the convenience.
Theory:
By Diffused Surface Illumination (DSI), we place Infrared (IR) Light-Emitting Diodes (LEDs) around the edges of a special type of acrylic known as Endlighten Acrylic. This acrylic is manufactured with special properties akin to having microscopic mirrors that evenly distribute the light across the surface of the acrylic. When your finger touches the surface, it creates a "blob" on the surface--it interrupts the evenness of the light. This blob is then to be read by the specialized camera and recorded as a "touch event".
Comparison to PixelSense:
The first PixelSense used a different technique called Rear-Diffused Illumination ( Rear DI). By Rear DI, the maker is required to use IR illuminators on the underside of the screen. Because of this, it is nearly impossible to create using LCDs due to the IR light reflecting from the back panel. Thus, people will usually use a short throw projector (usually more expensive if you want HD quality). All in all, DSI allows for higher visual quality at a reduced price. Input detection remains the same.
Why make one of these when you have a smartphone?
Typically, smartphones and tablets use either capacitive, resistive, or surface acoustic wave screens in order to detect user touch points. Capacitive uses the electrical properties of human bodies in order to detect user inputs at certain points (the skin of your fingertips). Thus, devices using this screen are highly sensitive and neglect pressure--meaning that users cannot use regular styluses or gloved hands. In addition to this, the cost for capacitive layering on a large scale project would be tremendous.
Resistive screens simply rely on pressure as they are built by layering screens. For resistive, there is an inconvenience in that building this setup would require the user to create layering patterns with little promise of multi-input detection.
In Surface Acoustic Wave (SAW) screens, when a panel is touched, the ultrasonic waves exerted in the panels are absorbed at those points and this registers a position for touch events to the controller for processing. This would be burdensome if applied to larger screens.
However, infrared technology holds the advantage of being able to deliver superior images without an overlay while granting the ability to detect most objects. Also, the ability to use fiducial markers (programmable blocks). Also, they tend to have the higher number of Maximum touch inputs.
What is the first thing to worry about after I have my design plan ready?
The first thing you should worry about is your LCD TV. You won't be able to use just any TV as there may be issues. I will explain this in more detail soon.
LCDs, the FFC Issue, and Before Ripping (Information)
The LCD is perhaps the most annoying hardware component regarding the setup of an LCD DSI table due to the FFC Issue--I will get to this soon. But first, a bit of insight: It is mildly important to understand how LCDs work for the purpose of explaining how the table works. Also, since we are taking one apart, it is important that we understand what each component does.
I. LCD theory
LCDs are an amazing piece of technology that remind us of how amazing and progressed our technological prowess has come. At the core, they rely on Liquid Crystals (LC). LCs are materials that are literally in a state between liquid and solid states of matter--this means that their particles maintain their orientation, yet they are able to move around to different positions akin to liquid state materials. Thus, they are highly sensitive to temperature changes ( they must be at an exact temperature to maintain their odd state). For LCD screens, LC particles are usually arranged in a twisted nematic phase. This only means that they are affected by electric current (nematic) and are naturally twisted. When we apply current to the LC substrate, the LCs untwist varying degrees depending on the voltage--which allows them to react predictably in controlling the passage of light. The orientation of the particles in the natural state depend on either a magnetic field or microscopic grooves. The change in orientation (for most LCDs) is described as either smetic c or chiral nematic. The smetic c orientation holds that the particles are layered in such a way that they tilt slightly in angles with each layer while the chiral orientation sees the molecules untwist slightly from layer to layer.
II. LCD creation
For the LCD screen construction there are 4 factors that are fundamental:
- Light can be Polarized
- LCs can transmit and change polarized light
- LC structure changes with current
- There are transparent substances that conduct electricity
There are many types of LCD screens such as Passive and Active Matrix LCDs, but since we are most likely only dealing with an Active Matrix, I will only explain this one. Active Matrix LCDs depend on Thin Film Transistors (TFT) which are essentially tiny switching transistors and capacitors. They are arranged in a matrix fashion on a glass substrate and to turn a pixel on, the row corresponding to that transistor is turned on and a charge is sent through that column. Since the other rows are off, only the capacitor at that row is receiving a charge--theoretically knowing that Capacitors can change current instantaneously, this is explained and we also factor that the charge can be held until the next refresh cycle. Controlling the voltage, one can untwist the LC enough to moderate the amount of light that passes. This allows for the creation of a gray-scale.
So how do we get color?
Each Pixel has 3 sub-pixels (red-blue-green{RGB}) and they each have their own transistor. each sub-pixel allows for 256 shades within their own gray-scale for a grand total of 16.8 million possible colors on your screen. this is true unless you have a fancy yellow pixel LCD display as we get the possible colors from 255^3 for the 3 sub-pixels.
How many transistors are we talking about?
Depends on the display. A 1024x768 TV has 1024 columns and 768 rows, so if we multiply that by 3 pixels, we get 2,359,296 TFTs etched onto the glass panel. If that doesn't impress you, wait until you see how thin the screen really is and what happens when you turn it on away from its casing!
Why should you care about all of this?
In building your table, you are going to have to make predictions and decisions. You won't always have a perfect display built. Sometimes, you're going to have to figure things out from the fundamentals in order to make sure you didn't just waste $$$$. Also, depending on your setup, you may have to predict the display size of your LCD based on the properties of the screen. More importantly, this is knowledge! so take it in!
III. Things I should worry about in regards to the TV I will use for the table:
- The FFC Issue as defined on the NUI Group Forums is when the FFC cables are either too small or fragile to be manipulated into building a Multi touch setup. The Flat Flex Cable is the cable that you will find connecting the LCD matrix to the Control boards. This can become a problem as they are hard to replace and if the get in the way of the construction, they may cause accidental damage. Fortunately, there are links to databases with TVs that do not have this problem and are perfect for Multi touch setups:
- Small LCDs: http://baseportal.com/cgi-bin/baseportal.pl?htx=/L...
- Large LCDs: http://wiki.nuigroup.com/Large_LCD_Monitors
- Alternatively, you could just stick with the Vizio as it is perfect
- The Aspect Ratio: You need to know your aspect ratio so that you can approximate the size of your LCD in order to create your chassis, or box. Most LCDs nowadays are 16:9 since this is the HDTV Standard. That is, 9 units of height for 16 units of width. Use similar triangles (yes, math) in order to determine the size of your LCD screen. Of course, this is only necessary in the case you do not want to rip the TV apart until you have everything else (Acrylic, wood, etc).
- Contrast Ratio: This is the measure between the brightest possible image and the darkest possible image. White/Black = Contrast Ratio. Unfortunately, there is no actual metric to standardize this so companies just exaggerate things such as 1000000000:1 (who knows what that means?) This matters only in terms of vanity--say you wanted the clearest possible image to be displayed on your LCD. An LCD with a high dynamic ratio is better than an LCD with a native that gives the same ratio; however, an LCD with a higher native ratio is superior. Native describes what the LCD is able to produce on its own and dynamic refers to using circuitry to assist the panel in accordance to ambient light.
What about glare when the LCD is laying down?
The LCD screen comes with a set viewing angle that ensures the best possible picture--most consumer don't know this and miss out on the full experience. A Bias angle is designed to offset this angle. Because of this, your angle of acceptable viewing is made much larger. There is still an area of unacceptable contrast, but you will most likely not reach it.
*You can learn a lot more about LCDs on HowStuffWorks.com:
Link: http://electronics.howstuffworks.com/lcd.htm
Steps in Determining Your LCD Screen Dimensions Before-hand (optional Math Information)
- The first thing to do in determining the LCD size is to find out what your aspect ratio is. The beautiful thing is that LCD retailers are required to release a specs sheet concerning the capabilities and basic information regarding the TV. In these, the most notable of specs is the aspect ratio--usually 5:4 or 16:9.
- Next, determine the size of your diagonal from these sheets. A display that is 47" isn't exactly 47" (it may actually be 46.85 or 46.96, which actually matters). Find this on your spec sheets and write it down.
- Next, use triangle-angle relationships to determine the angle opposite the horizontal and vertical measurement of your display using the aspect ratio.
- For example, a 16:9 ratio LCD will have 16 units of measure for the horizontal component and 9 for the vertical. The angle opposite is the given by cos (16/proper diagonal measure). For a television of a 46.96" diagonal, Cos(16/46.96") = 60.64224646 degrees, which makes sense.
- Next, setup a similar triangle diagram where the 16:9 is within the LCD. You are to find the X and Y components of the outer triangle. You use law of sines with the angle determined above to find this.
- For example, following our calculations, [Sin(60.642)/9] = [(x/46.96")/x] given that y = 9x/16.
- This will give you an approximation of 40.92918722/23.02266781 which falls under 16:9 ratio when reduced. This however, is not exact but fairly close.
Making a Box . . . ! (Hardware)
I. Things to take into consideration when building the box:
- Is your box the right size to hold your LCD screen? This is where aspect ratio comes into play. You need to make sure your layers won't fall through and that the LCD cables/components won't get in the way. Also, make sure you can make it mobile (it can fit through doors)
- Where will your computer be? You may not want to put the computer or the LCD components inside the box unless you are willing to include compartments on the underside for them as they will interrupt the image on the LCD and the camera's ability to track your fingers.
- Ventilation: The Cameras won't get too hot, but if you do decide to conceal the computer and LCD components (which does make it look nicer), you may want to invest on some ventilation. You can use a couple of in/outtake fans for this if you want.
- Are you accommodating for cabling? Make sure you have a plan in mind for neatly placing the cables in your table. One cable over the camera makes a significant amount of interference.
- Is your structure strong enough to hold the Screen layers? you have about 32-45 lbs of screen to hold and the additional weight due to people putting their hands on your table.
- A door? It may be a convenience to add a door so that you work with the LEDs and Cameras without having to remove your layers.
- Is your box closed? You should enclose your setup so that the white LED light do not escape your box
- Remember that Home Depot will cut the wood for you: You can get the exact amount you need and pay less. You must do this before checkout though.
II. Things we should discuss:
- Your LCD panel has a back plate that acts as a partially reflective mirror in a sense. On your TV, you will mostly likely find fluorescent tubes. These are known as the back-light and their sole purpose is to provide the base light that the LCD screen will use. The panel also contains the polarizers necessary for the LCs to control the light. We are going to replace this entirely with the LEDs. Why? The tubes are too weak to provide the lighting necessary for the LCD to work once the distance is increased--and you must increase the distance so that you may accommodate for your cameras. Also, the tubes are very fragile and risky. The white LEDs will be placed on the bottom of the box in order to make the screen show an image. Additionally, for this to work, they must be place around the cameras so that they are not in the way. the cameras should also be elevated so that you don't get too much light interference.
- You may want to first sketch your design or create a model using a CAD program. You can get Google Sketch-up and it will work wonders for free. It also has a built-in tutorial and gallery. You can find the Ideum MT50 table CAD file there too if you want a bit of inspiration.
- You MUST calculate the distance needed between your camera and screen in order to determine the size of your box. I used two cameras so that I could stitch the two feeds together and reduce my height by three times. Normally, this means having to calculate angles and ray diagrams. Thankfully, the NUI forums has a thread where Peau Productions created a lens calculator that may help simplify this. Please use this before purchasing your cameras or buying the wood. Also, consider looking through the projects on this forum.
- Consider putting the table on wheels. It will increase mobility and give your design an edge.
Ventilation Concepts and Laying the Support (Hardware)
Re-purposing Your Television--a Step by Step Guide (Hardware)
- a. Made sure your TV avoids the FFC issue
- b. Are comfortable enough to rip your TV apart.
- c. Understand that I am not to be held responsible for the damage you cause, nor the risks to your lives.
- d. Have read this section at least once.
- Of course, the first thing to do is turn off the power and remove the AC cable and gently lay the television screen-down on a flat and stable surface.
- With a screwdriver, unscrew the screws holding the back-plate in place and remove the back-plate. You should see something similar to the first picture above. Luckily, the Vizio I selected is very simple and doesn't have too many boards. In the case you didn't stick with the Vizio and opted for something else, here goes a breakdown of what you'll see:
- The Main Board: This is the most obvious piece you'll find. It is a circuit board that contains the audio and video inputs for the TV. You'll see the HDMI, USB, and S-video ports. This houses the CPU (on some TVs) and is also referred to as the Motherboard or the Main Module.
- Power Board: This is the odd looking circuit board. 98% of the time, it will be a brownish-yellow color. This one is the bulkiest/heaviest of the boards and houses a couple of capacitors, inductors, and heat-sinks. This is most obviously the power supply board as this is where the AC cord will connect to. Fun fact: Many times, people will replace a TV they believe is damaged when the only actual problem may be that a single Capacitor has blown. Seeing as how one of these may cost a couple of cents, it's funny how we'd pay over $400 for a $1 fix.
- Inverter Board: Long thin board along the edge of the display. Contains several transformers and performs logic operations regarding back-light and the display.
- T-Con Board: the Smallest board and is a Logic Board. Mostly contains the CPU (on some TVs) and processing chips.
- Speakers: you can either keep them or replace them with better ones. Choice is yours.
- LED for TV symbol: a simple LED that lights up as an indicator that the TV is on. You may want to keep this around.
- Circuit board for buttons: This is just a circuit board found on the side where the on/off, channel, volume, etc switches are. You may not need this.
- A set of long flat cables: these are the FFCs. The Vizio has 3 ft cables which is beyond perfect.
- Take a picture of everything. You have to remember each connection for each component so that it can reconnect.
- Safely remove all the components and unscrew the boards from the back. Remove the speakers and everything. The connectors are usually Zero Insertion Force (ZIF) connectors. Keep one end of both of the FFC cables connected to the Inverter Board.
- You will see a metal back-plate with holes. Through those holes, you will likely see white sheets. Those are the diffusers and we are trying to get to those. Unscrew the screws holding the plate together and remove the plate. To do this, we have to take the front plastic casing off so you'll have to flip the TV. Do this carefully as you do not want to risk damaging the LCD panel.
- Take the casing off and work your way unscrewing until you get to the LCD panel. Gently and safely remove the connectors to the panel. The green and brown connectors DO NOT come off. If it looks like it won't come off, it most likely won't. Be very careful here.
- Lay the LCD panel safely on a flat surface and wrap it with a blanket or something that will protect it. The panel is ~2 mm thick and thus, easily warped. Have a friend help you handle it so you distribute the weight evenly. Now think back to all that theory you read in the previous section and be amazed at how compact the panel is.
- Right under the LCD are the white diffusers. Take care as they are likely to be statically charged. The diffusers serve to evenly distribute the white light under the panel from the back-light. You may not need all three though. We opted to use the thickest one as the others did nothing for us. The case may be different for you. For now, just keep them until the testing of the LCD display.
- Do what you will with the back-light. If you opt for a size < 25" diagonal, you may actually make the factory light work and circumvent the cost of white LEDs. If not, do what you will with it--for the purpose of this table, it is no longer needed.
**Need more Pictures? This gentleman over at the NUI Forums was the one responsible for tipping us on the Vizio. He also did a tear down and included a couple of pictures worth taking a look at. I lost most of my pictures in transferring them from my phone so I used his as my first image on this section. Credit goes to him on that note.
Giving Life to Your Table (Hardware)
LED Setup (Hardware/Circuits)
I. Camera setup
- Assuming you calculated the height of your table based on your camera lens as I advised in the previous planning sections (or have prepared an appropriate platform for elevating the cameras to their needed height), place the cameras where they need to be. You need to perform the above calculations for centering your cameras as well.
- ****If you do not yet understand how to perform any of these equations, you can PM me and I will try to help. You simply use the FOV for the camera to determine a distance from the screen where the cameras can see portion desired from the screen. This should give you an idea of proper height and placement for the cameras.
- As pictured above, we used Duct Tape to keep the cameras steady. This is because we were still experimenting and making sure that our calculations were correct. The cameras I referenced you to from Peau Productions have certain sections for you to screw the in place when you are satisfied with their position. I should advise you not to do that until you have first configured all the software needed.
- Next, run the USB extensions to your computer. Make sure you keep the cables neatly pinned to the surface of the bottom so as to not have them interfere with the camera lens or the lights.
- We are concerned with the white LEDs here. Make sure you have the white LEDs on hand and not the IR.
- The reel is designed in a manner that you can cut the reel at certain points to make smaller sections. You can tell where these sections are because you can see a scissors symbol. Also, the symbols for positive (+) and GND (-) will meet their counterparts here.
- Determine your section size, cut, and paste to the bottom around the cameras as pictured above (the underside of the LEDs is a 3M adhesive).
- Solder the positive to the positive and the negative to the negative as trivial. The power adapter for the LEDs also needs to be connected to these in some way. You may solder it directly, buy a female jack, or just twist and tape.
Acrylic Setup/ IR Setup (Hardware)
I. Layering:
Now we will talk layering. This step is very important because depending on how you do this, your setup will yield differences in input detection, ghosting, and transparency. We must find a way to balance these three perfectly.
- ghosting refers to the "naturalness" of your interaction with the table. We say there is too much ghosting when the Acrylic layers are too thick or ordered in a way that your touch doesn't seem to feel right. If your touch layer is, for example, an inch from the projection layer, the effect of refraction will be heavier and you will feel as if your touches are off. This is a problem in that you won't get the full touchscreen experience.
- As far as input detection goes, what we are concerned with now is the positioning to yield maximum camera detection. There is a threshold where the layering is too off, prohibiting you from properly causing any photon collisions. When you press on the Acrylic, you force photons to collide and cause "explosions" in a way. Those explosions are termed and read as "blobs" by the software. If, for example, your resistant layer is too far from the camera or the IR layer, there will be no collisions read by the software.
- Transparency refers to the user's ability to see through the LCD. This can be used either creatively or it can be bad. The line between right and wrong is mainly blurred on this issue.
I will now share the setups we experimented with and allow you to pick the best choice:
*IR refers to Endlighten acrylic
*Support refers to extruded FF acrylic
- The original plan:
=======IR( 8-10 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (8 mm)________
.
.
.
. ~3 ft
.
.
.
_________|--camera--|___________
**Report: Heavy ghosting occurred due to the support being so thick. Also, the collisions in the IR wouldn't escape the 10 mm piece through the diffuse layer. Barely got blobs registered unless the the user increased pressure of touch (which could harm the integrity of the system) AND removed the protective layer. Also, the distance between the camera and layers was too great. Discovered that it works better if the camera distance was reduced, the thickness of the IR and support were reduced:
-------Protective (3 mm)----------
=======IR( 8 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (<8 (maybe 3) mm)________
. ~ 1-2 ft
_________|--camera--|___________
This seems to be what will work best.
2. The next setup worked due to the sturdiness of the box:
=======IR( 8 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
**********Diffuse( 1-2 mm)**********
_______Support (3-5 mm)________
.
.
.
.
.
~3ft
.
.
.
.
_________|--camera--|___________
***Report: Blobs were present but faint. some degree of pressure was needed to register input. Also, due to the sturdiness of the box, we did not have to worry about the layers warping. The initial distance tested was 3 ft. We assume that if the distance was decreased to 1.6 feet, the blobs would be read perfectly as the lens we purchased agreed to this mathematically. Only major problem would be the exclusion of the protective layer, yet the Endlighten fared to be unexpectedly durable so long as we occasionally wiped it down. I would recommend this setup.
3. Creative setup: This is my personal favorite:
=======IR( 8 mm)========
~~~~~~~LCD (2 mm)~~~~~~~~~
_______Support (3-5 mm)________
~1-2 ft
_________|--camera--|___________
**Report: Perfect blobs read. Additionally, decreasing the size from 3 ft to 1.6 ft would only make it better (allowing for us to place a protective layer). The downside is that since the diffuse layer has been removed, the transparency increases. We sacrifice transparency for perfect detection and virtually no ghosting. The transparency is not bad when the is no ambient light ( it is non intrusive). If the user wishes to use the table in an area with ambient light, this setup is not recommended as they will not see the screen performing at maximum quality. If the user is creative, they can show off the guts of the system and make it very presentable. You can attempt this:
-------Protective (3 mm)----------
=======IR( 8 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
_______Support (3 mm)________
. ~ 1-2 ft .
_________|--camera--|___________
The above is the example of the setup that we are moving toward at the moment for experimentation.
4. Setup that needs more exploring:
=======IR( 8-10 or less mm)==
~~~~~~~LCD (2 mm)~~~~~~~~~
. ~ 1-2 ft .
_________|--camera--|___________
**Report: Not recommended unless a support layer is added (LCDs are very fragile), but it is an interesting setup. received perfect blobs, no ghosting, yet great deal of transparency. Perhaps adding a layer of support and a thin diffuse layer will work.
III. Comments
A. The diffuser sheet we used was the thicker one that is packaged with the LCD as my teammate accidentally threw away the smaller ones. I am fairly certain that the thinner sheets would have worked best to reduce transparency in the case of the setups we built without diffuse layers.
B. I highly encourage the builder to read through other projects accomplished on the NUI group forums and formulate a design plan before starting the project.
C. I did not feel that I needed to thoroughly explain the re-assembly for the LCD panel onto the box. All that you have to do is place it down on top of your support layer and reconnect the appropriate FFC to the sockets on the T-con Board. It is important that you connect them to the sockets they correspond to. Also, make sure the sockets are clean and that the FFC are in correctly. If you place them backwards or incorrectly, you may see a column of dead pixels. This is nothing to be too alarmed about as you only need to fix the connection.
Did I Break It? Testing the TV (Hardware/Information)
I. Common troubleshoots:
- Twice-inverted letters and picture? Your LCD is backwards, simply flip the panel
- No picture? Check that the HDMI or video input of your choice is properly configured. Also, if you kept the LED indicator, check it to tell whether the TV should be on or not. you will notice that the range of the TV remote just increased by 7 times. You are welcome.
- Screen is black? Turn on white LEDs. If LEDs are on, make sure they are white and not IR. If still no picture, your Flat Flex Cables are connected in the wrong order.
- No sound? your speakers are connected on the wrong socket.
Understanding the Software (Information)
I. How the tracking software works:
The algorithm for tracking and maintaining touch events is a bit complex, yet very fascinating. Keep in mind that although Jeff Han pioneered the use of IR technology in his Frustrated Total Internal Reflection (FTIR) multi-touch displays just recently, the software techniques have been used for a long time.
The process involves a set of equations:
- The state equation X_k = X_(k-1) + VU_k +W_k
- Where X_k describes the propagation of the state variable for position X, X_(k-1) is the true position at time = k-1 of the event, W_k is a noise term, and VU_k is a velocity term of constant velocity V.
- The observation equation Y_k = X_k + n_k
- where n_k is the zero-mean Gaussian white noise.
* You can learn more about this on the NUI group's publication Multitouch Technologies available for free on the NUI group's forums or above as a .pdf file. This is a short two chapter book aimed to educate beginners before they risk it all in attempting to construct a multi-touch display. The full explanation of the above equation is included in this book as well.
Downloads
Choosing an OS (Information)
I. OS Breakdown:
- Microsoft's Windows
- Windows 8/ Windows 8.1/RT: Windows 8, RT, and 8.1 were optimized for the touchscreen apparatus. The metro offers a rich experience that feels like it was created specifically for a multi-touch device. Also, the corner sliding and huge amount of touchscreen apps already available through the windows store offer this OS a great advantage. We personally chose this OS because it was the easiest to set up. The NUI group; however, does not have too much documentation on this OS as it is not the preferred Windows OS. Also, a couple of changes in the coding made two library files be missing in this update--this only means that you'll find yourself downloading additional files an databases in order to compensate. Also, you must choose whether to overlay or keep the built-in touch system. The NUI community will push for the use of Touch Injector as a service to give the inputs functions rather than the established MT Vista. Also, since the three versions were written differently, the setup may be different. There are blogs whose users have modded the appropriate software in order to be optimized for your version. You may encounter problems such as no input being shown, the touch-tracker not showing up on start-up, and permission revocation. You must be the account's administrator and run the warez in this mode in order for things to work properly. Lastly, since the warez is open-source for the most part, you may have conflicts with your anti-virus application. Simply treat these as false positives and move on.
- Windows 7: It is smooth, fast, and perfect for multi-touch. Similar to the above version, you should try to build your setup in a way that the corners are free for touching as the start menu and the important apps for this are located on the task bar's corners by default. You will need to install a number of services in order to get your input recognized by the controller, but this has the most documentation to any other OS in the NUI forums. You should not have many problems dealing with this OS when it comes to multi-touch. Also, if you are a student, you may have access to the Microsoft Surface Developer Kit which will increase the functionality of you table. You can also experiment with building your own frameworks and graphical interfaces if you so desire--which is always a great thing. If you are an avid windows fanatic, I would recommend this OS as it is a blessing.
- Windows Vista/ XP: I grouped Vista with XP since after the service packs released for these OS', XP and Vista simply become Windows pre-7.1. I do believe their service has been discontinued (don't quote me on that) so I would not recommend these if you are worried about a longer-term experience with your table--I've not yet attempted to upgrade or downgrade an OS with the multi-touch Warez already installed. Other than that, you'll be finding a good amount of support in the NUI forums due to the newer software simply being a modded version of software originally for XP and Vista. You shouldn't have too many problems with these, but i recommend not to go here unless you are planning to run a virtual machine with a more updated OS--most apps have been optimized for the more recent OS' and I'd recommend starting at 7 if you want that old-fashioned Windows appeal.
- Linux OS:
- Canonical's Ubuntu (any release from 12.04 LTS to 13.10): When it comes to Linux OS', people seem to think you must be a hacker, or an overly obsessed computer maniac in order to understand what is going on--the movies seem to portray Linux as a completely text-based OS, and who wants that? Well, let me be the first to tell you that they are wrong! Ubuntu is a FREE (although I'd recommend leaving a donation to the company when downloading) OS distributed by Canonical that is beyond wonderful. It is classified under the "Debian-like" distributions as this is where it is sourced from. Ubuntu is virtually updated daily and provides the highest sense of security for the average user. It handles similarly to Windows, yet it has it's own advantages that a Windows OS could not provide you. A big pro is that you won't find your system being infected by viruses. Although you can still contract Trojans and such, your system will be unharmed as the viruses are written for Windows users--they won't open and execute on Ubuntu or any Linux distro (unless you use WINE, a windows emulator for Linux). If you dual boot with Windows, you can even use Ubuntu to rid your Windows partition of any infections it may have. Ubuntu 13.10 (dubbed "Saucy Salamander") was written with touchscreen devices in mind much like windows 8; however, the format of the graphical interface is non-intrusive, and the options are all equally optimized for the non-touch aficionado. Paired with a variety of apps that you can find in their software center, you can make this desktop exactly what you want it to be, giving you the ultimate freedom and maneuverability. So what's the problem with it then? It is very hard to find any documentation on setting up an Ubuntu multi-touch setup on the NUI forums. This is due to many of the programs that are needed for touch implementation being written with Windows users in mind. You'll have to find a replacement capture software--I've heard Cheese or VLC Media Player work well with Ubuntu in place of the CLeye for the PS3 eye, but you WILL have to experiment and go through a lot of hard work in order to get this setup to work. The few that I've seen are gorgeous, so it IS worth the trouble. I am personally planning on getting my table back soon in order to experiment with changing the OS back to Ubuntu. Also, while on Ubuntu, you'll find yourself dealing with the Terminal (Linux equivalent of the Command Prompt console or Powershell) more than anything else. Although everything gives you the option of graphical input, you may find the terminal to be more satisfying. If you want to learn about computers, processes and all that goes into providing a rich, safe, and efficient user experience, I'd say give any Linux distro a chance.
****You should probably keep in mind that although you can find a Linux replacement for any Windows program, you're not going to be Gaming too much on a Linux Machine. Most developers and gamers prefer the Windows setup for gaming. - Mint, Tails, Kubuntu, Xubuntu, Debian, or any of the other hundreds of Linux distros: I know I grouped a lot of OS' into one category--which should be a cardinal sin as each one handles and behaves differently. The issue is that I've only merely dabbled in these other OS' and haven't given myself much time to get fully acquainted with any other distro. Kubuntu and Xubuntu offer a more simplistic experience to Ubuntu and Tails is optimized for anonymity. I've not really heard of any members in the NUI forums dealing with these specifically, but I know that it can be done. I'm not able to fully give an opinion on these as I have not been so involved with them, but I have heard that Mint OS is a pretty good choice for those wanting to experience the feel of Linux.
- Canonical's Ubuntu (any release from 12.04 LTS to 13.10): When it comes to Linux OS', people seem to think you must be a hacker, or an overly obsessed computer maniac in order to understand what is going on--the movies seem to portray Linux as a completely text-based OS, and who wants that? Well, let me be the first to tell you that they are wrong! Ubuntu is a FREE (although I'd recommend leaving a donation to the company when downloading) OS distributed by Canonical that is beyond wonderful. It is classified under the "Debian-like" distributions as this is where it is sourced from. Ubuntu is virtually updated daily and provides the highest sense of security for the average user. It handles similarly to Windows, yet it has it's own advantages that a Windows OS could not provide you. A big pro is that you won't find your system being infected by viruses. Although you can still contract Trojans and such, your system will be unharmed as the viruses are written for Windows users--they won't open and execute on Ubuntu or any Linux distro (unless you use WINE, a windows emulator for Linux). If you dual boot with Windows, you can even use Ubuntu to rid your Windows partition of any infections it may have. Ubuntu 13.10 (dubbed "Saucy Salamander") was written with touchscreen devices in mind much like windows 8; however, the format of the graphical interface is non-intrusive, and the options are all equally optimized for the non-touch aficionado. Paired with a variety of apps that you can find in their software center, you can make this desktop exactly what you want it to be, giving you the ultimate freedom and maneuverability. So what's the problem with it then? It is very hard to find any documentation on setting up an Ubuntu multi-touch setup on the NUI forums. This is due to many of the programs that are needed for touch implementation being written with Windows users in mind. You'll have to find a replacement capture software--I've heard Cheese or VLC Media Player work well with Ubuntu in place of the CLeye for the PS3 eye, but you WILL have to experiment and go through a lot of hard work in order to get this setup to work. The few that I've seen are gorgeous, so it IS worth the trouble. I am personally planning on getting my table back soon in order to experiment with changing the OS back to Ubuntu. Also, while on Ubuntu, you'll find yourself dealing with the Terminal (Linux equivalent of the Command Prompt console or Powershell) more than anything else. Although everything gives you the option of graphical input, you may find the terminal to be more satisfying. If you want to learn about computers, processes and all that goes into providing a rich, safe, and efficient user experience, I'd say give any Linux distro a chance.
- Apple OS'
- I've personally never dealt with an Apple OS. I hear that the are Unix-like and have been developed from Unix, but I've never owned an Apple machine. However, there is some documentation on the NUI Forums and on the internet regarding Apple's many cat-themed OS'. I hear that a setup with these OS' runs similar to a desktop rather than an iPad, unless you can configure it in that manner. I am unfortunately uninformed as to the warez necessary or anything of the like regarding apple. It's not that I don't like them, but I've never had one so I cannot formulate a fair opinion on any of them. I do, however recognize the security of the software implemented in their devices though. Unlike Linux, you may still encounter some minor infections due to the code not being updated as frequently, but I'd say that for the regular user (not trying to access anything dangerous), this is a great choice.
So there you have it. Now all you have to do is choose an OS and get what you need to work things out. If you are not too much into computers, I'd recommend that you stick with the Windows setup (maybe Win 7). If you want a challenge--and want everything to be worthwhile (and don't want to spend money on an OS)--go with a Linux OS. If you are an Apple user, go with Apple!
**If you really want something interesting though, try investing some time into setting up an Android setup. Android is Unix-like so there have been some developers that have succeeded. Also, the developer kit is free and there is a rich community of developers for Android. You may have a hard time at first, but it should pay off in the end. Just an interesting thought for you.
CCV, Touch Injector and the Cleye + CCV Tests (Warez)
I. CCV:
CCV stands for Community Core Vision, the open-source and NUI group-developed tracking software at the basis for any multi-touch table you will find to be built in the NUI group forums. It comes with its own handy interface that luckily does not change too much in between version revision. I believe the CCV distro we are currently in is 1.5--this is the one we used since it has documentation and support for multi-camera setups. There is a mod in the NUI forums for 1.4 that allows it to accept multi-camera setups. The more cameras, the faster your table and the smaller you can build it. I've explained in a very layman's terms how this program works in the Understanding Software section. There are various replacements to this program, although I do not recommend going that route as the NUI forums specifically highlight CCV over all other programs.
The interface can be quite confusing to deal with. Seth Sandler, a contributor to the NUI group has posted a blog with a great description of the interface and how to navigate it here. Try to read through his site and discover more on the warez needed. I believe his is optimized for a Windows 7-based MT Mini--but the practicality is still essentially the same.
Other useful links for you:
- CCV source Code: http://nuicode.com/projects/tbeta/repository
- Sandler's blog also contains links to the downloads for Linux and Mac.
- CCV 1.5 download from NUI: http://nuigroup.com/forums/viewthread/13330/
- How to configure CLeye to CCV 1.5: http://nuigroup.com/forums/viewthread/14464/
- If you choose to go with 1.4, another user has made a blog detailing installation very clearly here.
*Make sure that the Blobs are perfect and perform as desired before moving onto calibration.
II. CLeye:
The Code Laboratories SDK for the PS3 eye. It comes with a variety of drivers and tools to sync and configure the PS3 eye to your setup via the computer. If you are working with CCV 1.5, I recommend downloading this first and making sure that your input can be read through this program. The graphical interface should display the capture of your blobs as you touch the table. Once you've got this done, you can proceed to CCV capture. There is a problem with registration regarding the CLeye though. The SDK requires that you buy a license for ~$35 in order to get the software to work. Luckily, there are a couple of ways to circumvent this; however, you will be making things harder for yourself as the process requires the handling and manipulation of DLL files. This could potentially mess up your software setup and force you to have to re-install everything all over again many times. You can find the drivers for the CLeye to be detailed on the main CCV 1.5 forum post given in the CCV section or here, and here is the CLeye site.
III. Touch Injector:
Since we're on Windows 8, we have to find a replacement for MT Vista that will work as a service for your inputs. For this, we use Touch Injector. This was also developed in the NUI group and handles very well. It also has quite a bit of documentation and there are tons of forum posts on debugging and troubleshooting.
Useful links:
- Touch Injector Download: http://nuigroup.com/forums/viewthread/14448/
- Project done with useful info on CLeye dlls and Touch Injector config: http://nuigroup.com/forums/viewthread/14648/
- links for a TUIO mouse driver that may replace Touch injector: http://nuigroup.com/forums/viewthread/14725/ --this will convert touch events into Windows touch events.
- Forum with valuable info regarding config: http://nuigroup.com/S=e0ec20a79ebf6558e095464ab473...
- Refer here for TUIO software
Calibration (Software)
Test Run (Software)
Done!!! (Happiness)
So you've finally completed your table. Feel free to leave any feedback on how this instructable could be made better. Also, feel free to post videos and pics of your setup in the comments below. I'll be sure to add more pics and a couple of videos once I get my setup modded.
If you realized anything from this instructable, let it be this: Multi-touch isn't just a toy, nor is it a technology that we should take too much for granted. Projects like these are gateways to the future of NUI technologies. Multi-touch facilitates the branching toward the naturalization of the human to computer interface. Children are able to become more engaged and willing to learn about technology in this manner--as we are a species that yearns for feedback from our senses--and the elderly and disabled peoples can find more comfort and relief in navigating these devices. More importantly, Multi-touch opens the door to greater technologies such as Pranav Mistry's SixthSense gestural interface (an instructable that may make it's way sooner than you may think) and a host of other interface revisions--some of which I am not able to fully reveal as I am planning on developing. The fact that you've completed this project is truly something to boast about.
**Update: I wanted to take a moment to thank all my viewers and subscribers for voting and allowing me to win my first contest! Can't wait to start on my next large-scale project--after I do some more finishing touches on this one of course.
Additional Information for Those on a Budget
- LCD TV: I can give you links to these three. They may soon be unavailable as I've recommended them to many other users, but you can always use their LED TV derivatives as well.
- Vizio has a great selection of LCD and LED TVs (I promise I'm not trying to sell you their products) And I tend to generally trust their TVs to comply and be cleared from the FFC issue. Here is a 23" and 24" i think would be worth taking a look at.
- Alternatively, you could go with this Samsung that seems to be pretty cheap.
- Remember that if you can't find those TVs in Amazon anymore, you can check eBay,and other sites. You could also just fall back to the other list I gave in the beginning and look for a TV that clears FFC issues.
-
Camera: one for $80
-
LEDs WHT: Depends on LCD size (but usually ~$24)
-
LEDs IR: Determine the size around the screen in meters before buying! this will be up to $99
-
wood: same as mine (price may go down depending on size)
-
computer : same
-
Endlighten: anywhere from $120-200 depending on the thickness you choose and polish.
-
LED power supplies: $60 from LED supplier.
- TV: Same as above
- Touch Frame: This will take care of your camera, LEDs, Acrylic for just $385 (we are going for 32 touches as this is a true multi touch setup). Be very careful because a display less than 24" will cost you a lot since they are not sold individually on a regular basis. above 24", you're safe! And yes, a 24" frame is good for the 23" TV.
-
Add in $50 on wood and lumber from your local hardware store.
-
Lastly, a computer $0-1000 depending on what you want and whether you already have one (not factored in).
- As far as LCD, the smaller the diagonal, the cheaper your table will be. So at a $300 budget (If you're not wanting to move past that) I'd get a 17-19" TV (usually $45 on eBay if you can find one). Keep in mind that you are sacrificing HD capability if you don't already own an HD-ready computer.
- A IR frame from PQLabs.com ($169) for your TV ( straight from the website as Peau will try to sell the smaller sizes in bulk). If you can't get your hand on one, consider the camera route (price goes down as display size goes down).
- Wood
- A computer
Of course, not all of us have a huge point to prove. Maybe you want to make a Multitouch table without having to spend so much money. I admit, as time has passed, it has become much easier to develop these devices for a smaller cost, and aside from the size of the display, you won't be losing much at all.
I. Options for building a Multitouch Table with a ~$600 budget
i) Parts if going the camera route:
ii) Parts If going the Frame route:
For a grand total of ~$600-700.
II. Options for a $400 or less budget:
All for ~$264. You may end up doing a lot of searching for a frame small enough, but that'll be your only worry.
III. FAQ
- Considering Projector setup? Want my thoughts on that? I'd say stay away from the projector setup as you'd be introducing too many dependent variables into your design. This includes:
- Projectors won't last as long as LCD TVs.
- HD Projectors are more expensive than LCD TVs nowadays.
- If you can't afford (or even find) a Projector with the exact throw distance you need, you'll be having to configure mirrors in order to manipulate your image optically.
- Not all mirrors are the same. Some have less ghosting than others, and a good mirror (hot mirror) is rarely cheap.
- You'll need to add a bit more layers to your setup.
- Considering an ordinary camera and just adding a floppy disk as the IR band-pass filter? I'd say don't do that either. If you want an optimal setup, THIS WILL NOT WORK. In the old days, it was the smart thing to do but now, we buy our cameras to match the IR wavelength exactly. Makeshift Band-pass filters usually fail at rapid and smooth input detection. Also, if configured incorrectly, nothing will work. In terms of the camera, the PS3 eye is THE camera of choice for the NUI group. All the drivers and protocols in CCV are optimized for it. It would be wise to stick with it.
- Do I have other options on the camera? Not any cheap ones.Point Grey is a popular option for those willing to go all out; however, if you buy a proper IR Band-pass filter and configure your own PS3 eye, you'd save quite a bit of money (~$20).
- What's the difference between LCD and LED TV? The only difference is the back-light. LCD uses Cold Cathode Fluorescent Lamps (CCFL) tubes and LED uses white LEDs. That's it!
- What's the theoretical layering for the Touch frames? The layering becomes much simpler since the frames take care of everything now. IR detectors and emitters surround the inner section of the frame--removing the need for most of the components. Thus, the layering reduces to:
"""""""Touch frame (usually comes with glass)"""""""
=================LCD================
~~~~~~~~~~~~~~~SUPPORT~~~~~~~~~~~~~~~
….........................Desired altitude............................
>>>>>>>>>>>>WHT LEDs>>>>>>>>>>>>>>>>
- Do I recommend the Frames over the camera route? For those who don't like having to troubleshoot and mess around with theoretical values too much--yes! The frames make this much easier since mostly everything has already been done for you. all you need to do is put it all together.