How to Build a LipSync

Touchscreen devices have become a staple piece of technology in this day and age. In 2016, upwards of 1.6 billion people in the world are using smartphones. Most people cannot get through the day without using their smartphone. It keeps us connected. It keeps us organized. It is an integral part of our social lives. For people in wheelchairs whom experience difficulties with fine upper body motor control, the usage of mobile devices can be very challenging. Statistics show there are over 1 million people in the United States and Canada that have very limited or no use of their hands, making touchscreen devices very difficult or impossible to use.

The "LipSync" is an assistive technology device which is being developed to allow quadriplegics and other people with limited hand use the ability to use touchscreen mobile devices by manipulation of a mouth-operated joystick with integrated sip and puff controls. The user is able to manipulate a cursor on their device screen using a mouth-operated joystick with integrated sip and puff controls to simulate the actions of "tap" and hitting the back button, respectively. With longer sips and longer puffs, additional secondary features are enabled including a "tap and drag", "long tap and drag" and the possibility of more specialized functions as per the user's needs. The LipSync is design specifically for portable devices, it does not require AC power, but it will work with any device including desktop and laptop computers that support mice through a universal serial bus (USB).

There are 2 main aspects we will be addressing in our project:

Developing an easy to build, but robust electronics assembly that novice to experienced makers can build.

Developing a device housing which can be 3D printed by makers either at home or at dedicated facilities.

FUTURE LIPSYNC PROJECT GOALS

Apr-Dec 2017: Support a network of makers to connect directly with people with disabilities and creating 150 LipSyncs through the Pacific Northwest

Jan-Dec 2018: Scale maker-disability matchmaking service across North America, to support 1,500 LipSyncs being created at the local level, as well as developing and applying other assistive technologies.

In addition, the Neil Squire Society has been working on a service delivery model for the technology. It will soon be launching the Makers Making Change initiative. Makers Make Change has the goal of creating one-on-one relationships between makers and people with disabilities. The Makers Making Change initiative will provide the infrastructure to connect makers, open assistive technologies, and people with disabilities in their community to make a local impact. Makers will be able to meet people with disabilities in their community and work with them to deliver a solution.

GET INVOLVED

If you want more information on the LipSync, track our development closer or work with us in creating (or getting) a LipSync, please connect with us. We have a newsletter and request form on our webpage: http://www.neilsquire.ca/lipsync

Download all the necessary files and directories to assemble a LipSync:

https://github.com/makersmakingchange/LipSync/arch...

LipSync-master\Hardware\Housing_design: Contains all 3D printing files in STL format and its respective print settings

LipSync-master\Hardware\PCB_design: Contains schematic and layout for the LipSync printed circuit boards

LipSync-master\Software\LipSync_Firmware: Contains microcontroller code LipSync-master\LipSync_BOM.csv: Contains LipSync Components in CSV format

Components should be ordered at least 2 weeks in advance to allow for sufficient shipping time. See below for the full list of components and its purchase links.

Component List:

1. [1] LipSync Circuit Board Pack

2. [1] Arduino Micro / Genuino Micro Microcontroller

3. [2] Right Angle Push Buttons

4. [2] 300 Ohm 1/4W Through Hole Resistors

5. [4] 10k Ohm Mini Axis Through Hole Resistors

6. [1] NXP Pressure Sensor [component label: MPXV7002DP]

7. [1] 0.01uF Ceramic Capacitor [capacitor label: 103]

8. [1] 1.0uF Ceramic Capacitor [capacitor label: 105]

9. [1] 470pF Ceramic Capacitor [capacitor label: 471]

10. [2] 17 Position Straight Female T/H Headers

11. [2] 3 Position Right Angle Male Headers

12. [2] 3 Position Straight Female Headers

13. [2] 4 Position Straight Female T/H Headers

14. [1] 10 Position Break Away Male Headers

15. [1] 5mm Bi-Colour T/H LED

16. [1] Shunt Connector

17. [1] 2N3906 PNP-Type T/H Transistor

18. [1] 6 Position Straight Female T/H Headers

19. [1] 1 Position Straight Female T/H Header

20. [1] Front Housing

21. [1] Joystick Front Plate

22. [1] Joystick Rocker

23. [1] Joystick Base Plate

24. [1] Rear Housing

25. [2] Buttons

26. [1] Mouth Piece

27. [1] Filter

28. [1] Luer Lock Fitting

29. [4] Force Resistive Sensors

30. [4] Screws Philips Head M3

31. [6] Compression Springs

32. [2] Standoffs M3 10mm

33. [2] Standoffs M3 15mm

34. [1] Luer Lock Nut

35. [4] Bumpers

36. [4] Nuts M3

37. [1] Tygon Tubing 3/32" ID x 45mm

38. [1] Light Pipe

39. [1] Threaded Insert 5/8"-27

40. [1] Threaded Adapter 5/8"-27

41. [1] USB Cable

42. [1] USB Adapter

43. [1] Cable Tie

To complete this LipSync Build, you will need the follow tools:

1. Small Spool of 60Pb/40Tn Solder Wire

2. Temperature Adjustable Soldering Iron (350°C)

3. Needle Nose Plier

4. Fine File or Sandpaper

5. Baby Powder

6. Super Glue

7. Medium Strength Threadlocker

8. Desolder Pump

9. Safety Glasses

10. Hobby Knife

11. Flush Cutters

12. ABS or High Strength PLA Filament 100g

The 3D Components include:

1 x Rear Housing

1 x Front Housing

1 x Joystick Rocker

1 x Joystick Base Plate

1 x Joystick Top Plate

These .stl files can be found on the hackaday site: hackaday.io/project/13424-lipsync

For this part you will need the following components:

1. 2x 300 Ohm Resistors
2. 2x Push Buttons
3. 10 Position Breakaway Headers
4. Shunt Connector
5. Transistor
6. Bi-Colour LED
7. 6 Position Female T/H Header
8. 2x Right Angle Male Headers
9. PCB Board 1

1. Snip the 300 Ohm Resistors using wire cutter
2. Using Needle Nose Pliers, bend the leads of the resistors at a 90 degree angle.
3. Insert the leads of the resistors into the holes of the PCB
4. Bend the leads on the other side to ensure they don't fall out
5. Flip the PCB and solder the resistors to the PCB
6. Snip the ends of the resistors, ensure flush cut (cut it as close to the PCB as you can).

1. Insert the leads of the buttons into the holes of the PCB
2. Flip the PCB and solder the push buttons to the PCB

1. Using a needle nose plier, snap away 2 of the 10 position header as shown.
2. Insert the 2 position breakaway headers.
3. Flip the PCB and solder the 2 position breakaway headers to the PCB

Carefully place the shunt connector on the 2 position break away headers

1. Snip the transistor using wire cutters
2. Place on board as shown
3. Insert the leads of the 2 position breakaway header into the holes of the PCB
4. Flip the PCB and solder the transistors to the PCB
5. Snip the ends of the transistor, ensure flush cut (cut it as close to the PCB as you can).

1. Snip the LED using wire cutters
2. Orienting it with the flatside facing you, bend leads of the LED at 90 degrees (as shown)
3. Insert the leads of the LED into the holes of the PCB, push all the way in
4. Flip the PCB and solder the LED to the PCB
5. Snip the ends of the LED, ensure flush cut (cut it as close to the PCB as you can).

1. Insert the leads of the 6 Position Female T/H Header into the holes of the PCB
2. Flip the PCB and solder one lead of header to the PCB
3. Flip the PCB and ensure that the header is sitting flat on the board (if it isn't, unsolder the one lead and fix)
4. Flip the PCB and solder the rest of the leads

1. Insert the leads of the 2 Right Angle Male Headers into the holes of the PCB
2. Flip the PCB and solder one lead of each header to the PCB
3. Flip the PCB and ensure that the headers are sitting flat on the board (if they aren't, unsolder the one lead and fix)
4. Flip the PCB and solder the rest of the leads

For this part you will need the following components:

• NXP Pressure Sensor
• 0.01uF Ceramic Capacitor [capacitor label: 103]
• 1.0uF Ceramic Capacitor [capacitor label: 105]
• 470pF Ceramic Capacitor [capacitor label: 471]
• PCB Board 2

1. Snip the leads of the capacitors using wire cutters
2. Place the the 103 to the far left, the 105 in the middle and the 471 to the far right.
3. Push them in all the way and ensure they sit flat on the PCB
4. Flip the PCB and solder the capacitors to the PCB
5. Snip the ends of the capacitors, ensure flush cut (cut it as close to the PCB as you can).

1. Apply a small amount of solder to one lead of the pressure sensor as shown
2. Apply a small amount of solder to the corresponding solder pad on the PCB
3. Place the pressure sensor on the PCB with all leads in line and on top of their corresponding solder pad
4. Place the solder tip on the lead to reflow the solder on the two components. You will be able to feel the pressure sensor should “drop” onto Board 2.
5. Apply solder to the rest of the leads and be careful to ensure that they have connection to the solder pad

For this part, you will need the following components:

1. PCB Board 1
2. PCB Board 2
3. 2x 4 Position Female T/H Headers
4. 2x 4 Position Breakaway Headers

1. Insert the 4 Position Breakaway Headers into the 4 Position Female T/H Headers as shown
2. Insert the leads of the 4 Position Female T/H Header into the holes of the PCB Board 2
3. Insert the leads of the 4 Position Breakaway Headers into the holes of PCB Board 1
4. Solder one lead of each of the 4 Position Breakaway Headers into Board 1
5. Solder one lead of each of the 4 Position Breakaway Headers into Board 2
6. Check to ensure that the headers are sitting flat on the boards (if they aren't, unsolder the one lead and fix)
7. Solder the rest of the leads on all the headers

For this part, you will need the following components:

1. PCB Board 1
2. 2x 17 Position Female T/H Headers
3. Arduino Genuino Microcontroller

1. Insert the microcontroller leads into the two 17 Position Female T/H Headers.
2. Carefully remove the Board 2 (with pressure sensor) as shown
3. Insert the leads of the 17 Position Female T/H Headers into the corresponding holes on the PCB as shown
4. Flip the PCB and solder one lead of each of the two headers to the PCB
5. Flip the PCB and ensure that the headers are sitting flat on the board (if they aren't, unsolder the one lead and fix)
6. Flip the PCB and solder the rest of the leads
7. Re-Insert Board 2

For this part, you will need the following components:

1. Joystick Front Plate
2. Joystick Base Plate
3. Joystick Rocker
4. 4x 10k Ohm Mini Resistors
5. 3 Position Straight Female Headers
6. Luer Lock Fitting
7. 4x Force Resistive Sensors
8. 4x Screws Philips Head M3
9. 4x Compression Springs
10. 2x Standoffs M3 10mm
11. 2x Standoffs M3 15mm
12. Luer Lock Nut
13. 4x Bumpers
14. 4x Nuts M3
15. Tygon Tubing 3/32" ID x 45mm
16. Sandpaper
17. Baby Powder
18. Super Glue

1. Snip the Mini 10K Ohm Resistors using wire cutter
2. Using Needle Nose Pliers, bend the leads of the resistors at a 90 degree angle.
3. Insert the leads of the resistors into the holes of the PCB
4. Bend the leads on the other side to ensure they don't fall out
5. Flip the PCB and solder the resistors to the PCB
6. Snip the ends of the resistors, ensure flush cut (cut it as close to the PCB as you can).

1. Using an exacto knife, carefully and thoroughly remove the support material from the 3D printed joystick parts

1. Place the 10mm standoffs into the raised slot as shown
2. Insert a nut on both of the standoffs and (using needle nose pliers) tighten the nuts
3. Apply super glue to the connecting areas of the nut and standoff
4. Wait 5 to 15 mins for the super glue to dry

1. Using fine sandpaper, sand the rounded surface of the rubber bumpers
2. Apply a small amount of baby powder to the sanded surface of each rubber bumper
3. Insert (twist in clockwise) the Luer Lock fitting into the rocker with the smaller end of the fitting coming out on the flat side of the rocker (as shown)
4. Insert the nut from the flat side of the joystick rocker
5. Ensure the nut sits flush to the rocker, once flush, twist the luer lock fitting in the rest of the way.
6. Apply a small amount of super glue to the connections between the luer fitting and nut, as well as the nut and rocker. - Wait 5 -15 minutes for the glue to dry.

1. Using fine sandpaper, sand the rounded surface of the rubber bumpers
2. Carefully remove the rubber bumpers from their packaging and place in the indented circles on the joystick rocker - ensure you place the higher part of the bumper towards the center.
3. Apply a small amount of baby powder to the sanded surface of each rubber bumper

1. Insert the leads of the two 3 Position Female T/H Headers into the holes of PCB Board 3
2. Flip the PCB and solder one lead of header to the PCB
3. Flip the PCB and ensure that the header is sitting flat on the board (if it isn't, unsolder the one lead and fix)
4. Flip the PCB and solder the rest of the leads

**WARNING** - Force Resistive Sensors (FSRs) are very fragile - Be careful not to touch with soldering iron or they will melt. Be careful not to bend at a radius of less than 2.45mm as this will cause the FSRs to malfunction. Be careful to solder the FSRs quickly as prolonged heat at the leads will conduct heat to the plastic of the FSR and cause it to melt.

1. Insert 1 FSR at a 30 Degree angle away from the centre of the board (as shown)
2. Hold the board with a clamp and point the out to the side (do not lay board down as this can cause the FSR to bend and break)
3. Quickly and carefully solder the the leads of the FSR
4. Repeat steps 1-3 for the next 3 FSRs

1. Insert 15mm Standoffs as shown
2. Place Board 3 onto the Joystick Base Plate
3. Insert 2 nuts onto the standoffs above Board 3 and tighten using needle nose pliers
4. Insert the head of the FSRs through the corresponding slots in the Joystick Base Plate as shown, ensure the black adhesive is facing the centre of the base plate.
5. Using an exacto knife, carefully remove the plastic covering (as shown) of the adhesive on the head of an FSR, be sure not to remove the black adhesive while trying to perform this step.
6. Carefully push down the head of the FSR so that the adhesive sticks to the Joystick Base Plate within the designated outline.
7. Repeat steps 6 and 7 for the rest of the FSRs
8. Apply super glue to the connection between the nuts and the standoffs, wait 5 to 15 minutes for the glue to dry

*Note* - Use the joystick stand holder to assist with assembly

1. Place Base Plate onto Joystick Stand Holder
2. Place Rocker (the side with the tygon tubing facing down) onto the Base Plate as shown
3. Place 4 springs into the indented circles
4. Place 4 screws into the Joystick Top Plate
5. Plate Joystick Top Plate onto of the Rocker + Springs
6. Tighten the screws until the Top Plate sits on the standoffs
7. Connect the 3 Position Right Angle Male headers from Board 1 into the 3 Position Straight Female Headers on Board 3, there is a slot in Board 1 which allows the tygon tubing to carefully be sent from one side of Board 1 to the other. (As shown)
8. Connect the tygon tubing to the outermost connector on the pressure sensor

For this step, you will need the following components:

1. Rear Housing
2. Front Housing
3. Fully Assembled Joystick (with Board 1, 2, and 3)
4. 2x 3D printed Buttons
5. 2x Springs
6. Light Pipe
7. Cable Tie
8. Filter
9. Mouth Piece
10. USB Cable

1. Insert the light pipe into the hole on the Rear Housing

1. Insert Cable Tie onto the Micro USB end of the USB cable
2. Cut the tail of the cable tie that is sticking out

1. Feed the USB Cable (Micro USB end first) half way through rear housing.

1. Using tweezers or needle nose pliers, insert the 2 springs into the corresponding slots in the Rear Housing
2. Insert the buttons, narrow end first, into the centre of the springs as shown

1. Connect the Micro USB side of the USB Cable to the Micro USB Port on the microcontroller of the Joystick
2. Slide the Joystick Assembly along the internal tracks inside the Rear Housing
3. Push in all the way until the Push Buttons on the Joystick Assembly sit inside the 3D printed buttons. - Check from the back by pushing the buttons, if they click well, then buttons have been correctly inserted into the 3D printed buttons - otherwise carefully push the assembly up and re-adjust
4. Twist on the the Front Housing onto the threads of the Rear Housing (ensure flush and tight fit)
5. Lightly twist on the filter (do not twist on too tight)
6. Lightly twist on the mouth piece (do not twist on too tight)
7. Insert the 5/8"-27 Threaded Insert

1. Download and install the latest version of Arduino IDE: https://www.arduino.cc/en/main/software
2. Download and save the latest LipSync microcontroller code: https://raw.githubusercontent.com/makersmakingchan... Right-click on the hyperlink of microcontroller code, select “Save Target As” (Internet explorer) or “Save Link As” (Chrome or Firefox) and save the microcontroller code (LipSync_Firmware) to the computer directory or folder of your choice.

3. Open the Arduino IDE and click on Tools -> Board -> Arduino/Genuino Micro

4. Connect to the PC using the LipSync USB cable.

5. Click on Tools -> Port

6. Click the right arrow at the top left of the Arduino IDE to upload the microcontroller
   code to the LipSync device.

7. Once upload is complete, a message saying “Upload Successful” will show at the
   bottom left of the Arduino IDE.

**Note** - You must calibrate your joystick before using it the first time.

Allow the LipSync to initialize for 3 seconds before moving the mouthpiece each time after it’s plugged into a device.

1. Press both speed adjustment buttons on the back of the LipSync for a few seconds until LED comes on

2. LED slowly blinks red & green 5 times

3. While LED blinks green 6 times, move the mouthpiece to the 12 o'clock position (up direction)

4. LED blinks red 1 time when position is measured

5. While LED blinks green 6 times, move the mouthpiece to the 3 o'clock position (right direction)

6. LED blinks red 1 time when position is measured

7. While LED blinks green 6 times, move the mouthpiece to the 6 o'clock position (down direction)

8. LED blinks red 1 time when position is measured

9. While LED blinks green 6 times, move the mouthpiece to the 9 o'clock position (left direction)

10. LED blinks red 1 time when position is measured

11. LED rapidly blinks red and green 6 times when calibration is complete