Kowtow Lamp

Introduction

Did you ever wish you had a Desklamp with some zaz and character? A sassy judging companion that responds to your ideas and questions your guility reading pleasures. Fear no more, we got you covered!

In this project you will make a Lamp that has the ability to move 4 Axis, detect Text Mediums and react if specific keywords are being detected.

The project was conducted as part of the "Computational Design

and Digital Fabrication" seminar in the ITECH masters program.

Fabian Eidner, Oliver Moldow and Aaron Wagner

This project can be built in many different ways, here are the parts we used, feel free to adjust and give it your own twist by using different sensory inputs or motors!

Main Electronics

1 x Arduino Uno R3 or generic equivalent - (Amazon)

1 x Mini Breadboard (Included in starter kit)

1 x AZDelivery HC-05 Bluetooth Modul - (Amazon)

1 x Webcam PCB (Disassembled from very cheap Trust 720p Webcam) (Amazon)

1 x Neopixel LED Ring (Amazon)

4 x Servo Motors

Optional

+ Soldering Iron

+ Multimeter

+ Servo Driver Board

+ Board to solder to

Lasercut Parts:

at least 320 x 600mm big Sheet (Plywood 3-4mm, or Plexi)

3D printed Parts:

• PLA or PETG
• 0.2 mm Layerheight
• 0.8 mm Wallthickness

Step 0: Part Preparation

Most non electronic Parts were cut from a 3mm plywood sheet (520 x 705mm used 320 x 600mm) using a Epilog Fusion Pro Lasercutter, but can also be cut from Paperboard using a cutting knife.

The rotating Baseplate (1. Axis) was 3d printed and greased using Graphite from a ordinary Penciltip. The gears for the first arm (2. Axis) and the Lamphead assembly were also 3d printed. The Parts were printed using PLA with a layerheight of 0.2mm and a 0.4mm nozzle with a infill density of 20 %,a wallthickness of 0.8mm was sufficient.

Step 1: The Stand

The stand mainly consists of the lasercut pieces which can easily assembled through press-fit connections. Located in the centre of the stand is a frame in which the first servo can be positioned.

The rest of the box should provide enough space to store the Arduino board including the electronics.

Step 2: Cover

To cover up the base and hide the electronics the top plate with circular hole is being mounted onto the base.

Step 3: Head of the Lamp

For now, we can put the base aside and can move on with the lampshade. Therefore, we attach the webcam on our 3d printed camera frame. On top of that goes the LED-Strip, so that the camera is surrounded by the circular LED. To give the head of our lamp the typical desk-lamp look we covered the lights & camera with a paper lamp shade.

Step 4: Short Arm

Eventually the head of the lamp can be mounted onto the shorter arm of the lamp. Integrated in the centre of shorter arm is a hole, designated for the second servo motor.

Step 5: Long Arm

The 3d-printed circular base plate builds the base for the lamp. It also builds the base for the third servo motor which is placed on top of the base plate into the servo frame. Attached to this servo is the smaller gear. The longer arm of the lamp is vertically oriented with the larger gear being at the bottom the forth and last servo-motor inserted into the hole at the top.

Step 6: Finish

As shown on the picture, the two arms can now be connected at the location of the fourth servo.

Eventually, the entire lamp can be placed into the socket of the base plate, with both gears going into each other. 

Theory

The sensory input for this project was the video input, or the eye of the lamp, from the disassembled webcam. There are a plethora of ways to convert and create visual stimulation, it would have been possible to just have a color or distance sensor and detect if a object is placed on the table or a book with a specific color. The goal for us though was to get more familiar with Computer Vision and to explore the anthropomorphic realm. A Lamp that could not only detect a object or colour but react to information normally only available to us humans, would be able to convey a lot more character.

Tools:

#1 Optical character recognition

In order to be able to extract the Text Information from our Webcam Footage we need to utilize an OCR or Optical character recognition software.

An open source implementation of an OCR is the tesseract originally developed by Google.

For many custom implementations the tesseract is built from Source luckily for our purpose we can use the Windows / Linux implementation from University of Mannheim, which was built for the purpose of digitizing historical german newspapers, as of today over 700000 pages are available.

Windows version:

https://github.com/UB-Mannheim/tesseract/wiki 

As mentioned in their installation the preselected directory should ideally be used for the installation:

C:\Program Files\Tesseract-OCR

The installation path will be important to later access the tesseract.exe within python.

# 2 Coding Environment

The machine vision part of this project was done using python, the openCV library and an OCR implementation and can be done using any coding environment.  

This implementation was done using Anaconda:

https://www.anaconda.com/

We strongly recommend creating a new virtual environment for installing the python libraries:

After installation we can open up the “anaconda powershell prompt” and create a new virtual environment “myenv” can be replaced with whatever you want to name your environment:

conda create --name myenv 

...waiting...

proceed ([y]/n)? 

confirm by typing “y” and pressing enter.

...waiting...

Now we have to activate the environment:

conda activate myenv 

We can proceed by installing the libraries and and IDE:

IDE:

conda install spyder 

...waiting...

 proceed ([y]/n)? 

...waiting...

Libraries:

Open CV (cv2) Library

pip install opencv-python

...waiting...

 proceed ([y]/n)? 

...waiting...

Pytesseract (pytesseract) Library

pip install pytesseract

...waiting...

 proceed ([y]/n)? 

...waiting...

Introduction:

The interaction between python and the arduino bluetooth modul can be rather tricky, thats why we are going to go over both processes seperatly at first to check that both sides are working, after that is the case we can combine both parts to get the full pipeline.

Bluetooth modul:

Wiring:

The wiring for the Bluetooth modul is really straight forward as shown in the picture. The RX and TX pin can go to any digital PINs on the Arduino, we encountered two issues which we want to share with you. The first one was using the 0 and 1 PIN it might just have been faulty hardware on our end but we were not able to receive any data while using those two PINs, we ended up using PIN 10 (RX) and 11 (TX).

Another issue we encountered was not being able to upload scrips while the HC-05 was plugged in to the arduino, unplugging the RX and TX PINs always fixed that issue.

Arduino:

After wiring the HC-05 Modul we can come to the Ardunio software side. As soon as the Arduino is plugged in the Bluetooth moduls LED should start flashing quite fast and continously.

The "bluetooth_test.ino" Script is an adaption of the simple blinking example script using the inbuilt LED. The script listens for input from the Modul if the received data stored in "c" is equal to one the LED turns on for a few seconds and turns of again.

For that we need to send data through Python, for now we will only compile and upload the script onto the Arduino.

Windows Bluetooth Connection

The HC-05 Modul is password protected, which means we need to connect to it manually first to than be able to get the COM port it is on and be able to connect to it through Python.

For that we go to the Bluetooth Settings in Windows (The Ardunio should be powered on and have the Script on it) and add a new Device, after a few seconds the HC-05 Modul should appear as a device, clicking on it we are asked to enter a password, which by default should be set to "1234", very secure!

After entering the password and having added the new device we can go into "more bluetooth options" and click on the "COM" Tab, you should see the HC modul twice one port is going to be the input and one the output. We are mostly insteresed in the outgoing COM port, not it down we are going to need this one for the Python script!

Python:

Now it is going to get interesting, for this section we expect red and done the Steps in the previous Python section:

1. Downloaded anaconda / another Python distribution
2. Created a virtual Environment (not nececarry but strongly recommended)
3. Installed atleast the pySerial Library
4. Opend the BluetoothScriptMod.py in a IDE of your choice (PyCharm, Spyder, Jupyter nootbook, Atom, etc.)

The code itself is still very simple to just check the connection to the HC Modul, the script gets a connection with the HC modul and then has a simple user input, if the user inputs a 1 it sends a 1 to the arduino where we can use that input for a corresponding reaction like a blinking LED.

So change the COM port to the outgoing COM port you noted down earlier. And run the script, you should see a User Input question popping up in the consol / output of your IDE.

Once the lamp is assembled (left), we can now continue to set up the Arduino circuit (right).

The hardware consists of 4 servos that are connect to the Arduino board. A capacitor (100 μF) prevents rapid discharges and ensures smoother servo-motions.

To supply all components with 5V we used a 5V Voltage regulator to convert the voltage of our 9V batterie.

The webcam is directly connected to the computer.

Python:

The combined code looks very similar to what we discussed in the previous sections we will quickly go over it again to explain what is where:

Line 1-14: Importing the necessary Libraries and define the function to send data to arduino

Line 15-20: Define the User inputs, cooldowntimer between Reactions, Words to react to, set the iterationCounter to the cooldowntimer for the first run through

Line 21-28 Establish Bluetooth connection with outgoing COM port, Start tesseract from the File path where we installed the OCR (Mannheim)

Line 29-35 Capture Video from specified Webcam (0) is your first "standard webcam" (1) the second etc. if it cant be opend through an error

Line 36-56 Main loop create a variable to store text data, read in the image data from webcam, perform Computer vision text detection on image and store text data in imgchar variable, if the iteration its on exceds the cooldowntimer look for keywords in the imgchar string, if keyword is detected call "retrieveData" function we defined at the start and send corresponding data to arduino, finally "cv2.imshow" shows the webcam feed in a window, the framerate is going to be slow because of the textdetection running for every frame.

Line 57-End cv2.waitKey is waiting for a specific press of a Key on the Keyboard in our case "ord('q')" so the Key Q on your Keyboard, if that Key is pressed while the User is in the Webcam window the program breaks out of the while loop and closes the bluetooth connection stops the webcam capture and closes all open windows (Lamp O Vision)

Ardunio:

The Arduino code can directly be uploaded to the Arduino, if you want to change any reaction you can do so by changing the values in the function.