TeleShaper: Distant Design Collaboration With ESP32/8266

The aim of this project is to integrate material manipulation at all stages of the design process, making the development more interactive. Due to the contemporary situation of remote interaction and travelling restrictions, this project explores the possibility of a remote physical collaboration at any stage of the design process, irrespective of the location of the contributors (even across continents)

Our project has three parts which are connected over internet:

1. The origami surface to be manipulated.

2. Remote Controller 1: Made of copper strips and paper

3. Remote Controller 2: Made of fabric and conducting threads.

The project was undertaken as part of the Computational Design and Digital Fabrication seminar in the ITECH masters program.

Contributors:

• Carolina Leite Vieira
• Pelin Asa
• Pinaki Mohanty

Common

• ESP32 or ESP8266
• Jumper wires
• Voltage regulator (optional, helpful for externally powering ESP32)
• Power banks or 9V batteries (optional, if powering externally)
• Scissors

Surface Manipulation Setup

• Origami Paper
• 5 Servo Motors (SG90)
• 9V battery (recommended to power the motors separately)
• Cardboard
• Paper Clips
• Strings
• Aluminum or Plastic Box frame with connectors
• Long copper wires
• Paper Straws
• Double-sided Tape

Controller 1

• Copper tape
• Paper
• LEDs (for checking the touchpads are working)
• ~220 Ohm resistors (for LEDs)

Controller 2

• Conductive thread
• Fabric (for sewing the threads onto)
• Needle
• LEDs (for checking the touchpads are working)
• ~220 Ohm resistors (for LEDs)

• To install your respective board to Arduino IDE go to File>Preferences
• In the ‘Additional Boards Manager URLs” section enter:
  • For ESP32: https://dl.espressif.com/dl/package_esp32_index.json
  • For ESP8266: https://dl.espressif.com/dl/package_esp32_index.json

• Go to Tools>Board>Boards Manager
  • For ESP32: Install esp32 by Espressif Systems
  • For ESP8266: Install esp8266 by ESP8266 Community

• Connect your ESP to your computer with a micro USB cable
• Go to Tools>Board and select your respective board (ESP32 Dev Module or ESP8266)
• Change the upload speed to 115200
• Select your COM port
• If the port does not come up you might need to check your USB cable or install the CP210x VCP driver from: https://www.silabs.com/developers/usb-to-uart-bridge-vcp-drivers

• To check if your ESP connects to Wi-Fi you can upload the web server example from File>Examples>WiFi>SimpleWifiServer and change the ssid and password lines to you
• If uploading fails hold the Boot/IOO button when you see Connecting…. and click EN button once uploaded
• Open the Serial Monitor at baud rate of 115200 and check if you are connected to your Wi-Fi network

Debugging:

Brownout detector was triggered error: This error is caused by low voltage. It might be caused by your power source. If you are powering through micro USB, try a shorter cable. If powering externally check your voltage and try different batteries, you can supply 5 - 10V to 5V pin on ESP32 or up to 3.3V to 3.3V pin. Never power your ESP with 2 sources at the same time or supply more than 3.3V to the 3.3V pin.

You can also disable the brownout detector by uncommenting WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0) on line 68 and the necessary libraries on lines 11 and 12 in the touch sensors code.

To connect our devices over different networks we used the Blynk platform (blynk.io). Through the Blynk server you can control a device connected to Wi-Fi with another device on a different network anywhere in the world with almost no delay in response time.

• Install the Blynk app on your phone and create an account
• Create a new project and add devices selecting your specific board (ESP32 Dev board or ESP8266) You will need to add each device that will be used in the project and receive their authorization tokens in the email. Activate the project by clicking the play symbol.
• Go to Tools>Manage Libraries and install the Blynk library
• In the codes below, change your Wi-Fi SSID and password and authorization tokens. Touch sensors code is the same for both copper tape and conductive thread controllers. For these, don't forget to set the authorization token of the controlled device in the BLYNK_CONNECTED function.
• In the servo code for the controlled service, you just need to enter your device's own authorization token.
• Bridge widget part in the code creates a bridge to enable device-to-device communication. You can add as many bridges as you want according to how many virtual pins you need to send separate data to. You can write to the virtual pins with myBridge.virtualWrite(myVirtualPin, value) command.
• You can add different widgets in the project page in the app to control your device from there or check that the code is sending data to your virtual ports. To read the value sent to the virtual pin from the controller device, add Blynk.virtualWrite(myVirtualPin, value) command.
• You can first try your setup by connecting a push pin to the controller device and turning on and off an LED connected to the controlled device.
• When you upload the code, open your serial monitor with baud rate 115200 and check that you are connected to Wi-Fi and the server. You can also check it on the app; if they are all connected there should not be a red number displayed in the boards symbol in the app as shown in the screenshot above.

Since the ESP32 is not as standardized as the Arduino, the first step in the development of the touchpad is to determine which pins allow touch detection. The specific position varies between suppliers, but the analog pins will remain the same: GPIO4, GPIO2, GPIO15, GPIO32, GPIO33, GPIO27, GPIO14, GPIO12, GPIO13. For the use of the LED, there is no restriction of which pins can be used.
The pins used in this project were:

Touch Pin 1: GPIO4

Touch Pin 2: GPIO32

Touch Pin 3: GPIO33

Touch Pin 4: GPIO27

Touch Pin 5: GPIO14

LED Pin 1: GPIO16;

LED Pin 2: GPIO23;

LED Pin 3: GPIO22; LED Pin 4: GPIO21; LED Pin 5: GPIO2;

For creating the touchpad, we used copper tape over a normal paper to create touch buttons. Any shape can be done, but it's important to point out that the buttons should not overlap each other.
We connected the 5 touch pins from the ESP32 with each copper tape button. For fixing the wires over the copper, we simply used another piece of copper tape over the jumper wires.

The codes and the pins for the fabric touchpad are the same as the copper one. You can sew the threads in any shape you want, just make sure the ones for different pins do not touch each other.

By sewing them in a grid you can code different x and y coordinate values and also activate 2 LEDs/Servos together by pressing the connection points. You can tie a metal ring at the end of the threads to connect the jumper wires to and then attach the jumper to its pin on ESP.

You can adjust the sensitivity by changing the threshold value in the code or adding a high value resistor between the thread and the jumper.

We chose water bomb tessellation for the surface manipulation since it's quite flexible and can assume curvature in multiple directions. This kind of tessellation is quite unique and we followed this video tutorial to make one:

A comparatively thinner paper was used for the free flow of the surface. The top flatter portions were then reinforced by gluing pieces of cardboard as shown in the picture.

A boxed frame was set up on which the 5 servos were mounted using double-sided tape. You can also adopt better strategies for mounting. The paper straws were used to loop anchor the strings, which was in turn attached to the horn of the servo.

The strings from the servos were used to suspend the origami shape with the help of paper clips.

We used ESP8266 (NodeMCU) for this side of the project, you can also use ESP32, the only different is using the ESP32 library in the code. The servo signals are connected to the following pins:

• GPIO5 - D1
• GPIO4 - D2
• GPIO13 - D7
• GPIO14 - D5
• GPIO15-D8

The code reads the signals from the 5 virtual pins written on it by the controllers and writes them in turn onto the 5 physical pins of the ESP listed above to signal the servos to rotate. The authorization key used should be generated in the Blynk app as explained before.

In this project, each of the three devices we used were in different countries and we controlled the servo-powered origami service together at the same time with two touchpad controllers. You can first try controlling the servos with each touchpad separately and then try them together.