Talking Head: a Talking Hat

Animatronics and puppets have been a part of the entertainment industry for a very long time. In 1961 Walt Disney and his "imagineers" started developing animatronics for the entertainment industry; their first project was a 9" tall dancing man figurine. As robotics advances more electronic actors can be seen in film. For example, E.T. the Extra Terrestrial and Jaws were both robotic creations.

While it may be expensive and complicated to enter into the world of advanced animatronics, it doesn't take much to fabricate a simple creation. Depending on what the outer shell is, a simple mouth can give the illusion of life to an inanimate object! Puppets are normally controlled with a person's hand, but they could just as easily be controlled with a cheap motor. This project serves as an introduction to analyzing sounds and controlling a motor based on what sounds are playing through a talking hat! The hat used for this project is a top hat, but one could easily change what the exterior is with a bit of planning and forethought; the most difficult portion of this construction is the electronics and programming, and one of those is already done for you!

Tools

• Hot Glue Gun/Glue
• Utility Knife
• Simple Soldering Kit
• Sewing Supplies
• Scissors
• Tape Measure
• Marker/Pen/Pencil
• Any Electrical Wiring Tools (Side cutters, wire strippers, needle nose pliers)
• 1" Width Masking Tape

Materials

• Felt Top Hat
• 1 Servo Motor
• Some cardboard (enough to cover 1 cross section of the hat interior)
• Zipper
• Adafruit 4 Ohm 3W Mono Speaker
• ESP32 Dev Board
• MP3 Player Board
• Class D Amp Board
• AUX Splitter
• AUX Cable
• 3x AA Battery Holder
• Half Size Breadboard and Wires
• Heat Shrink Tubing Set
• Micro USB Cable (For Programming the ESP32)

1.1 Cut out the mouth

• Mark a path approximately 1" from the top of the hat around 1/2 of the hate circumference. 1" tape can be used to help measure this.
• Use an exacto knife to make 3 cuts:
  • 2 will run from the top of the hat vertically downward 1", one on each side at the marks
  • 1 will follow the bottom line of the guide tape, connecting the previous 2

*Be mindful of where you are cutting; do not cut yourself!*

1.2 Cut inside lining of fold

• Fold the flap of the hat back and cut the inside lining of the fold. This allows the fold to be more flexible and allow the mouth to open easier.

1.3 Cardboard layer/mounting plate

• Use a piece of scratch paper to create a template for the electronics mount.
  • Placing the paper on top of the hat and creating a rough sketch of the hat shape is a good place to start.
• Cut that paper out and see how it fits inside the hat. The goal is to have the paper template rest close to the hole of the hat
  • If it is too small, get a new piece of paper and try again.
• Transfer the template to cardboard.
• Press the cardboard into the hat.
  • If the cardboard rests 7/8" to 1" under the bottom lip of the jaw, it is in the correct spot.
  • If the cardboard rests below the previously mentioned range, the template is too large.
    • In this case remove the cardboard, cut where the cardboard is not seating nicely, and place it back in.
    • Repeat until the cardboard rests in the proper position.
  • If the cardboard rests above the previously mentioned range, the cardboard is too small and you will need to cut a new piece.

You can always take material off, but you cannot put material back on. It is generally better to cut it too large and remove material as opposed to cutting entirely new pieces.

1.4 Zipper/Back cut

• Make a cut similar to the cut for the mouth in the back of the hat for back access to the electronics.
  • This can be done in a similar way using tape as a guide to cut a straight line.
  • This cut should be at the top of the hat, but not all the way. Remember there is some material inside the hat to provide sturdiness; the cut needs to be made right below that material. A safe thickness is 1/4".
• Sew a zipper to this cut to help close/open the back access to the hat.

• Fold the flap back and cut the inside lining of the fold. This allows the fold to be more flexible and allow the flap to open easier.

*Be careful not to burn yourself!*

2.1 Cut the cables to size

• Cut the cable about 3-4" away from each end (The cable(s) in the picture are too long)

2.2 Stripping and preparing the wires

• Remove about 1/2" of the outer rubber coating from the non-connector end of one of the cables.
• Cut away any copper strand or foil shielding if your cable has it.
• Untwist and spread apart each of the three interior wires (red, white, green) as seen in the third picture.
• Using a lighter, carefully burn away the thin cotton strands as well as the enamel coating the individual copper strands by quickly running the flame through the spread strands.
• The color from the ends of the wires should be removed, as seen in the fifth picture.

2.3 Resistors and heat shrink

• Cut two resistors to have their leads slightly longer than those in the sixth picture.
• Twist the end of the red wire around one of the ends of one of the resistors and solder the resistor to the wire.
• Then cut a section of heat shrink tubing to fit over the connection between the resistor and the wire.
• Shrink the tubing by quickly and carefully raking it through the flame of a lighter.
• Repeat with the white wire and another resistor.
• Once complete, twist the spare leads of the resistors together and solder them as seen in picture 9.
• Cut the male end of a jumper wire off with 3/4" of wire attached to the end.
• Strip 1/4" of the end of the wire and solder it to the connected resistors as in picture 10.
• Cover the connection with heat shrink tubing and shrink it (less tubing should be used than in the picture - will be fixed for the final draft.)

2.4 Ground and finishing up

• Cut off another male jumper connector with 3/4" of wire attached and strip off 1/4" of the end of the wire as before.
• Splice the wire end of the jumper with the green wire of the AUX cable and solder them together as shown in picture 12.
• Cover the connection with heat shrink tubing again and shrink the tubing.
• A larger piece of tubing can be slid over both wires and shrunk to cover all remaining exposed wires, as in picture 13.

2.5 Repeat

• Repeat steps 2.2-2.4 for the other 3-4" of cable, HOWEVER this time, omit the resistors and connect the red and white wires together directly. The cable with the resistors will be used to connect to the amplifier, and the cable without the resistors will be used to connect to the ESP32.

3.1 Speaker Prep

• Cut two wires abut 8" each, and strip 1/4" from each end of both.
• Solder one end of each wire to a terminal as shown in picture 1.
• Insert the other end of each wire into its respective screw terminal on the amplifier board (The speaker and the amp board have a + and - symbol to signify which terminal is which. Match the symbol on one to the symbol on the other.)

3.2 ESP32 and MP3 Board

• Connect the ESP32 to the far right side of the breadboard as shown in picture 3.
• If the pins on your MP3 board come at a 90 degree angle as ours did, using a pair of pliers gently bend the pins out to match how they are in picture 4.
• Insert the MP3 board into the far left of the breadboard, in the fourth row down, as seen in picture 5.

3.3 Breadboard connections

• Connect a jumper wire across the inner busses of the breadboard as in picture 6.
• Make the two connections shown in picture 7 using small preformed wires, or jumper wires if it is all you have.
• Connect the RX and TX pins of the MP3 board to pins 18 and 19 on the ESP32 respectively.
• Place a mall piece of electrical tape on the breadboard in the spot shown in picture 9.
• Additionally, make the two connections (the small blue and green wires) seen in picture 9.
• Insert the amplifier board face down into the breadboard in the spot show in pictures 10 and 11.

3.4 Audio Connections

• Connect the AUX splitter to the output of the MP3 board.
• Connect one of the custom AUX cables to one end of the splitter.
• Connect the green wire to the Audio In (-) terminal on the amplifier board, and the black (red/white) wire to the Audio In (+) terminal as seen in pictures 13 and 14.
• Connect the other custom AUX cable to the other output of the aux splitter.
• Connect the green wire to the inner ground bus of the breadboard and the black (red/white) wire to pin 35 of the ESP32 as seen in picture 15.

3.5 Servo Connection

• Connect jumper wires to each wire of the servo as in picture 16.
• Connect the brown wire of the servo (using the jumper) to the top inner ground bus as seen in picture 17 as a black jumper wire.
• Connect the red wire of the servo (using the jumper) to the top +V bus of the breadboard as seen in picture 17 as a red wire.
• Connect the orange wire of the servo (using the jumper) to pin 21 of the ESP32 as seen in picture 17 as the leftmost yellow wire.

3.6 Battery and Final Prep

• Connect the positive (red) wire of the battery holder to the top +V bus of the breaboard, and the negative (black) wire of the battery holder to the top inner ground bus as shown in picture 18.
• Connect a jumper wire from the top +V bus to the 5V VIN pin of the ESP32 as seen in picture 19, and connect a small preformed wire or jumper wire from a GND pin on the ESP32 to either inner ground bus on the breadboard.
• (These connections were made out of order when taking pictures and the ordering will be fixed).
• Insert 3 AA batteries into the battery holder, and connect the ESP32 to a PC or laptop with a micro USB cable.

4.1 Download the Code

• Download the linked .ino file
  • If the .ino file will not open in Arduino, open the file in Notepad and copy/paste the code into a new Arduino sketch or copy the code from below.

#include <BLEDevice.h>
#include <BLEUtils.h>
#include <BLEServer.h>
#include <Servo_ESP32.h>

// See the following for generating UUIDs:
// https://www.uuidgenerator.net/

std::string value;

#define audioIn 35
//#define audioOut 7

#define SERVICE_UUID        "4fafc201-1fb5-459e-8fcc-c5c9c331914b"
#define CHARACTERISTIC_UUID "beb5483e-36e1-4688-b7f5-ea07361b26a8"

//Commands
#define NEXT_SONG 0x01
#define PREV_SONG 0x02
#define SELECT_SONG 0x03
#define VOL_UP 0x04
#define VOL_DOWN 0x05
#define SET_VOL 0x06
#define PLAY_1ST_SONG 0x08
#define SELECT_STORAGE 0x09
#define TF_CARD 0x02  //TF Card
#define SLEEP 0x0A
#define WAKEUP 0x0B
#define RESET 0x0C
#define PLAY 0x0D
#define PAUSE 0x0E
#define SONG_IN_FOLDER 0x0F
#define STOP 0x16
#define PLAY_FOLDER 0x17
#define SHUFFLE 0x18
#define SINGLE_CYCLE 0x19
#define TOGGLE_DAC 0x1A
#define OFF 0x00
#define ON 0x01
#define PLAY_AT_VOL 0x22

static int8_t Send_buf[8] = {0};

static const int servoPin = 21; //printed G18 on the board
Servo_ESP32 myservo;

char commandRecieve;

  int audioRead = 0;
  int checkSum = 0;
  int countCycles = 0;

class MyCallbacks: public BLECharacteristicCallbacks
{
  void onWrite(BLECharacteristic *pCharacteristic)
  {
    value = pCharacteristic->getValue();

    if (value.length() > 0)
    {
      Serial.println("*********");
      Serial.print("New value: ");
      for (int i = 0; i < value.length(); i++)
      {
        Serial.print(value[i]);
      }

      Serial.println();
      Serial.println("*********");
    }
  }
};

void setup()
{
  Serial.begin(115200);
  Serial.println("1- Download and install an BLE scanner app in your phone");
  Serial.println("2- Scan for BLE devices in the app");
  Serial.println("3- Connect to ESP32-BLE_Server");
  Serial.println("4- Go to CUSTOM CHARACTERISTIC in CUSTOM SERVICE and write something");
  Serial.println("5- See the magic =)");
  BLEDevice::init("ESP32-BLE-Server");
  BLEServer *pServer = BLEDevice::createServer();

  BLEService *pService = pServer->createService(SERVICE_UUID);

  BLECharacteristic *pCharacteristic = pService->createCharacteristic(
                                         CHARACTERISTIC_UUID,
                                         BLECharacteristic::PROPERTY_READ |
                                         BLECharacteristic::PROPERTY_WRITE
                                       );

  pCharacteristic->setCallbacks(new MyCallbacks());
  pCharacteristic->setValue("Hello World");


  pService->start();

  BLEAdvertising *pAdvertising = pServer->getAdvertising();
  pAdvertising->start();
  myservo.attach(servoPin);
  
  Serial2.begin(9600, SERIAL_8N1, 16, 17);
  delay(500);

  sendCommand(SELECT_STORAGE, 0, TF_CARD);
  delay(200);

  //sendCommand(PLAY_AT_VOL, 30, 1);
  Serial.println();
  menu('h', 0);
}

 char c;  // char from Serial
 char cmd=' ';
 char cmd1=' '; 
 

void loop()
{
  //if (value[0] != commandRecieve){
  while(value[0] != 0){
    c = value[0];
    //commandRecieve = c;
    decode_c(); // Decode c.
    for(int i = 0; i < value.length(); i++){
      value[i] = value[i+1];
    }
  }
  audioRead = analogRead(audioIn);
  if(audioRead != 0){
  Serial.print(audioRead);
  Serial.print("\n");
  }
  delayMicroseconds(500);
  if (audioRead != 0){
    myservo.write(90);
    delay(33);
  }
  else{
    myservo.write(180);
  }
}


void sendCommand(byte command, byte option1, byte option2){
  delay(20);

  Send_buf[0] = 0x7E;
  Send_buf[1] = 0xFF;
  Send_buf[2] = 0x06;
  Send_buf[3] = command;
  Send_buf[4] = 0x00;
  Send_buf[5] = option1;
  Send_buf[6] = option2;
  Send_buf[7] = 0xEF;
  for(uint8_t i=0; i<8; i++){
    Serial2.write(Send_buf[i]);
  }
}

void menu(char op, int nval){
  // Menu
  switch (op){
    case '?':
    case 'h':
        Serial.println("SerialMP3Player Basic Commands:");
        Serial.println(" ? - Display Menu options. ");
        Serial.println(" P0_ - Play _ file");
        Serial.println(" F0_ - Play _ folder");
        Serial.println(" S0_ - Play _ file in loop");
        Serial.println(" V0_ - Play _ file, volume 30");
        Serial.println(" p - Play");
        Serial.println(" a - pause");
        Serial.println(" s - stop ");
        Serial.println(" > - Next");
        Serial.println(" < - Previous");
        Serial.println(" + - Volume UP");
        Serial.println(" - - Volume DOWN");
        Serial.println(" v_ - Set Volume to _");
        Serial.println(" c - Shuffle");
        Serial.println(" q - Loop 0-ON 1-OFF");
        Serial.println(" x_ - Loop _ folder");
        Serial.println(" t - Mute");
        Serial.println(" r - Reset");
        Serial.println(" e - Sleep");
        Serial.println(" w - Wake up");
        break;

    case 'P':
        Serial.print("Play Track ");
        Serial.print(nval);
        Serial.println();
        sendCommand(PLAY, 0, nval);
        break;

    case 'F':
        Serial.print("Play Folder ");
        Serial.print(nval);
        Serial.println();
        sendCommand(PLAY_FOLDER, 0, nval);
        break;

    case 'S':
        Serial.println("Play loop");
        sendCommand(PLAY_1ST_SONG, 0, nval);
        break;

    case 'V':
        Serial.println("Play file at 30 volume");
        sendCommand(PLAY_AT_VOL, 30, nval);
        break;


    case 'p':
        Serial.println("Play");
        sendCommand(PLAY, 0, 0);
        break;

    case 'a':
        Serial.println("Pause");
        sendCommand(PAUSE, 0, 0);
        break;

    case 's':
        Serial.println("Stop");
        sendCommand(STOP, 0, 0);
        break;

    case '>':
        Serial.println("Next");
        sendCommand(NEXT_SONG, 0, 0);
        break;

    case '<':
        Serial.println("Previous");
        sendCommand(PREV_SONG, 0, 0);
        break;

    case '+':
        Serial.println("Volume UP");
        sendCommand(VOL_UP, 0, 0);
        break;

    case '-':
        Serial.println("Volume Down");
        sendCommand(VOL_DOWN, 0, 0);
        break;

    case 'v':
        Serial.print("Set Volume to ");
        Serial.print(nval);
        Serial.println();
        sendCommand(SET_VOL, 0, nval);
        break;

    case 'c':
        Serial.println("Shuffle Play");
        sendCommand(SHUFFLE, 0, 0);
        break;

    case 'q':
        Serial.println("Toggle Loop Mode");
        sendCommand(SINGLE_CYCLE, 0, nval);
        if (nval == 1){
          Serial.println("OFF");
        }
        else if (nval == 0){
          Serial.println("ON");
        }
        break;

    case 'x':
        Serial.println("Loop Folder");
        sendCommand(PLAY_FOLDER, 0, nval);
        break;

    case 't':
        Serial.println("Muted");
        sendCommand(SET_VOL, 0, 0);
        break;

    case 'r':
        Serial.println("Reset");
        sendCommand(RESET, 0, 0);
        break;

    case 'e':
        Serial.println("Sleep");
        sendCommand(SLEEP, 0, 0);
        break;

    case 'w':
        Serial.println("Wake up");
        sendCommand(WAKEUP, 0, 0);
        break;
  }
}

void decode_c(){
  // Decode c looking for a specific command or a digit

  // if c is a 'v', 'P', 'F', 'S' or 'V' wait for the number XX
  if (c=='v' || c=='P' || c=='F' || c=='S' || c=='V'){
    cmd=c;
  }else{
    // maybe c is part of XX number
    if(c>='0' && c<='9'){
      // if c is a digit
      if(cmd1==' '){
        // if cmd1 is empty then c is the first digit
        cmd1 = c;
      }else{
        // if cmd1 is not empty c is the second digit
        menu(cmd, ((cmd1-'0')*10)+(c-'0'));
        cmd = ' ';
        cmd1 = ' ';
      }
    }else{
      // c is not a digit nor 'v', 'P', 'F' or 'S' so just call menu(c, nval);
      menu(c, 0);
    }
  }
}

4.2 Install the ESP32 Board Library

• Linked below is a great tutorial for this process
• https://randomnerdtutorials.com/installing-the-esp...

4.3 Install the Servo Library

• Download the zip file at the bottom of this page.
• Hover over Include Library, click on Add .ZIP Library..., and select the downloaded .zip file.

4.4 Selecting the ESP32 Board

• In the top left corner of the Arduino IDE, click Tools, the hover over Board, then ESP32 Arduino and select NodeMCU-32S.

4.5 Upload the Code

• With the ESP32 plugged into the PC, click the upload (right pointing arrow) button and watch the progress in the terminal in the bottom of the screen.
• When the text turns red and says "Connecting ........____....." press and hold the button on the ESP32 labeled "100" until the dots and dashes stop moving and the text changes.
• Once the terminal displays "Leaving... Hard resetting via RTS pin...," the program is ready to test.

4.6 Uploading Audio

• To get MP3 files onto the MP3 player, a Micro SD card is required.
• Using a USB adapter, MP3 files can be dragged onto the card and renamed with a 3 digit prefix indicating it's placement among the rest of the tracks.
• For example, if your first file was titled "This Must Be The Place.mp3", you would rename it to "001This Must Be The Place.mp3" and so on.

5.1 Download the BLE Terminal App

• On iPhone: Download "BLE Scanner 4.0" from the app store

5.2 Set Up a Name Filter

• There may be too many bluetooth devices around you to be able to easily find the ESP32 BLE server.
• To create a name filter, click on the three horizontal bars in the top left corner of the screen, bringing up the menu.
• Click on Filter. Then, click were it says "E.G. Fitness Tracker" and replace that with ESP32.
• Ensure the slider to the right is green and hit Apply Filter before clicking back over the menu, selecting BLE Scanner to return to the main view.
• Now, you should only see devices with "ESP32" in the name, likely to be just the one you are using.

5.3 Communicating with the ESP32

• Click the green Connect button next to your ESP32 in the app.
• Under Services should be an option that says CUSTOM SERVICE. Click on it, and then click on the option that says Write,Read in blue text.
• Lastly, click on the option that says Write Value in blue text. This text box is where you can send commands to the ESP32.
• Click on the option in the popup that says Text and it should be highlighted in blue.
• A list of commands can be seen in picture 10.

5.4 Play Some Music! (Or Dialogue)

• Ensuring that you have MP3 files on the SD card in your MP3 player, (See Step 4.6) typing "p" into the text box in the app and sending it to the ESP32 should begin playback of the first track.
• If you don't hear anything at this stage, ensure that the filesystem of your SD card is FAT32.
• If the issue persists, double check your wiring. If the issue still persists, swap the TX and RX cables going from the MP3 player, only on the side of the MP3 player.
• For some reason, some MP3 boards in this style come with what are conventionally called TX and RX flipped.
• See Section 7 for a full list of commands.

If everything at this stage works (the servo motor should also be moving in time with the dialogue or music), continue on to mounting. If not, proceed to the Troubleshooting section at the end (To be added).

6.1 Preparing the Servo

• Create/find a shim to hold the servo at a proper height to properly attach to the top of the hat. The shim can be made out of multiple pieces of cardboard glued together, some plywood, or whatever else can be found lying around to hold the servo at a good height
• Glue this shim to the cardboard at a distance where the servo arm will be rotating about the jawline cut, and glue the servo to the shim.

6.2 Mounting Electronics to Cardboard

• Cut a small slot out of the front of your cardboard to make room for the speaker to sit and insert the cardboard into
• Attach electronics to the back half of the cardboard. Keep in mind where wires will need to be, so wires don't get pressed against the sides and forced into bad positions.

6.3 Mounting the Speaker

• Make an outline slightly smaller than the speaker on the hat. The speaker will be glued to the sides of this hole, so make sure there is sufficient material to attach the speaker to.
  
  • Sound does not travel through solid objects very well, so creating a hole and covering it with fabric matching the hat will help enable sound to travel easier.
• Cut out the outline of the speaker from the hat.
• Glue the speaker into the previously-created hole. Pay attention to where wires are traveling; make sure the wires are comfortably routed inside the hat and not forced into bad positions.
• Cut a piece of fabric matching the hat's material large enough to fill the hole in front of the speaker and glue this fabric over the hole.

6.4 Inserting Cardboard with Electronics

• Put the cardboard plate into the hat and connect all the electronics before securing the cardboard in place.
• Once all the electronics are connected, hot glue the cardboard plate ⅞" to 1" under the bottom lip of the jaw.
• Glue the arm of the servo to the top of the hat.

6.5 Hiding the Electronics

• Use a piece of fabric to cover the electronics. Glue part of the fabric on the roof of the hat's mouth, and glue the other end to the cardboard plate along the ends of the mouth.
  • This fabric can be any color! Red would work to mimic the inside of the mouth, but a dark grey/black would work well.

7.1 Testing the Assembled Hat

• Repeat the steps from sections 5.3 and 5.4.
• Everything should be working the same, with the top of the hat moving instead of just the arm of the servo.

7.2 Hat Controls

• The commands available for the hat can be found in the code itself, as shown here.

• Serial.println(" ? - Display Menu options. ");
  Serial.println(" P0_ - Play _ file");
  Serial.println(" F0_ - Play _ folder");
  Serial.println(" S0_ - Play _ file in loop");
  Serial.println(" V0_ - Play _ file, volume 30");
  Serial.println(" p - Play");
  Serial.println(" a - pause");
  Serial.println(" s - stop ");
  Serial.println(" > - Next");
  Serial.println(" < - Previous");
  Serial.println(" + - Volume UP");
  Serial.println(" - - Volume DOWN");
  Serial.println(" v_ - Set Volume to _");
  Serial.println(" c - Shuffle");
  Serial.println(" q - Loop 0-ON 1-OFF");
  Serial.println(" x_ - Loop _ folder");
  Serial.println(" t - Mute");
  Serial.println(" r - Reset");
  Serial.println(" e - Sleep");
  Serial.println(" w - Wake up");

You may have noticed a pair of googly eyes appear on the hat partway through the project; this didn't need to be its own step, as customization is entirely personal preference! Put some goggles on the hat, choose a different kind of hat to put the motors in, put the hat on wheels, it doesn't matter what happens to the exterior because it won't change the functionality of the rest of the project (as long as it doesn't weigh too much or damage electronics).

As stated in the introduction to this Instructable, the main focus of this project was on the audio analysis and motor control. The beauty of that is this system can be put into almost any small-scale project you could want! Talking fish, puppets, stuffed animals, the possibilities are finite but large! The hat as an exterior is a suggestion and something easy to follow, but do not feel limited by this project.

• One of the most likely issues you may run into is being unable to play anything from the MP3 board. With the exact ESP32 board we used, following our schematics should go without error. However, if you are using a board with different pinouts, you may think the RX pin of the MP3 player board should connect to the TX pin on the ESP32 and vise versa, and you'd typically be right. However, this MP3 board seems to have a long running issue where the RX and TX pin labels are swapped, and you actually need to connect what is labeled "RX" on the MP3 board to what is labeled "RX" on the ESP32.
• If you notice a loud hissing or popping coming from the speaker, turn everything off immediately and ensure all the connections from the MP3 player are correct, especially the ground or green cables.
• If the servo is making a loud grinding or similar noise when idle, the servo is improperly mounted and the hat requires more force than the servo can handle to be held close. Ensure the servo's body does not prevent the hat from closing fully, and hat the servo arm does not get caught on the hat. A wide popsicle stick can be glued down the center of the "roof of the mouth" of the hat for extra support and stability.