Billy Bass Can Speak (Arti-fish-al Intelligence)

Billy Bass’ moving and speaking are controlled by a TV remote control. Also, switches allow switching between that and his original singing and dancing routine. If you don’t want the switching option in your project, the instructions describe how to bypass it. An Arduino microcontroller with an IR sensor catches signals from a TV remote and moves Billy’s motors. A second Arduino processes the audio. It’s needed because the SD Card Shield and the Motor Drive Shield use some of the same Arduino pins.

The intelligence is arti-fish-al,

even though it’s superficial.

I think it is beneficial.

The code is not efficient,

for I am not proficient.

Somehow, it is sufficient.

Detailed instructions are provided for the 9 steps in the project.

1. Open up Billy’s Case and Extend the Wires for Billy’s Motors
2. Connect and Test the Arduino for the Motors
3. Connect the IR Receiver and Find the Codes from a TV Remote Control
4. Test How the TV Remote Controls Billy’s Motors
5. Record Billy’s Comments on an SD Card
6. Connect and Test the SD Card Shield in the Second Arduino
7. Make the Circuit for the Second Arduino
8. Make the Second Arduino Talk
9. Attach the Arduinos to Billy Bass

PARTS

1. 2 Arduino Uno
2. 2 USB cables for the Arduinos
3. 1 L293D Motor Drive Shield Version 1 or 2 (I used Version 1)
4. 1 SD Card Shield
5. 1 IR receiver, such as 1838 IR Infrared 37.9 KHz receiver
6. 1 TV remote control
7. 1 Breadboard
8. 1 Plastic mounting plate for breadboard and Arduino
9. Jumper wires 
10. Colored wires, 22 AWG, solid strand
11. Heat-shrink tubing
12. 1 4P2T (4 pole double throw) switch
13. 1 1P2T (1 pole double throw) switch
14. 1 3.5 mm audio jack (female)
15. 1 10 nF capacitor
16. 1 100 nF (0.1 uF) capacitor
17. 1 10 K resistor
18. 1 Powered audio amplifier, such as Anker SoundCore mini Model A3101
19. 1 Audio cable with 3.5 mm male connectors at both ends
20. 3 bolts, 4-40, ½ inch (13 mm) for Step 9
21. 4 thin twist-ties for Step 9
22. 1 female 3-pin connector, pin spacing 1/10 inch (2.5 mm), for Step 9. (photo is in Step 9). My connector came from some old equipment. On the Internet there are similar connectors called JST PH Cable Socket.
23. Electrical tape or some other black tape for Step 9
24. 1 small, white, sticky label or paper and clear tape for Step 9

TOOLS

1. Small screwdrivers
2. Needle-nose pliers
3. Drill and drill bits
4. Wire stripper
5. Soldering iron and solder
6. Scriber or some other sharp, pointed object to make a mark on plastic
7. Pen or magic marker
8. Scissors

Billy Bass’ case has 6 screws on the back. Unscrew them and pull the back off. The front side has a circuit board, a push button and a sensor that are visible from the outside, and one motor with mechanical parts to move the head and tail. The motor for the mouth is inside the fish. The back has a 4-battery holder, a speaker, and a switch visible from the outside. The battery holder has a 4½ volt connection as well as a 6 volt.

The motors move body parts out and springs return them when a motor’s power is removed. There is only one motor for the head and tail. When the motor turns in one direction, the head moves. And when it turns in the opposite direction, the tail moves. So, at any specific time, only the head or the tail can be moved, not both at the same time.

The wires to the mouth motor are red and grey. The wires to the head/tail motor are orange and black. Both pairs of wires come from the same connector on the circuit board. The wires need to be cut near their middle in order to connect them to the Arduino motor drive shield. The red wire from the battery holder also needs to be cut near its middle. The black wire from the battery holder is not cut.

The current for the motors can be measured while Billy is playing his song. And temporarily connecting the cut wires together, the voltage can be measured during the song. The results were as follows: 

Head/tail: Highest was +5.9 volts and lowest, -4.0 volts. Highest current was 0.32 amps.

Mouth: Highest was +5.23 volts and lowest, 0.0. Highest current was 0.16 amps.

The voltage and current varied significantly as the body parts moved, probably because my multimeter couldn’t keep up with the rapid changes as the parts moved. And maybe Billy’s circuit sometimes provides lower voltages for smaller movement.

These measurements show that Billy’s motors are within the limit of the L293D motor drive shield, which is 0.6 amps in each of 2 channels. 

Holes need to be drilled in the back of Billy’s case to hold the audio jack and for 1 or 2 switches, depending on whether you are installing the 4P2T switch that switches between the TV remote option and his original singing and dancing routine. The photo shows good places for the holes. The best places for the top and bottom holes are exactly where 2 small, square holes are located. I don’t know why they are there, and I drilled them out. The holes are ¼ inches (6 mm) for my switches. A larger hole is also needed to allow wires to be fed from the inside of Billy’s case to the Arduinos. It is also shown in the photo.

Cutting New Wires

Wires need to be soldered to Billy’s wires to make them longer. If you are not installing the 4P2T switch, wires are needed for the 4 wires from Billy’s motors to the motor shield, a red wire from the battery holder wire to an on/off switch, and a red wire from the on/off switch to the motor shield. If you are installing that switch, wires are also needed for the 5 wires from Billy’s circuit board. It is easiest to use wires of the same color as Billy’s. The length of the new wires depends on exactly where you cut Billy’s wires, and the wires should be long enough to allow for some slack when Billy’s case is open. For my project, 8 new wires are 4 inches (10 cm). With the red power wires, 1 new wire is 3 inches and the other is 5 inches.

In my project, the 4 wires from the 4P2T switch to the Arduino’s motor shield are 6 inches (15 cm), the red power wire from the 1P2T switch is 8 inches (20 cm), and the black wire from the battery holder to the motor shield is 10 inches (25 cm). 

Wires are also needed for the audio jack. The white and green wires from the audio jack are 10 inches (25 cm). (I used a white wire because I ran out of black wire.)

Soldering the Wires

I found that soldering the wires should be done for each group of 4 wires separately. After stripping about ¼ inch (6 mm) of insulation off each end of each wire, place the new wires for Billy’s circuit board into the switch terminals that are farthest from the edge of Billy’s case, and solder them. Then, cut 4 pieces of heat-shrink tubing ¾-inch (2 cm) long and slide them over the wires. Attach the loose end of the wires to Billy’s wires for the circuit board, and solder them together. Slide the heat-shrink tubing over the soldered connection and shrink the tubing.

The wires to Billy’s motors should be done next, attaching to the switch’s middle terminals.

Then, the wires to the Arduino, attaching to the remaining terminals.

The red wires for the 1P2T switch should be soldered in the same way, making sure that the new wire for Billy’s circuit board is in the switch terminal that is farthest from the edge of Billy’s case, and the wire to the battery holder is in the middle terminal. 

Soldering the black wire from the battery holder to the Arduinos is tricky. The terminal on the battery holder does not have a hole. The existing wires (a black one and a blue one in my Billy) are soldered to the flat terminal. The new wire needs to be soldered there also, and, of course, the old wires detach from the terminal as soon as heat is applied to solder the new wire. I had to remove the old solder and some insulation from the old wires, and twist the 3 wires together before soldering them to the terminal. I had to use tape to hold the wires tightly to the terminal and then apply the solder with one hand and hold the soldering iron in the other hand.

The terminals for the left and right audio channels of the audio jack should both be soldered to the green wire. The white wire is for the sleeve (ground) terminal. (I used a white wire because I ran out of black wire.)

The last step is to guide the wires through the hole made to pass them to the Arduinos. To make the testing easier, I connected the wires to the Arduinos temporarily with alligator clips and jumper wires.

The L293D Motor Drive Shield sits on top of the Arduino Uno. After lining up the pins carefully, press the Shield onto the Uno firmly. I used Version 1 of the Shield, which was the only one available in the local electronics store. If you have Version 2, the connections for the project should be similar.

The wiring from the Arduino to Billy’s motors is shown in the photo. The wires in the photo are the same colour as the wires to Billy’s motors, and they come from one side of the switches that were soldered in the previous step.

There is a very small jumper on 2 pins labelled PWR, beside the EXT_PWR screw-terminals. This jumper needs to be removed because external power is being provided to the motors from Billy’s batteries. The power from the Arduino via its USB cable does not provide sufficient current. If the jumper is not removed, the external power is connected to the Arduino’s power from the USB cable, and this could cause problems.

When power is applied to the motor drive, a green LED on the board turns on.

The Adafruit Motor library must be installed before the Arduino is able to drive the motors. There are different libraries for the Version 1 Shield (AF_Motor V1) and for Version 2 (AF_Motor V2). The steps to install the library are as follows: 

1. Plug the Arduino into one of the USB ports of your computer.
2. Open the Arduino app on the computer. A blank sketch appears. In the bottom right corner, the sketch should show the model of the Arduino board and the port it’s plugged into. If the info isn’t there, click “Tools” (it’s in the top left corner of the computer screen), then move the cursor to “Boards” (in the list that appears), click “Arduino AVR Boards” and click the model of Arduino you are using. And then choose the port by clicking “Port” just below “Boards”, and click the port name, which probably has “usbmodem” in the name.
3. Click “Tools” (it’s in the top left corner of the computer screen).
4. Click “Manage Libraries...” (in the list that appears).
5. Wait a few seconds until a blinking vertical line appears in the top area of the Library Manager. Then type AF_Motor V1 for Version1 or type AF_Motor V2 for Version 2. After a few seconds, the info appears. 
6. Click inside the “Select version” box to choose the most recent one.
7. Click “Install”.

The sketch for testing the motors is named “Billy-Bass-Serial-Monitor” and can be downloaded. The link is below. The sketch operates the motors in a 2-motor Bigmouth Billy Bass. It tests the use of the Serial Monitor to control the mouth, head and tail. Entering H and Carriage Return (the Enter key) on the computer keyboard turns the head, h returns the head, T bends the tail, t returns the tail, M opens the mouth, and m closes the mouth. CR (the Enter key) must be pressed after each letter.

Here are the steps to test Billy’s motors:

1. Download file “Billy-Bass-Serial-Monitor”. The link is below.
2. Open it in the Arduino app.
3. Click “Verify” (the tick-mark at the top left of the sketch). No errors (in red colour) should show at the bottom.
4. Click “Upload” (the right arrow at top left, beside “Verify”).
5. Turn on the Serial Monitor (click “Tools” and then click “Serial Monitor”).
6. On the keyboard, type a letter and Carriage Return, and watch Billy.
7. Type other letters and Carriage Returns to check the other motions.
8. The Serial Monitor should show messages.
9. When finished, quit the Arduino app and disconnect the Arduino from the computer.

A screenshot of the Serial Monitor is shown in one of the photos.

My Billy Bass doesn’t open his mouth as much as the Billy Bass in Youtube videos when my Billy is playing his song and also when I ran the test. It’s tempting to take part of Billy’s skin off his head to check if something can be fixed, but I worry about wrecking something if I try to do that.

The connections for the IR Receiver are easy, but involve some soldering. First, solder a short, yellow jumper wire in the hole for pin #2 on the motor drive shield. It’s located in one corner beside the screw-terminal used for the grey wire in the photo. Then, on the other side of those 5 screw-terminals, are 5 holes labeled “Gnd” and 5 labeled “+5”. Solder a short, red jumper wire in one of the “+5” holes and a short, black jumper wire in one of the “Gnd” holes.

The IR receiver goes into the breadboard as shown in the photo. It has 3 pins: “-”. “+”. and “S”. 

The soldered red and black jumper wires should now be pushed into the breadboard, red into the red ”+” and black into the blue “-”. Additional red and black jumper wires then go from there to the IR receiver. The soldered yellow jumper wire goes to the “S” of the IR receiver.

The IRremote library must be installed before the Arduino is able to recognize the signals from a TV remote control. The steps to make it happen are the same as before. 

The sketch for finding the TV-remote codes is named “SimpleReceiver-changed” and can be downloaded. The link is below. It is the SimpleReceiver example contained in the IRremote library with a few changes for the Billy Bass project: Changed NEC TV remote to SAMSUNG, changed Serial.begin to 9600, and changed some statements to comments, in order to stop displaying data for an UNKNOWN TV remote and IR noise.

Here are the steps to find the codes for your TV remote:

1. Download the 2 files needed for my project sketch and save them in a new folder named “SimpleReceiver-changed”. The links are below.
2. Open it in the Arduino app.
3. In the definitions of the protocols of the TV remote, near the beginning of the sketch, make a comment of the SAMSUNG protocol by adding “//” at the beginning, and remove the “//” for your TV remote.
4. Click “Verify”.
5. Click “Upload”.
6. Turn on the Serial Monitor.
7. Point the TV remote at the IR sensor and press a key very briefly. The data for that key should appear on the Serial Monitor. If you press a key for more than very briefly, the data will be repeated. Press other keys you may want to use.
8. When finished, quit the Arduino app and disconnect the Arduino from the computer.

A screenshot of the Serial Monitor for my project is shown in one of the photos. In the screenshot, “Command=” shows the signal code. The codes I chose for my Samsung TV remote are below:

TV-remote  Signal    

   Key        Code     Function

  VOL +      0x7 Open the mouth

  VOL -       0xB   Close the mouth

  MUTE      0xF   Turn the head

CH LIST   0x6B   Return the head

  CH /\       0x12   Bend the tail

  CH \/       0x10   Return the tail

MEDIA.P  0x8C Turn the head, say “Hello” and flap the tail twice

 MENU      0x1A  Turn the head, say “Good to see you”

 SLEEP     0x3  Turn the head, say “Bye” and flap the tail twice

After you have the signal codes that you would like to use from your TV remote, the project sketch needs to be changed to use those codes instead of the Samsung codes.

1. Download the 2 files needed for my project sketch and save them in a new folder named “Billy-Bass-motors”. The links are below.
2. Open it in the Arduino app.
3. In the definitions of the protocols of the TV remote, near the beginning of the sketch, make a comment of the SAMSUNG protocol by adding “//” at the beginning, and remove the “//” for your TV remote.
4. Near the end of the sketch, change the codes to ones that you want to use, in the    “if (IrReceiver.decodedIRData.command == )“ statements.
5. Click “Verify”.
6. Click “Upload”.
7. Turn on the Serial Monitor.
8. Point the TV remote at the IR sensor, press a key very briefly and watch Billy. 
9. Messages should appear on the Serial Monitor. If you press a key for more than very briefly, some data will be repeated. Press other keys you decided to use.
10. When finished, quit the Arduino app and disconnect the Arduino from the computer.

A screenshot of the Serial Monitor for my project is shown in one of the photos.

Audio files may be recorded on your computer using a microphone or converted from text. An audio file then needs to be converted to .wav format for Arduino to be able to play it. The file name is a maximum of 8 characters followed by “.wav”. Arduino can play a .wav file in the following format: bit resolution: 8 Bit, sampling rate: 16000 Hz, audio channel: mono, PCM format: PCM unsigned 8-bit.

There are various apps available for a computer for recording audio from a microphone. The one I used is Audacity. Conveniently, it can save the result as a .wav file. It also can import audio files of various types and adjust them using the “Effects” option.

To use text and convert it to speech, one website is https://ttsmp3.com There are various languages and accents to choose from. The result can be downloaded to your computer as MP3 files. I have not tried it.

There are a few ways to convert any audio file into .wav format, such as using the Audacity app on your computer, or the website. https://audio.online-convert.com/convert-to-wav. 

For testing purposes, a few .wav files may be downloaded. The links are below. When being saved on the SD Card, they must be saved as individual files. 

The steps for using Audacity are as follows:

1. Open the Audacity app. 
2. Look at the “Project Rate” (at bottom left corner) and set it to 16000.
3. Click in “Click to Start Monitoring” (beside the microphone icon at the very top).
4. Speak the words you want to record, to check how loud they are. The Recording Meter should go into the yellow range but not into red. Move the mic slider (below the Meter) if needed.
5. Click the red Record button to start recording and the black Stop button to stop.
6. Click the green Play button to listen to the recording. Move the speaker slider if needed.
7. If the recording is not satisfactory, delete it by clicking on Select > All, and then press the delete key on the keyboard. Record it again.
8. Click Tracks > Mix > Mix Stereo Down to Mono.
9. To delete part of the recording, such as some of the part after the spoken words, click and drag from the spot, to the end of the recording, and press the delete key on the keyboard.
10. There are many Effects that could be applied. One I used was Bass and Treble, because my voice sounded muffled. Click Effect > Bass and Treble, move the Treble slider to 10.0 and the Bass to -4.0, click “Enable” if there is no tickmark there, and click Start Playback to hear the result. If the result is satisfactory, click “Apply”. Click “Close”.
11. Another effect I used was “Normalize”, to get all the phrases to the same volume level. “Normalize” adjusts the volume of a recording such that the maximum volume is a specific level below the red level on the Meter. A setting of -3.0 dB is good. to do this, click Effect > Normalize, then set the dB adjustment level, and click “OK”. Click “Close”.
12. Click the green Play button to check the recording.
13. Click File > Export > Export as WAV and enter the info for where you want to save it. The file name must be 8 characters or less, followed by .wav. Also, check that the Encoding is set to “Unsigned 8-bit PCM”. Then click “Save”.
14. To do another recording, click Tracks > Remove Tracks. Click Tracks > Add New > Stereo Track. And go back to step 3.

Before using an SD card, it must be formatted. Small-capacity SD Cards, 32MB or less, are not supported by the Arduino SD library. (Such cards are formatted as FAT12, while larger capacities are FAT16 or FAT32, depending on the capacity.) Formatting is possible on a computer, smart phone or digital camera. The steps on an iMac computer are as follows:

1. Insert the SD card into the slot on your computer, with the label side towards you.
2. Click Finder in the Dock at the bottom left of the computer display.
3. Click Applications in the sidebar at the left.
4. Double-click Utilities.
5. Double-click Disk Utility.app.
6. The SD card should appear in the sidebar on the left. If not, choose View > Show All Devices.
7. In the sidebar, select the storage device (APPLE SD Card Reader).
8. Click Erase (at the top).
9. In the Scheme pop-up menu, choose “Master Boot Record”. If this isn’t in the pop-up menu, make sure you selected the “APPLE SD Card Reader”.
10. In the Format pop-up menu, choose “MS-DOS ( FAT)”.
11. Enter a Name, maximum 8 characters.
12. Click “Erase”. 
13. When the erase process is complete, click “Done”.
14. Quit the Disk Utility.

Steps to store Billy’s comments on the SD card:

1. Open the place you saved Billy’s .wav files.
2. Highlight the files. (Click the first file in the list. While pressing “shift” on the keyboard, click the last file.)
3. Click Edit > Copy … Items.
4. While pressing “control” on the keyboard, click the SD Card icon on the desktop.
5. In the pop-up menu, click “Paste … Items”.
6. When it is complete, to look at the contents of the SD Card, double-click the SD Card icon.
7. Get the same pop-up menu again and click “Eject”.
8. When the SD Card icon disappears, remove the SD Card from the computer’s slot.

A second Arduino is needed because the SD Card Shield and the Motor Drive Shield use some of the same Arduino pins.

The SD Card Shield sits on top of the Arduino Uno. After lining up the pins carefully, press the Shield onto the Uno firmly.

The SD library must be installed on the computer. The steps to make it happen are the same as before (in Step 2).  

The library has examples for using an SD Card. The “CardInfo” example will display what we want. It can be accessed as shown below:

1. Connect the Arduino to the computer.
2. Insert the SD Card into the SD Card Shield.
3. Click “File” (it’s in the top left corner of the computer screen).
4. Move the cursor down to “Examples” and then to the right.
5. Move the cursor down to “SD” (it’s near the end of the list) and to the right again.
6. Click “CardInfo”. The example sketch appears.
7. Click “Verify” (the tick-mark at the top left of the sketch). 
8. Turn on the Serial Monitor (click “Tools” and then click “Serial Monitor”).
9. Click “Upload” (the right arrow at top left, beside “Verify”).
10. When finished, quit the Arduino app. 
11. Disconnect the Arduino from the computer.

A screenshot of the first part of the Serial Monitor is shown in one of the photos. There were many files in the list, and Billy’s .wav audio test files appeared at the end. The creation date and time for these files are when the SD Card was formatted, and so, I suppose they are part of the card format. 

The circuit is quite simple. The same breadboard is used for both Arduinos.

A black ground wire goes from the second Arduino to the ground from the first Arduino. 

The IR receiver will be used by both Arduinos. So, there will be 2 yellow signal wires, one for each Arduino. A yellow signal wire from the right pin of the IR receiver goes to pin 2 on the second Arduino. The first Arduino will continue to provide the 5 Volts for the IR receiver in the final setup. 

The audio output on pin 9 may be connected directly to an earphone or to a power audio amplifier. To avoid clicks during the speaking, a low pass filter should connect the power amplifier. The circuit for this filter is shown below:

   Pin 9 >------------||-------------|__10k__|-------+-------------> to Power amplifier (green wire)

(green wire) 100nF=0.1uF

== 10 nF

   GND pin >------------------------------------------+-------------> to Power amplifier (black or white wire)

  (black wire)

Download the 2 files needed for my project sketch and save them in a new folder named “Billy-Bass-comments”. The links are below.

Connect the second Arduino to the computer and open the “Billy-Bass-comments” sketch. The TMRpcm library must be installed before the Arduino is able to process the files on the SD Card. The steps to make it happen are the same as before (in Step 2). 

After opening the sketch in the Arduino app, it needs to be changed to use the codes from your TV remote instead of the Samsung codes. This is described in Step 4.

There are 3 comments in the sketch. The codes are shown below:

TV-remote  Signal    

   Key        Code     Function

 MEDIA.    0x8C Say “Hello”

 MENU     0x1A   Say “Good to see you”

 SLEEP    0x3     Say “Bye”

The sketch will say “hello” when the Arduino starts. When you press one of the keys on your TV remote, the earphone or the power audio amplifier should speak the comment.

A screenshot of the Serial Monitor is shown in the photo.

At this point, the two Arduinos have the proper sketches saved in them. They could be connected to the computer or to battery packs that have USB ports. The project should work now.

You could probably input the Ardionos’ power from Billy’s batteries instead of external power, but I didn’t try that.

You could add more comments and make appropriate changes to both sketches, using the existing code as a guide.

The last thing to do, is to attach the Arduinos to Billy Bass and move the IR receiver to a place that will be visible to the TV remote. 

The Arduino used for the SD card can be attached to the back of Billy’s case, near the top. The steps are as follows:

1. Take out the jumper wires between the Arduino and the breadboard.
2. Hold the Arduino between the 2 screw-holes that attach the back and front of Billy’s case, and scratch 3 marks through the holes in the Arduino, to show where to drill holes. 
3. Drill the 3 holes, slightly larger than ⅛ inch (3 mm) to allow for the small inaccuracies when drilling the holes to match the Arduino’s holes.
4. Apply 3 bolts and nuts to hold the Arduino.

Attaching the other Arduino is more complicated. The photos show the final result and the steps are described below:

1. Drill 2 holes, ⅛ inch, on the left side and 2 on the right, one above the other, near the bottom of the back of the case, between the existing screw-holes and the legs of the stand. 
2. Put a twist-tie thru the holes on each side, with the lower part of the twist-tie longer than the top. These twist-ties will go thru holes in the plastic mounting plate for the breadboard and Arduino.
3. You may have to drill holes in the 4 corners of the mounting plate.
4. Attach the back of Billy’s case to the front, using the 6 screws.
5. Check if Billy’s batteries need replacing. The battery holder will not be easy to access after the mounting plate is attached to Billy.
6. Drill 1 hole near the bottom of each of Billy’s 2 legs of the stand.
7. On the breadboard, move the IR jumper wires to the other side of the breadboard, close to the audio circuit.
8. My 3-pin cable connector has stranded wires. I soldered short pieces of solid wire to allow pushing them into the breadboard.
9. Take the IR receiver out of the breadboard and place it in the 3-pin connector.
10. Place the wires from the 3-pin connector into the proper places on the breadboard.
11. Place the wires that are coming out of Billy, into the proper places of the Arduinos and the breadboard.
12. Slide the mounting plate into place and attach it using the twist-ties.
13. Place the jumper wires that connect the SD Arduino to the breadboard.
14. Attach the IR receiver to the edge of Billy’s case using black tape.

The last item is labeling the switches and audio jack, using a sticky label or paper and clear tape.