Positive Response Chatbot

This Instructables show how to build a simple chatbot pet.

It is motion responsive and gives positive message responses while chatting.

You'd be willing to chat with it for half an hour. Or you can leave it standby near you, sometimes it will give you some positive message responses while you chat with others.

• Arduino Nano RP2040 Connect dev board
• JR6001 voice module
• Motor Driver module
• Electronic Pet bird, dancing robot or all other moving robot

Many times, chatbot pets just need companionship rather than answering your questions intelligently. One of a good example is Haro in Gundum Anime. It can give random positive message response is already very good enough.

• Arduino IDE compatible
• Speedy speech recognition, offline engine preferred
• Voice output
• positive message response
• motion response

Note:

Arduino IDE just my personal preference, if you choose other platform like micro-python, then the hardware selection and software design may be very different.

If you browse the Arduino Official Store, you can find a Speech Recognition Engine for their dev board. It is a pre-trained offline engine so it can help to development a chatbot quickly.

There are various pre-trained speech recognition engine in the market but seldom support Cantonese language. So this time I will use this engine with Cantonese as a demo.

The engine can support Nano 33 BLE Sense Rev2, Portenta H7, Nicla Vision and Nano RP2040 Connect. I picked Nano RP2040 Connect in this project.

Ref.:

https://store-usa.arduino.cc/products/speech-recognition-engine

The Arduino Speech Recognition Engine can run in trial mode before any payment.

Here are the steps to try it:

1. Install DSpotterSDK_Maker_RP2040 Library in Arduino IDE Library Manager
2. Follow the steps to get the dev board serial number: https://tool.cyberon.com.tw/ArduinoDSpotterAuth/DocHowToGetBoardSerialNumber.php
3. Start the trial at: https://tool.cyberon.com.tw/ArduinoDSpotterAuth/CTMain.php
4. Fill the email, serial number and select desired language
5. Decide a Trigger words, this is the hint you start the conversation, just like "Hey Siri" or "Hello Google"
6. This engine can recognize at most 20 Commands, you can treat it as 20 keywords in your conversation that the chatbot will response
7. Confirm filled details and wait the reply email
8. Open example in Arduino IDE File Menu -> Examples -> DSpotterSDK_Maker_RP2040 -> Voice Recognition
9. Save the example
10. Copy the email attachment to the example folder
11. Change follow 2 #include line to the new file names

#include "CybLicense.h"

and

#include "Model_L0.h"             // The packed level zero model file.

Then upload the program to test. As show in the above video, trial mode requires wait 20 seconds in the middle.

After speech recognition, we need voice output response.

The RP2040 dev board can develop with 2 board cores.

The earlephilpower's arduino-pico board core support PWMAudio and I2S output. However, Arduino Speech Recognition Engine requires Arduino official Mbed OS board core.

So we need another way for voice output.

Direct playing voice with MCU may not the best option. Decoding and playing voice use up the processing power, and the voice clip use up the limited storage easily. Better alternative is offload the job to a voice module. The voice clips, in MP3 format, preloaded in the voice module so you don't need to concern the storage and how to decode it. And it also built-in amplifier, simply connect it to a speaker and send a command will play the voice loudly.

Note:

I am using voice module JR6001, if you are using another voice module please follow the data sheet modify the send message function accordingly.

Most robot toy use a DC motor emulate bionic motions. Simply use a motor driver module can reproduce a motion response easily.

Since Arduino Nano RP2040 Connect is highly integrated many feature in one dev board. The wire connection in this project is relatively simple. Just few wires connect to 2 modules.

Here are the connection summary:

RP2040    Voice Module    Motor Module
======    ============    ============
GND    -> GND          -> GND
3V3    -> VCC
GPIO15 -> BUSY
TX     -> RX
RX     -> TX
          SPK+/SPK-                     -> Speaker
GPIO28                 -> IN2
GPIO29                 -> IN1
VIN                    -> VCC
                          OUT1/OUT2     -> Motor

I have uploaded my sample code to the Github:

https://github.com/moononournation/PositiveResponseChatbot.git

However, the engine model needs to match the dev board serial number. You need import your own trial or release license and generated model files before using it.

This is the most important step in this project. I would like to have positive message response for each keyword, or called command in this engine. To avoid message repeat too much in the conversation, each keyword can have multiple response and appear randomly. At the same time some message can reuse in multiple keywords. All relationship managed in the responseMap.h.

Here are some response example:

• "Good Morning" will reply "Good Morning"
• "Hello" will say back "Hello", "How are you?" or "Let's play together!"
• "It's great!" will echo "great!" or "I think so."
• "I am cold" will considerate "wear more clothes"
• "It taste good" will reply "I wanna taste it"

After decided the response message map, it's time to make voice clips. You can record clips one by one with recorder app. Or you can generate it by TTS engine.

I invited a little girl help me record it and use Audacity tune to a higher pitch. Then export all voice clips in MP3 format and name it in number according to the response map. At last copy all files to the voice module.

Note:

JR6001 requires all voice clips name in 2 digits number and place into the ZH folder.

If you are using Mac OS, you need to double check with Terminal no any "." leading files in the folder:

ls -la ZH

The electronic part and the software are ready, but we still need a robot look. There are many off the shelf robot toy in the market, simply buy one or reuse one in your hand can do the job ;P

• Battery powered
• Attach it to the shoulder
• Refine the response map
• and more...