Speech Recognizer

Hi everyone..........

This is my second instructable that I am posting.

So welcome all.....

In this Instructable I am going to teach you about how to build up a voice recognizer using an arduino board.

So I think you have an experience with arduino boards before.If not , it is not at all a big huge problem here.But I recommend you to get used to it as it is very interesting to play with it and to create some cool projects out of it according to your creativity and knowledge on it.

So for the people who do not have a previous experience using arduino :

Arduino is an open source computer hardware which is manufactured by a company that have a large community of designers and manufacturers.it can be considered as like a small computer that can be used to control other electronic circuits .

Arduino is programmed in an environment developed by themselves which can be easily downloaded from their website.

• Just search in the google "download arduino"

• Click on "Arduino - Software"

• You will be able to see "Download the Arduino IDE"

• Choose depending on you operating System

• Download and install it

So you have successfully installed the software and you can write your code for arduino and with the help of a cable you can connect the arduino board to the computer and can insert the code .

• Inexpensive

Arduino boards are inexpensive on comparison with other micro-controller platforms. It will cost just around $50.

• Cross-platform

The Software for Arduino works on Windows, Macintosh OS, and Linux operating systems. If we think of other micro-controller systems it will work only in Windows or in other words limited only to windows.

• Open source and extensible software

The software is open source , so that people began to study about it deeply and to include libraries(which includes a set of functions for it's operation) of other programming languages.

• Simple and easy programming environment

It is easy to use the Arduino IDE(software which we have already discussed about....) for people including beginners , since a large amount of resources are provided by Arduino themselves which is available in the internet for free.So feel free to learn more about it.

• Open source and extensible hardware

The plans of the Arduino boards are published under a Creative Commons license, so people who have an experience in circuit designing can make their own version of the module,they also have the right to extend the technology and can improve by adding features to it.

So I have already told that this project is mainly focussed on voice recognition using the Arduino and to allow it to carry out some tasks.

Talking more clearly ........

It pick-up the sound signals provided by the user , which can be detected by blinking of the LED after that is is converted to synthesized speech.

The main components required for this project are :

• Arduino Due x 1

• Spark Fun Electret Microphone Breakout x 1

• Spark Fun Mono Audio Amp Breakout x 1

• Speaker: 0.25W, 8 ohms x 1
  

• Breadboard x 1
  

• 5 mm LED: Red x 3
  

• Resistor 330 ohm x 3

• Jumper wires x 1

• Soldering Iron x 1

• BitVoicer Server

It is a speech recognition and synthesis server for speech automation.

1. Audio waves are being found out then it captures these waves and and are amplified by Sparkfun Electret Breakout board.

2. The amplified signal attained from the above process will be digitalized and buffered/stored in the Arduino board using its analog-to-digital converter (ADC) present in it.

3. The audio samples will be provided to BitVoicer Server using the Arduino serial port present.

4. BitVoicer Server will process the audio stream and then it recognize the speech it contains.

5. The recognized speech will be mapped to the commands that has already been defined by itself before ,then it will be sent back to the Arduino. If one of the commands consists in synthesizing speech, BitVoicer Server will prepare the audio stream and send it to the Arduino.

6. The Arduino will identify the commands provided and perform the certain appropriate action. If an audio stream is received, it will be queued into the BVS Speaker class and played using the DUE DAC and DMA.

7. The SparkFun Mono Audio Amplifier will amplify the DAC signal so it can drive an 8 Ohm speaker and can be heard through it.

• The very first step is to wire different components in the breadboard and also with the arduino board as shown in the picture.

• Remember the arduino board used here is DUE ,there are different other models that have been manufactured by Arduino each works on different voltage levels.

• Most of the Arduino boards works at 5 V ,but the DUE runs at 3.3 V.

• The DUE already uses a 3.3 V analog reference so you do not need a jumper to the AREF pin.

• Oh sorry, I forgot to say an AREF pin is an "ANALOG REFERENCE PIN" which is present in an arduino board as shown in the next figure(It is an arduino UNO but it similar in the similar site in the case of DUE).

• The AREF pin on the DUE is connected to the micro-controller through a resistor bridge.

• To use the AREF pin, resistor R1 must be de-soldered from the PCB[Printed Circuit Board].

So we have to upload the code onto the Arduino Board , so that it can work depending the instructions provided in the code.

It is very easy to do this.I will explain each one of them in detail, about what they do and how they work.

How to Install a Library ?

So before that we have to know how to install BitVoicer Server libraries into the Arduino IDE ,.that onto the arduino software.

• So for that open the Arduino IDE

• On the top panel click on "Sketch"

• Then Click on "Include Library"

• On that click on "Manage Library"

• Then the library manager will open and we can see a list of libraries that is ready for installation or those which have already been installed.

• Search for the library to install and then select the version number.

• Here we are installing BitVoicer Server libraries , which is necessary for this project.

How to Import a .zip library ?

• Libraries can also be distributed as a ZIP file or folder.

• The name of the folder is the name of the library.

• Inside the folder will be a .cpp file, a .h file and often a keywords.txt file, examples folder, and other files required by the library.

• From the version 1.0.5 of Arduino IDE , you can install 3rd party libraries inside it.

• Do not unzip the downloaded library, leave it as is.

For that go to sketch > Include Library > Add .zip Library

Select the location of the .zip file and open it.

Return to the Sketch > Import Library menu.

If it is properly imported then that library is to be seen at the bottom of the drop-down menu when you navigate.

This is the program which has to be uploaded in the Arduino.

It can be done simply by connecting the Arduino Board to the computer and upload it to the board.

Now let us look at what each of the functions written in the code actually do ...........

1. Library references and variable declaration

Before talking about this we have to know and understand some basic terminologies.These include :

• BVSP
  • It is a library that gives us almost every resources which is required to exchange information with the BitVoicer Server .

• There is a protocol present known to be as the BitVoicer Server Protocol which is implemented through the class of of BVSP. This is required in order to interact with the server.

• BVSMic
  • It is a library which implements all the things which is required to record the audio using Analog-to-Digital Converter(ADC) of the Arduino.

• This audio is stored in the class internal buffer and they can be retrieved and then it can be sent to the speech recognition engines which is available on BitVoicer Server.

• BVSSpeaker
  • It is a library which contain all the essential resources which is required to reproduce audio streams sent out from the BitVoicer Server.

• For that the Arduino board must have an in-built Digital-to-Analog Converter(DAC).

• Arduino DUE is the only Arduino board that have an integrated DAC.

• BVSP, BVSMic, BVSSpeaker and DAC libraries,the reference to these are written on the first four lines which makes up the staring of the program.

• When you install BitVoicer Server you can find BitSophia which provides all these four libraries.

• When the user add a reference to the BVSSpeaker library the DAC library which have been earlier mentioned will be automatically invoked.

• The BVSP class is used to communicate with BitVoicer Server

• The BVSMic class is used to capture and store audio

• BVSSpeaker class is used to reproduce audio using the Arduino DUE DAC.

2. setup function

The setup function is used to carry out certain actions like :

• To set pin modes and their initial states

• To initialise serial communication

• To initialise BVSP class

• To initialise BVSMic class

• To initialise BVSSpeaker class

• It also sets “event handlers” (function pointers) for the frameReceived, modeChanged and streamReceived events of the BVSP class.

It carries out five major operations :

1. keepAlive() function

This function is to request the server about the status info.

2. receive() function

This function is to check whether the server have sent out any data or not.If Server sent any data it will process it.

3. isSREAvailable() , startRecording() , stopRecording() and sendStream() functions

These functions are used to control the various settings on recording the audio and after when it attains the audio it will sent this audio to the BitVoicer Server.

4. play() function

This function is used to play the audio which is been queued at BVSSpeaker class.

5. playNextLEDNote()

This function is used to control how the Led should blink.

6. BVSP_frameReceived function

This function is called every time when the receive() function starts to identify that one complete frame has been received.Here we are running the commands attained from BitVoicer Server. The commands that controls the blinking of LEDs are of 2 Bytes .In that first byte indicate the pin and second byte indicates pin value.Here we are using the analogWrite() function to set the appropriate value to the pin. At that time we also have to check whether playLEDNotes command, which is of Byte type, has been received. If it has been received, I set playLEDNotes to true and it will monitor and marks the current time. This time will be used by the playNextLEDNote function to synchronize the LEDs with the song.

7. BVSP_modeChanged function

This function is called every time when the receive() function identifies a change in mode in the outbound direction (Server --> Arduino). BitVoicer Server can send framed data or audio to the Arduino. Before the communication goes from one mode to another, BitVoicer Server sends a signal. The BVSP class identifies this signal and raises or flags the modeChanged event. In the BVSP_modeChanged function, if the user detect the communication is going from stream mode to framed mode, he/she will know the audio has ended so the user can tell the BVSSpeaker class to stop playing audio.

8. BVSP_streamReceived function

This function is called every time when the receive() function identifies that audio samples have been received. It simply retrieve the audio and queue them into the BVSSpeaker class so the play() function can reproduce them.

9. playNextLEDNote function

This function only runs if the BVSP_frameReceived function identifies the playLEDNotes command. It controls and synchronizes the LEDs with the audio sent from BitVoicer Server. To synchronize the LEDs with the audio and know the correct timing, a free software Sonic Visualizer can be used. It allows us to watch the audio waves so the person can can tell when a piano key was been pressed.

We have now set upped BitVoicer Server to work with the Arduino.

There are four main solution objects for a BitVoicer Server : Locations, Devices, BinaryData and Voice Schemas.

Let us look at these in detail :

Locations

It represent the physical location at which the device is being installed.

We can create a location called Home.

Devices

They are considered to be as the clients of BitVoicer Server.

Like creating a location we can create a Mixed device , for easiness let us name it as ArduinoDUE .

Sometimes some buffer overflows can happen so to eliminate it I had to limit the Data Rate in the communication settings to 8000 samples per second.

BinaryData is a type of command BitVoicer Server can send to client devices. They are actually byte arrays you can link to commands.

When BitVoicer Server recognizes speech related to that command, it sends the byte array to the target device.

So for that reason I have created one BinaryData object to each pin value and named them ArduinoDUEGreenLedOn, ArduinoDUEGreenLedOff and so on.

So I had to create 18 BinaryData objects , so I suggest you download and import the objects from the VoiceSchema.sof file which is provided below.

So what is a Voice Schema ?

Voice Schemas are where everything comes together. the major role of them is to define how the sentences should be recognized and what all commands need to be run.

For each sentence, you can define as many commands as you need and the order they will be executed.

You can also define delays between each of the commands provided.

BitVoicer Server supports only 8-bit mono PCM audio (8000 samples per second) so there will be a need to convert the audio file to this format,there are so many online conversion tolls present today and I recommend https://audio.online-convert.com/convert-to-wav.

You can import (Importing Solution Objects) all solution objects I used in this project from the files below.

One of them contains the DUE Device and the other contains the Voice Schema and its Commands.

There you go !!!!!

You have made an awesome project and you can talk to it .

So start talking....................

You can blink up the LEDs and at the same time you can say it to sings a song if it need, its code has already been provided.

So I have completed my Second Instructable !!!!!!!!!!

Yeah......

I think everyone understood it .....

If any one have any queries please feel free to ask me.

I will come up with an excellent Instructable next time.....

Bye...

See You soon...............