Home Automation Using Raspberry Pi Matrix Voice and Snips

Introduction

Home automation allows us to control household electrical appliances like light, door, fan, and so on. Home automation not only refers to reduce human efforts but also energy efficiency and time saving.

I modified this project and will add some more features soon

Light control using snips

The aim of the project is to implement home automation using Raspberry Pi, Matrix Voice and Snips platform. In this project, Raspberry Pi is connected with Matrix Voice via GPIO pins and all the five lights (such as bedroom, kitchen lights) which we are controlling through voice command are interfaced with expansion GPIO pins of Matrix Voice. We are providing input(voice command) through Matrix voice mic and taking output (response) as light on/off from GPIO of Matrix Voice.

1. Setup Matrix OS on Personal computer and RPi.

2. Setup Matrix core on RPi so that we can install Snips

3. Installed Snips in RPi using client tool on personal computer.

4. Make an app on Snips platform and deploy it in RPi.

5. Make a four scripts and check it.

1. Matrix Voice ==> $55

2. Raspberry Pi 3 B+ ==> $35

3. 5V 2.5A Power Supply Adapter ==> $4.78

4. 5V Relay single channel ==> $4

5. 16 GB SD card ==> $7.68

Total cost of project is around $ 106

The Matrix Voice is a development board for building sound driven behaviors and interfaces. It is an user-friendly tool for simple to complex Internet of Things (IoT) voice app creation. It comes under two versions

1. Matrix Voice version

2. Matrix Voice ESP32 version

Both versions of these boards run the same on a Raspberry Pi, however, the Matrix Voice ESP32 version has the option to run standalone by programming the ESP32 module.

Here I am using Matrix Voice

Matrix Voice expansion GPIO

DF## pins are equivalent to GPIO## pins.

Raspberry Pi GPIO

Note:- The expansion GPIOs on the Matrix Voice are not connected to the RPi GPIOs. The expansion GPIOs mostly come from the FPGA.

Before installing Matrix OS make sure you have setup Matrix voice. For that you need

• Raspberry Pi 3 Model B+5V
• 2.5A Micro USB Power Supply
• Matrix voice

1. Insert flashed micro SD card into Raspberry Pi

2. Attach Matrix Voice onto Raspberry Pi GPIO pins

3. Power Raspberry Pi with micro USB power supply

Once done next is installing Matrix OS on your personal computer. You can use Windows or macOS or Linux. In this project I am going to use Linux (Ubuntu) to install Matrix OS

Before starting installing make sure you have installed NodeJS and npm

1. To install NodeJS, go to NodeJS and download Linux binaries(x64) of version 10.15.1 and then extract it with

tar -xvf node-v10.15.1-linux-x64.tar.xz <br>cd node-v10.15.1-linux-x64
sudo cp -R * /usr/local/ 
sudo apt-get update 
sudo apt-get upgrade

and check NodeJS and npm version

node -v

npm -v

Now we are ready to install Matrix OS

2. MOS contains a command line interface tool for controlling and managing your Matrix devices. To install it execute the following command in your terminal.

sudo npm install -g matrix-cli

3. Client needed to be configured by registering and then logging into a Matrix Labs account.
To register to the matrix account use command

matrix register

and then write username and password

For login use command

matrix login

4. With an account logged into the CLI tool, you can now register your matrix device. This will prompt you to enter a name and description for the device.

matrix register device

After the device is created, a set of unique login credentials will be generated. Save your device id and device secret as you will not be able to see the secret for this device again.

Now we have successfully installed MOS on our machine next step is RPi setup.

Before starting installation make sure you have installed NodeJS and NPM.

wget https://nodejs.org/dist/v10.15.1/node-v10.15.1-linux-armv7l.tar.xz

tar -xvf node-v10.15.1-linux-armv7l.tar.xz

cd node-v10.15.1-linux-armv7l 

sudo cp -R * /usr/local/

Then update the npm

sudo npm install npm@latest -g

1. Access the terminal of your Raspberry Pi via SSH or directly access it by connecting a screen, mouse, and keyboard. Then run the following commands to go into your home directory and install MOS on the Raspberry Pi.

cd ~/
curl <a href="https://raw.githubusercontent.com/matrix-io/matrix-creator-quickstart/master/install.sh" rel="nofollow"> https://raw.githubusercontent.com/matrix-io/matri...</a> | sh

After installing it will automatically reboot RPi.

2. Linking Device To Matrix Labs Account

To link your Matrix device to your Matrix Labs account, you need to create a file name.envrc in the home directory of the Raspberry Pi and then paste the device credentials you saved earlier while registering new device.

nano ~/.envrc

~/.envrc file look like this

export MATRIX_DEVICE_ID=dc7a2a81bf24

export MATRIX_DEVICE_SECRET=08621018e9d77h15i5n0t4r3alz0f06cd4f7e5544272u

3. Once file is created you need to make the credentials visible to the shell by running the following command. MOS will then be able to read it and properly register your Matrix device.

source ~/.envrc

4. Start MOS with the following command

node ~/matrix-os/index.js

If you see matrix OS is running you are done to check everything. Go back to your desktop.

1. Open the terminal on your Desktop and write the following command.

matrix use YOUR_DEVICE_NAME_OR_ID

This selects the Matrix device you want to interact with.

2. Once your device is selected, run these commands to verify that everything has installed properly. We are going to use hello world example Make sure node ~/matrix-os/index.js is active on RPi.

matrix install HelloWorld

matrix start HelloWorld

After running the start command you will see rainbow LED sequence.

matrix stop HelloWorld

1. Add the Matrix repository and key.

curl <a href="https://apt.matrix.one/doc/apt-key.gpg" rel="nofollow"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -

echo "deb <a href="https://apt.matrix.one/raspbian" rel="nofollow"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list

sudo apt-get update

 sudo apt-get upgrade

2. Install the the Matrix CORE packages.

sudo apt-get install matrixio-malos

sudo reboot

3. To install ZeroMQ

echo "deb <a href="http://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/" rel="nofollow"> http://download.opensuse.org/repositories/network...</a> ./" | sudo tee /etc/apt/sources.list.d/zeromq.list

wget <a href="https://download.opensuse.org/repositories/network:/messaging:/zeromq:/release-stable/Debian_9.0/Release.key" rel="nofollow"> https://download.opensuse.org/repositories/networ...</a> -O- | sudo apt-key add

4. JavaScript setup
Create a node project folder in the home directory of RPi

cd ~/

mkdir js-matrix-core-app (whatever name you want)

 cd js-matrix-core-app

 npm init

5. Installing npm Packages for ZMQ and Protocol Buffers
To install the ZMQ and MATRIX Protocol Buffers npm packages. Make sure you are in folder (name you give). Here I gave it js-matrix-core-app. This allows you to interact with Matrix Core through Node.js.

npm install zeromq --save

npm install matrix-protos --save

Now you are done with Matrix core

Snips is an AI voice platform for connected devices. It is an end-to-end solution with Speech Recognition and Natural Language Understanding, enabling anyone to add a customizable voice assistant to their product. Snips is the only deep-learning based voice platform that runs fully on-device. It can run without the need for a server or the cloud. It offers Privacy by Design.

For this make sure all the steps mentioned above are working
I. Raspberry Pi

1. To Install the matrix Kernel Modules. Run the following commands in your Raspberry Pi's terminal to add the matrix repository & key and update your repository packages.

curl <a href="https://apt.matrix.one/doc/apt-key.gpg"> https://apt.matrix.one/doc/apt-key.gpg </a> | sudo apt-key add -

echo "deb <a href="https://apt.matrix.one/raspbian"> https://apt.matrix.one/raspbian </a> $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list

sudo apt-get update 

sudo apt-get upgrade 

sudo reboot

2. To Install the Matrix Kernel modules. Use the command below

sudo apt install matrixio-kernel-modules

This allows the microphones on your matrix device to register as an ALSA microphone on your Raspberry Pi.

sudo reboot

3. Next command will test your microphones by raising your output volume, creating a 5 second long audio recording, and then playing that recording. Make sure you have select audio jack as output for that use command

sudo raspi-config

then blue screen appears, go to advanced option then go to audio and select analog jack.

amixer set PCM 100% && arecord recording.wav -f S16_LE -r 16000 -d 5 && aplay recording.wav

If you are able to hear the audio then you are done.Next we are going to install the SAM tool on desktop

II.Installing the SAM CLI Tool on Personal Computer(Ubuntu)
We are using the SAM CLI tool to create, manage and deploy our Snips assistants, to configure hardware, to view the logs of assistants on the Raspberry Pi, as well as to run your application locally for quick prototyping.

1. Install SAM into your computer with the following command.

sudo npm install -g snips-sam

2. Installing Snips.ai through SAM
We are going to install Snips.ai on our Raspberry Pi, Snips will be installed through the SAM CLI tool from our personal computer.

sam devices

This command should log all of the Raspberry Pis on your network with their IP address and hostname.
Note:- sam devices not required if you already know your RPi IP address Connect to your Raspberry Pi using command

sam connect ip address of Pi

you'll then be prompted to insert a username and password of RPi.

After you are connected use the init command from your personal computer to install Snips onto your Raspberry Pi.

sudo sam init

III. Configuring Snips.ai on Raspberry Pi
Move on to your Pi with Snips.ai properly installed, the next step is to edit the snips.toml file for configuring the mics. Open terminal and use command to open the snips.toml file

sudo nano /etc/snips.toml

Scroll down to where you see [snips-audio-server] and replace

# mike = "Built-in Microphone"<br>with the following:
mike = "MATRIXIO SOUND: - (hw:2,0)"

IV. Testing Snips.ai on personal Computer(Ubuntu)
Through your personal computer's terminal, you can use SAM to test the speaker and microphone of your Raspberry Pi.

1. To Test Raspberry Pi's speaker

sam test speaker

2. To Test the microphones on your Matrix device

sam test microphone

If both commands seem to work, you've properly configured Snips.ai on your Raspberry Pi.
If there is some problem use this command

sam setup audio 

Then it will ask whether you are using snips kit or not (I choose no) Then it will ask to select microphone. Here I am using Matrix Voice microphone Then it will ask to select speaker. Here I am using RPi audio jack as speaker.

To test Snips.ai is properly working by installing a demo assistant for the weather or not.

sam install demo

After installing as you see snips assistant is running. Now, test the assistant by saying

"Hey Snips!" 

"What will the weather be like in Madagascar in two days?"

The response from Snips should be

"You asked for the weather in Madagascar in two days."

Congratulations we have successfully installed Snips on RPi.

Making an app on Snips console

1. Creating A Snips Assistant & App

Sign into your Snips.ai and create an assistant. After sign in you page will like this

2. Create a new assistant

give name to an assistant, select language and then create it.Once created you will see page like this

3. Then click on add an app

Then select create a new app and give it a name whatever you want and create it

Once created you will see page like this

4. After that open the app, click on edit app.

You will see page like this. As you see I already created intent (lights). You will see no intent on your page.

5. Create a new intent

After creating you will see page like this.

6. Give training examples

7. Then create a new slots and give it name whatever you want. Here I am using three slots

• switch for on and off state
• room for different rooms such as bedroom, bathroom etc
• device for devices such as light, fan etc

8. Then give slot type name. Here I am building custom slot type.

• For room slot

After that add slot values "bathroom", "bedroom" and "kitchen"

• For switch slot

After that add slot values "on" and "off".

• For device slot

Give any name whatever you want.

After that add slot values such as "light", "fan" etc

9. Close the app, in order to tell which is slot double click on off and on and select slot name(you provided earlier). Then save it and you can test it by deploying it

Then save it and you can test it in window provided right hand side of the page.

Then deploy it

1. From your computer's terminal, sign in through the Sam CLI Tool

sam login

Login using your snips account

2. Connect to your Raspberry Pi. When prompted for a username & password, you can press the enter key to insert the Raspberry Pi credentials.

sam connect YOUR.PI.IP.HERE

3. Copy the cmd given in snips console which deploying an app and paste in your terminal of your computer

Assistant will be used to listen and respond to events from your Snips assistant

1. Go to the directory which we made during JavaScript setup in Matrix core. cd ~/js-matrix-core-app and create 4 new JavaScript files.

 touch assistant.js relay.js everloop.js func.js

2. Install MQTT to listen in on events from Snips.

npm install mqtt --save

3. Open everloop.js using any editorand paste the code given in program. Code is used to control the LEDs on the MATRIX Creator/Voice.

4. Open relay.js using any editor and paste the code given in program. It is used to turn the relay on and off.

5. Open assistant.js using any editor and paste the code given in program. It combines all the code from everloop.js & relay.js to control the everloop and relay switch through voice. It shows you how to listen to lightState intents and read slots.

var snipsUserName = 'ruchir'; // username of snips account 

var lightState = 'hermes/intent/'+snipsUserName+':lightState'; // name of intent you give while creating an intent

Once done now it is time to check.Open your personal computer(Ubuntu) terminal and run

sam watch

On the other hand, open RPi terminal and go to the directory

cd js-matrix-core-app/

node assistant.js  // run it

Relay ==> Matrix Voice

• 5V ==> 5V
• GND ==>GND
• Signal1 ==> GPIO0 (DF0)
• Signal2 ==> GPIO1 (DF1)
• Signal3 ==> GPIO2 (DF2)
• Signal4 ==> GPIO3 (DF3)
• Signal5 ==> GPIO5 (DF5)

Video