Detecting Emergencials Situations - Qualcomm Dragonboard 410c

Looking for security systems who works to monitoring emergencials situations, it's possible to notice that is too hard to process all the information recorded. Thinking about that, we decided to use our knowledge in audio/image processing, sensors and actuators to create one complete system that makes possible to predict situations where people's lives are in danger.

This project haver local sensor and remote devices to collect data and send to the dragonboard, which has processing power capable of extracting important information from the data received.

The remote device is an Arduino board with a module HC-06 turning possible to tranfer all the informations, and a low cost wide net capable of processing great amount of data.

First of all, you must to decide which sensors and actuatos that you're going to use, and make the archtecture sketch.

In our case, we're using these sensors conected in ARDUINO Pro Mini, listed below:

• PIR (Passive Infrared -- Presence sensor) http://www.dx.com/p/pyroelectric-infrared-pir-mot...
• DHT 11 (Humidity and temperature sensor) http://www.dx.com/p/dht11-digital-temperature-and...
• CO Sensor (Carbon Monoxide Sensor) http://www.dx.com/p/gas-sensor-mq-2-mq-3-mq-4-mq-...
• Noise sensor http://www.dx.com/p/5v-acoustic-sound-sensor-modu...

Actuators:

• motor servo http://www.dx.com/p/sg90-universal-9g-servo-motor...
• buzzer http://www.dx.com/p/diy-2-pin-ac-dc-buzzer-black-...

Communication:

• Bluetooth Module HC-06 http://www.dx.com/p/hc-06-serial-port-passthrough-...

To the Dragonboard 410c, we will have some sensors and softwares to process all the data inputs:

Sensors:

• DHT 11 http://www.dx.com/p/dht11-digital-temperature-and-...
• Sunlight Sensor http://www.ebay.com/itm/UV-Sunlight-Sensor-Grove-/...

Actuators:

• Relay http://www.ebay.com/itm/103020005-Seeed-Technolog...
• Led status https://www.seeedstudio.com/Grove-Green-LED-p-114...
• Buzzer https://www.seeedstudio.com/Grove-Buzzer-p-768.ht...

Now it's time to connect all the following components to the Arduino Board, creating a device that will receive the data from the ambience (noise, humidity, temperature, etc), and send to the Dragonboard by the bluetooth module HC-06.

It's necessary to pay attention on the conections, because all the sensor has specifcs places to conect.

On the system, it's possible to have more than one device to collect data. The more devices you have installed in the environment, more accurate the diagnostics generated by data processing is. Since it will be possible to extract a wider range of information that may be useful.

We decided to utilize an arduino board because has more compatible sensors, and it's possible to install this remote devices in diferents places, collecting more informations.

The local device is the DragonBoard 410c, wich process audio, video, digital and analog informations with your powerful SnapDragon 410 processor.

Placing of the components (Remote Devide)

Wich one piece has some pins that must to be connectected in the right pins on the arduino pro mini board.

Bluetooth module HC-06 has 4 pins:

• TX (Transmissor) -> connected on the RX Arduino's pin
• RX (Receiver) -> connected on the TX Arduino's pin
• VCC -> connected on the 5v
• GND

DHT 11 Sensor has 4 pins (but only 3 in use):

• Signal -> conected on a digital pin
• VCC -> connected on the 5v
• GND

PIR Sensor has 3 pins:

• Signal -> connected on a digital pin
• VCC -> connected on the 5v
• GND

Gas sensor (MQ) has 4 pins:

• Digital OUT -> conected on a digital pin (if you want a digital information)
• Analog OUT -> in our case, we're using this connected on a analog pin
• VCC -> connected on the 5v
• GND

Noise sensor (KY-038) has 3 pins:

• Signal -> connected on a analog pin
• VCC -> connected on the 5v
• GND

Code for Arduino remote Device:

/*<br> * Arduino send data through Blutooth
 * 
 * The value of sensors are read, concatened on 
 * String and send via serial port.
 */
 
#include "DHT.h"

#define DHTPIN  3
#define DHTTYPE DHT22
#define PIRPIN  9
#define COPIN   A6

DHT dht(DHTPIN, DHTTYPE);
float humidaty, temperature;
boolean pir = 0;
int co, mic;
String msg = "";
char nome[40];

void setup() {
  Serial.begin(9600);
  dht.begin();
}

void loop() {
  humidaty = dht.readHumidity();
  temperature = dht.readTemperature();
  pir = digitalRead(PIRPIN);
  co = analogRead(COPIN);
  mic = analogRead(A0); 
  msg = "#;" +String(humidaty) + ";" + String(temperature) +";"+ String(mic) +";"+ String(pir)+ ";" + String(co) + ";#" +"\n";
  Serial.print(msg);
  delay(2000);
}

Code explanation:

All the pins used in the Arduino are quoted at the beginning of the code and the respective libraries required for the operation of the sensors are initialized. All data will be passed to respective variables which will receive the values read from each sensor every 2000 milliseconds, then all of them are Concatenated in a String, then it is written in Serial. From there it is very easy of the pyton code present in the DragonBoard to capture such data.

To process all the data received and control the security system, is necessary use some softwares and libraries in the Qualcomm DragonBoard 410c.

In this especific project we're using:

Softwares:

• Python
• Arduino

Plataforms:

• Amazon AWS -> online server
• Phant -> Host data service

Libraries:

• OpenCV - Video Processing (https://opencv-python-tutroals.readthedocs.io/en/latest/)
• PyAudio - Audio Processing (https://people.csail.mit.edu/hubert/pyaudio/)
• Wave (https://www.physionet.org/physiotools/wave-installation.shtm)
• AudioOp (https://docs.python.org9http://scikit-learn.org/stable/install.html/2/library/audioop.html)
• Numpy (http://www.numpy.org)
• SciKit1 - Train and predict the machine learning (http://scikit-learn.org/stable/install.html)
• cPickle - Save the machine learning parameters (https://pymotw.com/2/pickle/)
• MRAA - Use the GPIOs (http://iotdk.intel.com/docs/master/mraa/python/)
• UPM - Use the GPIOs(https://github.com/intel-iot-devkit/upm)
• PySerial - Use to serial comunication with Bluetooth device (https://pythonhosted.org/pyserial/)

First of all you need to get the IP address from the Dragonboard, in order to do that, you need to turn on the DragonBoard connected with a mouse, a keyboard and a HDMI monitor. When the board turned on you need to connect to a network, than you go to the terminal and run the command:

sudo ifconfig

after that you can get the IP address.

With the IP address you can access the Dragonboard via SHH, to do that you need open a terminal in a computer connected in the same network as the board. In the terminal you can run the command:

 ssh linaro@{IP}

(you should replace {IP} with the IP address that you get in the Dragonboard).

The first lib you need to install is the mraa lib. In order to do that you need to run the following command in the terminal:

 sudo add-apt-repository ppa:mraa/mraa && sudo apt-ge;t update && sudo apt-get install libmraa1 libmraa-dev mraa-tools python-mraa python3-mraa

To install opencv for python you only need to run the command:

sudo apt-get install python-opencv

To install PyAudio you need to run the command:

sudo apt-get install python-pyaudio python3-pyaudio

The libs WAVE and AudioOp are already installed in the board. To install numpy you need to run the command:

sudo apt-get install python-numpy python-scipy

The last lib that you have to install is the scikit, to install it you need to have pip installed. Than you only need to run the command:

pip install scikit-lear

DragonBoard Connection with the Arduino by way of Bluetooth

The Bluetooth module (HC-06) was initially connected to the Arduino Nano according to the following example:

Using the Linaro (Operating System Used in the current project in DragonBoard) graphical interface, on the right side of the bottom bar click on the Bluetooth symbol and then click on "Setup New Device" and configure with your Bluetooth module leaving it paired. Verify that your module is actually connected by clicking on the Bluetooth symbol again, click on "Devices..." and see if the name of your device is listed and connected. Now select your device in the "Bluetooth Devices" screen and right click on it and note the port that your Bluetooth module is connected to (ex.: "rfcomm0"). Note: The port name that your device is connected to will be important for the next step to enable data exchange.

Establishing DragonBoard Data Exchange and Bluetooth

Basically we follow the step by step of the link: http://www.uugear.com/portfolio/bluetooth-communi... but we did not perform the part of the pairing only the execution of the python codes and Arduino. In python was used the serial library that is initialized in the port connected to bluetooth, hence the python code read the data of the sensors that are connected to the arduino through the bluetooth module.

To make the connections betwen the dragonboard and the componnents, we are using a type of shield called by Mezannine, developed by 96boards.

Using this shield, connect peripherals becomes much easier.

The connectors uses is from the grove's development kit, so it's just using an especif cable that connects both ways,

All the parts can be easily found in this website: https://www.seeedstudio.com/category/Grove-c-45.ht...

We are using this kits bellow:

• Grove Relay
• Grove Sunlight sensor
• Grove led socket
• Grove temp&humi sensor
• Grove Buzzer

The part of the program in the DragonBoard was coded in Python and the program used on the Arduino was developed in C++. Every 2 minutes the Arduino reads all the sensor attached to it. Than the Arduino send the reading to the DragonBoard by Bluetooth. The DragonBoard combines the reading came from the Arduino with the reading that it makes by the Mezzanine shield with the features from the audio and the video samples.

With this data, the Board try to predict if it is occurring an emergency situation. The Board send to the Amazon Web Service using the Phant the raw data and the prediction that it made. If the board predict that it is occurring a strange situation it try to warn the user blinking a led and buzzer in the Mezzanine and show at web application. In the web application it also possible to see the raw data to understand what is happening in this area.