CLOUD MONITOR With AWS & ARDUINO - Electric Boy

It is a simple project - turn a light on when something goes wrong... Becoming increasingly numb towards notifications with so many dashboards on our computers these days, how can we make sure we don't miss the really important ones. The answer is a physical status indicator. Or more specific to the task, a Cloud Monitor, that can sit on your desk - always in view. As the name suggest, the monitor will help to keep an eye on the health of your cloud services (...or anything else really, the sky is the limit, excuse the pun). Even you, like me, need to make one? Even if not, you may have an idea for a your future IoT project.

Well, if you're ready, let's start!

• COMPONENTS AND SUPPLIES

_ Arduino Micro e Genuino Micro (1 unit) ...or any smallish Arduino compatible - in my case a freetronics LeoStick (http://www.freetronics.com.au/collections/arduino/products/leostick)

_ ThingM BlinkM - I2C Controlled RGB LED (1 unit)

_ Mini cloud light (1 unit) ...or any other translucent vessel of your choice

_ USB-A to B Cable (1 unit) ...or any old USB cable with a type-A plug

• TOOLS NEEDED

_ Soldering iron (generic)

• APPS & ONLINE SERVICE

_ Amazon Web Services AWS Lambda (https://aws.amazon.com/it/lambda/)

_ Amazon Web Services AWS IoT (https://aws.amazon.com/it/iot/)

The night light already comes with a LED built in - cold white in my case. I thought it would be nice to indicate different status by different colours. So I only kept the cloud shaped casing. For the brains of the operation I picked the smallest Arduino compatible I had available: The Freetronics LeoStick has been my preferred prototyping platform for years and I have plenty spares. It comes loaded with good stuff: a piezo speaker, two RGB LEDs (one is tied to power, RX and TX though) and best of all, you can simply plug it into a USB port - no external FTDI or cable needed. It is also tiny yet breadboard compatible.

Why didn't I choose an ESP8266? To be truely wireless, you might as well cut the power cord - which makes things a little more complicated for adding a battery and inconvenience of recharging. Since the cloud monitor will sit next to my computer it is much easier to use USB power. Also setting up the Wi-Fi connection isn't always straight forward.
Based on the ATmega32u4, the Arduino Micro and LeoStick are sharing the oddity of having I2C data on D2 and clock on D3. This becomes relevant when connecting the BlinkM RGB LED. Unlike the common Atmega328 boards where you can simply plug the BlinkM shield into the headers A2..A5, this won't work here (I didn't bother with the soft I2C library).

By desoldering the male headers VCC and GND on the BlinkM, I could then extend those with wire and keep everything in a plug-able little package.
The BlinkM has its own micro controller on board and allows for advanced applications: e.g. play scripted colour patterns without an Arduino connected. I almost feel a WS2812 (Adafruits NeoPixels are great) would have served me better - alas I had none available. To finish up the hardware bit, I cut the opposite end of the male type-A USB plug, threaded it through a pre-drilled hole near the base of the cloud light and soldered the wires to the LeoStick (red:5V, white:Data-, green:Data+, black:Ground).

The only strong requirement I imposed on myself, was to have the monitor run behind a firewall. Though a crucial feature, this made web hooks for event changes impractical. A polling mechanism is costly in terms of TCP traffic and may delay events depending on polling frequency.

The solution is found in WebSockets which provide full-duplex communication. Amazons IoT service provides a message broker that supports MQTT over WebSockets. As it turns out, the service can be called without having to configure Things, Shadows, Policies or Rules.

There is a device SDK available for the Arduino Yún and some efforts are made of porting the SDK to other platforms like the ESP8266. But because the monitor will always be connected by serial interface, I decided early on to have a NodeJS application (run on the desktop computer) to implement the client API and use the Arduino only to receive and display colour codes. That way changes can be easily made in JavaScript, without having to bother with firmware uploads.
For testing a little example infrastructure is needed. Let's say we have a load balancer enabled across availability zones that does health checks on a web server instance and auto scaling policies based on CPU load. The corresponding CloudFormation template can be ▶️ viewed in the Designer or ▶️ created directly from the console. Note: some of the services in this stack may incur charges.

I extended the template with properties for the Lambda function and necessary permissions. Later require the IoT REST API endpoint to be inserted as a parameter. To automate this, I wrote a small shell script that uses the CLI to request the ARN (> aws iot describe-endpoint) and then calls create-stack with the parameter in-line. Or you can still do it by hand:

// RETRIVE IoT REST API ENDPOINT

> aws iot describe-endpoint

// CREATE STACK
> aws cloudformation create-stack --stack-name MiniCloudMonitor --template-body file://cfn-template.json --parameters ParameterKey=IotRestApiEndpoint,ParameterValue={IoT_REST_API_ENDPOINT} --capabilities CAPABILITY_NAMED_IAM

// DELETE STACK
> aws cloudformation delete-stack --stack-name MiniCloudMonitor

Ideally I should use the same alarm thresholds that triggers the auto scaling, for also calling the Lambda function and that way update the status of the monitor. Currently this is only possible when using SNS as an intermediate. At the time this extra layer felt redundant and I decided to use CloudWatch EC2 lifecycle Rules to call the Lambda directly. Still, I want to explore the option of SNS → Lambda in the future.

I started by writing the Arduino Sketch. The main loop() is reading Chars from the serial connection and building a String until it receives a newline character. It is then assumed a hex colour code was sent and the appropriate I2C command is written to the BlinkM LED. This is not so much about efficiency as convenience. The complete sources for this Sketch and other files can be obtained on GitHub. Following are some relevant code snippets:

void loop() {

while (Serial.available()) {

char inChar = (char)Serial.read();

if (inChar == '\n') {

long number = strtol(inputString.c_str(), NULL, 16);

byte r = number >> 16;

byte g = number >> 8 & 0xFF;

byte b = number & 0xFF;

BlinkM_fadeToRGB(blinkm_addr, r, g, b);

inputString = "";

} else {

inputString += inChar;

}

}

}

The NodeJS App has to implement interfaces to AWS and Arduino. Later can be accomplished in just a few lines of code when using the excellent serialportpackage:

var serialport = require('serialport');
port = new serialport(

PORT_COM_NAME, {

baudRate: SERIAL_BAUD_RATE

});

port.on('open', function() {

...

});

port.on('error', function(err) {

...

});

Connecting to AWS IoT demands hardly much effort either. The only pitfall is to know that using MQTT+WebSockets over port 443 requires authentication through Access Keys. The SDK will read these from the environment variables. It may be necessary to explicitly export AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

var awsiot = require('aws-iot-device-sdk');
var device = awsiot.device({

clientId: 'MiniCloudMonitor-' + (Math.floor((Math.random() * 100000) + 1)),

region: AWS_REGION,

protocol: 'wss',

port: 443,

debug: true

});

device.on('connect', function() {

...

device.subscribe(MQTT_TOPIC);

});

device.on('message', function(topic, payload) {

if (port && payload && topic == MQTT_TOPIC) {

var message = JSON.parse(payload);

if (message.hasOwnProperty(MQTT_JSON_KEY))

{ return;

}

}

});

The Lambda function accepts a colour code as an input parameter - not pretty, but very flexible at this stage. To be able to publish to the MQTT topic, it instantiates an IotData object, which does require the IoT REST API endpoint. The CloudFormation template took care of that during creation of the stack.

var AWS = require('aws-sdk');
var mqtt = new AWS.IotData({

endpoint: process.env.MQTT_ENDPOINT});

exports.handler = function(event, context, callback) {

var params = {

topic: process.env.MQTT_TOPIC,

payload: '{\"colour\":\"' + event.colour + '\"}',

qos: 0

};

mqtt.publish(params, function(err, data) {

callback(err);

});

};

I truly enjoyed bringing a virtual event "born" in the cloud into the physical world. And as my little pet project it was heaps of fun. To take this to the next level I would consider...

• improving on robustness and exception handling
• explore better ways to integrate AWS cloud metrics
• experiment with more physical indicators like gauges, bar graphs, ...
• have the option to move to other platforms like Azure, Google, Heroku, ...
• monitor application specific events for Jenkins, GitHub, ...

I hope you enjoyed reading this guide and maybe even picked up something new along the way. If you can think of a different/better way to do things please share it in the comments below. And of course, if you spotted mistakes a heads up would be highly appreciated. Thanks for your time.