Robust Remote Solar Monitoring

This is a project for a real-world prototype that I’ve developed for a charitable organisation to try and help them. The organisation helps people in the poorest parts of the world with clean water and solar electricity. The prototype is of an end-to-end remote monitoring solution for their deployments so I'd class it as intermediate-advanced (but with enough Googling, I think even a beginner could do it :)).

Problem Statement: The organisation is unaware of the health and usage of many of their 300+ solar systems deployed in some of the most rural areas of the world. So when a problem arises, they are often informed of it after many weeks or months depriving the locals of essential energy. Even when they are informed of a failure, it's not possible to know which component is failing without visiting the site (often a long and arduous task). Also, it's not possible to know why it failed as there's no history of usage patterns and environmental conditions that would otherwise help to determine the probable cause of the failure.

Aim: To build a robust prototype that can be installed at remote sites and data gathered and sent to a platform for conversion, analysis and trend recognition.

Proposed Solution: Use a general purpose MCU and connect it to various parts of the solar system to sense electrical characteristics and environment conditions throughout the day and transmit them regularly back to a central location. This will then be analysed for failures and potential issues and automatically alert when relevant.

Architecture

The first diagram is the same as in the previous step but I've put the technologies and systems I'll be using for each component so we've got a reference point whilst going through this Instructable (and this Instructable, as much as it is fun, is somewhat long).

We'll be using an Intel Edison as the General Purpose MCU installed on the module. By itself, it's a tiny module but we'll be mounting it on a (very handy) Arduino Breakout board to speed up development. It's got 6 analog inputs, which is good as we'll need at least 4 analog inputs. You can install full-blown Linux (almost) on the Edison and work from there but I wanted to work directly with bare metal as 1) it's more fun, 2) I'd like to have the option open of frequently shutting down and starting up the device, 3) I don't want to have the 4GB of onboard flash memory to be eaten up by an OS, 4) I'm thinking of moving this to a RTOS at some future stage. A more comprehensive tutorial on the Edison can be found on this highly recommended set of videos.

The GSM module we're using for transmitting data is an AdaFruit SIM808 board as it has good support, lots of documentation and built-in GPS. Cool. Because this will be deployed in rural areas, there's a 50/50 chance that we'll be able to send data via 2G/EDGE so for maximum effectiveness, we'd like a fallback so if 2G is not available, resort to sending an SMS with highly compressed data.

Circuit Design

The second shows the essential components of the design on breadboard. The top and bottom power rails are running at different potentials so it's important to bear that in mind when plugging in components. The top power rail is designed to run at anything from 8V - 30V depending on the battery being used in the solar system.

The bottom power rail is a fixed 5V.

To step down the voltage to 5V, a linear regulator or voltage divider would be too inefficient as we're already in an power starved environment. Instead, I'm using this buck converter with an input range of 10-35V and fixed output of 5V. Another nice thing about this fixed output is we can plug it directly into the 5V port on the Arduino and bypass the internal (inefficient) regulator.

A temperature sensor is crucial in this environment as battery life is affected significantly by ambient temperature. Having an accurate idea of temperatures where systems are deployed would help quantify the impact they're having on the (often expensive) gel batteries. I'm using one from Seeed's Grove Starter Kit.

We'll be detecting current using an ACS715 from Pololu - which can detect 0-30A and convert it into an analog output.

2 voltage dividers are in play here to detect voltages from the battery and the PV array. Resistance on both paths is greater than 1 MOhm so losses are relatively negligible, especially given the batteries used in the system are ~100Ah.

The GSM/GPS module is mounted like an Arduino shield on the Edison breakout board. 2 Antennas, an external 3.7V Lipo battery and a SIM card are required for the module to work as we need.

The complete list of hardware needed for the setup :-

• Intel Edison with Arduino breakout board
• SIM808 module
• SIM Card
• 3.7 Lipo battery terminated with a JST connector
• Temperature sensor (I use this one)
• Button (I use this one)
• GPS Antenna with u.Fl connector (or with u.Fl - SMA Adapter)
• GSM Antenna with u.Fl connector (or with u.Fl - SMA Adapter). Quad-band if you also want to use this in Africa.

First, we attach the headers that came with the SIM808 module to the Edison Arduino breakout board.

Then, put the SIM808 module on top and solder it in place.

The SIM808 communicates via Serial and the Edison has a Serial port. Great, that sounds perfect for a hardware serial connection (I'll talk about why we can't use Software Serial in the next Step). Well, it's not quite perfect. If you tried using it now, there'd be no communication between the 2. The problem is the SIM808 board we're using uses pins 2 and 3 for RX and TX respectively. These pins are tied at the PCB-level so there's no way to change that.

The Edison breakout board uses pins 0 and 1 for RX and TX respectively.

So we need to tie SIM808-RX(pin 2) to Edison-TX(pin 1) and SIM808-TX(pin 3) to Edison-RX(pin 0). More soldering. Mine looks a bit messy as I managed to accidentally connect RX-RX and TX-TX (desoldering is painful - don't make the same mistake!)

The other consequence of this is that we can't use digital pins 2 and 3 for anything else any more but luckily that's somewhat OK as there's another 4 pins still available to us.

Use a multimeter to check continuity between these pins on the breakout board (and no continuity between TX and RX).

Without going further, we could do some basic testing now and see if the module's working. So if you want to do that, go ahead and skip to the Software step and then come back and continue with the next hardware setup step.

The next step is to insert the SIM card, plug in the 3.7 Lipo battery and attach the antennas - but be careful with the u.Fl connectors as it's easy to put too much force. I found the best way is to rotate them while putting a little pressure on them until they slot in.

For the SIM card, the holder on the board is a Mini SIM holder (the largest of the 3 sizes you can get). Although you can get an adapter, I only had a micro SIM on me and I didn't want to wait for 2 days while I got an adapter delivered so for testing I aligned my micro SIM carefully in the slot and closed the lid and it worked. Sweet. (I imagine this would work for a nano SIM too).

Measuring Battery Voltage

We need 2 voltage divider circuits - 1 for measuring Battery voltage and 1 for the PV voltage.

For the battery, the solar system setup will either be a 12V or 24V setup so we can expect a range 8-30V. For this, we'll use a voltage divider circuit with R1 = 1 MOhm and R2 = 147 KOhm. So for 30V, the max output will be ~4V (use this helpful site to see this for yourself). In the picture, you'll see for R2 I've used 2 resistors 100 KOhm + 47 KOhm.

With this combination, I've tried to maximise the resistance to lower wastage but not too low so as to compromise the accuracy of the readings. Before I plug the output of this into A0, let's make sure the voltage divider circuit is working as expected and there's no mistakes before we damage our board! Using an A/C -> D/C main adapter with adjustable VOut, I check that when I crank the input voltage up to 12V, I see 1-2V on the output pin of the voltage divider. So it works. Nice.

Measuring PV Voltage

The solar panel array's open circuit voltage (a measure of the theoretical maximum possible voltage) is 80V so we'll use R1 = 1 MOhm and R2 = 47 KOhm which, for 80V, will give a maximum output of ~4V. We'll plug this output into A2. Again, test the circuit as above before plugging it in to the board.

Measuring Temperature

I'm using the Seeed Grove Temperature sensor to measure ambient temperature at the deployment site. It uses a Grove connector but as we're not using a Grove base/shield/board, I wire in a connector into the Grove's GND, VCC (it's safe at 5V) and output pin (there's 2 pins, we can ignore the other one as we only expect 1 output). This will go to A5.

Measuring Current

For measuring current - and hence usage of the system - I'm using an ACS715 current sensor which can measure 0-30A. Sweet. It's pretty straightforward, it has 3 outputs - GND, VCC (again, safe at 5V) and OUT. OUT will go to pin A1 .

All of these inputs produce analog readings in the range 0-1023 and in the software, we'll need to scale and/or convert these.

Also, don't solder these in yet or anything. In the final step, we're going to put this on a veroboard/stripboard and make it truly portable. Sweet action.

Before loading the custom software, let's test the SIM808 board just on it's own to make sure it's interfaced correctly.

Assuming the Intel Edison drivers and Arduino board setup is complete :-

1. Download the test harness from here

2. We're going to be using hardware serial on the SIM808's ports (configured by default as port 2 and 3) as software serial isn't currently supported. So we need to remove these lines :-

#include SoftwareSerial.h

SoftwareSerial fonaSS = SoftwareSerial(FONA_TX, FONA_RX);<br>

SoftwareSerial *fonaSerial = &fonaSS;

and replace with

HardwareSerial *fonaSerial = &Serial1;

3. Before attaching the board to the USB port on our PC, we need to attach the external power supply aswell or we'll get a power surge on the USB as the combined unit now draws more current than the port can supply and we'll get a warning (see picture). Some older machines may not have USB surge protection so be careful to do this step first before connecting or the port could get blown.

4. We'll now run the test sketch and get a menu where we can test various functions like checking battery voltage, making a phone call, sending SMS, sending a HTTP POST to a website etc. Even without a SIM, antennas and external LiPo battery attached, I can test the ADC value (option 'a') to make sure everything is smooth. Remember to switch the baud rate on the Arduino Serial Monitor to 115200 or you'll just see a bunch of junk.

I've uploaded the custom software I created for this monitoring module here. There's 2 variables that should be modified before using this in a live environment :-

char* _dataDestinationUrl     = "__YOUR_URL_HERE.bluemix.net";

This is the Url to send data to - if you're using BlueMix (see my step later on setting up the data pipeline), the format will be similar to the above.

char* _smsDestination         = "+44_YOUR_NUMBER_HERE";

This is the number to send SMS' to when it fails to send to the URL above via GPRS (again, see the data pipeline setup to get this number).

In a nutshell, the crux of the algorithm in the custom software works as follows :-

1. Every hour*, run a cycle. In a cycle, do the following
   
   1. Take the following readings every* second for 1* minute
      
      1. Battery voltage
      2. PV voltage
      3. Inverter output (A/C current)
      4. Temperature
2. If this is the last cycle in a day (i.e. this this runs once every 24 hours)
   
   1. Transmit readings data
      
      1. Attempt GPRS transmit to website*
      2. On failure, compress data further and attempt SMS transmit to a number*
3. If this is the first cycle in a week (i.e. this runs once every 24*7 hours)
   
   1. Enable GPS module
   2. Wait 5* minutes
   3. Retrieve data from GPS module

   4. Transmit GPS data

      1. On failure, attempt SMS transmit to a number*
      2. Attempt GPRS transmit to website*
4. Charge the external LiPo battery if it's below a threshold*

* These are all (easily) configurable parameters

Of course, while testing I don't want to wait an hour before I get a reading or a week to get a GPS reading! So there's 2 #defines that can be toggled while developing/testing :-

#define DEBUG 

Having this defined a) speeds up time so 1 hour is ~1 second, b) outputs log statements and c) data isn't sent anywhere (so no mobile credits are wasted while testing).

#define RUNTIME_TEST 

Having this defined only speeds up time so it will deliver actual data to the server.

Needless to say, don't leave these in when deploying. That would be an expensive fail.

Getting there!

As mentioned earlier, I attempt to transmit the data via GPRS and, if that fails, via SMS.

To collect the data that comes in via GPRS, I signed up for an IBM Bluemix cloud account. This lets me host the service to capture the data, setup a MongoDB instance, store the data received and have a dashboard to analyse the data. I managed to set up the infrastructure side in about an hour so it's not too bad. I'd have liked to use the IDE for the Compose For MongoDB service by BlueMix that allows you to manage your MongoDB instance online (seemed somewhat like PhpMyAdmin for MySQL databases) but unfortunately it requires a credit card to sign up so I instead decided to build my database in scripts (I've uploaded a test script in the repository to load mock data - see next step).

Bluemix Setup

To setup a Bluemix instance (it's free for 30 days) :-

Start here, enter your details, hit confirm in the email they send you and then login.

You'll be on the "Apps" dashboard - from here choose "Create Application" (Don't hit "Create a Cloud Foundry App" as that'll create a Java-based app).

Head down to Cloud Foundry Apps and choose a language (I chose PHP)

Enter an App Name and Host Name (The Host Name will be used in the sub-domain given by Bluemix. E.g. if you type numaansAwesomeTest, your dashboard will be at numaansAwesomeTest.mybluemix.net). Hit Create.

Follow the instructions for downloading the BlueMix CLI and CF CLI and make sure you can see the Hello World page when you navigate to yourAppName.mybluemix.net.

You should also now go back to the Arduino IDE sketch and enter this Url for the destination for where your data will go:

<br>char* _dataDestinationUrl     = "yourAppName.mybluemix.net/handleData.php"

MongoDB Setup

Once that's working, we're going to set up a MongoDB instance. Go back to your Dashboard, hit 'Create Service' and select the Compose For MongoDB service.

You can leave the Service Name and Credential Name as-is but be sure to change the 'Connect To' option on the LHS to your application. Hit Create.

You should now see your MongoDB instance being provisioned along with 3 tabs at the top.

Go to the Connections tab and connect it to the PHP/other app instance you created earlier.

Once connected, it'll show up in the Connections tab - click into it. Head down to the Connections tab and next to the Compose For MongoDB service, hit Show Credentials.

Navigate to the bottom where it says uri and keep a note of that string - we'll need it later.

Twilio Setup

To collect SMS data, I signed up for a Twilio account (they have a free trial) which has a pretty decent API for listening to messages received. When I get a callback to say an SMS has been retrieved, we'll use a Twilio webhook to send it to our Bluemix cloud instance to persist it.

After signing up and logging in, head over to Phone Numbers on the LHS navigation.

Click Get Started on the screen that comes up and click Get Your First Twilio Number.

Head over to the settings for your new number, scroll down 'Messaging' and for the input box next to 'A Message Comes In', make sure Webhook is selected from the drop-down and enter your BlueMix url - something like

yourAppName.mybluemix.net/handleData.php

You can test it working by sending it an SMS from your phone to the number and you'll see it come up in the logs.

Again, go back to the sketch and update the following line with your Twilio-assigned number :-

char* _smsDestination = "+44_YOUR_NUMBER_HERE"

With our services all in place, we need to download our code onto them and see some analytics ! Here's an introductory page to IBM's Analytics platform.

First up, download the code from the Github page.

The most important files are

- mongo.PROD.config : This contains the connection string for our MongoDB instance (I'm planning to have this auto-build with a build server that'll choose the right environment but for now, I'm using PROD)

- handleData.php : Endpoint for our modules to transmit data to

- index.php : Default homepage that shows a dashboard with data received

- loadTestData.php : Loads test data into our database. This blats all existing data so use with Caution.

With the folder downloaded, open up the file mongo.PROD.config in a text editor like notepad. Find the MongoDB connection string from the previous step and enter it where it says :-

{ "connectionString" : _YOUR_CONNECTION_STRING_HERE_ }

Now upload the software using the bluemix cli executables we downloaded earlier. Just navigate to the root folder using a terminal (cmd.exe) and enter the following command :-

cf push rrm -m 512m

If that succeeds, navigate to the following :-

your-app-name.mybluemix.net/loadTestData.php

That should show "Completed Successfully" if it did. If for some reason that didn't work, a good place to start is the Troubleshooting section on the BlueMix site.

Next, let's see the dashboard ! Head over to your homepage :-

your-app-name.mybluemix.net

and you should see the dashboard. Awesome sauce.

Phew, all done. Almost :)

I wanted to get this module ready for deployment, so this section is for doing exactly that so I guess it's "optional".

I use the following items to make this prototype deployable :-

Switching (buck) regulator

ABS Case

Binding posts

To make this deployable, we need to mount the module on a board, sort out how it'll be powered, expose the connections safely and encase the entire thing.

I'm using a blue, translucent ABS case 7.5" x 4.5" x 2.5" to house the assembly. The Edison+GSM module, external 3.7V LiPo battery, resistors (for the voltage divider circuit), current and temperature sensors are all soldered on to a veroboard. If you've not come across them, veroboards are awesome. I love them. All the horizontal holes on a veroboard have a copper track running through them so they're all connected. Which means we can have 2 components at opposite ends of the board but still be connected and not have to do any wiring. I use this property abundantly when connecting the components.

I use a drill to make holes for the binding posts on the case and a 5mm and 10mm drill bit work great for making just the right sized holes (not my finest drill work admittedly but I had to wrestle quite a bit with the plastic). I also used a contact adhesive to make it more secure and, although somewhat temporary, for a prototype it works.

When deployed, we'll be connecting our module to the batteries in a solar system - these are either 12V (1 battery) or 24V (2 batteries in series), depending on the need for that location/village. We need to step this down to 5V and, as mentioned in the Design step, I'm using a switching regulator for that.

I was going to use a ring terminal for the external connections but 1) in a solar system setup, binding posts made more sense and 2) I didn't have ring terminal connectors on me at the time so these would have to do.

The order of the terminals looks odd but their order is for good reason. The charge controller used in the solar systems is a Victron one (the 24/800 one specifically) and I've matched the order of terminals to theirs so there's no messy crossover when wiring up. You can see them side by side in the pics.

In the final picture, you can see the setup alongside a Charge Controller and Inverter. The next step is to fix this in a large metal portable power box that'll also contain batteries, lightning arrestors etc. as seen in the last pic.

Thanks for watching !

PS: If you're doing this (or a similar) project for a charity, drop me a line for any (free) help I could give.