The New Hub - ESPNow to MQTT/HTTP

***** My new version Hub 3 is out, check it out here. *****

A small but crucial part of my ultra low sensor system is the hub. The purpose of the hub is to take the ESPNow messages coming from my sensors and push them over Wifi to an MQTT server.

If you have been following me, you might recall my original hub which was composed of 2 subsystems. The receiver and the gateway.

The receiver had one simple job: listen and filter ESPNow messages and send them over serial, to the gateway. The gateway, however, was a bit more complicated. It was responsible for multiple things including parsing and saving the ESPNow messages, maintaining a wifi connection, publishing MQTT messages and serving the setup and web portal.

Originally, I used 2 wemos d1 minis: one as a receiver and one as a gateway and sandwiched them together. It was very simple and did the job. However, I felt it needed to be enhanced with more peripherals: like a screen, an led or a buzzer and the firmware also needed more options and a UI makeover.

This is the new Hub. I added an OLED screen at the top and a buzzer and LED indicator at the bottom - as you can see above. I used the OLED display for so many things, including the boots sequence information ( Wifi and MQTT connection info). I have also used it to display the progress of over the air updates. I really liked that one. And when the device is idle, it shows the total number of connected sensors and the IP address. And each time the hub gets a new update from an sensor, it displays it. So this is a door sensor example and this is a climate sensor one. The OLED was a very useful addition. It is truly versatile.

The other addition was the buzzer and the LED. They are used for things that need more urgent attention. This switch on the right hand side can toggle between audible or visual alerts. So I can make it beep or flash. I learned from the first hub that having constant beeping around you is very annoying so that is why I added that switch for an option for a quiet mode. I added those two, mainly to get alerts when the batteries run low. This allows me to replace the batteries when needed.

A shoutout to PCBWay for providing these PCBs for my project. I have been using their service for over 3 years and had nothing but great experience and quality. If you are a hobbyist and in the market for PCBs, I highly encourage you to give them a visit. Get 10 PCBs for $5 USD at https://www.pcbway.com

Another major update is the firmware. I almost rebuilt it from the ground up. This started when I had issues fine tuning the portal library I used. So I wrote my custom code to deploy the setup portal and serve the web pages. You can check out a previous video for an intro on that subject. The HTML coding also got more structured. I started using frameworks like Bootstrap and JQuery. The above is the new UI. It is also more responsive and user friendly. And it looks way better if you ask me.

On the main page, I get to see all the discovered sensors at a glance. For each sensor, the ID is displayed at the top. This will later be used to identify it. This is basically the MAC address of the sensor.

The sensor status is right below it. In the case of a switch, this shows OPEN or CLOSED. And in the case of a temperature sensor, it displays the temperature and the humidity reading when available.

Further below is the battery voltage and a small icon reflecting the battery level. And all the way at the bottom, it shows when it was the last updated.

At the top right corner of the screen, I can access the device settings page. This is where all the settings can be changed, like the wifi credentials, MQTT info and the temperature unit. This can toggle between Celsius or Fahrenheit. It is worth noting that the sensors themselves send the readings in Celsius, but it is the hub that does the conversion on the fly before displaying them or sending them externally.

For MQTT, the sensor updates are published to a unique topic using this format. The sensor ID is the ID I showed earlier, which is created from the sensor's MAC address. The payload is JSON formatted and is different depending on the sensor type.

So far, I have 2 types: switch and climate. Switch is used for sensors that are either OPEN or CLOSE type, like a door or a motion sensor. And climate is used with the temperature and humidity sensors types.

Obviously with the addition of the climate sensor, I needed to adjust the ESPNow data structure I was using slightly. Before, it only had Boolean variable for the on/off sensors. But now, it has 2 extra values to be able to handle the climate type.

One thing I am experimenting with is 'auto discovery and setup' with external system using MQTT. For example, in Home Assistant, you can send the attributes of a sensor over MQTT and it gets added automatically without any setup. I tested it and it is working very well and I might explore this more.

A new addition to this firmware is the HTTP integration which is very similar to the MQTT except that it needs to be pulled using a URL. Accessing the URL and passing it the sensor ID, it returns a JSON with the sensor details just like the MQTT payload.

And that pretty much wraps up the new hub. This was very exciting and it has taken me weeks … to design, build, test and assemble. What I am really missing now is a 3D printed case!

You can buy the Hub at my store - https://store.mrdiy.ca/p/thehub_ver2

But if you are planning to build one, here are the BIN files for the Gateway (for ESP-12F 4MB FS:none OTA:~1019KB) and the Receiver ( ESP01 1MB FS: none OTA:~502KB)

Thanks for reading!

If you found this useful, maybe consider visiting my YouTube channel and say hello. I will be posting future updates about this project.