Wall Switches, Philips Hue & NodeMCU

Here's a way to turn your Philips Hue lights on or off using your existing wall switches and wiring -- but without interrupting household AC power to the lights and thereby losing app and voice control of them. The project also allows you to retain several other benefits of the Hue system that are lost when the lights lose AC power because they have been switched off.

If desired, additional control beyond on/off can easily be added to the wall switches once the basic project is complete.

Project budget: $10.

To every thing there is a season. A time to be born, and a time to die; a time to weep, and a time to laugh; a time to break down, and a time to build up; a time to ..., well you get the idea. It goes on a bit. In 2014, it was a time to remodel our kitchen.

Perhaps because of the eye-watering total expense, every incremental cost seemed laughably trivial. This is how I, known far and wide for being, um, careful with money, came to install eight Philips Hue down lights for general lighting and three Hue Lightstrips for cabinet illumination. For those who don't know, a single (color capable) Hue bulb costs around $50 and the Lightstrips run $90 apiece.

But what lights!! This was in early 2014 mind, when Hue was relatively new and Philips had the smart bulb space pretty much to itself. Friends and enemies alike grudgingly admired them and I could see they secretly wished they too had spent a fraction of their Subzero budget on such wonders. And all this before I fired up the Hue Disco app in full-on strobe mode (watch the vid!). Or perhaps that was just me, a child of the '70s.

So the Philips Hue system ranks up there with the wheel in terms of human ingenuity and imagination. After three years in service, the lights have performed pretty much flawlessly and the pain of the Visa bill that arrived a month after their purchase is but a distant memory. Philips has continued to improve its control app (no need to use third-party ones now, except of course the mighty Disco) and I see they are on to their third generation of the system, which I assume is even better than mine.

There was just one problem. While domestic lighting had now evolved into a new, transcendent form, we humans remained mired in the Old Ways. A Hue light requires no physical on/off switch to do its thing. A tappity-tap on the app or, latterly, a request to Alexa or Google Assistant is all one needs. If, that is, some family member hasn't turned off power to the lights via the legacy wall switches. Then you are forced, caveman-style, to get off your butt and flip the switch on the wall again. The horror.

Almost as bad, if power is interrupted via a wall switch and then restored, the Hue lights come back on at full bright white, not the artfully chosen dimming level and color that you had selected earlier. A real dinner-party buzz-kill. Plus, wall switches are dumb, really dumb. If they are left in the "on" state, but someone has turned off the lights using an app or voice control, toggling a switch would actually kill power to the lights and nothing would happen until you toggled the switch again. Annoying!

Finally, using the wall switches meant the lights would just, well, turn on, or turn off. Just like they did in 1980. Or 1880. Lights should fade on and off in a pleasing transition, like Someone out there originally intended. No, really. Think sunrise and sunset.

What to do then? For years I pondered the question. One option was to simply remove the switches. Or use some prophylactic switch cover such as this. But the former approach risked family rebellion and the latter was embarrassingly low-tech. And though it might take a couple of years in Guantanamo for me to admit it, controlling lights with wall switches was, well, pretty handy sometimes.

Philips itself has recognized the problem and its boffins have come up with a rather inelegant solution, albeit one that eliminates the need to fiddle with 120/240VAC. Really, though, some battery-powered plastic gizmo strangely reminiscent of the Clapper to control those beautiful lights? Not on my watch! Also, your original wall switches are still out there, lurking, ready to kill power to your lights and ruin your day.

No, I had to adapt the wall switches and wiring we already had. There had to be a way. Reader, there is!

Just for the record, I have no association with Philips and I very much doubt they would approve or endorse the bumble-headed project that follows.

With a project budget of $10 you can probably guess the component list is going to be short. In fact, it comprises just one thing: an ESP8266-based NodeMCU. Well under $10 even with Amazon Prime. Buy a few and have them shipped from China and you can have them for less than $5 a throw. That's less than a plastic cover for the switches!

Now, I'm being a little flexible with the truth here, in the grand tradition of all project budgets. You will also need a few other bits and bobs, notably a micro USB cord and some form of power supply (old iPhone charger cube would be perfect). You will also need some low-voltage hook-up wire and connectors. But if you are still reading you probably have this stuff lying around.

And of course, I am not including the Hue system itself, nor the wall switches and associated wiring which I'm assuming you already have.

If you've ever installed a switch or re-wired an outlet, you almost certainly have all the tools you need:

• Screwdrivers, pliers, wire strippers -- ideally ones designed for electrical work
• Voltage detector -- I really like this one
• Digital multimeter -- may or may not be required but will certainly make life easier if troubleshooting is necessary
• Depending on where and how you install the NodeMCU, you may need a drill, dremel, drywall saw, putty knife and such-like

This project requires some background knowledge which I shall airily gloss over to avoid repetition with the resources linked below. In particular, unless you are already familiar with the topic, you will need to review:

• A getting started guide for the NodeMCU. There are a few very good YouTube videos on this -- here's one -- and a few Instructables -- I think this one covers what you need to know. Key steps are installing the appropriate driver on your computer and, assuming you will be programming using the Arduino IDE, preparing the IDE for the NodeMCU. Both steps are covered in these resources. When I bought my first NodeMCU, I had trouble finding a driver that worked on my iMac and eventually used the one for OS-X Sierra here. That was at the beginning of 2017, however, and I think the official driver for Sierra has been updated several times since then. If you do run into problems with your computer not recognizing or seeing the NodeMCU though, don't give up! I can almost guarantee some diligent Googling and/or experimentation will solve your particular issue. Just the other day, for example, I found my MacBook Air wouldn't upload code to the NodeMCU (I'd previously just been using the iMac). After 30 minutes of frustration I tried changing the Board to version 0.9 in the IDE, even though my NodeMCU was 1.0 and my iMac had worked fine with that. Miracles, the code uploaded via my MacBook! Why? Who knows!!?

• The Philips Hue Developer Program getting started guide. You don't have to read the rest of the material on the developer program site for this project although it does have plenty of useful information for more advanced interaction with Hue. If you are planning on controlling more than one light, though, do take a look at the page discussing Hue groups.

• I stumbled on this site, which in fact was my specific inspiration for this project. It covers much of the same information you'll find on the Philips developer site but goes on to specifically discuss controlling Hue using an Arduino (in this case using an ethernet shield). I adapted the sample code found on this site for the "QuickStart" version of the coding described in Step 7.

• How three-way and four-way switches are wired. This is not required if you currently control your Hue lights from just one location with a single pole toggle switch (i.e., one with only two terminals to make or break the 120//240 or whatever VAC lighting circuit). Even in this case, however, a basic understanding of household wiring in your neck of the woods is required to accomplish this project safely.

• Lastly, a basic understanding of the Arduino world is assumed because in all likelihood you will need to modify my code to a greater or lesser extent to suit your particular needs (though the code itself is very simple).

After spending just a few minutes reviewing the background material referenced in Step 3, you will see that sending the Hue bridge some basic commands from a NodeMCU, or indeed from any other internet enabled Arduino, is pretty straightforward.

And if you are fortunate enough to be turning your Hue lights on and off from one location, it is likely you will immediately see how you can re-wire the wall switch at that location to leave power to the Hue lights on all the time and have your Node MCU handle the actual on/off business. If not, here's what you do: (FROM HERE, YOU ARE PROCEEDING AT YOUR OWN RISK: HOUSEHOLD POWER CAN KILL YOU!)

• Turn off the power at the breaker panel. Unless you are opening the main breaker make sure you have turned off ALL circuits that might be routed through your wall switch workbox. In many cases, even if you control your Hue lights from only one location that workbox may have wiring from separate circuits unrelated to your Hue lights.
• Using your voltage detector, check that you have indeed turned off all the power to the switch workbox.
• Disconnect the wires from the two terminals on the switch (in the U.S. they will in all likelihood be black) and connect them together with a wire nut.
• Connect a length of hook-up wire to each of the two terminals -- long enough to reach wherever you are going to locate your NodeMCU (one idea would be to install a low-voltage workbox adjacent to the wall switch workbox).
• Connect one of the hook-up wires (doesn't matter which one) to a ground pin of the NodeMCU.
• Connect the other wire to pin D1 of the NodeMCU.
• Et voila! When the switch is flipped pin D1, configured as an input and pulled up to 3.3V, is grounded and the code sends the appropriate command to the Hue bridge.

When we did the kitchen remodel I also went a little crazy on the electrical side. As you can see from the video, our kitchen is by no means large, but it is accessed from five locations. At four of these locations, I installed a switch to control the Hue downlights. For those with electrical form, and those who have diligently completed the background reading, this means the lighting circuit included two three-way switches and two four-way switches. Even if you only switch the lights from two locations, however, the "Simple Case" solution will not work. Somehow, we need a way to tell the NodeMCU that a switch located away from its own location has been toggled.

Of course, if I had thought about all this while the kitchen walls were exposed it would have been easy to run some control wire between the switches and back to the NodeMCU rather than ( or as well as) running the 14-3 cable. Now, with the walls closed up, it was not immediately obvious to me how I could send a signal from all four switches back to the NodeMCU.

The switches were all connected together, but only in an arrangement that would guarantee 120VAC power would always be available to one of the two traveler terminals on the three-way switch connected to the load (i.e., the lights).

Taking small bites out of the problem, the first step was to ensure power to the lights was always on. In my case, that was easy because the workbox with the three-way load switch had another switch for a separate circuit, so I could simply disconnect the black wire from the black, common terminal of the three-way Hue switch and connect it to the incoming hot wire of the other circuit. You may not be so lucky, in which case you will need to figure out another way of getting constant power to the lights.

Back at the other three-way switch, I disconnected the incoming hot wire from the common terminal of that switch (in my case, it was just a jumper wire that was taking 120V from another incoming hot wire in the same workbox). If your disconnected hot wire is actually entering the workbox hot you should try and disconnect it from power or at the very least securely cap it off.

Now, since all four switches were always connected to each other via the travelers, I had in effect exactly the same situation as in the Simple Case -- two "empty" terminals, one that would normally be connected to line power, the other to the load. The only difference was the two terminals were on separate switches on opposite sides of the room. If only there were another unused wire running between the two workboxes I could position the NodeMCU next to the load workbox, connect one end of the "spare" wire to the NodeMCUs ground and the other end of the "spare" wire to the common terminal of the other three way switch. That way, on each flip of any of the four switches the common terminal of the load three-way switch would float or connect to the NodeMCU ground -- and if D1 were connected to the common terminal of that switch we had totally replicated the Simple Case.

A wire, a wire, my Kingdom for a wire!

Doh!! There WAS a wire - the lighting circuit neutral, tucked away, forgotten, at the back of all the workboxes along with all the other neutrals still serving a useful purpose on other circuits. Disconnect that neutral (it served no purpose, now the lights were receiving constant power via another circuit), reconnect the incoming and outgoing neutral wire in the two four-way switch boxes et Voila!

For when I am no longer around, I attached a little flag to this neutral and/or what was now the Hue low voltage DC switch in each of the workboxes explaining the situation. While perhaps not code compliant, it is difficult to see how the arrangement is unsafe in any way, though I'd be glad to hear any contrary view.

A circuit diagram showing modifications to the original wiring is attached (note the diagram uses D7 as the sense pin of the Node MCU, which is mapped to pin 13 in the Arduino IDE; the rest of this narrative and the code assumes you will use D1, mapped to pin 5: either pin will work).

As I mentioned, although my kitchen wiring scheme was relatively complicated, I had the advantage of having installed it myself, so vaguely remembered how it was arranged. In most cases, you will not know which switch is which so your first order of business is to identify which is the "line" three-way, which is the "load" three way and which, if any, are four-way switches. For this, you will need to turn the power back on briefly, and have all switch plates removed. With the lights OFF, the "line" three-way will be a switch with three terminals and will be hot at the common (black) terminal. Use your voltage detector! The load switch will also have three terminals but be dead at the common terminal. Any remaining switches will be four-way (and have four terminals).

Note it is possible other wiring schemes have been used so if what I've just described doesn't match what you are seeing your only option is to delve deeper into into the site I linked to earlier, or use other resources, or call a competent electrician for help. One reason I've been so wordy though is to try and explain the underlying concept to non-electricians. If you understand that, you will likely be able to adapt your precise wiring arrangement to reach the same result.

Here's where you need to get creative for a neat final job -- and each situation will be unique. The NodeMCU is small, so in most cases it can be stashed away inconspicuously somewhere close to the load switch workbox (it is not a good idea, and probably not electrical code-compliant, to put it in the workbox itself). As already mentioned, one solution would be to install a low voltage workbox next to the switch workbox (or install a combi box) so the wires can be hidden and kept nice and short.

The other consideration is power to the NodeMCU. An arrangement by which an iPhone cube charger or similar is used in or near the workbox suggests itself. I was lucky again in this -- I could run the hookup wires from the load switch workbox straight out the back and into a slide-out spice cabinet on the other side of the wall, taking power from an outlet hidden behind the adjacent refrigerator. This meant everything was out-of-sight and yet easily accessible for code updates, maintenance, etc. Also I could admire my handiwork while cooking.

I tried two ways to code this project. Both work. The fastest way is to use the approach in this step. An alternative approach (only slightly slower) is to use the "Rules" approach described in the next step. The only difference is that the QuickStart is pretty much self contained while the Rules approach uses more tools from the Philips developer site. I had hoped that using the Rules approach would help with the latency issue discussed below but if it makes a difference it is hardly noticeable. My project is using the Rules approach because my gut tells me it is more robust. But I have no proof. Another advantage of using Rules though is that you can add additional control attributes without modifying the NodeMCU code.

In either case the code is pretty simple. I don't think it requires much explication beyond the embedded comments. The QuickStart code is available here (the file named KitchenOnOffGH1.1.ino).

In the main loop the NodeMCU is simply looking for a state change on D1 (mapped to pin 5 in the Arduino IDE). At the same time, it is polling the Hue bridge to check on the status of the lights. This is necessary because simply keeping track of the state of the lights internally is not possible -- the lights can be turned on or off via an app or voice control for example. I've limited the polling to once every second, though it didn't seem to affect performance to run it more regularly -- nor improve performance to run it less frequently. The code assumes the operation of any of the switches is a desire to change the lights from the current state, and sends the appropriate command (on or off) to the bridge. This command can also include many other lighting attributes -- brightness, color, transition time, etc. -- but it seems to me in most cases those attributes are best handled via an app or voice control. I was also nervous about loading up the command with more instructions, though a little experimentation didn't really suggest this was a problem in practice.

One issue I did run into was latency - at first it could take a couple of seconds for the lights to respond to a state change at the switch. This results in the untutored, or the impatient, toggling the light switch furiously while nothing seems to be happening. This in turn can cause a bit of a meltdown on the part of the NodeMCU and/or Hue bridge as they struggle manfully to understand what the heck the user is requesting.

After a bit of research, I think the latency is to do with how the NodeMCU sends client.print() packets. There's some splother on the interwaves about this, not much of which I understood. I did find that if I reduced the number of client.print() lines to the absolute minimum (by including in single statements what you'd normally send separately as variables) the latency was substantially reduced (but still at the one second mark). I've left the (now unused) variables and old client.print() statements in the code as comments so you can fiddle with them if you want. (Also, the file named KitchenOnOffGH1.0.ino is the original version with the longer delay if it helps you to compare).

A one second delay doesn't bother me -- when do you ever need instantaneous light, really? We are just conditioned to expect it. It's still orders of magnitude faster than asking Google or Alexa to switch on the lights. The delay also adds some mystery to the uninitiated as to what is going on. It reminds me of the delay after pressing the overhead light switch in some newer airliners. My wife takes a different view, however, and in early user trials managed to confuse the system regularly by world-class toggling.

I couldn't completely solve the latency issue but I did solve the system confusion problem by causing the NodeMCU to ignore any state change within 2.5 seconds of the first one. This allows Management to toggle away to her heart's content until the lights come on. I picked 2.5 seconds because this is well beyond maximum latency plus time to stop toggling after the lights react, thereby avoiding something akin to pilot-induced oscillation.

The "Rules" code is here (the file named KitchenOnOffRulesGH1.1.ino, with the original, non-optimized version being 1.0).

Using this approach, the task of capturing the state of the lights is taken over by the Hue bridge, as is deciding whether the wall switch state change is commanding an "on" or "off" signal. This eliminates the need for the NodeMCU to poll the bridge to check on status.

Offloading these tasks simplifies the NodeMCU code but does require a couple of additional steps using the Philips Hue API.

First, you need to follow the steps in the link to create a generic IP sensor.

Then you need to create a couple of rules that refer to that sensor. Follow the steps in the previous link for creating rules but refer to the screen shots of my "on" and "off" rules (attached) to get a better idea what you need to put in the message body. Obviously, you will need to change the sensor id number and the reference to your light and/or group ids. As mentioned in the comments included in the QuickStart code, note the different syntax for referring to individual lights rather than groups. In these screenshots, that means in the "actions" section changing "groups" to "lights" and "action" to "state".

That's about it, unless you run into problems...

Unlike many of my projects, this one went pretty smoothly once I'd had the, ahem, light bulb moment about the neutral wire.

I did run into an odd problem with very sporadic uncommanded light state changes that seemed to relate somehow to using our refrigerator icemaker/water dispenser. It didn't happen every time the icemaker was used, but annoyingly seemed to happen mostly when Management was getting ice late in the evening. So pretty much a worst-case scenario.

The NodeMCU is taking power from the same circuit as the fridge and my whole set-up (NodeMCU and hookup wire) is in very close proximity to the fridge. So my theory is that somehow using the icemaker will sometimes trigger the fridge's compressor motor and induce a spurious signal to the NodeMCU, or perhaps that the motor causes a power conditioning problem for the NodeMCU.

With that working theory some obvious solutions present themselves -- plugging the NodeMCU power supply into a separate circuit, using a different power supply and/or routing the hookup wire away from the fridge all seem reasonable options to me.

For the moment, though, I've simply included a confirmatory step in Version 1.1 of the code to check that a state change has indeed occurred, even though this inevitably increases latency very slightly (25ms). I haven't had the thing running long enough to be certain this has fixed things but will update if not.

Obviously you will not be faced with this precise circumstance but I mention it in case you experience something similar.