Open Source Smart Home With Touchscreen Control Panel

Using a Raspberry Pi and 7" touchscreen with nymea:core and nymea:app to build a smart home system with touch control panel.

The story

For a while now I'm using nymea as my smart home solution with nymea:core on a Raspberry Pi and nymea:app on my phone. Initially I wanted to wall-mount a phone or tablet but results weren't that great. Phones are built for a pocket and that causes a lot of small fiddly issues when used in a different context. One example is waking up the screen: Phones typically have a hardware button on the side to turn on the screen which gets a real challenge to do if the thing sticks on a wall. Next thing is that mobile operating systems don't like apps running all the time in kiosk mode, yes, Android has a feature to lock an app to foreground but there are still cases where it wouldn't do the right thing. There's more such issues I've encountered while trying. While it would probably be possible to get an Android device to cater for the use case it seemed a lot of fiddling against the OS. So eventually I decided to look for an alternative and ended up with a Raspberry Pi with a 7" touch screen panel.

The setup

In my place I use a Raspberry Pi with nymea:core which is placed to cover the most part of my home (including balcony) with Bluetooth so sensors are within a reachable distance. The control panel however, should be placed in a different location where it can easily be reached and used. For this reason, I now have 2 Raspberry Pis set up, one for the core and one for the touch panel frontend. If you want to build this too, it can be created just the same with nymea:core running on the same Raspberry Pi as nymea:app. To ease up this article I'm going to continue with describing a single-RPi system. If you'd like to connect multiple touch panels to the same nymea:core or prefer such a distributed as I have, just follow the :core or :app steps on different Raspberry Pis and repeat as appropriate.

For connected things I mostly use Philips Hue light bulbs (also some others, but that's a whole new topic :) ), have some Xiaomi Flower Care sensors for preventing my lemon tree from drying out, connected my Kodi media center to it (to dim down lights while I watch a movie) and included a bunch of online services in the setup, like push notifications, weather services and what not. Given this article focuses on the touch screen control panel I won't go into details on my general nymea setup but rather continue with setting up the display. I've added some few screenshots to give some insight.

Assembling the Raspberry Pi and display in that particular case as I have is not hard, but somewhat fiddly. Turns out if you first mount the Raspberry Pi on the back of the display and then try to put it into the casing it's gonna get very tricky to fit it in without using too much force. Instead I figured out that it's much easier to first just plug the cables, then place the RPi in the casing at its final position and lastly plug the display in. Then tighten all the screws. However, note that with this casing it is not possible to add/remove the SD card once it's mounted. So I would recommend to first set up the basic system on the SD card using the Raspberry Pi without the casing, perhaps even using a regular monitor on the HDMI port if you have one at hand. The 7" touch panel should do too, however, put it on the desk next to the Raspberry Pi for now. Once you are confident the system on the SD card works good enough and can be accessed through SSH (more on that later) it's time to put the case together.

Preparing the Raspberry Pi SD card

Download the nymea:core image from the nymea wiki. Preparing the SD card depends a bit on your OS but there are plenty of instructions out there on how to write an SD card image. I for one tend to use dd as described in the nymea wiki right below the download links, but if you prefer something more user friendly and available on all major operating systems, Etcher should work fine too.When done, insert the SD card in the Raspberry Pi and boot it up. If you want to use an ethernet cable just plug one in now. If you want to use WiFi, you can install nymea:app on your iOS or Android phone and set up WiFi credentials from there without having to plug a keyboard to the Raspberry Pi. Use the "Wireless setup" menu entry in the app's loading screen for that.

Installing nymea:app

The nymea:core image for the Raspberry Pi only has the nymea:core preinstalled. At this point it should already work from your phone. You can play around a bit and check things out. However, we're interested in running the app on the touch panel, aren't we? :)

There is a nymea:app desktop snap package available in the Ubuntu Core Snap store. It is built as a GNU/Linux desktop app but as there are armhf builds available it will also install on a Raspberry Pi. However, it will only install a start menu entry by default. We need to do some tweaking to actually start it in kiosk mode without the need of an entire desktop.The nymea:core image comes with SSH preinstalled and enabled. Use nymea:app on your phone to discover the Raspberry Pi in your network and obtain the IP address. Once found, log in the Raspberry Pi using SSH (replace 192.168.0.100 with the IP of your Raspberry Pi):

$ ssh nymea@192.168.0.100 <br>nymea@192.168.0.1's password: nymea

Once logged in, install the app

$ sudo snap install nymea-app 

Ok, if that went well, here comes the more tricky part :)

Making it autostart in kiosk mode

As we don't want an entire desktop, I tried a few simple window managers and had the best success with using lightdm with autologin into an openbox session. So let's install lightdm and openbox:

$ sudo apt-get install lightdm openbox 

Now edit /etc/lightdm/lightdm.conf. My favorite editor is nano:

$ sudo nano /etc/lightdm/lightdm.con 

Go to the [Seat:*] section and change the following settings (remove the # at the start of the line too):

user-session=openboxautologin-user=nymea 

Create an openbox autostart file in ~/.config/openbox/autostart with the following content:

/snap/bin/nymea-app 

Now enable lightdm autostart:

sudo systemctl enable lightdm --now 

Tweaks and cleanup

Rotate screen:

By default, the Raspberry Pi touchscreen in this casing is rotated upside down. You can fix that with adding the following line to /boot/config.txt

lcd_rotate=2 

Fix graphics performance:

nymea:app is written in Qt with QtQuick2 which makes heavy use of OpenGL (ES2). For some reason, that is not enabled by default in the Raspberry Pi images so graphincs performance is somewhat bad with nymea:app. To fix this, run

sudo raspi-config 

go to Advanced Options → GL Driver and select the GL (Fake KMS)driver. I didn't have great success with the non-fake KMS driver... In theory it should work I think but the fake KMS certainly is good enough. Alternatively you can just edit /boot/config.txt again and append this line at the end:

dtoverlay=vc4-fkms-v3d 

Hiding the mouse cursor:

By default, the X11 server used will paint a mouse cursor whenever you touch the screen. Given we use a touch screen that doesn't make much sense. To hide that, open /etc/lightdm/lightdm.conf again and edit the following line (removing the # and appending the -nocursor at the end):

xserver-command=X -nocursor 

Turning the display off instead of just blanking it:

Now, this is the most tricky tweak, but it's gonna be the last needed one, I promise :) By default, the Raspberry Pi will just blank the screen when idle. This makes sense with regular monitors, but again, doesn't make much sense with this touch screen model. We want the screen to turn off completely when in idle instead of being lit and just painting it black. The screen's power can be turned on and off with a sysfs file, namely /sys/class/backlight/rpi_backlight/bl_power. We can manually turn it on and off with writing 0 or 1 into this file. In order to do so automatically when the screen would blank/unblank, let's install xscreensaver and wrap its execution in a script that hooks into the appropriate places:

$ sudo apt-get install xscreensaver 

Now create a file /usr/local/bin/xscreensaver-wrapper.sh with content:

#!/bin/bash<br>bl_on() {
  sudo /usr/local/bin/screenon.sh
}

bl_off() {
  sudo /usr/local/bin/screenoff.sh
}

process() {
  while read line; do
    case "$line" in
      UNBLANK*)
        bl_on
        ;;
      BLANK*)
        bl_off
        ;;
      RUN*)
        bl_off
        ;;
    esac  done
}

/usr/bin/xscreensaver &
/usr/bin/xscreensaver-command -watch | process

and make it executable with:

$ sudo chmod +x /usr/local/bin/xscreensaver-wrapper.sh 

Next create two more files, /usr/local/bin/screenon.sh with content

echo 1 | sudo tee /sys/class/backlight/rpi_backlight/bl_power 

and /usr/local/bin/screenoff.sh with content

echo 0 | sudo tee /sys/class/backlight/rpi_backlight/bl_power

Make both of them executable again using:

$ sudo chmod +x /usr/local/bin/screenon.sh <br>$ sudo chmod +x /usr/local/bin/screenoff.sh 

Last bit missing is to grant permissions to the user nymea to execute those sudo commands without asking for a password. For that, edit /etc/sudoers, add the 2 lines as shown below (IMPORTANT: Pay proper attention to get this right. Messing up the sudoers file might lock you out of the system and requires some deeper Linux knowledge or a reflash to fix it):

...
# Allow members of group sudo to execute any command
%sudo ALL=(ALL:ALL) ALL
nymea ALL=(ALL:ALL) NOPASSWD: /usr/local/bin/screenoff.sh
nymea ALL=(ALL:ALL) NOPASSWD: /usr/local/bin/screenon.sh
# See sudoers(5) for more information on "#include" directives:
... 

Edit ~/.config/openbox/autostart again and insert this line at the beginning in order to invoke xscreensaver along with nymea:app.

/usr/local/bin/xscreensaver-wrapper.sh & 

Step 5: Done

Reboot and be happy!