Raspberry Pi - Visual Touchpad / Second Display

This instructable will show how to add a small (usually SPI based) touch screen to almost any pi project and mirror the HDMI out to it as a secondary display. We will also show how to accept touch input on this second display from the touch screen sensor and controller. This can replace the external mouse and lets us use the small LCD as a visual touchpad along with the HDMI out! (or as a small portable touch-display on its own...) This is an inexpensive way to add portable I/O.

If you have a Raspberry Pi project you feel would benefit from the addition of a small touch-screen, read on!

You will need:

• A Raspberry Pi Zero, 2 or 3
• An LCD touchscreen Display (usually 2.2-3.6 inches, must be fbtft compatible, see notes and links below)
• An HDMI monitor/Cable
• Keyboard/Mouse
• Power Supply - 5V MicroUSB

Note- Everything aside from the LCD touchscreen itself is pretty much what you'd need for any Raspberry Pi project. If you already have a Raspberry Pi project and just want to add a small touchscreen to it, the only part you'll need is the touchscreen itself.

Compatible Touchscreens

There are a lot of different touch screens out there and not all of them are compatible with the fbtft driver we're using. When shopping for a touchscreen, most displays from Adafruit, PiTFT, Tontec, Sainsmart can be made to work. However, it's best to check before buying. The most definitive and up-to-date list of compatible displays will always be in the source code itself-

https://github.com/raspberrypi/linux/blob/rpi-4.4....

You can read througfh the fbtft_device_display displays struct to see what displays have support. If in doubt you can also ask the vendor if their display is fbtft compatible.

Also you can tell a lot by looking at the back of the display. Avoid any TFT that has 3x 74HC5950 chips on it. (It has no real driver IC, those are just serial to parallel shift registers... ) See pics for examples of a display that works and one to avoid.

This assumes a basic familiarity with the Raspberry Pi and the Raspbian OS. First you must install Raspbian onto your microSD. See here for help - https://www.instructables.com/id/Install-Raspbian-... Make sure you've got a mouse keyboard and network connection and boot the Pi. (For the Pi zero you can either use a USB network adapter or you can set up the OS image on a Raspi 2 then boot to the zero later...)

Once you have booted the OS, open a terminal and type -

sudo apt-get update

sudo apt-get upgrade

sudo rpi-update

Now you have the latest kernel and fbtft drivers. Next let'e enable SPI. From the same command prompt run -

sudo raspi-config

select 9 ADVANCED OPTIONS

select A5 SPI

select YES

Now reboot the pi with -

sudo reboot

Next we'll need to setup the modules in /etc/modules. Usually there are 3-

• fb_WHATEVER_YOUR_HARDWARE_IS
• fbtft
• fbtft_device

In order to do that you need to know the model of your TFT. (i.e. - Adafruit 3.2 inches SPI). It may also help to know the driver IC number (usually the largest IC on the back PCB of the TFT). Once you have that you can look for your TFT here - https://github.com/raspberrypi/linux/blob/rpi-4.4...

On that page - search for the text 'fbtft_device_display displays' and you will find a list of supported displays. In my case the display is a Tontec 3.5" TFT with an ili9486 IC. Looking in that page i find the display definition- [the portions in bold are what we'll need to complete the set-up...]

.name = "tontec35_9486", /* boards after 02 July 2014 */
.spi = &(struct spi_board_info) {

.modalias = "fb_ili9486",

.max_speed_hz = 128000000,

.mode = SPI_MODE_3,

.platform_data = &(struct fbtft_platform_data) {

.display = {

.buswidth = 8,

.backlight = 1,

},

.bgr = true,

.gpios = (const struct fbtft_gpio []) {

{ "reset", 15 },

{ "dc", 25 },

{ "led_", 18 },

{},

},

}

}

The only things we need to know from this are the 'name' (in this case - tontec35_9486) and the 'modalias' setting which is the hardware module we need to load. We can see that this is fb_ili9486. So now we can edit /etc/modules -

sudo nano /etc/modules

and add the following 4 lines (though as discussed - line 2 will change depending on .modalias from above)-

spi_bcm2835

fb_ili9486

fbtft

fbtft_device

Now hit [CTRL+X] to save the file.

We are almost done with the driver. But in order for the framebuffer to load, we must also set the module options. (This is usually the name of the device as it appears in https://github.com/raspberrypi/linux/blob/rpi-4.4... and the optionally the rotation.) These are set by creating a file in the /etc/modprobe.d directory (see photos)-

sudo nano /etc/modprobe.d/fbtft_device.conf

and editing it to contain the following all on one line(again my TFT was a tontec35_9486, edit the portion in bold for your TFT)-

options fbtft_device name=tontec35_9486 rotate=90

and save the file with [CTRL+X].

At this point the kernel drivers are now installed (and hopefully configured). the next step is to install and test the TFT!

In order to install and test the TFT we simply turn off the pi and plug in the display to the GPIO connector (making sure that pin 1 is aligned properly!). Now turn on the pi. You should see the hdmi display boot normally. If plugged in properly, the TFT will first go WHITE(power on), then go BLACK(driver load). If either of these things does not happen STOP!

Troubleshooting-

• If the display does not go white or the pi does not boot, then verify that the display is plugged in properly (i.e. not backwards...) and that the pins are in their proper plug. (i.e. not shifted over...)
• If the display does not go black by the time xorg boots up on the HDMI monitor, then either the 'name' is wrong in /etc/modprobe.d/fbtft_device.conf or the wrong fb_ module is being loaded in /etc/modules or both... (use a magnifier and read the number on the ICs on back of the TFT. You should be able to match one on them to a fb_ file in the kernel source...)

If you have a black screen you can now test the framebuffer. We'll do that by adding a console, first edit /boot/cmdline.txt -

sudo nano /boot/cmdline.txt

and add the test fbcon=map:10 right before the text 'rootwait'. My edited cmdline.txt is -

dwc_otg.lpm_enable=0 console=ttyAMA0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline fbcon=map:10 rootwait

Now you can reboot and if successful, you should see a the console scroll by at boot up on the TFT! (see photo) To log onto the TFT console you can go [CTRL+ALT+F1]. The next step is to duplicate the HDMI display to the TFT.

en

Now we can copy the hdmi to the TFT. We do that with a program called fbcp. In order to build fbcp we need a couple of things. So first we open a command prompt and enter -

sudo apt-get install build-essential cmake git

and follow the prompts. Now that we have cmake and such we can get the source code and compile it. In the same terminal execute the following commands-

git clone https://github.com/tasanakorn/rpi-fbcp.git

cd rpi-fbcp

mkdir build

cd build

cmake ../

make

sudo install fbcp /usr/local/bin/fbcp

If all went well you should now have fbcp installed. To test it simply open a terminal and enter -

fbcp &

You should now see your hdmi display mirrored onto your TFT. To make this persist across reboots one can edit /etc/rc.local

sudo nano /etc/rc.local

and add the line -

fbcp &

before the line -

exit 0

Now if all this is working it's time to set up the touchscreen!

***Also if you are going to use fbcp I you may wish to remove the fbcon=map:10 entry from /boot/cmdline.txt

Now we will enable the touchscreen for the TFT. On the display I have, the driver is an ADS7846. To use this we will set up the device tree overlay for that hardware. To do this the one thing we need to know is which GPIO the pendown event is connected to. You will possibly need to look at the tech specs or pinout for the display to figure that one out. On my Tontec display the touchscreen is connected to GPIO 4.

To set up the overlay we need to edit the file /boot/config.txt -

sudo nano /boot/config.txt

and add the device tree overlay line (change penirq in bold below to the correct GPIO for your touchscreen, See Picture) -

dtoverlay=ads7846,penirq=4,speed=2000000,penirq_pull=2,swapxy=1,xohms=60

and save with [CTRL+O].

Next we install some packages-

sudo apt-get install evtest xinput tslib libts-bin xinput-calibrator

Once those install, we create the xorg config file at /usr/share/X11/xorg.conf.d/99-calibration.conf

sudo nano /usr/share/X11/xorg.conf.d/99-calibration.conf

And add the following text-

Section "InputClass"

Identifier "calibration"

MatchProduct "ADS7846 Touchscreen"

Option "Calibration" "3850 100 3900 150"

Option "SwapAxes" "1"

EndSection

now type [CTRL+O] to save. (then [CTRL+X] to exit)

Now we reboot to test the configuration. Once xorg loads, the touchscreen should register touches. Don't worry if the axes are swapped or inverted. We can fix that next. as long as the touchscreen registers we are happy!

Troubleshooting-

If the touchscreen does not register follow these steps- (Note also that thing vary slightly by touchscreen driver as well. If there is a dtoverlay then info will be available in the /boot/overlays directory of your pi.)

1. Touchscreen does not register touch - 99% of the time the penirq is wrong, but check all the cables too
   
2. swiping up down moves the cursor right-left - the axes are swapped
3. swiping left moves the cursor right (and vice-versa) - the x axis is inverted
4. swiping up moves the cursor down (and vice-versa) - the Y axis is inverted
5. the cursor is not where I touch the screen - the panel needs to be calibrated

• For problem 1 - double check the penirq pin for the TFT you have, then check and make sure the touchscreen is not damaged and that the ribbon cable is plugged in correctly to the TFT driver board
• For problem 2 - invert the SwapAxes option in /usr/share/X11/xorg.conf.d/99-calibration.conf and the SwapXY option in /boot/config.txt
• For problem 3/4 - to reverse an axis - swap out the large and small calibration numbers in /usr/share/X11/xorg.conf.d/99-calibration.conf (or just calibrate the screen following the steps below and copy/paste those values into your config)
  

To Calibrate the screen-

In order to calibrate the screen we run xinput-calibrator to do that open a terminal in xorg and type -

DISPLAY=:0.0 xinput_calibrator

follow the instructions clicking each of the 4 red cross-hairs with a stylus on the TFT and note the values shown in the terminal (see photo). Those values can now be entered in the Option "Calibration" line of /usr/share/X11/xorg.conf.d/99-calibration.conf. (See Video)

Now reboot and you should have a fully functional mini display/touchpad!