ESP32 Touch Display Multipage Macropad

This instructable builds on my quick-start guide for Waveshare's ESP32 -S3-2.8 inch touch display

This instructable covers the following:

1. Introduction to EEZ Studio
2. USB HID with ESP32(S3)

I am assuming you have followed all the steps in the quick-start guide and have the demo program running

You don't need any additional hardware (I will not be showing how to use the onboard GPIO pins in this instructable)

You need EEZ Studio, which can be downloaded from:

1. https://www.envox.eu/support-us/

I will use the Windows version because I am using a Windows machine, but I assume the steps will be the same for other platforms.

The LVGL documentation is essential to do anything:

1. https://docs.lvgl.io/8.3/

(Press Ctrl+S to save after every step or any action)

Zip folders, code files, and images can be found in my github repository

(Refer to Image 1 of this step)

1. We will need to create an LVGL V8.x project.
2. You can save the project in any folder as long as you can access it and have read/write permissions
3. I am calling my Project 'InstructablesDemo' (avoid spaces in your project names. Use underscores instead)

(Refer to Image 2 of this step)

1. After you press Create Project, you should see something like in Image 2
2. Select Main from the Pages section on the left, and you should see a populated properties menu on the right.
3. Under Properties>POSITION and SIZE, change the Width and Height to 240 and 320, respectively.

Note: Our project will be in the Portrait orientation. If you want to use the make something for landscape orientation, you would swap 240 and 320.

The fields for Width and Height will most likely be highlighted in red for you. We will solve that right away.

(Refer to Image 3 of this step)

1. Open the Settings menu by pressing the gear icon and write the same values for the display's width and height

The Width and Height fields should not be highlighted in red anymore

(Refer to Images 4 and 5 of this step)

Click on the Hello World label widget under the Widgets Structure pane and drag it to the centre of the screen

(Refer to Images 6 and 7 of this step)

1. Open the Settings menu again
2. Change the shown field to lvgl.h

(Refer to Image 1 of this step)

1. Press the Check button to verify that everything is correct.
2. If everything is correct, you will see no warnings or errors.

(Refer to Image 2, 3, and 4 of this step)

1. Press the Build button to generate the code files for your GUI
2. You should see a src folder with a 'ui' folder inside it in your project directory

Start the Arduino IDE and open the sketchbook where the demo code was run in the quick-start guide

(Refer to Images 1 and 2 of this step)

1. Create a new sketch. This will automatically create a sketch folder as well.
2. My sketch name is Instructables_EEZ_Demo (and I suggest you use the same name as you will need to copy-paste some code files to keep things quick)
3. Paste the contents of 'Instructables EEZ Demo Base.zip' into your sketch folder. I am assuming you used the same sketch name as I did. It's the same files as in LVGL_Arduino from the quick start guide without the unnecessary gyro, sd, audio, etc, files.

You can upload the code to see if everything is running. You will see a Hello Arduino message (sort of)

(Refer to Images 3 and 4)

1. Paste the ui folder created in Step 2 in the libraries folder
2. As in image 4, add the two lines of code and try compiling and uploading it.

You should now see the same thing as in Image 5 of this step

Go back to your project and delete the Hello World label.

I found this floating Pikachu GIF and resized it to 240x320 with Ezgif. To save space, I also reduced the number of frames.

You can find the GIF split into frames in the 'Pikachu Image Frames.zip' on the linked GitHub repository.

(Refer to Images 1 and 2 of this step)

1. Click on the blue arrow to add images. The images need to be added individually.
2. Select 'INDEXED 4 BIT' for the color format
3. Keep the bitmap names the same as the image names. For example: Frame0

(Refer to Image 3 of this step)

1. After you add all the Frames, your section should look like mine, pointed with a blue arrow.
2. Next, drag and drop an AnimationImage widget on the screen and resize it to cover the entire space.
3. Rename the widget to PikachuBackground

(Refer to Images 4, 5, and 6 of this step)

1. Add each image(frame) by pressing the '+' button and selecting the image (frame) you added before
2. After adding all the images, you should have something that looks like Image 6 of this step
3. Your Pikachu should now be floating.
4. Repeat Infinite should be on. Duration can be adjusted to change the speed of the gif animation

Save, verify, and press the build button. Then, paste the UI folder into the screen, as in Step 3, and upload the code.

The result should look like Image 7 (of course, your Pikachu should be floating)

Note: You can download the code files for this step from the GitHub repository. Download 'Instructables Pikachu Clock.zip'

I will also include the UI folder in this zip. You should cut and paste the UI folder into the libraries folder. Then, you can create a new Arduino Sketch and folder and copy the contents of my ino files to that sketch. Finally, you should paste the other code files into the folder as your Arduino sketch.

(Refer to Image 1 of this step)

1. Add a text label and place it in the center of the screen above Pikachu
2. Clear the 'Text' field (not necessary)
3. I named my label 'TimeLabel'

(Refer to Image 2 of this step)

1. Change the ALIGN field of the label to CENTER
2. Change the font to MONTSERRAT_30

(Refer to Images 3 and 4 of this step)

1. Add the lines highlighted with the red marker.
2. Replace your SSID and password in the ssid and password fields.
3. Change the value in timeClient.setTimeOffset() as per your time zone
4. We will not be using the onboard RTC.

If you upload the code now, you may get an error regarding the font used

(Refer to images 5 and 6 of this step)

1. To solve this error, go to the Lvgl folder in your libraries folder
2. Lvgl->src->lv_conf.h
3. Open the file in something like VSCode. Use Ctrl+F and search 'font'
4. You will find a list of define statements for font. Change the 0 to 1 for all the fonts we are interested in using.

After uploading, you should see the time over your Pikachu.

You can play with positioning the text label, font styles, and other options.

See mine in image 7.

Notes about the code:

1) I wanted my time on two different lines and formatted the string as such. See the following lines in the code:

formattedDate = timeClient.getFormattedTime().substring(0, 5);

String formattedTime = formattedDate.substring(0, 2) + "\n" + formattedDate.substring(2, 5);

You can change the format as per your liking.

2) Notice 'objects.time_label'. This is the name EEZ Studio gave to our 'TimeLabel'

All widgets used can be accessed using objects.(widget name). You can get the names via the Arduino IDE's auto-complete or by going through the screens.c file in the ui folder.

3) Animations are CPU intensive and are the reason why we had to use RTOS to split the time calculation and lvgl_update tasks

It is time to make an additional page for our LVGL GUI. This page will have basic music controls for your computer's media playback.

First, we will add a button on our Main page to take us to the Music Page

(Refer to Images 1 to 6 for this part)

1. Place a button widget wherever you like. I renamed my button widget to 'MusicPgBtn'
2. Notice that the Button widget has a nested Label widget. Click on the label widget and change its text to anything you like. For our purpose, writing 'Music' there makes sense. You do need to give the label a name.
3. Under the Label's STYLE, go to DEFAULT->TEXT->Color, and change it to #000000 (hex code for black). The text's opacity needs to be 255
4. Add the cloud bitmap like how we added the Pikachu image frames. Download the cloud image present in the folder 'Main Page Button.zip' from the GitHub repository. I stored the bitmap as 'Cloud'
5. Go back to the Button widget. Go to DEFAULT->BACKGROUND and make the changes as shown in Image 4
6. Still, for the MusicPgBtn, go to DEFAULT->SHADOW and change the opacity to 0. (Refer to Image 5)
7. Your GUI should now look like mine in Image 6. Try moving the Music label up/down with your arrow keys.

Now, we will start adding our Music Page.

(Refer to Images 7 to 9)

1. In the top-left window, where the 'Pages' column is, press the '+' button to add a new page (Refer to image 7)
2. You will be greeted with a New Page dialog box to enter the name for the page. I will call this page Music.
3. You should get a blank page similar to our starting Main page.

We will now go back to the Main page and configure our MusicPgBtn to take us to the Music Page when it is pressed.

(Refer to Images 1 to 3)

1. Go to the 'User Action' window. It is next to the Pages window. (Refer to Image 1 of this step)
2. Press the '+' button to add a User action. Give the action whatever name you like. I will call it 'GoToMusicPg' (Refer to Image 2)
3. It doesn't have that many properties. You can write a description for it. (Refer to Image 3 of this step)

We will now add an event handler to our MusicPgBtn

(Refer to Images 4 to 6)

1. Go to the Main screen, select our Button->EVENTS->'+'. (Refer to Image 4)
2. You should see a 'New Event Handler' Dialog. You can select your desired action from the Action dropdown. Since we only have one action for now, it will select that one. (Refer to Image 5)
3. You should see something similar to Image 6 of this step

Verify everything and press the build button.

(Refer to images 7 to 10)

1. Go to your ui folder. You should now see an actions.h file (Refer to Image 7)
2. Open the actions.h file in your preferred text editor. VSCode, in my case. From the ui folder, also open the file screens.h.
3. In the screens.h file, you should see the screen IDs. (Refer to Image 8)
4. Create an actions.c file in the same ui folder and add the code lines shown in Image 9. It is also pasted below. (Refer to Image 9)
5. You can get the event handler/action method name from the action header file. (Refer to Image 10)

(actions.c code)

#include "actions.h"

#include "ui.h"

void action_go_to_music_pg(lv_event_t * e){

loadScreen(SCREEN_ID_MUSIC);

}

Copy and paste the ui folder into the library folder like before.

The Arduino sketch requires minor changes. Why should we update the time when we are not even on the music page? We will execute the logic in the time task only if the active screen is the main screen.

Change the time_update_task to:

void time_update_task(void *pvParameters)

{

static String lastFormattedTime = "";

while (1)

{

if(lv_scr_act()==objects.main){

timeClient.update();

formattedDate = timeClient.getFormattedTime().substring(0, 5);

String formattedTime = formattedDate.substring(0, 2) + "\n" + formattedDate.substring(2, 5);

if (formattedTime != lastFormattedTime) {

formattedTime.toCharArray(timeArray, sizeof(timeArray));

lv_label_set_text(objects.time_label, timeArray);

lastFormattedTime = formattedTime;

}

}

vTaskDelay(pdMS_TO_TICKS(1000)); // Update time every second

}

}

The changes have been bolded.

Upload the code to your display. You should see the same floating Pikachu, but now with our Music button below it. (Refer to Image 11).

Upon pressing the button, you should see a blank screen. (Refer to Image 12)

We will now design our music page by adding a background, widgets, and other stuff. I will keep this one very simple.

(The code files and ui folder for this step can be found in the Instructables Final Project.zip)

You can use the concepts covered earlier to add a short animated background and/or buttons with an image background.

Instead of the design, I would rather show you how to trigger media control via digital button presses.

(Refer to Image 1)

1. I added four buttons:
2. LVGL Name: PlayPauseBtn (For play and pause)
3. LVGL Name: VolUpBtn (For volume up)
4. LVGL Name: VolDnBtn (For volume down)
5. LVGL Name: GoToMainBtn (For taking us back to the main page)

(Refer to Image 2)

1. Create a user action like before. I am calling mine 'GoToMainPg'
2. Add it under the even handlers of the 'GoToMainBtn'

(Refer to Image 3)

1. Create three more actions:
2. PlayPauseAction
3. VolUpAction
4. VolDownAction
5. Add these to the PlayPauseBtn, VolUpBtn, and VolDnBtn, respectively.
6. Verify and press the build button.

(Refer to Images 4 and 5)

1. Open the actions.h and actions.c file again. The header file should have a lot more event functions now, and the .c file (the source file) should be the same as the previous step.
2. Update actions.c file to now load the main screen (Refer to Image 5)

(Updated actions.c code)

#include "actions.h"

#include "ui.h"

void action_go_to_music_pg(lv_event_t * e){

loadScreen(SCREEN_ID_MUSIC);

}

void action_go_to_main_pg(lv_event_t * e){

loadScreen(SCREEN_ID_MAIN);

}

(Changes have been bolded)

Copy and paste the ui folder in the libraries folder like before

Open the Arduino sketch again. I suggest you also keep the actions.h file to copy the events method signatures more easily

1. We need to define the other events as well

(Refer to Image 6 and 7)

1. We need to define the highlighted events. Copy these to the Arduino sketch. (The extern keyword should not be present in the method signatures in the Arduino sketch)

(Refer to Images 8 to 11)

1. Add the required libraries shown in Image 8. You must install these libraries if you don't have them already.

#include <actions.h>

#include "USB.h"

#include "USBHIDKeyboard.h"

#include "USBHIDConsumerControl.h"

1. Add the following lines outside of the setup() function:

USBHIDKeyboard keyboard;

USBHIDConsumerControl consumerControl;

1. In the setup() function, add the following lines:

USB.begin();

consumerControl.begin();

1. Now, we can define the events

void action_play_pause_action(lv_event_t * e){

consumerControl.press(CONSUMER_CONTROL_PLAY_PAUSE);

consumerControl.release();

}

void action_vol_up_action(lv_event_t * e){

consumerControl.press(CONSUMER_CONTROL_VOLUME_INCREMENT);

consumerControl.release();

}

void action_vol_down_action(lv_event_t * e){

consumerControl.press(CONSUMER_CONTROL_VOLUME_DECREMENT);

consumerControl.release();

}

In the Arduino IDE, change the following setting: Tools->USB CDC On Boot->Enabled (Refer to Image 12)

Verify and upload the code to your display.

You should now be able to press the buttons on the second page to control the volume and play or pause any song or video.

To see my results, check images 13 and 14.

I hope you learned something from this instructable.

Let me know if you would like me to make anything clearer.

Common troubleshooting tips:

1. If you are ever having trouble uploading your code to the display, please try to put the device into bootloader mode. (Learn how to do this in the FAQ section of the wiki)
2. If a button is not executing the expected action, recheck if you have assigned the relevant event to it. (I have made this several times, even while writing this instructable.)