AI-driven IoT Shopping Assistant W/ ChatGPT
by Kutluhan Aktar in Circuits > Electronics
1797 Views, 23 Favorites, 0 Comments
AI-driven IoT Shopping Assistant W/ ChatGPT
Activate your cart with a QR code sent via email by the web app, update product info by scanning barcodes, and get shopping tips from ChatGPT.
Supplies
1 x WIZnet W5300 TOE SHIELD
1 x STM32 Nucleo-144 NUCLEO-F439ZI
1 x DFRobot GM77 Barcode and QR Code Scanning Module
1 x DFRobot Fermion: 1.51” OLED Transparent Display
1 x DFRobot Tiny (Embedded) Thermal Printer
1 x LattePanda 3 Delta 864
1 x DFRobot 8.9” 1920x1200 IPS Touch Display
1 x Anycubic Kobra 2
1 x Keyes 10mm RGB LED Module (140C05)
3 x Button (6x6)
1 x Xiaomi 20000 mAh 3 Pro Type-C Power Bank
2 x USB Buck-Boost Converter Board
1 x Breadboard
1 x Ethernet Cable
1 x Jumper Wires
Story
In light of recent developments in machine learning and artificial intelligence, various brands aimed to implement AI-based solutions to improve the shopping experience and increase their profit margin by tailoring unique suggestions and advertisements depending on the targeted user's profile and preferences. Especially e-commerce companies, platforms, and marketplaces benefited from AI algorithms to autonomously identify the most optimal ad placements for individual customers.
Even though there are some pioneering methods to combine physical store shopping with AI-based e-commerce features, such as Amazon Go cashierless convenience stores, these self-checkout stations are expensive investments for local stores since they require renovating (remodeling) store layouts or paying monthly fees to cloud services. Furthermore, local businesses may need to construct data sets for niche market products since AI algorithms require broad data sets for methods based on computer vision, sensor fusion, and deep learning.
After scrutinizing the recent research papers and the granted patents on e-commerce methods, I noticed there are nearly no appliances focusing on merging physical store shopping with AI-based solutions without altering existing conditions for local businesses. Therefore, I decided to build a budget-friendly and accessible AIoT shopping assistant utilizing the most common and recognizable product identification method — barcodes — to bring AI-based e-commerce features to physical store shopping.
Since I wanted to build a shopping assistant providing a wholesome physical shopping experience with the most prominent e-commerce features, I decided to develop a full-fledged e-commerce web application from scratch in PHP, HTML, JavaScript, CSS, and MySQL. Since I developed this complementing web application to bridge the gap between physical store shopping and e-commerce, it is capable of communicating with the shopping assistant (device), obtaining product information from barcodes, and generating AI-based recommendations consecutively.
The complementing e-commerce web application allows the customer to:
- Create a user account identified with the unique 12-digit token
- Obtain the generated account verification QR code via an HTML email, including the unique user token
- Add or remove product information via barcodes scanned by the shopping assistant
- Display the current product list in the cart and the total cart price
- Inspect ChatGPT-powered recommendations for each product in the cart
- Place an order by making a payment via credit/debit card
- Get the payment confirmation QR code via an HTML email
- Check purchased items from previous orders
As suitable for local businesses, I decided to install an Apache HTTP Server (XAMPP) on LattePanda 3 Delta, which also has a MariaDB database, to host my web application. Since I focused on creating a low-budget but state-of-the-art AIoT shopping assistant, I decided to employ the OpenAI API to generate ChatGPT-powered recommendations related to the obtained product information by barcodes via the Open Food Facts JSON API. I also employed Brevo's Email API to send HTML emails directly from localhost to inform the customer of the generated unique verification QR codes.
After developing my e-commerce web application successfully, I started to work on building the shopping assistant (device). Since Wi-Fi and Bluetooth transmissions may not be suitable options for a shopping assistant requiring as minimal latency as possible to communicate with the e-commerce application, I decided to utilize WIZnet's W5300 TOE SHIELD providing high-speed bus communication and reliable Ethernet transmission speeds. Since W5300 TOE SHIELD cannot perform without an STM32 Nucleo-144 board, I decided to utilize the NUCLEO-F439ZI development board and program it in MicroPython.
Since I needed the shopping assistant to scan barcodes and QR codes simultaneously to recognize customers, activate/deactivate carts, and send the scanned product barcode with the selected command (add/remove) to the e-commerce web application, I connected a GM77 barcode and QR code scanner to NUCLEO-F439ZI. Also, I connected an SSD1309 transparent display and an RGB LED so as to inform the customer of the device status. Furthermore, I added a tiny (embedded) thermal printer to directly notify the customer when the customer requests a device status report or scans the payment confirmation QR code. In that regard, I wanted to enrich the correlation between physical store shopping and e-commerce.
Lastly, to make the shopping assistant as robust and sturdy as possible while being employed as a self-checkout station, I designed a checkout counter-themed case, a removable top cover with snap-fit joints, and handles for moving the shopping assistant if necessary.
So, this is my project in a nutshell 😃
In the following steps, you can find more detailed information on coding, utilizing QR codes as an authentication method, obtaining product information via barcodes, generating ChatGPT-powered product recommendations, developing a full-fledged e-commerce web application, and communicating with the web application via W5300.
🎁🎨 Huge thanks to WIZnet for providing me with a W5300 TOE SHIELD.
🎁🎨 Huge thanks to DFRobot for sponsoring these products:
⭐ GM77 Barcode and QR Code Scanning Module | Inspect
⭐ Fermion: 1.51” OLED Transparent Display | Inspect
⭐ Tiny (Embedded) Thermal Printer | Inspect
⭐ LattePanda 3 Delta 864 | Inspect
⭐ DFRobot 8.9" 1920x1200 IPS Touch Display | Inspect
🎁🎨 Also, huge thanks to Anycubic for sponsoring a brand-new Anycubic Kobra 2.
Designing and Printing a Checkout Counter-themed Case
Since I focused on building a budget-friendly and accessible shopping assistant that scans product barcodes, communicates the e-commerce web application, and activates/deactivates the user's cart on the web app via QR codes, I decided to design a stylish and compact case allowing the customer to inspect the device status, scan product barcodes effortlessly, and receive the printed notifications by the embedded thermal printer. To avoid overexposure to dust and prevent loose wire connections, I added a removable top cover with snap-fit joints. Then, I added a slot to integrate the tiny thermal printer and two handles to position the removable top cover easily while performing maintenance. Also, I decided to emboss the WIZnet logo, the ChatGPT logo, and the shopping cart symbol on the removable top cover to highlight the capabilities of the shopping assistant.
Since I needed to fix the position of the scanner module to get better results, I added a two-sided protrusion on the front of the main case. Also, I thought providing a small gift for a new customer trying this shopping assistant would be cool, so I decided to print a Togepi figure to demonstrate my idea, which is a Fairy-type Pokémon and is considered to be a symbol of good luck and fortune :)
I designed the main case and the removable top cover in Autodesk Fusion 360. You can download their STL files below.
For the Togepi figure added as a gift for a first-time customer trying the shopping assistant, I utilized this model from Thingiverse:
Then, I sliced all 3D models (STL files) in Ultimaker Cura.
Since I wanted to create an attractive counter structure for the main case and apply a modern retail store theme representing a flourishing business, I utilized these PLA filaments:
- ePLA-Matte Dark Grey
- Bone White (Togepi)
Finally, I printed all parts (models) with my brand-new Anycubic Kobra 2 3D Printer.
Since Anycubic Kobra 2 is budget-friendly and specifically designed for high-speed printing, I highly recommend Anycubic Kobra 2 if you are a maker or hobbyist needing to print multiple prototypes before finalizing a complex project.
Thanks to its upgraded direct extruder, Anycubic Kobra 2 provides 150mm/s recommended print speed (up to 250mm/s) and dual-gear filament feeding. Also, it provides a cooling fan with an optimized dissipation design to support rapid cooling complementing the fast printing experience. Since the Z-axis has a double-threaded rod structure, it flattens the building platform and reduces the printing layers, even at a higher speed.
Furthermore, Anycubic Kobra 2 provides a magnetic suction platform on the heated bed for the scratch-resistant spring steel build plate allowing the user to remove prints without any struggle. Most importantly, you can level the bed automatically via its user-friendly LeviQ 2.0 automatic bed leveling system. Also, it has a smart filament runout sensor and the resume printing function for power failures.
#️⃣ First of all, install the gantry and the spring steel build plate.
#️⃣ Install the print head, the touch screen, and the filament runout sensor.
#️⃣ Connect the stepper, switch, screen, and print head cables. Then, attach the filament tube.
#️⃣ If the print head is shaking, adjust the hexagonal isolation column under the print head.
#️⃣ Go to Prepare➡ Leveling ➡ Auto-leveling to initiate the LeviQ 2.0 automatic bed leveling system.
#️⃣ After preheating and wiping the nozzle, Anycubic Kobra 2 probes the predefined points to level the bed.
#️⃣ Finally, fix the filament tube with the cable clips, install the filament holder, and insert the filament into the extruder.
#️⃣ Since Anycubic Kobra 2 is not officially supported by Cura yet, download the latest PrusaSlicer version and import the printer profile (configuration) file provided by Anycubic.
#️⃣ Then, create a custom printer profile on Cura for Anycubic Kobra 2 and change Start G-code and End G-code.
#️⃣ Based on the provided Start G-code and End G-code in the configuration file, I modified new Start G-code and End G-code compatible with Cura.
Start G-code:
G90 ; use absolute coordinates
M83 ; extruder relative mode
G28 ; move X/Y/Z to min endstops
G1 Z2.0 F3000 ; lift nozzle a bit
G92 E0 ; Reset Extruder
G1 X10.1 Y20 Z0.28 F5000.0 ; Move to start position
G1 X10.1 Y200.0 Z0.28 F1500.0 E15 ; Draw the first line
G1 X10.4 Y200.0 Z0.28 F5000.0 ; Move to side a little
G1 X10.4 Y20 Z0.28 F1500.0 E30 ; Draw the second line
G92 E0 ; zero the extruded length again
G1 E-2 F500 ; Retract a little
M117
G21 ; set units to millimeters
G90 ; use absolute coordinates
M82 ; use absolute distances for extrusion
G92 E0
M107
End G-code:
M104 S0 ; Extruder off
M140 S0 ; Heatbed off
M107 ; Fan off
G91 ; relative positioning
G1 E-5 F3000
G1 Z+0.3 F3000 ; lift print head
G28 X0 F3000
M84 ; disable stepper motors
#️⃣ Finally, adjust the official printer settings depending on the filament type while copying them from PrusaSlicer to Cura.
Assembling the Case and Making Connections & Adjustments
// Connections
// W5300 TOE SHIELD :
// GM77 Barcode and QR Code Scanner
// PA3 --------------------------- TX
// PA2 --------------------------- RX
// Tiny (Embedded) Thermal Printer
// PG9 --------------------------- TX
// PG14 --------------------------- RX
// SSD1309 OLED Transparent Display
// PA7 --------------------------- MOSI
// PA5 --------------------------- SCLK
// PA4 --------------------------- DC
// PC7 --------------------------- RES
// PC6 --------------------------- CS
// Control Button (A)
// PC0 --------------------------- +
// Control Button (B)
// PC3 --------------------------- +
// Control Button (C)
// PC2 --------------------------- +
// Keyes 10mm RGB LED Module (140C05)
// PA1 --------------------------- R
// PF9 --------------------------- G
// PA0 --------------------------- B
Since W5300 TOE SHIELD cannot perform without an STM32 Nucleo-144 board, I needed to attach W5300 TOE SHIELD to an STM32 development board supporting W5300 MicroPython software. As of now, W5300 TOE SHIELD is compatible with these Nucleo-144 development boards for programming in MicroPython:
- NUCLEO-F429ZI
- NUCLEO-F439ZI
- NUCLEO-F722ZE
- NUCLEO-F756ZG
- NUCLEO-F767ZI
Since NUCLEO-F429ZI is not available in my country, I decided to utilize NUCLEO-F439ZI, which shares the same pin assignments and structure with NUCLEO-F429ZI.
To utilize the ST-LINK features, WIZnet states that the ST-LINK pin connections of the STM32 Nucleo-144 board must be manually altered due to the fact that the FMC (Flexible Memory Controller) data pin to control the W5300 chip overlaps with the default ST-LINK pin.
#️⃣ First of all, remove the SB5 and SB6 resistors, which are the solder bridges for the ST-LINK-USART, from the top of the STM32 Nucleo-144 board with a soldering iron. Also, solder double-row female pin headers to the top of the STM32 board in order to attach W5300 TOE SHIELD.
#️⃣ Then, after attaching W5300 TOE SHIELD to the STM32 Nucleo-144 board, connect the PC10 and PC11 pins of W5300 TOE SHIELD to the RX and TX pins of the CN5 connector on the STM32 board via jumper wires.
- PC10 ➡ RX (CN5)
- PC11 ➡ TX (CN5)
After attaching W5300 TOE SHIELD to NUCLEO-F439ZI successfully by making manual adjustments on the board, I proceeded to connect the remaining components to W5300 TOE SHIELD via its Arduino-compatible pin layout on the top of the shield.
To scan product barcodes and the QR codes generated by the e-commerce web application, I connected a GM77 barcode and QR code scanner to W5300 TOE SHIELD. Since W5300 TOE SHIELD cannot directly supply the GM77 barcode and QR code scanner due to its demanding current loads, I connected a USB buck-boost converter board to my Xiaomi power bank to obtain stable 5V to power the scanner module.
As mentioned in the user manual of the GM77 scanner module, it does not transfer data with the built-in UART (serial) communication channel out of the box. Therefore, I needed to scan a configuration QR code provided by the manual to activate the built-in TTL 232 interface.
After activating the built-in UART (serial) interface of the scanner module, it starts to transfer data packets via serial communication immediately:
- Baud rate ➡ 9600
- Data bit ➡ 8
- Stop bit ➡ 1
Then, I connected a tiny (embedded) thermal printer to directly inform the customer when the customer requests a device status report or scans the payment confirmation QR code. Since W5300 TOE SHIELD cannot power the tiny (embedded) thermal printer due to its operating voltage, I connected a USB buck-boost converter board to my Xiaomi power bank to obtain stable 12V to supply the thermal printer. The higher input voltage (voltage range is 5~9V) means faster printing speed and more clear printed records. I utilized 12V above the recommended 9V threshold due to my power bank's current limitation.
To inform the customer of the device status, I added an SSD1309 OLED transparent screen and a 10mm common anode RGB LED module (Keyes). Since the SSD1309 screen provides a connector for the GDI display interface, I was able to connect it effortlessly.
Finally, I added three control buttons to let the customer send commands to the e-commerce web application and request a device status report printed by the thermal printer.
After completing breadboard connections and adjustments successfully, I made the breadboard connection points rigid by utilizing a hot glue gun.
After printing all parts (models), I fastened the components to their corresponding slots on the main case.
I placed the W5300 TOE SHIELD attached to NUCLEO-F439ZI and connected an Ethernet cable to the shield via the built-in RJ-45 connector. Then, I affixed the tiny thermal printer to the top of the removable cover.
Finally, I attached the removable top cover to the main case via the snap-fit joints.
As mentioned earlier, I also printed a Togepi figure, representing good luck and fortune, as a small gift for the first-time customers trying the shopping assistant :)
Creating a Brevo Account to Send HTML Emails From Localhost
Since I wanted to provide a wholesome e-commerce customer experience with this shopping assistant, I decided to make it able to send account verification and payment confirmation HTML emails, including the generated unique QR codes, to the customer's registered email address on the database. Nonetheless, I did not want to make this feature dependent on a paid email forwarder or cloud service since I built this budget-friendly device for local businesses. Therefore, I decided to send HTML emails directly from localhost via Brevo's Email API.
Brevo's Email API can handle up to 120,000 emails a minute with 99.8% successful API calls and has free of charge plan for relatively small projects like this. Even though Brevo provides official libraries (packages) for the most common programming languages, I decided to make cURL calls (HTTP requests) to Brevo's Email API in PHP. In that regard, I had the flexibility to utilize only the supported functions I needed.
#️⃣ First of all, sign up for Brevo and create a new account with the free plan. If required, you can add your company name and website, as did I.
#️⃣ After verifying the email address with which you signed in, go to Senders & IP ➡ Senders. Then, create a new sender with the selected email address.
#️⃣ To verify the new sender, click the verification link sent to the registered account email.
#️⃣ Finally, go to SMTP & API ➡ API Keys and click the Generate a new API Key button to create an API key for utilizing Brevo's Email API.
Creating an OpenAI Account to Generate Credentials for ChatGPT
As mentioned earlier, I focused on creating a low-budget but state-of-the-art AIoT shopping assistant. Therefore, I did not want to inundate shop owners to construct valid data sets for their products to generate AI-powered suggestions. After inspecting recent developments in AI-based recommendation systems, I decided to employ the OpenAI API to generate ChatGPT-powered recommendations related to the given product.
Since ChatGPT is the centerpiece of this shopping assistant, I needed to integrate ChatGPT into my e-commerce web application via the OpenAI API to minimize latency and provide a better user experience.
Although OpenAI provides official libraries in different programming languages for the OpenAI API, I decided to make cURL calls (HTTP requests) to utilize the API functions since I only needed to employ one of the built-in algorithms — gpt-3.5-turbo.
#️⃣ First of all, sign up for OpenAI to create a free account.
#️⃣ Even though a free account has limited tokens for API calls, it is more than enough for generating insightful product recommendations.
#️⃣ Then, go to API keys and click the Create new secret key button.
#️⃣ After creating the secret key, you can make direct API requests to the OpenAI API so as to generate ChatGPT-powered product recommendations.
Developing a Full-fledged E-commerce Web Application in PHP, JavaScript, CSS, and MySQL
To provide an exceptional e-commerce customer experience while physical store shopping, I developed a full-fledged web application from scratch in PHP, HTML, JavaScript, CSS, and MySQL.
First of all, the e-commerce web application allows the customer to create a user account by entering the required information on the home page via the sign-up form. Then, the web application generates a 12-digit unique account (user) token, creates a QR code with the generated user token that activates the user's cart on the shopping assistant (device), and sends an HTML email, including the account verification QR code, to the customer's registered email address. After storing the given account information in a particular MySQL database table, the web application creates a unique database table for the new user by utilizing the unique user token.
After activating the user cart via the verification QR code, the web application can receive the scanned product barcode via an HTTP GET request from the shopping assistant (device) and obtain the product information by barcode via the Open Food Facts JSON API. Depending on the selected command by the customer, the web application changes the product list in the cart by updating the product information stored in the user's database table. Also, the web application updates the user dashboard (interface) automatically to display the latest product information stored in the user's database table as an HTML table. On the user dashboard, the web application lets the customer inspect the current product list in the cart with the total cart price, generate ChatGPT-powered recommendations for each product, go to the checkout page, display the account verification QR code, and examine the previous orders.
After placing an order successfully by paying the cart amount on the checkout page, the web application generates a payment confirmation QR code with the user token and sends it via an HTML email to the customer's registered email address.
As shown below, the e-commerce web application consists of two folders and thirteen code files:
- /assets
- -- background.jpg
- -- barcode.php
- -- class.php
- -- credit.jpg
- -- dashboard_updates.php
- -- icon.png
- -- index.css
- -- index.js
- -- new_account.php
- /ChatGPT
- -- icon.png
- -- index.css
- -- index.php
- checkout.php
- dashboard.php
- index.php
- login.php
- previous_orders.php
For each section of the web application, I will talk about the related code files in the following steps.
📁 class.php
In the class.php file, I created a class named user and a child class named product to bundle the following functions under a specific structure.
⭐ Define the user class and its functions.
class user {
public $conn;
private $Brevo_API_URL = "https://api.brevo.com/v3/smtp/email";
private $Brevo_API_Key = '<_API_Key_>';
private $Brevo_email = 'freelance@theamplituhedron.com';
private $Brevo_email_name = 'AIoT Shopping Assistant';
public function __init__($conn){
$this->conn = $conn;
}
⭐ In the add_new_account function:
⭐ Check if the provided user information corresponds to an existing account stored in the users MySQL database table.
⭐ Confirm the given account password by comparing provided form parameters.
⭐ Obtain the unique account (user) token — 12 digits.
⭐ Generate an account verification QR code from the generated user token by merely making a URL GET request to Google Charts.
user%<_user_token_>
⭐ Insert new user information into the users MySQL database table.
⭐ Create a unique MySQL database table with the unique user token.
⭐ Send a verification email to the user, including the account verification QR code.
⭐ Set the required session variables.
⭐ If there is no error, redirect the customer to the user interface (dashboard).
public function add_new_account($firstname, $lastname, $email, $username, $password, $c_password){
// Check for existing users.
$existing_sql = "SELECT * FROM `users` WHERE `username`='$username' OR `email`='$email'";
$existing_sql_result = mysqli_query($this->conn, $existing_sql);
$existing_sql_check = mysqli_num_rows($existing_sql_result);
if($existing_sql_check > 0){
header('Location: ../?userAlreadyExists');
exit();
}
// Confirm the given account password.
if($password != $c_password){
header('Location: ../?wrongPassword');
exit();
}
// Obtain the unique user token — 12 digits.
$user_token = $this->generate_token(12, $username);
// Create a QR code from the given username and the generated user token.
$qr_code = "https://chart.googleapis.com/chart?cht=qr&chs=450x450&chl=user%".$user_token."&choe=UTF-8";
// Insert new user information into the users MySQL database table.
$insert_sql = "INSERT INTO `users` (`firstname`, `lastname`, `username`, `password`, `email`, `token`, `qr_code`, `successful_order`)
VALUES ('$firstname', '$lastname', '$username', '$password', '$email', '$user_token', '$qr_code', 1)";
mysqli_query($this->conn, $insert_sql);
// Create a unique MySQL database table for the registered account.
$new_table = $this->create_products_table($user_token);
if(!$new_table){ header('Location: ../?mysqlServerFailed'); exit(); }
// Send a confirmation email to the user, including the verification QR code.
$this->send_confirmation_email($email, "Verify Your Account", $firstname.' '.$lastname, $qr_code);
// Set the required session variables.
$_SESSION["name"] = $firstname.' '.$lastname;
$_SESSION["username"] = $username;
$_SESSION["email"] = $email;
$_SESSION["user_token"] = $user_token;
$_SESSION["qr_code"] = $qr_code;
// If there is no error, go to the user interface (dashboard).
header('Location: ../dashboard.php');
exit();
}
⭐ In the user_login_request function:
⭐ Check whether the given account information is accurate when the user requests to log into an existing account.
⭐ After logging into the account successfully, set the required session variables.
⭐ If there is no error, redirect the customer to the user interface (dashboard).
public function user_login_request($u_username, $u_password){
// Check whether the given account information is accurate.
$account_sql = "SELECT * FROM `users` WHERE `username`='$u_username' AND `password`='$u_password'";
$account_sql_result = mysqli_query($this->conn, $account_sql);
$account_sql_check = mysqli_num_rows($account_sql_result);
if($account_sql_check > 0){
if($row = mysqli_fetch_assoc($account_sql_result)){
// Set the required session variables.
$_SESSION["name"] = $row['firstname'].' '.$row['lastname'];
$_SESSION["username"] = $row['username'];
$_SESSION["email"] = $row['email'];
$_SESSION["user_token"] = $row['token'];
$_SESSION["qr_code"] = $row['qr_code'];
// If there is no error, go to the user interface (dashboard).
header('Location: ../dashboard.php');
exit();
}
}else{
header('Location: ../login.php?noAccountFound');
exit();
}
}
⭐ In the generate_token function:
⭐ Define the lowercase, uppercase, number, and symbol characters to create the main string.
⭐ Derive a unique key from the main string by utilizing the random_int function and add the given username to the generated key to create the unique user token.
private function generate_token($len, $username){
// Define the main string.
$lowercase = "abcdefghijklmnopqrstuvwxyz"; $uppercase = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"; $number = "0123456789"; $symbol = "*()[]{}#$?!";
$main = $lowercase.$uppercase.$number.$symbol;
// Derive the user token from the main string.
$token = "";
for ($i=0; $i<$len; $i++){ $token .= $main[random_int(0, (strlen($main)-1))]; }
return $username."_".$token;
}
⭐ In the create_products_table, create a unique MySQL database table by using the user token for the newly registered user.
private function create_products_table($table){
// Create a new database table.
$sql_create = "CREATE TABLE `$table`(
id int AUTO_INCREMENT PRIMARY KEY NOT NULL,
product_barcode varchar(255) NOT NULL,
product_name varchar(255) NOT NULL,
product_ingredients varchar(255) NOT NULL,
product_price int NOT NULL,
cart_number int NOT NULL,
order_number int NOT NULL
);";
if(mysqli_query($this->conn, $sql_create)){ return true; } else{ return false; }
}
⭐ In the send_Brevo_email function:
⭐ Define the POST data parameters required by Brevo's Email API in the JSON format.
⭐ Define the required HTML headers, including the API key, for authentification.
⭐ Send an HTML email with the given content by making a cURL call (HTTP request) to Brevo's Email API.
⭐ Execute the defined cURL call with the given parameters.
public function send_Brevo_email($to_email, $subject, $name, $html_content){
// Define POST data parameters in the JSON format.
$data = '{
"sender":{
"name":"'.$this->Brevo_email_name.'",
"email":"'.$this->Brevo_email.'"
},
"to":[
{
"email":"'.$to_email.'",
"name":"'.$name.'"
}
],
"subject":"'.$subject.'",
"htmlContent":"'.$html_content.'"
}';
// Define the required HTML headers.
$headers = array('accept: application/json', 'api-key:'.$this->Brevo_API_Key, 'content-type: application/json');
// Send an HTML email via Brevo's Email API by making a cURL call.
$curl = curl_init();
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_URL, $this->Brevo_API_URL);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
// Execute the defined cURL call.
$result = curl_exec($curl);
if(!$result){ header('Location: ../?emailServerFailed'); exit(); }
curl_close($curl);
}
⭐ In the send_confirmation_email function:
⭐ Define the HTML content of the account verification email, including the unique account verification QR code.
⭐ Transfer the given HTML email to the user's registered email address via Brevo's Email API.
private function send_confirmation_email($to_email, $subject, $name, $qr_code){
// Define the HTML message (content) of the email.
$html_content = '<html><head><style>h1{text-align:center;font-weight:bold;color:#505F4E;font-size:40px;}a{text-decoration:none;color:#9BB5CE;font-size:18px;font-weight:bold;}div{display:block;background-color:#F9E5C9;text-align:center;border:50px solid #5C5B57;}div p{font-size:25px;color:#505F4E;font-weight:bold;}@media only screen and (max-width: 600px){h1{font-size:20px;}a{font-size:9px;}div{border:10px solid #5C5B57;}div p{font-size:12px;}}</style></head><body><div><h1>Thanks for trying AIoT Shopping Assistant 😊</h1><img src=\"'.$qr_code.'\" alt=\"QR_CODE\" /><p>Please scan the account QR code with the shopping assistant to activate your cart 🛍️</p><a href=\"http://192.168.1.22/AIoT_Shopping_Assistant/\">➡️ Go to your Dashboard<br><br></a></div></body></html>';
// Transfer the HTML email.
$this->send_Brevo_email($to_email, $subject, $name, $html_content);
}
⭐ Define the product class and its functions, extending the user class.
private $OPENAI_API_KEY = "<_OPENAI_API_KEY_>";
private $OPENAI_ENDPOINT = "https://api.openai.com/v1/chat/completions";
⭐ In the get_product_info function:
⭐ Make an HTTP GET request to the Open Food Facts JSON API with the given barcode.
⭐ Then, decode the received JSON object to obtain the product name, ingredients, and price.
public function get_product_info($barcode){
// Make an HTTP GET request to the Open Food Facts JSON API.
// Then, decode the received JSON object.
$data = json_decode(file_get_contents("https://world.openfoodfacts.org/api/v0/product/".$barcode.".json", TRUE));
$product_info = array(
"name" => $data->product->product_name,
"ingredients" => (is_null($data->product->ingredients_text_en) || $data->product->ingredients_text_en == "") ? "Not Found" : $data->product->ingredients_text_en,
"price" => (int)$data->product->product_quantity / 100
);
return $product_info;
}
⭐ In the get_current_products function:
⭐ Obtain the current order tag (number) from the users database table.
⭐ Fetch all registered product information as the current cart list, sharing the latest order number.
⭐ Store the fetched product information as arrays.
⭐ Calculate the total cart price (amount).
⭐ Then, return the generated arrays and the estimated total cart price as a list.
public function get_current_products($table){
$total_price = 0;
// Obtain the current order tag (number).
$order_number = $this->get_order_number($table);
// Obtain all registered product information of the current cart as a list.
$p_barcode = []; $p_name = []; $p_ingredients = []; $p_price = []; $p_number = [];
$sql_list = "SELECT * FROM `$table` WHERE `order_number`='$order_number' ORDER BY `id` ASC";
$result = mysqli_query($this->conn, $sql_list);
$check = mysqli_num_rows($result);
if($check > 0){
while($row = mysqli_fetch_assoc($result)){
// Store the fetched product information as arrays.
array_push($p_barcode, $row["product_barcode"]);
array_push($p_name, $row["product_name"]);
array_push($p_ingredients, $row["product_ingredients"]);
array_push($p_price, $row["product_price"]);
array_push($p_number, $row["cart_number"]);
// Calculate the total cart price (amount).
$price = $row["product_price"] * $row["cart_number"];
$total_price+=$price;
}
return array($p_barcode, $p_name, $p_ingredients, $p_price, $p_number, array("total_price" => $total_price));
}else{
return array(["Not Found!"], ["Not Found!"], ["Not Found!"], ["Not Found!"], ["Not Found!"], array("total_price" => 0));
}
}
⭐ In the get_previous_orders function:
⭐ Obtain the current order tag (number) from the users database table.
⭐ If there are any previous orders, return the purchased products as an HTML list for each order by utilizing the given order numbers.
public function get_previous_orders($table){
// Obtain the current order tag (number).
$order_number = $this->get_order_number($table);
// If there are any previous orders, return the purchased products as an HTML list for each order.
if($order_number == 1){
echo '<h1><i class="fa-solid fa-circle-xmark"></i> No previous order was found!</h1>';
}else{
$list = "";
for($i=1;$i<$order_number;$i++){
$sql = "SELECT * FROM `$table` WHERE `order_number`='$i' ORDER BY `id` ASC";
$result = mysqli_query($this->conn, $sql);
$check = mysqli_num_rows($result);
if($check > 0){
while($row = mysqli_fetch_assoc($result)){
$line = '<li>'.$row["product_name"].' ['.$row["product_barcode"].'] <span><i class="fa-solid fa-xmark"></i></span>'.$row["cart_number"].'</li>';
$list.=$line;
}
}
echo '<div class="orders"><h2><i class="fa-solid fa-cash-register"></i> Order ['.$i.']</h2><ul>'.$list.'</ul></div>';
$list = "";
}
}
}
⭐ In the user_checkout function:
⭐ Generate a payment confirmation QR code from the generated user token by merely making a URL GET request to Google Charts.
finished<_user_token_>
⭐ Update the successful order number for the customer in the users database table after the checkout process.
⭐ Send a confirmation email to the user, including the payment confirmation QR code.
⭐ If there is no error, redirect the customer to the user interface (dashboard).
public function user_checkout($table, $email, $name){
// Create a QR code from the user token and the given command.
$qr_text = 'finished%'.$table;
$qr_code = "https://chart.googleapis.com/chart?cht=qr&chs=450x450&chl=".$qr_text."&choe=UTF-8";
// Update the successful order number after the checkout process.
$sql = "UPDATE `users` SET `successful_order`=`successful_order`+1 WHERE `token` = '$table'";
mysqli_query($this->conn, $sql);
// Send a notification email to the user, including the unique payment QR code.
$this->send_payment_email($email, "Order Successful", $name, $qr_code);
// If there is no error, go to the user interface (dashboard).
header('Location: ./dashboard.php?paymentCompleted');
exit();
}
⭐ In the send_payment_email function:
⭐ Define the HTML content of the payment confirmation email, including the unique payment confirmation QR code.
⭐ Transfer the given HTML email to the user's registered email address via Brevo's Email API.
private function send_payment_email($to_email, $subject, $name, $qr_code){
// Define the HTML message (content) of the email.
$html_content = '<html><head><style>h1{text-align:center;font-weight:bold;color:#505F4E;font-size:40px;}a{text-decoration:none;color:#9BB5CE;font-size:18px;font-weight:bold;}div{display:block;background-color:#F9E5C9;text-align:center;border:50px solid #5C5B57;}div p{font-size:25px;color:#505F4E;font-weight:bold;}@media only screen and (max-width: 600px){h1{font-size:20px;}a{font-size:9px;}div{border:10px solid #5C5B57;}div p{font-size:12px;}}</style></head><body><div><h1>Thanks for your order 😊👍</h1><img src=\"'.$qr_code.'\" alt=\"QR_CODE\" /><p>Please scan your payment QR code with the shopping assistant to complete your order 💲✅</p><a href=\"http://192.168.1.22/AIoT_Shopping_Assistant/\">➡️ Go to your Dashboard<br><br></a></div></body></html>';
// Transfer the HTML email.
$this->send_Brevo_email($to_email, $subject, $name, $html_content);
}
⭐ In the chatgpt_get_suggestion function:
⭐ Define the questions related to the given product information.
⭐ Define the POST data parameters required by the OpenAI API in the JSON format.
⭐ Define the required HTML headers, including the OpenAI API key, for authentification.
⭐ Obtain ChatGPT-powered product suggestions by making a cURL call (HTTP request) to the OpenAI API.
⭐ Execute the defined cURL call with the given parameters.
⭐ After getting the response from the OpenAI API, decode the received JSON object to obtain the recommendations generated by ChatGPT — gpt-3.5-turbo.
⭐ Then, modify the obtained suggestions to add line breaks for each answer by utilizing the added question numbers.
⭐ Return the modified suggestions and the defined product questions.
public function chatgpt_get_suggestion($product){
// Define the questions related to the given product.
$questions = array(
"What is the nutritional value of ".$product."?",
"What should I purchase with ".$product."?",
"Can you teach me a recipe with ".$product."?",
"How should I serve ".$product."?",
"Is there a more affordable and healthy option than ".$product."?"
);
// Define POST data parameters in the JSON format.
$data = '{
"model": "gpt-3.5-turbo",
"messages": [
{"role": "user", "content": "'.$questions[0].'"},
{"role": "user", "content": "'.$questions[1].'"},
{"role": "user", "content": "'.$questions[2].'"},
{"role": "user", "content": "'.$questions[3].'"},
{"role": "user", "content": "'.$questions[4].'"},
{"role": "user", "content": "Please add the exact question at the beginning of the answer with the question number."}
],
"temperature": 0.7
}';
// Define the required HTML headers.
$headers = array('Authorization: Bearer '.$this->OPENAI_API_KEY, 'Content-Type: application/json');
// Obtain product suggestions from ChatGPT by making a cURL call to the OpenAI API.
$curl = curl_init();
curl_setopt($curl, CURLOPT_POST, 1);
curl_setopt($curl, CURLOPT_POSTFIELDS, $data);
curl_setopt($curl, CURLOPT_URL, $this->OPENAI_ENDPOINT);
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
// Execute the defined cURL call.
$result = curl_exec($curl);
if(!$result){ header('Location: ../?ChatGPTServerFailed'); exit(); }
curl_close($curl);
// Decode the received JSON object to obtain suggestions generated by ChatGPT.
$res = json_decode($result);
$suggestions = $res->choices[0]->message->content;
// Modify the obtained suggestions to add line breaks.
$modified_suggestions = $suggestions;
$modified_suggestions = str_replace('1. '.$questions[0], "<h2>Suggestions</h2>", $modified_suggestions);
for($i=1;$i<count($questions);$i++){
$modified_suggestions = str_replace(strval($i+1).'. '.$questions[$i], "<br><br>", $modified_suggestions);
}
// Return the modified suggestions and the defined product questions.
return array($modified_suggestions, $questions);
}
⭐ In the insert_product function:
⭐ Obtain the current order tag (number) from the users database table.
⭐ Check whether the given product barcode is in the user's database table with the recent order tag.
⭐ If the given product is already in the current cart (table), update the product amount (cart number).
⭐ If not, insert the new product information into the user's database table with the recent order tag.
public function insert_product($table, $barcode, $name, $ingredients, $price){
// Obtain the current order tag (number).
$order_number = $this->get_order_number($table);
// Check whether the given product is in the user's database table or not.
if($this->check_product($table, $barcode, $order_number)){
// If the given product is already in the cart (table), update the product amount (cart number).
$sql_update = "UPDATE `$table` SET `cart_number`=cart_number+1 WHERE `product_barcode` = '$barcode'";
if(mysqli_query($this->conn, $sql_update)){ return true; } else{ return false; }
}else{
// If not, insert the new product information into the user's database table.
$sql_insert = "INSERT INTO `$table` (`product_barcode`, `product_name`, `product_ingredients`, `product_price`, `cart_number`, `order_number`)
VALUES('$barcode', '$name', '$ingredients', '$price', 1, '$order_number');
";
if(mysqli_query($this->conn, $sql_insert)){ return true; } else{ return false; }
}
}
⭐ In the delete_product function:
⭐ Obtain the current order tag (number) from the users database table.
⭐ Check whether the given product barcode is in the user's database table with the recent order tag.
⭐ If so, remove the given product from the current cart (table).
public function delete_product($table, $barcode){
// Obtain the current order tag (number).
$order_number = $this->get_order_number($table);
// Check whether the given product is in the user's database table or not.
if($this->check_product($table, $barcode, $order_number)){
// Remove the given product from the cart (table).
$sql_delete = "DELETE FROM `$table` WHERE `product_barcode`='$barcode' AND `order_number` = '$order_number'";
if(mysqli_query($this->conn, $sql_delete)){ return true; } else{ return false; }
}
}
⭐ In the check_table function, check if there is an existing user database table named with the given user token.
public function check_table($table){
$sql = "SELECT * FROM `information_schema`.`TABLES` WHERE `table_name` = '$table' limit 1";
$sql_result = mysqli_query($this->conn, $sql);
$sql_check = mysqli_num_rows($sql_result);
if($sql_check > 0){ return true; } else{ return false; }
}
⭐ In the check_product function, check whether the given product barcode is in the user's database table with the given order tag.
private function check_product($table, $barcode, $order_number){
$sql = "SELECT * FROM `$table` WHERE `product_barcode` = '$barcode' AND `order_number` = '$order_number'";
$sql_result = mysqli_query($this->conn, $sql);
$sql_check = mysqli_num_rows($sql_result);
if($sql_check > 0){ return true; } else{ return false; }
}
⭐ In the get_order_number function, obtain the current order tag (number) from the users database table via the given user token.
private function get_order_number($token){
$order_number = 0;
$sql = "SELECT * FROM `users` WHERE `token` = '$token'";
$sql_result = mysqli_query($this->conn, $sql);
$sql_check = mysqli_num_rows($sql_result);
if($sql_check > 0){
if($row = mysqli_fetch_assoc($sql_result)){
$order_number = $row["successful_order"];
}
return $order_number;
}
}
⭐ Define the required MariaDB database connection settings for LattePanda 3 Delta 864.
$server = array(
"name" => "localhost",
"username" => "root",
"password" => "",
"database" => "shopping_assistant_users"
);
$conn = mysqli_connect($server["name"], $server["username"], $server["password"], $server["database"]);
Using the Open Food Facts JSON API to Get Product Information by Barcode
Open Food Facts is created by a non-profit association and provides a worldwide food products database. Also, the Open Food Facts JSON API lets the user search a product by barcode and obtain provided product characteristics by making an HTTP GET request.
Although Open Food Facts only covers food products, it has a vast database, including products worldwide. Therefore, I decided to utilize the Open Food Facts JSON API to obtain product information by barcode to demonstrate the features of my shopping assistant.
If you want to obtain product information by barcodes of products other than foods, you can utilize some overall product database services with paid subscription plans, such as Barcode Lookup.
📁 barcode.php
⭐ Include the class.php file.
include_once "class.php";
⭐ Define the _product object of the product class extending the user class.
$_product = new product();
$_product->__init__($conn);
⭐ If all required HTML (GET) parameters are received:
⭐ Check whether the given table is in the MariaDB (MySQL) database.
⭐ If so, make an HTTP GET request to the Open Food Facts JSON API to obtain the product information with the given barcode.
⭐ According to the selected command by the user, add or remove the given product information to/from the user's database table.
if(isset($_GET["table"]) && isset($_GET["barcode"]) && isset($_GET["com"])){
// Check whether the given table is in the MySQL database.
if($_product->check_table($_GET["table"])){
// Make an HTTP GET request to the Open Food Facts JSON API to obtain the product information with the given barcode.
$product_info = $_product->get_product_info($_GET["barcode"]);
// According to the selected command by the user, add or remove the given product to/from the user's database table.
if($_GET["com"] == "add"){
$_product->insert_product($_GET["table"], $_GET["barcode"], $product_info["name"], $product_info["ingredients"], $product_info["price"]);
echo "Given Product Added to the Cart Successfully!";
}else if($_GET["com"] == "remove"){
$_product->delete_product($_GET["table"], $_GET["barcode"]);
echo "Given Product Removed from the Cart Successfully!";
}
}else{
echo "No Table Found!";
}
}
Designing the Home Page and the Sign-in Page
📁 new_account.php
⭐ Include the class.php file.
include_once "class.php";
⭐ Define the _user object of the user class.
$_user = new user();
$_user->__init__($conn);
⭐ Create a new user account if the customer fills out the sign-up form accurately.
if(isset($_GET['firstname']) && isset($_GET['lastname']) && isset($_GET['email']) && isset($_GET['username']) && isset($_GET['password']) && isset($_GET['c_password'])){
$_user->add_new_account($_GET['firstname'], $_GET['lastname'], $_GET['email'], $_GET['username'], $_GET['password'], $_GET['c_password']);
}
⭐ Sign into an existing account if the user fills out the sign-in form correctly.
if(isset($_GET['u_username']) && isset($_GET['u_password'])){
$_user->user_login_request($_GET['u_username'], $_GET['u_password']);
}
📁 index.php
⭐ If the user has already signed in, redirect the customer to the user interface (dashboard).
if(isset($_SESSION["username"]) && isset($_SESSION["email"])){
header('Location: ./dashboard.php');
exit();
}
⭐ Design the sign-up form and the home page of the e-commerce web application.
You can inspect and download the index.php file below.
📁 login.php
⭐ If the user has already signed in, redirect the customer to the user interface (dashboard).
if(isset($_SESSION["username"]) && isset($_SESSION["email"])){
header('Location: ./dashboard.php');
exit();
}
⭐ Design the sign-in page for returning customers.
📁 index.css
You can inspect and download the index.css file below.
Designing the User Interface (dashboard) Showing the Account QR Code
📁 dashboard.php
⭐ If the user signed into an existing account successfully, proceed to load the user dashboard.
if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
header('Location: ./');
exit();
}
⭐ If the user requests to log out, remove and destroy all session variables.
⭐ Then, redirect the user to the home page.
if(isset($_GET["logout"])){
// Remove and destroy all session variables.
session_unset();
session_destroy();
// Go to the home page.
header('Location: ./');
exit();
}
⭐ Design the user dashboard (interface) consists of:
- a self-updating product list in the current cart,
- total price indicator and the Checkout button,
- the unique user (account) token,
- the Logout button,
- the Account Information section,
- the Previous Orders button.
⭐ Design the hidden checkout form to pass the estimated total cart amount to the checkout page.
<form id="hidden_checkout" action="checkout.php" method="post" target="_blank" style="display:none;">
<input id="checkout_price" name="checkout_price" type="hidden" value="$0">
</form>
📁 dashboard_updates.php
⭐ Include the class.php file.
⭐ If the user did not sign into an existing account, redirect the user to the home page.
include_once "class.php";
if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
header('Location: ./');
exit();
}
⭐ Define the _product object of the product class extending the user class.
$_product = new product();
$_product->__init__($conn);
⭐ If the update URL (GET) parameter is received:
⭐ Retrieve the product information of the current cart from the user's database table as a list.
⭐ Generate HTML table rows consisting of the retrieved product information and the Ask ChatGPT button redirecting to the ChatGPT recommendations page of the product.
⭐ Then, create a JSON object from the generated HTML table rows and the evaluated total cart price.
⭐ Print the created JSON object.
if(isset($_GET["update"])){
$p_barcode = []; $p_name = []; $p_ingredients = []; $p_price = []; $p_number = []; $total_price = [];
list($p_barcode, $p_name, $p_ingredients, $p_price, $p_number, $total_price) = $_product->get_current_products($_SESSION["user_token"]);
$list = "<tr><th>Product</th><th>Barcode</th><th>Ingredients</th><th>Price($)</th><th>Unit</th><th>Counsel</th></tr>";
for($i=0; $i<count($p_barcode); $i++){
$list .= '<tr>
<td>'.$p_name[$i].'</td>
<td>'.$p_barcode[$i].'</td>
<td>'.$p_ingredients[$i].'</td>
<td>'.$p_price[$i].'</td>
<td>'.$p_number[$i].'</td>
<td><a href="ChatGPT/?product='.$p_name[$i].'" target="_blank"><button><i class="fa-solid fa-comment-dots"></i> Ask ChatGPT</button></a></td>
</tr>
';
}
// Create a JSON object from the generated HTML table rows consisting of the product information and the evaluated total cart price.
$data = array("list" => $list, "total_price" => "$".$total_price["total_price"]);
$j_data = json_encode($data);
// Return the recently generated JSON object.
echo($j_data);
}
📁 index.js (jQuery and AJAX)
⭐ Every 5 seconds, make an HTTP GET request to the dashboard_updates.php file.
⭐ Then, decode the received JSON object to obtain the HTML table rows consisting of the product information of the current cart from the user's database table and the estimated total cart price.
⭐ Assign the obtained table rows and the total cart price to the corresponding HTML elements on the user dashboard so as to showcase the recent products added to the cart and inform the customer of the calculated total cart price (amount).
setInterval(function(){
$.ajax({
url: "./assets/dashboard_updates.php?update",
type: "GET",
success: (response) => {
// Decode the obtained JSON object.
const data = JSON.parse(response);
// Assign the HTML table rows as the current product list in the cart.
$(".products table").html(data.list);
// Assign the evaluated total cart price (amount).
$("#total_price").html(data.total_price);
}
});
}, 5000);
⭐ When the user clicks the Checkout button, redirect the customer to the checkout page and transfer the evaluated total cart price via the hidden HTML form.
$(".info").on("click", "#checkout", () => {
var total_price = $("#total_price").text();
$("#checkout_price").val(total_price);
$("#hidden_checkout").submit();
});
Displaying the Previous Orders When Requested
📁 previous_orders.php
⭐ Include the class.php file.
include_once "assets/class.php";
⭐ Define the _product object of the product class extending the user class.
$_product = new product();
$_product->__init__($conn);
⭐ If the user signed into an existing account successfully, proceed to load the previous orders page.
if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
header('Location: ./');
exit();
}
⭐ Design the previous orders page to display the purchased product list of each previous order as an HTML list.
$_product->get_previous_orders($_SESSION["user_token"]);
Designing the Checkout (payment) Page
After creating the checkout page and HTML form to get the payment information (credit/debit card), I did not integrate a payment processing service since I do not have partner access as a business owner to general payment services, such as Visa.
Nonetheless, it is easy to send the received payment information to a payment processing service if you have a local business with partner access, as explained at Visa Checkout.
📁 checkout.php
⭐ Include the class.php file.
include_once "assets/class.php";
⭐ Define the _product object of the product class extending the user class.
$_product = new product();
$_product->__init__($conn);
⭐ If the user signed into an existing account successfully, proceed to load the checkout page.
if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
header('Location: ./');
exit();
}
⭐ If the user places an order by entering the requested credit/debit card information, execute the user_checkout function, explained in the previous steps.
if(isset($_GET["c_number"]) && isset($_GET["c_name"]) && isset($_GET["c_date"]) && isset($_GET["c_verify"])){
$_product->user_checkout($_SESSION["user_token"], $_SESSION["email"], $_SESSION["name"]);
}
⭐ Design the checkout page and the checkout HTML form.
⭐ Check whether the total cart price variable is received via the hidden HTML form on the dashboard page.
if(isset($_POST["checkout_price"])){
echo $_POST["checkout_price"];
}else{
echo "$0";
}
Integrating ChatGPT Into the Web Application Via the OpenAI API
As explained in the previous steps, I employed the OpenAI API to generate ChatGPT-powered recommendations for the given product instead of coercing local businesses to construct large data sets of their products.
In that regard, I was able to provide insightful product suggestions by merely making API calls to the OpenAI API.
While making a cURL call (HTTP request) to the OpenAI API, the web application passes five questions for the gpt-3.5-turbo model:
- What is the nutritional value of <_product_>?
- What should I purchase with <_product_>?
- Can you teach me a recipe with <_product_>?
- How should I serve <_product_>?
- Is there a more affordable and healthy option than <_product_>?
Since ChatGPT accepts system commands in the text format to alter the generated response structure, I added a simple command to add the exact question with the question number before each answer. In that regard, I was able to modify the retrieved response from ChatGPT to add line breaks between answers.
{"role": "user", "content": "Please add the exact question at the beginning of the answer with the question number."}
📁 index.php
⭐ Include the class.php file.
include_once "../assets/class.php";
⭐ Define the _product object of the product class extending the user class.
$_product = new product();
$_product->__init__($conn);
⭐ If the user signed into an existing account successfully, proceed to load the ChatGPT recommendations page.
if(!isset($_SESSION["username"]) && !isset($_SESSION["email"])){
header('Location: ./');
exit();
}
⭐ If the customer requests to get ChatGPT-powered recommendations for an eligible product, execute the chatgpt_get_suggestion function so as to make a cURL call (HTTP request) to the OpenAI API in order to get suggestions regarding the given product from ChatGPT.
$suggestions = "<h2>Please enter a product name.</h2>";
$questions = ["Please enter a product name."];
if(isset($_GET["product"]) && $_GET["product"] != "Not Found!"){
list($suggestions, $questions) = $_product->chatgpt_get_suggestion($_GET["product"]);
}
⭐ Design the ChatGPT recommendations page by utilizing the ChatGPT theme.
⭐ Print the defined questions and the modified ChatGPT-powered answers to inform the customer.
for($i=0;$i<count($questions);$i++){
echo '<h2><i class="fa-regular fa-comment-dots"></i> '.$questions[$i].'</h2>';
}
...
echo $suggestions;
📁 index.css
You can download and inspect the index.css file of the ChatGPT folder below.
Setting and Running the E-commerce Web Application on LattePanda 3 Delta
Since I wanted to build a budget-friendly AIoT shopping assistant not dependent on cloud or hosting services, I decided to host my web application on LattePanda 3 Delta 864. Therefore, I needed to set up a LAMP web server.
LattePanda 3 Delta is a pocket-sized hackable computer that provides ultra performance with the Intel 11th-generation Celeron N5105 processor.
Plausibly, LattePanda 3 Delta can run the XAMPP application. So, it is effortless to create a server with a MariaDB database on LattePanda 3 Delta.
Also, I utilized a DFRobot 8.9" IPS touch display to demonstrate my e-commerce web application features.
#️⃣ First of all, install and set up the XAMPP application.
#️⃣ Then, go to the XAMPP Control Panel and click the MySQL Admin button.
#️⃣ Once the phpMyAdmin tool pops up, create a new database named shopping_assistant_users.
#️⃣ After adding the database successfully, go to the SQL section to create a MySQL database table named users with the required data fields.
CREATE TABLE `users`(
id int AUTO_INCREMENT PRIMARY KEY NOT NULL,
firstname varchar(255) NOT NULL,
lastname varchar(255) NOT NULL,
username varchar(255) NOT NULL,
`password` varchar(255) NOT NULL,
email varchar(255) NOT NULL,
token varchar(255) NOT NULL,
qr_code varchar(255) NOT NULL,
successful_order int NOT NULL
);
Configuring NUCLEO-F439ZI With the Modified W5300 TOE SHIELD Firmware
Since the STM32 Nucleo-144 boards need to be flashed with the compatible Wiznet5K firmware to control W5300 TOE SHIELD and program it in MicroPython, I had to upload the official Wiznet5K firmware to the NUCLEO-F439ZI development board.
Plausibly, WIZnet provides official pre-built firmwares for these Nucleo-144 boards:
- NUCLEO_F429ZI
- NUCLEO_F439ZI
- NUCLEO_F722ZE
- NUCLEO_F756ZG
- NUCLEO_F767ZI
Although there is a pre-built firmware for NUCLEO-F439ZI, I decided to compile my Wiznet5K MicroPython firmware in order to change some minor pin settings due to overlapping.
You can download my updated firmware below — firmware_updated.hex.
Since I have a secondary operating system (Ubuntu) on my LattePanda 3 Delta, I was able to use it to build my Wiznet5K firmware.
#️⃣ First, open the terminal to install the required Python modules.
sudo apt-get install git
sudo apt-get install make
sudo apt-get install gcc
sudo apt-get install gcc-arm-none-eabi
#️⃣ Then, clone the official Wiznet5K MicroPython project from GitHub by entering the following Git command.
git clone https://github.com/Wiznet/W5300-TOE-MicroPython.git
#️⃣ After installing the MicroPython project, I modified some pin settings for NUCLEO-F439ZI in the mpconfigboard.h file.
/libraries/ports/stm32/boards/NUCLEO_F439ZI/mpconfigboard.h
#️⃣ Finally, I built my modified firmware for the NUCLEO-F439ZI development board attached to W5300 TOE SHIELD. Do not forget to change the board name and the chip model for different combinations.
cd W5300-TOE-MicroPython/libraries/mpy-cross
make
cd ../ports/stm32
make BOARD=NUCLEO-F439ZI MICROPY_PY_WIZNET5K=5300
#️⃣ Navigate to the build-NUCLEO-F439ZI folder available in /libraries/ports/stm32, and the compiled firmware file should be there — firmware.hex.
After building my firmware version successfully, I employed the STM32CubeProgrammer on Windows to upload the modified firmware to NUCLEO-F439ZI.
#️⃣ First, click the Open File button to open the modified firmware file — firmware_updated.hex.
#️⃣ Then, connect NUCLEO-F439ZI to the computer via a USB cable and click the Connect button.
#️⃣ Finally, click the Download button to flash NUCLEO-F439ZI with the given firmware.
Installing and Modifying MicroPython Libraries on NUCLEO-F439ZI
After flashing my Wiznet5K firmware to the NUCLEO-F439ZI development board, I utilized Thonny to program NUCLEO-F439ZI attached to W5300 TOE SHIELD.
However, since the Wiznet5K firmware does not include the required modules for some components, I needed to upload modules to NUCLEO-F439ZI and modify them if the default settings are not compatible with W5300 TOE SHIELD.
⚠️ Notice:
Even though the W5300 TOE SHIELD datasheet mentions one available hardware serial port (USART6), NUCLEO-F439ZI has two eligible hardware serial ports (USART2 and USART6), as shown in the STM32 Nucleo-144 user manual. Therefore, I was able to connect two components (the scanner module and the thermal printer), requiring serial communication.
#️⃣ To program NUCLEO-F439ZI on Thonny, go to Tools ➡ Select interpreter... and choose MicroPython (generic) with the given STLink Virtual COM Port.
#️⃣ Then, via Thonny's file explorer, select the MicroPython device's flash folder to upload or update files.
#️⃣ To make HTTP requests easily, upload the urequests.py file (module) to NUCLEO-F439ZI via Thonny's file explorer.
#️⃣ To get the pre-defined functions for the W5300 chip, upload the wiznet_conf.py file.
#️⃣ To control the SSD1309 transparent display, upload the ssd1309.py file.
#️⃣ To display fonts provided by the ssd1309 module on the transparent screen, upload the xglcd_font.py file.
#️⃣ To control the tiny (embedded) thermal printer, upload the Adafruit_Thermal.py file.
Since this module utilizes USART1 as the default hardware serial port and it is not available on NUCLEO-F439ZI, I needed to modify the Adafruit_Thermal.py file to utilize USART6 instead. I also added the required port settings for the thermal printer.
self.uart = UART(6, 9600, bits=8, parity=None, stop=1)
I also modified the data_end parameter for the printed bitmap images to align them perfectly.
data_end = header.file_size - 0
#️⃣ Finally, create a folder named assets on the device to upload the required image and font files.
You can download the assets folder with the added files below.
Programming NUCLEO-F439ZI to Scan Data and Communicate With the E-commerce Application
After uploading all required MicroPython modules and files to NUCLEO-F439ZI, I created the main.py file to program W5300 TOE SHIELD. Since NUCLEO-F439ZI runs the main.py file automatically, I did not encounter any problems while initiating the shopping assistant (device).
⭐ Include the required modules.
⭐ Define the shopping_assistant class and its functions.
⭐ In the __init__ function:
⭐ Define the W5300 object and the static IP address settings depending on your network settings.
⭐ Define the required print and hardware serial port settings (USART6) for the tiny (embedded) thermal printer.
⭐ Define the required hardware serial port settings (USART2) for the GM77 barcode and QR code scanner.
⭐ Define the required SPI settings for the SSD1309 OLED transparent display.
⭐ Get the fonts stored in the assets folder.
⭐ Define the control button pins.
⭐ Define the RGB LED pins.
⭐ Initialize the SSD1309 OLED transparent display.
class shopping_assistant:
def __init__(self):
# Define the W5300 object and the static IP address settings.
self.w5300 = wiznet5k_w5300()
self.w5300.w5300_set_ip('0.0.0.0','0.0.0.0','0.0.0.0','0.0.0.0')
# Define the required print and hardware serial port settings for the tiny (embedded) thermal printer.
self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
# Define the required hardware serial port settings for the GM77 barcode and QR code scanner.
self.scanner = UART(2, 9600, bits=8, parity=None, stop=1)
self.user_token = ""
self.new_product_barcode = ""
# Define the required settings for the SSD1309 OLED transparent display. (SCK: PA5, MOSI: PA7)
spi = SPI(1, SPI.MASTER, baudrate=10000000)
self.display = Display(spi, dc=Pin("A4"), cs=Pin("C6"), rst=Pin("C7"))
# Define the given fonts.
self.bold = XglcdFont('assets/Unispace12x24.c', 12, 24)
self.light = XglcdFont('assets/Bally5x8.c', 5, 8)
# Define the control button pins.
self.button_A = Pin("C0", Pin.IN, Pin.PULL_UP)
self.button_B = Pin("C3", Pin.IN, Pin.PULL_UP)
self.button_C = Pin("C2", Pin.IN, Pin.PULL_UP)
# Define the RGB LED pins.
self.red = Pin("A1", Pin.OUT_PP)
self.blue = Pin("A0", Pin.OUT_PP)
self.green = Pin("F9", Pin.OUT_PP)
self.adjust_color([0,0,0])
sleep(2)
# Initialize the SSD1309 OLED transparent display.
self.product_menu_activate = False
self.change_layout("home")
⭐ In the read_QR_barcode function:
⭐ If the scanner module reads a barcode or QR code successfully, get the data packet via serial communication.
⭐ Decode and modify the received data packet to obtain the scanned information.
⭐ If the scanned information includes the user command and the unique user token, activate the cart and register the given user token.
⭐ If the scanned information includes the finished command and the received user token corresponds to the registered user token, discard the registered user token to deactivate the cart.
⭐ Then, notify the customer via the thermal printer.
⭐ If the cart is activated and the scanned information includes a product barcode, save the given product barcode and initiate the product menu (interface) to allow the customer to select a command (add or remove) before transferring the given barcode to the e-commerce web application.
def read_QR_barcode(self):
scanned_data = self.scanner.readline()
# Decode and modify the received data packet to obtain the user (account) token with the given command or the new product barcode.
if(type(scanned_data) is not type(None)):
decoded_data = scanned_data.decode("utf_8")
decoded_data = decoded_data.replace("\r", "")
print("\nScanned: " + decoded_data)
# Get the user token.
if(decoded_data.find("user%") >= 0):
self.user_token = decoded_data.split("%")[1]
print("\nYour cart registered successfully!")
print("Registered Token: " + self.user_token)
self.change_layout("register")
sleep(10)
self.change_layout("scan")
# After getting the finished command, discard the registered user token to deactivate the cart.
elif(decoded_data.find("finished%") >= 0):
given_token = decoded_data.split("%")[1]
if(given_token == self.user_token):
print("\nPayment Received Successfully!")
print("Your cart discarded successfully!")
self.change_layout("payment")
sleep(2)
# Notify the customer via the thermal printer.
self.print_status(True)
self.change_layout("home")
self.user_token = ""
# Get the product barcode.
else:
if(self.user_token != ""):
self.new_product_barcode = decoded_data
print("New Product Barcode: " + self.new_product_barcode)
self.product_menu_activate = True
if(self.product_menu_activate == True):
self.change_layout("barcode")
while(self.product_menu_activate == True):
self.product_menu()
else:
print("\nPlease scan your unique account QR code to register your cart.")
sleep(1)
⭐ In the make_get_request function:
⭐ Make an HTTP GET request to the e-commerce web application by utilizing the registered user token, the given barcode, and the selected command by the customer.
http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table=kutlu123_!t3*b1450zKD&barcode=80135876&com=add
http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table=kutlu123_!t3*b1450zKD&barcode=80135876&com=remove
⭐ Then, print the response of the web application.
def make_get_request(self, barcode, com="add"):
path = "http://192.168.1.22/AIoT_Shopping_Assistant/assets/barcode.php?table={}&barcode={}&com={}".format(self.user_token, barcode, com)
# Make an HTTP GET request.
request = urequests.get(path)
print("\nURL => "+path)
print("App Response => ")
print(request.text)
sleep(1)
⭐ In the product_menu function:
⭐ If the user presses the control button (A), make an HTTP GET request to the web application by adding the add command.
⭐ If the user presses the control button (B), make an HTTP GET request to the web application by adding the remove command.
⭐ After making the request, close the product menu (interface).
def product_menu(self):
print("Press: Button (A) -> Add | Button (B) -> Remove")
sleep(1)
if(self.button_A.value() == False):
self.make_get_request(self.new_product_barcode, "add")
self.change_layout("add")
sleep(10)
self.change_layout("scan")
self.product_menu_activate = False
if(self.button_B.value() == False):
self.make_get_request(self.new_product_barcode, "remove")
self.change_layout("remove")
sleep(10)
self.change_layout("scan")
self.product_menu_activate = False
⭐ In the print_status function:
⭐ If the user presses the control button (C), inform the user of the current device status by printing a receipt via the tiny (embedded) thermal printer.
⭐ Change the thermal printer hardware settings (heat time, heat dots, heat interval) to obtain smoother prints depending on the targeted print feature (image, text, inverted, etc.).
⭐ Also, if the customer places an order successfully and scans the payment confirmation QR code, notify the user by printing an invoice via the thermal printer.
def print_status(self, payment=False):
if(self.button_C.value() == False):
print("\nPrinting the device status...")
# Change the thermal printer hardware settings to obtain smoother prints depending on the targeted feature.
if(self.user_token == ""):
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_chatgpt.bmp')
self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
self.printer.justify('C')
self.printer.setSize('L')
self.printer.println("AIoT")
self.printer.println("Shopping")
self.printer.println("Assistant")
self.printer.println("Status\n\n")
self.printer.justify('L')
self.printer.setSize('S')
self.printer.boldOn()
self.printer.println("Please scan")
self.printer.println("your unique")
self.printer.println("account QR code")
self.printer.println("to activate your")
self.printer.println("cart and start")
self.printer.println("shopping!\n\n")
self.printer.boldOff()
self.printer.justify('R')
self.printer.setSize('M')
self.printer.inverseOn()
self.printer.println(" Have a ")
self.printer.println(" Great ")
self.printer.println(" Day :) \n")
self.printer.inverseOff()
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_scan.bmp')
self.printer.feed(5)
elif(self.user_token != ""):
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_chatgpt.bmp')
self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
self.printer.justify('C')
self.printer.setSize('L')
self.printer.println("AIoT")
self.printer.println("Shopping")
self.printer.println("Assistant")
self.printer.println("Status\n\n")
self.printer.justify('L')
self.printer.setSize('S')
self.printer.boldOn()
self.printer.println("Your cart is")
self.printer.println("registered and")
self.printer.println("initialized")
self.printer.println("successfully.")
self.printer.println("Please scan a")
self.printer.println("product barcode")
self.printer.println("to change")
self.printer.println("the cart")
self.printer.println("items!\n\n")
self.printer.println("Registered Token:")
self.printer.doubleHeightOn()
self.printer.println(self.user_token)
self.printer.println("\n")
self.printer.doubleHeightOff()
self.printer.boldOff()
self.printer.justify('R')
self.printer.setSize('M')
self.printer.inverseOn()
self.printer.println(" Have a ")
self.printer.println(" Great ")
self.printer.println(" Day :) \n")
self.printer.inverseOff()
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_cart.bmp')
self.printer.feed(5)
# If the customer places an order successfully:
if(payment == True):
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_chatgpt.bmp')
self.printer = Adafruit_Thermal(bus=6, heattime=120, heatdots=5, heatinterval=40)
self.printer.justify('C')
self.printer.setSize('L')
self.printer.println("AIoT")
self.printer.println("Shopping")
self.printer.println("Assistant")
self.printer.println("Status\n\n")
self.printer.justify('L')
self.printer.setSize('S')
self.printer.boldOn()
self.printer.println("Payment for")
self.printer.println("your latest")
self.printer.println("order is received")
self.printer.println("successfully via")
self.printer.println("your dashboard")
self.printer.println("on the web")
self.printer.println("application.")
self.printer.println("Please scan")
self.printer.println("your account")
self.printer.println("QR code to")
self.printer.println("activate your")
self.printer.println("cart for")
self.printer.println("your next")
self.printer.println("order!\n\n")
self.printer.println("Removed Token:")
self.printer.doubleHeightOn()
self.printer.println(self.user_token)
self.printer.println("\n")
self.printer.doubleHeightOff()
self.printer.boldOff()
self.printer.justify('R')
self.printer.setSize('M')
self.printer.inverseOn()
self.printer.println(" Have a ")
self.printer.println(" Great ")
self.printer.println(" Day :) \n")
self.printer.inverseOff()
self.printer = Adafruit_Thermal(bus=6, heattime=155, heatdots=1, heatinterval=1)
self.printer.printBMPImage('assets/printer_payment.bmp')
self.printer.feed(5)
⭐ In the change_layout function:
⭐ Change the layout on the SSD1309 OLED transparent display depending on the ongoing operation.
⭐ Display notifications with the assigned monochrome images.
⭐ Adjust the RGB LED color for each layout differently.
def change_layout(self, activated):
self.display.clear_buffers()
if(activated == "home"):
self.adjust_color([0,1,0])
self.display.draw_bitmap("assets/cart.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Please scan", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "your unique", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "account QR ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "code to ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "activate ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "your cart! ", self.light, invert=True)
elif(activated == "scan"):
self.adjust_color([0,1,1])
self.display.draw_bitmap("assets/chatgpt.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Please scan", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "a product ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "barcode to ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "change ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "the cart ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "items! ", self.light, invert=True)
elif(activated == "register"):
self.adjust_color([0,0,1])
self.display.draw_bitmap("assets/registered.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Account QR ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "code ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "registered ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "and your ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "cart ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "is ready! ", self.light, invert=True)
elif(activated == "barcode"):
self.adjust_color([1,0,1])
self.display.draw_bitmap("assets/barcode.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Press: ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), " ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "Button (A) ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "-> Add ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "Button (B) ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "-> Remove ", self.light, invert=True)
elif(activated == "add"):
self.adjust_color([0,0,1])
self.display.draw_bitmap("assets/add.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Given ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "product ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "added to ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "your ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "registered ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "cart! ", self.light, invert=True)
elif(activated == "remove"):
self.adjust_color([1,0,0])
self.display.draw_bitmap("assets/remove.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Given ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "product ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "removed ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "from your ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "registered ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "cart! ", self.light, invert=True)
elif(activated == "payment"):
self.adjust_color([1,1,1])
self.display.draw_bitmap("assets/payment.mono", 5, (self.display.height-48) // 2, 48, 48, invert=True)
self.display.draw_text(48+10, (self.display.height-48) // 2, "Order ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*1), "payment ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*2), "received ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*3), "and your ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*4), "cart ", self.light, invert=True)
self.display.draw_text(48+10, ((self.display.height-48) // 2) + (self.light.height*5), "deactivated", self.light, invert=True)
self.display.present()
sleep(1)
⭐ Define the assistant object of the shopping_assistant class.
⭐ Finally, initiate the AIoT shopping assistant (device).
assistant = shopping_assistant()
while True:
assistant.start()
⭐ Furthermore, I modified the w5300_set_ip function in the wiznet_conf.py file so as to print the Ethernet connection status on the shell.
def w5300_set_ip (self, ip_addr, gw_addr, netmask, dns_svr):
self.nic.ifconfig((ip_addr, gw_addr, netmask, dns_svr))
print("W5300 STATIC IP: ", self.nic.ifconfig())
if(self.nic.isconnected() == False):
print("\nW5300: Ethernet connection problem!")
else:
print("\nW5300: Connected successfully!")
pyb.delay(500)
Providing a Wholesome E-commerce Customer Experience With AI-based Suggestions
After programming W5300 TOE SHIELD attached to NUCLEO-F439ZI in MicroPython and hosting the e-commerce web application on LattePanda 3 Delta, the AIoT shopping assistant is ready to help customers while physical store shopping.
🛍️📩📲 When the customer provides the required information on the sign-up form on the home page, the e-commerce web application generates a unique account (user) token, creates a unique database table for the new user, and adds the provided information to the MariaDB database.
🛍️📩📲 After saving new user information to the MariaDB database, the e-commerce application generates an account verification QR code and sends an HTML email to the customer's registered email address via Brevo's Email API, including the account verification QR code.
🛍️📩📲 Then, the e-commerce application redirects the customer to the user dashboard.
🛍️📩📲 If there is no product in the cart, the e-commerce application informs the customer on the user dashboard and the ChatGPT recommendations page.
🛍️📩📲 The shopping assistant (device) waits for the customer to scan the unique account verification QR code to activate the cart. Also, it turns the RGB LED to blue to inform the user the fact that the cart is not activated yet.
🛍️📩📲 After scanning the account verification QR code, the shopping assistant registers the obtained user token and turns the RGB LED to green.
🛍️📩📲 After activating the cart, the shopping assistant waits for the customer to scan a product barcode and turns the RGB LED to cyan.
🛍️📩📲 When the customer scans a product barcode, the shopping assistant initiates the product menu (interface) to allow the customer to select a command before sending the scanned product barcode to the e-commerce application. Also, it turns the RGB LED to yellow until the customer chooses a command.
🛍️📩📲 After scanning a product barcode, if the user presses the control button (A), the shopping assistant turns the RGB LED to green and makes an HTTP GET request to the e-commerce application by adding the add command.
🛍️📩📲 Then, the e-commerce application obtains the product information of the given barcode by employing the Open Food Facts JSON API and saves the obtained product information to the user's unique database table.
🛍️📩📲 If the product is already added to the cart, the application increments the given product's cart number by 1.
🛍️📩📲 After scanning a product barcode, if the user presses the control button (B), the shopping assistant turns the RGB LED to red and makes an HTTP GET request to the e-commerce application by adding the remove command.
🛍️📩📲 Then, the e-commerce application discards the given product from the user's database table.
🛍️📩📲 According to the received HTTP GET requests from the shopping assistant, the e-commerce application changes the product information in the user's database table.
🛍️📩📲 Every 5 seconds, the e-commerce application updates the HTML table rows consisting of the product information of the current cart in the user's database table and the estimated total cart price to inform the user automatically.
🛍️📩📲 After adding products to the cart, the e-commerce application allows the customer to generate ChatGPT-powered suggestions for each product.
🛍️📩📲 When the customer clicks the Ask ChatGPT button, the e-commerce application makes a cURL call to the OpenAI API to generate AI-based recommendations for the given product.
🛍️📩📲 After decoding the API response, the e-commerce application redirects the customer to the ChatGPT recommendations page consisting of the product questions and the provided ChatGPT suggestions.
📌 ChatGPT Recommendations Page for Nutella
➡️ Questions
- What is the nutritional value of Nutella?
- What should I purchase with Nutella?
- Can you teach me a recipe with Nutella?
- How should I serve Nutella?
- Is there a more affordable and healthy option than Nutella?
➡️ Suggestions
- Nutella is a hazelnut spread that primarily consists of sugar, palm oil, hazelnuts, cocoa powder, and skim milk powder. While it is delicious, it is important to note that Nutella is high in calories and sugar. The exact nutritional values can vary depending on the brand and serving size, but on average, a 2-tablespoon (37g) serving of Nutella contains approximately: - Calories: 200 - Fat: 12 grams (including 4 grams of saturated fat) - Carbohydrates: 21 grams (including 20 grams of sugar) - Protein: 2 grams - Fiber: 1 gram - Sodium: 15 milligrams
- Nutella pairs well with many different foods. Here are some popular options to enjoy with Nutella: - Bread or toast: Spread Nutella on a slice of bread or toast for a quick and satisfying snack. - Pancakes or waffles: Use Nutella as a topping for pancakes or waffles, along with some fresh fruits or whipped cream. - Fruit: Dip slices of fruits like strawberries, bananas, or apples into Nutella for a sweet and fruity treat. - Pretzels or crackers: Spread Nutella on pretzels or crackers for a salty-sweet combination. - Ice cream: Drizzle Nutella over a scoop of vanilla ice cream for a decadent dessert.
- Certainly! Here's a simple recipe for Nutella Mug Cake: Ingredients: - 4 tablespoons all-purpose flour - 4 tablespoons Nutella - 3 tablespoons milk - 1/4 teaspoon baking powder - Optional toppings: whipped cream, chopped nuts, or chocolate chips Instructions: 1. In a microwave-safe mug, whisk together the Nutella and milk until smooth. 2. Add the flour and baking powder to the mug and mix until well combined. 3. Microwave the mug on high for approximately 1 minute and 30 seconds, or until the cake has risen and is cooked through. 4. Let the mug cake cool for a few minutes and then add your desired toppings. 5. Enjoy the warm and gooey Nutella mug cake!
- Nutella can be served in various ways. Here are a few suggestions: - Spread it on toast, bagels, or croissants for a quick and delicious breakfast. - Use it as a filling for crepes, pastries, or donuts. - Pair it with fruits like strawberries, bananas, or apples for a sweet and indulgent dip. - Incorporate it into baked goods like cookies, brownies, or cakes for added richness and flavor. - Enjoy it straight from the jar with a spoon (though moderation is key!).
- If you're looking for a more affordable and healthier alternative to Nutella, you can consider making your own hazelnut spread at home. Here's a simple recipe: Ingredients: - 1 cup hazelnuts (roasted and peeled) - 3 tablespoons cocoa powder - 2-3 tablespoons honey or maple syrup (adjust to taste) - 1/4 teaspoon vanilla extract - 1-2 tablespoons vegetable oil (for desired consistency) Instructions: 1. In a food processor, blend the hazelnuts until they turn into a smooth paste. 2. Add the cocoa powder, honey or maple syrup, and vanilla extract to the hazelnut paste and blend until well combined. 3. Gradually add vegetable oil while blending until you achieve the desired consistency. 4. Transfer the homemade hazelnut spread to a jar or container and store in the refrigerator. This homemade version allows you to control the ingredients and adjust the sweetness to your preference, making it a more affordable and potentially healthier option compared to store-bought Nutella.
🛍️📩📲 When the customer clicks the Checkout button, the e-commerce application redirects the user to the checkout page and transfers the evaluated total cart price (amount).
🛍️📩📲 Then, if the customer provides the required credit/debit card information correctly, the e-commerce application places the order, generates a unique payment confirmation QR code, updates the order number in the database, and redirects the customer to the user dashboard.
🛍️📩📲 Since the order number is updated, the e-commerce application is ready to add products to the cart again for the next order.
🛍️📩📲 Then, the application sends an HTML email to the customer's registered email address, including the payment confirmation QR code.
🛍️📩📲 When the customer scans the payment confirmation QR code, the shopping assistant confirms the payment, deactivates the cart, turns the RGB LED to white, and prints an invoice via the tiny thermal printer to inform the user.
🛍️📩📲 When the customer clicks the Previous Orders button, the e-commerce application redirects the customer to the previous orders page. The application is capable of displaying all previous orders as separate HTML lists since it saves product information with the corresponding order number.
🛍️📩📲 If there are no previous orders, the application notifies the customer on the previous orders page.
🛍️📩📲 If the customer clicks the Logout button, the e-commerce application signs out of the user account and redirects the customer to the home page.
🛍️📩📲 On the home page, the e-commerce application redirects the user to the sign-in page when the customer clicks Already have an account?
🛍️📩📲 If the user provides accurate account information on the sign-in form, the e-commerce application signs into an existing account and redirects the customer to the user dashboard.
🛍️📩📲 While the shopping assistant operates, NUCLEO-F439ZI prints notifications, the Ethernet connection status, the scanned information, and the device status on the shell for debugging.
🛍️📩📲 Furthermore, if the customer presses the control button (C), the shopping assistant prints a receipt via the thermal printer to inform the user of the current device status.
As mentioned earlier, I added a small gift for a first-time customer trying the shopping assistant. I printed a Togepi figure with my 3D printer as the gift since it represents good luck and fortune :)
Videos and Conclusion
Further Discussions
By applying budget-friendly shopping assistants providing AI-based product recommendations, we can achieve to:
🛍️📩📲 bridge the gap between e-commerce and physical store shopping,
🛍️📩📲 enrich the customer experience,
🛍️📩📲 increase the profit margin,
🛍️📩📲 improve local businesses' marketing strategies,
🛍️📩📲 reach the target audience effortlessly.