Gesture Control Mouse

by Shahbaz Hashmi Ansari in Circuits > Gadgets

328 Views, 1 Favorites, 0 Comments

Gesture Control Mouse

Greetings everyone, and welcome to my Instructables tutorial. Today, I'll guide you through the process of creating an Gesture Control Mouse.

Project Overview:

In this project, a gesture-controlled mouse is developed using a Node MCU, MPU 6050, and a flex/bend sensor. The MPU 6050, leveraging its gyro and accelerometer, moves the mouse pointer in all directions, while the bend sensor triggers a right-click when bent. Initially prototyped on a breadboard, the system is tested in scenarios like playing Asphalt 8 game and scrolling the roboattic Lab website. This innovative, sensor-based design is lightweight and runs efficiently on low-spec PCs, unlike camera-based gesture systems that require heavy computational resources, making it an ideal, accessible choice for college or school projects.

Before beginning a huge shoutout to JLCMC for sponsoring.

Now, let's get started with our project!

Supplies

Electronic Components Required:

Node MCU
MPU 6050
47 KΩ
Flex/Bend Sensor
Breadboard
Dotted PCB Board
18650 Battery
18650 Battery Holder

Additional Tools:

Soldering Iron
Hot Glue
Cutter

Softwares:

Arduino IDE
VS Code

Breadboard Assembly

What You Need to Do:

First, gently pop the Node MCU and MPU 6050 onto the breadboard—don't worry, they fit in nicely!
Next, connect the VCC and GND pins of the MPU 6050 to the 3.3V and ground pins on your Node MCU respectively.
Then, hook up the communication lines: connect SDA to the Node MCU's D2 pin and SCL to its D1 pin. These connections are super important because they let the components “talk” to each other.

A Quick Tip:

Make sure you double-check every wire connection. Trust me, a loose wire can become a real headache later on!

Installing Required Libraries

To ensure smooth execution of your Python and Arduino code, install the necessary libraries as outlined below.

Python Libraries:

For Python, create a `requirements.txt` file with the following contents:

requests

numpy

pyautogui

Then, install all required packages using the following command in your terminal or command prompt:

pip install -r requirements.txt

Arduino Libraries:

For your .ino (Arduino) code, you need to install the following libraries:

1. ESP8266WiFi

This library allows the ESP8266 module to connect to Wi-Fi.

Installation Steps:

Open Arduino IDE.
Go to File > Preferences.
In the "Additional Board Manager URLs" field, add:

http://arduino.esp8266.com/stable/package_esp8266com_index.json

Go to Tools > Board > Boards Manager.
Search for ESP8266 and install the package ESP8266 by ESP8266 Community.
This will also install the ESP8266WiFi library automatically.

2. Adafruit_MPU6050

Supports the MPU6050 accelerometer and gyroscope sensor.

Installation Steps:

Open Arduino IDE.
Go to Sketch > Include Library > Manage Libraries.
In the Library Manager, search for Adafruit MPU6050.
Click Install.
For more details, visit the Adafruit MPU6050 GitHub repository.

3. Adafruit_Sensor

Provides a unified sensor interface.

Installation Steps:

Open Arduino IDE.
Go to Sketch > Include Library > Manage Libraries.
Search for Adafruit Unified Sensor.
Click Install.

Basic Code to Fetch API

Okay, now it’s time to wake everything up with some basic code! Here’s what you do:

What You Need to Do:

Open up your Arduino IDE and upload the basic code to your Node MCU.

#include <ESP8266WiFi.h>

#include <ESP8266WebServer.h>

#include <Wire.h>

#include <Adafruit_MPU6050.h>

#include <Adafruit_Sensor.h>

// WiFi credentials

const char* ssid = "YourWiFiName"; // Replace with your WiFi network name

const char* password = "YourPassword"; // Replace with your WiFi password

ESP8266WebServer server(80);

Adafruit_MPU6050 mpu;

void setup() {

Serial.begin(115200);

Wire.begin(D2, D1); // SDA=D2, SCL=D1

// Initialize MPU6050

if (!mpu.begin()) {

Serial.println("Failed to find MPU6050 chip");

while (1) {

delay(10);

}

// Configure sensor settings

mpu.setAccelerometerRange(MPU6050_RANGE_8_G);

mpu.setGyroRange(MPU6050_RANGE_500_DEG);

mpu.setFilterBandwidth(MPU6050_BAND_21_HZ);

// Connect to WiFi

WiFi.begin(ssid, password);

Serial.print("Connecting to WiFi");

while (WiFi.status() != WL_CONNECTED) {

delay(500);

Serial.print(".");

}

Serial.println("");

Serial.print("Connected to ");

Serial.println(ssid);

Serial.print("IP address: ");

Serial.println(WiFi.localIP());

// Define server routes

server.on("/", handleRoot);

server.on("/data", handleData);

// Start server

server.begin();

Serial.println("HTTP server started");

}

void loop() {

server.handleClient();

}

void handleRoot() {

String html = "<!DOCTYPE html>\n";

html += "<html>\n";

html += "<head>\n";

html += "<title>MPU6050 Sensor Data</title>\n";

html += "</head>\n";

html += "<body>\n";

html += "<h1>MPU6050 Sensor Web Server</h1>\n";

html += "<p>Use the /data endpoint to get sensor readings in JSON format</p>\n";

html += "<p>Server IP: " + WiFi.localIP().toString() + "</p>\n";

html += "</body>\n";

html += "</html>\n";

server.send(200, "text/html", html);

}

void handleData() {

sensors_event_t a, g, temp;

mpu.getEvent(&a, &g, &temp);

// Create JSON response

String json = "{";

json += "\"accelerometer\": {";

json += "\"x\": " + String(a.acceleration.x) + ", ";

json += "\"y\": " + String(a.acceleration.y) + ", ";

json += "\"z\": " + String(a.acceleration.z);

json += "}, ";

json += "\"gyroscope\": {";

json += "\"x\": " + String(g.gyro.x) + ", ";

json += "\"y\": " + String(g.gyro.y) + ", ";

json += "\"z\": " + String(g.gyro.z);

json += "}, ";

json += "\"temperature\": " + String(temp.temperature);

json += "}";

server.send(200, "application/json", json);

}

Once the code is uploaded, check the serial monitor. You should see a local server IP appear. That’s your magic number!
Open up your VS Code and copy & paste this code:

import requests

import json

import time

from colorama import Fore, Style, init

init()

def fetch_data(ip_address):

try:

response = requests.get(f"http://{ip_address}/data", timeout=2)

if response.status_code == 200:

return json.loads(response.text)

else:

print(f"{Fore.RED}Error: Received status code {response.status_code}{Style.RESET_ALL}")

return None

except requests.exceptions.RequestException as e:

print(f"{Fore.RED}Error connecting to server: {e}{Style.RESET_ALL}")

return None

def print_data(data):

if data is None:

return

print("\033c", end="") # Clear the terminal screen

print(f"{Fore.CYAN}===== MPU6050 Sensor Data ====={Style.RESET_ALL}")

print(f"{Fore.GREEN}Acceleration (m/s²):{Style.RESET_ALL}")

print(f" X: {data['accel_x']:.4f}")

print(f" Y: {data['accel_y']:.4f}")

print(f" Z: {data['accel_z']:.4f}")

print(f"{Fore.GREEN}Gyroscope (rad/s):{Style.RESET_ALL}")

print(f" X: {data['gyro_x']:.4f}")

print(f" Y: {data['gyro_y']:.4f}")

print(f" Z: {data['gyro_z']:.4f}")

print(f"{Fore.YELLOW}Temperature: {data['temp']:.2f}°C{Style.RESET_ALL}")

print(f"{Fore.CYAN}============================={Style.RESET_ALL}")

def main():

# Ask for the NodeMCU IP address

ip_address = input("Enter the NodeMCU IP address: ")

print(f"Connecting to NodeMCU at {ip_address}...")

print("Press Ctrl+C to stop")

try:

while True:

data = fetch_data(ip_address)

print_data(data)

time.sleep(0.1)

except KeyboardInterrupt:

print("\nExiting program")

if __name__ == "__main__":

main()

Now, take that IP address and paste it into your Python code in VS Code. Hit the run button, and boom—you should see the MPU 6050’s gyro and accelerometer data streaming in your terminal.

Elevate Your Electronic Projects - JLCMC

JLCMC is your one-stop shop for all electronic manufacturing needs, offering an extensive catalog of nearly 600,000 SKUs that cover hardware, mechanical, electronic, and automation components. Their commitment to guaranteeing genuine products, rapid shipping (with most in-stock items dispatched within 24 hours), and competitive pricing truly sets them apart. In addition, their exceptional customer service ensures you always get exactly what you need to bring your projects to life.

To show their support for our community, JLCMC is offering an exclusive $19 discount coupon. This is the perfect opportunity to save on high-quality components for your next project. Don’t miss out—visit https://jlcmc.com/?from=RBL to explore their amazing range of products and grab your discount coupon today!

Let’s Make the Mouse Cursor Move

Alright, now for the really fun part—moving the cursor with your hand! Here’s what you need to do:

First, update your Node MCU code and upload it.

#include <ESP8266WiFi.h>

#include <ESP8266WebServer.h>

#include <Wire.h>

#include <Adafruit_MPU6050.h>

#include <Adafruit_Sensor.h>

// WiFi credentials

const char* ssid = "YourWiFiName"; // Replace with your WiFi network name

const char* password = "YourPassword"; // Replace with your WiFi password

// Use static IP to prevent IP address changes

IPAddress staticIP(192, 168, 31, 212); // The IP you want to assign to NodeMCU

IPAddress gateway(192, 168, 31, 1); // Your router's IP address (typical gateway)

IPAddress subnet(255, 255, 255, 0); // Subnet mask

IPAddress dns(8, 8, 8, 8); // DNS (Google's DNS)

ESP8266WebServer server(80);

Adafruit_MPU6050 mpu;

void setup() {

Serial.begin(115200);

delay(100); // Short delay for serial to initialize

Serial.println("\n\nMPU6050 Web Server Starting...");

// Initialize I2C for MPU6050

Wire.begin(D2, D1); // SDA=D2, SCL=D1

// Initialize MPU6050

if (!mpu.begin()) {

Serial.println("Failed to find MPU6050 chip");

Serial.println("Please check your wiring!");

while (1) {

delay(10);

}

Serial.println("MPU6050 Found!");

// Configure sensor settings

mpu.setAccelerometerRange(MPU6050_RANGE_8_G);

mpu.setGyroRange(MPU6050_RANGE_500_DEG);

mpu.setFilterBandwidth(MPU6050_BAND_21_HZ);

// Configure WiFi in station mode with static IP

WiFi.mode(WIFI_STA);

WiFi.config(staticIP, gateway, subnet, dns); // Set static IP

WiFi.begin(ssid, password);

Serial.println("Connecting to WiFi...");

// Wait for connection with timeout

int timeout = 0;

while (WiFi.status() != WL_CONNECTED && timeout < 20) {

delay(500);

Serial.print(".");

timeout++;

}

if (WiFi.status() != WL_CONNECTED) {

Serial.println("\nFailed to connect to WiFi! Check credentials and try again.");

while(1) {

delay(1000);

}

Serial.println("");

Serial.print("Connected to WiFi network: ");

Serial.println(ssid);

Serial.print("IP address: ");

Serial.println(WiFi.localIP());

// Define server routes

server.on("/", HTTP_GET, handleRoot);

server.on("/data", HTTP_GET, handleData);

server.onNotFound(handleNotFound);

// Start server

server.begin();

Serial.println("HTTP server started");

Serial.println("Web interface available at http://" + WiFi.localIP().toString());

Serial.println("Data endpoint available at http://" + WiFi.localIP().toString() + "/data");

}

void loop() {

server.handleClient();

delay(2); // Small delay to improve stability

}

void handleRoot() {

String html = "<!DOCTYPE html>\n";

html += "<html>\n";

html += "<head>\n";

html += "<title>MPU6050 Sensor Data</title>\n";

html += "<meta http-equiv='refresh' content='5'>\n"; // Auto-refresh page every 5 seconds

html += "<style>\n";

html += "body { font-family: Arial, sans-serif; margin: 20px; }\n";

html += "h1 { color: #0066cc; }\n";

html += "p { margin: 10px 0; }\n";

html += ".data { font-family: monospace; background-color: #f0f0f0; padding: 10px; border-radius: 5px; }\n";

html += "</style>\n";

html += "</head>\n";

html += "<body>\n";

html += "<h1>MPU6050 Sensor Web Server</h1>\n";

html += "<p>Server is running successfully!</p>\n";

html += "<p>Server IP: <strong>" + WiFi.localIP().toString() + "</strong></p>\n";

html += "<p>Get raw data at: <a href='/data'>/data</a> (JSON format)</p>\n";

// Get current sensor data

sensors_event_t a, g, temp;

mpu.getEvent(&a, &g, &temp);

html += "<div class='data'>\n";

html += "<h2>Current Sensor Data:</h2>\n";

html += "<p>Accelerometer (m/s²):<br>\n";

html += "X: " + String(a.acceleration.x) + "<br>\n";

html += "Y: " + String(a.acceleration.y) + "<br>\n";

html += "Z: " + String(a.acceleration.z) + "</p>\n";

html += "<p>Gyroscope (rad/s):<br>\n";

html += "X: " + String(g.gyro.x) + "<br>\n";

html += "Y: " + String(g.gyro.y) + "<br>\n";

html += "Z: " + String(g.gyro.z) + "</p>\n";

html += "<p>Temperature: " + String(temp.temperature) + " °C</p>\n";

html += "</div>\n";

html += "</body>\n";

html += "</html>\n";

server.send(200, "text/html", html);

}

void handleData() {

sensors_event_t a, g, temp;

mpu.getEvent(&a, &g, &temp);

// Enable CORS to allow requests from any origin

server.sendHeader("Access-Control-Allow-Origin", "*");

server.sendHeader("Access-Control-Allow-Methods", "GET");

server.sendHeader("Access-Control-Max-Age", "10000");

server.sendHeader("Access-Control-Allow-Headers", "Content-Type");

// Create JSON response

String json = "{";

json += "\"accelerometer\": {";

json += "\"x\": " + String(a.acceleration.x) + ", ";

json += "\"y\": " + String(a.acceleration.y) + ", ";

json += "\"z\": " + String(a.acceleration.z);

json += "}, ";

json += "\"gyroscope\": {";

json += "\"x\": " + String(g.gyro.x) + ", ";

json += "\"y\": " + String(g.gyro.y) + ", ";

json += "\"z\": " + String(g.gyro.z);

json += "}, ";

json += "\"temperature\": " + String(temp.temperature);

json += "}";

server.send(200, "application/json", json);

}

void handleNotFound() {

String message = "File Not Found\n\n";

message += "URI: ";

message += server.uri();

message += "\nMethod: ";

message += (server.method() == HTTP_GET) ? "GET" : "POST";

message += "\nArguments: ";

message += server.args();

message += "\n";

for (uint8_t i = 0; i < server.args(); i++) {

message += " " + server.argName(i) + ": " + server.arg(i) + "\n";

}

server.send(404, "text/plain", message);

}

Open up your VS Code and copy & paste this code. We’re using pyautogui here to map the MPU 6050’s gyro data directly to your cursor movement. Basically, when you tilt the sensor left, the cursor moves left, and when you tilt right, it follows along.

import requests

import json

import time

import numpy as np

import pyautogui

import math

from collections import deque

import threading

import tkinter as tk

from tkinter import ttk

# Configuration

NODEMCU_IP = '192.168.XX.XXX' # Your NodeMCU IP

DATA_URL = f'http://{NODEMCU_IP}/data'

CONNECTION_TIMEOUT = 3 # Increase timeout to 3 seconds

MAX_RETRIES = 5 # Maximum number of retries on connection failure

# Settings for mouse control

MOUSE_SENSITIVITY_X = 50.0 # Higher value = less sensitive

MOUSE_SENSITIVITY_Y = 50.0

GYRO_THRESHOLD = 0.05 # Minimum gyro movement to register (eliminates jitter)

SCREEN_WIDTH, SCREEN_HEIGHT = pyautogui.size()

SMOOTHING_WINDOW = 5 # Number of samples for smoothing

# For click detection

CLICK_THRESHOLD = 2.0 # Threshold for click detection (acceleration magnitude)

CLICK_COOLDOWN = 1.0 # Seconds between clicks to avoid accidental double-clicks

# For safety (prevent mouse from going crazy)

pyautogui.FAILSAFE = True # Move mouse to corner to abort

# For UI

UPDATE_INTERVAL = 50 # milliseconds

# Global variables

last_click_time = 0

is_paused = False

is_running = True

show_debug = True

baseline_accel = {"x": 0, "y": 0, "z": 0}

baseline_gyro = {"x": 0, "y": 0, "z": 0}

calibration_samples = 20

mouse_pos = {"x": SCREEN_WIDTH // 2, "y": SCREEN_HEIGHT // 2}

# Smoothing buffers

accel_history = {"x": deque(maxlen=SMOOTHING_WINDOW),

"y": deque(maxlen=SMOOTHING_WINDOW),

"z": deque(maxlen=SMOOTHING_WINDOW)}

gyro_history = {"x": deque(maxlen=SMOOTHING_WINDOW),

"y": deque(maxlen=SMOOTHING_WINDOW),

"z": deque(maxlen=SMOOTHING_WINDOW)}

# Create UI

root = tk.Tk()

root.title("Gesture Mouse Control")

root.geometry("600x400")

root.protocol("WM_DELETE_WINDOW", lambda: set_running(False))

def set_running(state):

global is_running

is_running = state

if not state:

root.destroy()

def toggle_pause():

global is_paused

is_paused = not is_paused

pause_btn.config(text="Resume" if is_paused else "Pause")

status_var.set("PAUSED" if is_paused else "RUNNING")

def toggle_debug():

global show_debug

show_debug = not show_debug

debug_btn.config(text="Hide Debug" if show_debug else "Show Debug")

def start_calibration():

global baseline_accel, baseline_gyro

status_var.set("Calibrating... Don't move the sensor!")

root.update()

# Reset baselines

baseline_accel = {"x": 0, "y": 0, "z": 0}

baseline_gyro = {"x": 0, "y": 0, "z": 0}

# Collect samples

accel_samples = {"x": [], "y": [], "z": []}

gyro_samples = {"x": [], "y": [], "z": []}

for _ in range(calibration_samples):

data = fetch_data()

if data:

for axis in ["x", "y", "z"]:

accel_samples[axis].append(data["accelerometer"][axis])

gyro_samples[axis].append(data["gyroscope"][axis])

time.sleep(0.1)

# Calculate averages

for axis in ["x", "y", "z"]:

if accel_samples[axis]: # Check if we got samples

baseline_accel[axis] = sum(accel_samples[axis]) / len(accel_samples[axis])

baseline_gyro[axis] = sum(gyro_samples[axis]) / len(gyro_samples[axis])

status_var.set("Calibration complete")

sensitivity_x_var.set(MOUSE_SENSITIVITY_X)

sensitivity_y_var.set(MOUSE_SENSITIVITY_Y)

# Reset position to center

global mouse_pos

mouse_pos = {"x": SCREEN_WIDTH // 2, "y": SCREEN_HEIGHT // 2}

pyautogui.moveTo(mouse_pos["x"], mouse_pos["y"])

def update_sensitivity():

global MOUSE_SENSITIVITY_X, MOUSE_SENSITIVITY_Y

try:

MOUSE_SENSITIVITY_X = float(sensitivity_x_var.get())

MOUSE_SENSITIVITY_Y = float(sensitivity_y_var.get())

except ValueError:

pass # Ignore invalid inputs

def fetch_data():

"""Fetch data from NodeMCU with retries"""

for attempt in range(MAX_RETRIES):

try:

response = requests.get(DATA_URL, timeout=CONNECTION_TIMEOUT)

if response.status_code == 200:

status_var.set("Connected")

return json.loads(response.text)

else:

status_var.set(f"Error: Server returned {response.status_code}")

time.sleep(0.5)

except requests.exceptions.RequestException as e:

status_var.set(f"Connection error: {type(e).__name__}")

time.sleep(0.5)

return None

def smooth_data(data, history_buffer):

"""Apply smoothing to sensor data"""

history_buffer.append(data)

return sum(history_buffer) / len(history_buffer)

def detect_click(accel_data):

"""Detect a click gesture based on acceleration"""

global last_click_time

current_time = time.time()

# Calculate acceleration magnitude

accel_mag = math.sqrt(accel_data["x"]**2 + accel_data["y"]**2 + accel_data["z"]**2)

# Check if it exceeds threshold and enough time has passed since last click

if accel_mag > CLICK_THRESHOLD and current_time - last_click_time > CLICK_COOLDOWN:

last_click_time = current_time

return True

return False

def process_sensor_data(data):

"""Process sensor data and control mouse"""

global mouse_pos

if not data or is_paused:

return

# Get accelerometer and gyroscope data

accel = data["accelerometer"]

gyro = data["gyroscope"]

# Apply calibration

calibrated_accel = {

"x": accel["x"] - baseline_accel["x"],

"y": accel["y"] - baseline_accel["y"],

"z": accel["z"] - baseline_accel["z"]

}

calibrated_gyro = {

"x": gyro["x"] - baseline_gyro["x"],

"y": gyro["y"] - baseline_gyro["y"],

"z": gyro["z"] - baseline_gyro["z"]

}

# Apply smoothing

for axis in ["x", "y", "z"]:

calibrated_accel[axis] = smooth_data(calibrated_accel[axis], accel_history[axis])

calibrated_gyro[axis] = smooth_data(calibrated_gyro[axis], gyro_history[axis])

# Calculate mouse movement - using gyro for more precise control

# Invert axes as needed based on sensor orientation

dx = -calibrated_gyro["y"] if abs(calibrated_gyro["y"]) > GYRO_THRESHOLD else 0

dy = calibrated_gyro["x"] if abs(calibrated_gyro["x"]) > GYRO_THRESHOLD else 0

# Scale the movement

dx = dx * (SCREEN_WIDTH / MOUSE_SENSITIVITY_X)

dy = dy * (SCREEN_HEIGHT / MOUSE_SENSITIVITY_Y)

# Update position

mouse_pos["x"] += dx

mouse_pos["y"] += dy

# Constrain to screen boundaries

mouse_pos["x"] = max(0, min(SCREEN_WIDTH - 1, mouse_pos["x"]))

mouse_pos["y"] = max(0, min(SCREEN_HEIGHT - 1, mouse_pos["y"]))

# Move the mouse

pyautogui.moveTo(mouse_pos["x"], mouse_pos["y"])

# Check for click gesture

if detect_click(calibrated_accel):

pyautogui.click()

click_label.config(text="CLICK!")

root.after(500, lambda: click_label.config(text=""))

# Update UI

if show_debug:

accel_label.config(text=f"Accel: X: {calibrated_accel['x']:.2f}, Y: {calibrated_accel['y']:.2f}, Z: {calibrated_accel['z']:.2f}")

gyro_label.config(text=f"Gyro: X: {calibrated_gyro['x']:.2f}, Y: {calibrated_gyro['y']:.2f}, Z: {calibrated_gyro['z']:.2f}")

mouse_label.config(text=f"Mouse: X: {mouse_pos['x']:.0f}, Y: {mouse_pos['y']:.0f}")

def update_ui():

"""Update UI and process data periodically"""

if is_running:

data = fetch_data()

if data:

process_sensor_data(data)

root.after(UPDATE_INTERVAL, update_ui)

# Create UI elements

frame = ttk.Frame(root, padding="10")

frame.pack(fill=tk.BOTH, expand=True)

# Status

status_frame = ttk.LabelFrame(frame, text="Status")

status_frame.pack(fill=tk.X, expand=True, pady=5)

status_var = tk.StringVar(value="Initializing...")

status_label = ttk.Label(status_frame, textvariable=status_var, font=("Arial", 12))

status_label.pack(pady=5)

click_label = ttk.Label(status_frame, text="", font=("Arial", 16, "bold"), foreground="red")

click_label.pack()

# Debug info

debug_frame = ttk.LabelFrame(frame, text="Sensor Data")

debug_frame.pack(fill=tk.X, expand=True, pady=5)

accel_label = ttk.Label(debug_frame, text="Accel: X: 0.00, Y: 0.00, Z: 0.00")

accel_label.pack(anchor=tk.W)

gyro_label = ttk.Label(debug_frame, text="Gyro: X: 0.00, Y: 0.00, Z: 0.00")

gyro_label.pack(anchor=tk.W)

mouse_label = ttk.Label(debug_frame, text="Mouse: X: 0, Y: 0")

mouse_label.pack(anchor=tk.W)

# Settings

settings_frame = ttk.LabelFrame(frame, text="Settings")

settings_frame.pack(fill=tk.X, expand=True, pady=5)

# X sensitivity

sensitivity_x_frame = ttk.Frame(settings_frame)

sensitivity_x_frame.pack(fill=tk.X, expand=True, pady=2)

ttk.Label(sensitivity_x_frame, text="X Sensitivity:").pack(side=tk.LEFT)

sensitivity_x_var = tk.StringVar(value=str(MOUSE_SENSITIVITY_X))

sensitivity_x_entry = ttk.Entry(sensitivity_x_frame, textvariable=sensitivity_x_var, width=8)

sensitivity_x_entry.pack(side=tk.LEFT, padx=5)

# Y sensitivity

sensitivity_y_frame = ttk.Frame(settings_frame)

sensitivity_y_frame.pack(fill=tk.X, expand=True, pady=2)

ttk.Label(sensitivity_y_frame, text="Y Sensitivity:").pack(side=tk.LEFT)

sensitivity_y_var = tk.StringVar(value=str(MOUSE_SENSITIVITY_Y))

sensitivity_y_entry = ttk.Entry(sensitivity_y_frame, textvariable=sensitivity_y_var, width=8)

sensitivity_y_entry.pack(side=tk.LEFT, padx=5)

# Apply button

ttk.Button(settings_frame, text="Apply Settings", command=update_sensitivity).pack(pady=5)

# Buttons

buttons_frame = ttk.Frame(frame)

buttons_frame.pack(fill=tk.X, expand=True, pady=10)

calibrate_btn = ttk.Button(buttons_frame, text="Calibrate", command=start_calibration)

calibrate_btn.pack(side=tk.LEFT, padx=5)

pause_btn = ttk.Button(buttons_frame, text="Pause", command=toggle_pause)

pause_btn.pack(side=tk.LEFT, padx=5)

debug_btn = ttk.Button(buttons_frame, text="Hide Debug", command=toggle_debug)

debug_btn.pack(side=tk.LEFT, padx=5)

ttk.Button(buttons_frame, text="Exit", command=lambda: set_running(False)).pack(side=tk.RIGHT, padx=5)

# Instructions

instructions = """

INSTRUCTIONS:

1. Click 'Calibrate' and keep the sensor still

2. Tilt to move cursor

3. Quick shake for mouse click

4. Adjust sensitivity if needed

"""

ttk.Label(frame, text=instructions, justify=tk.LEFT).pack(anchor=tk.W, pady=5)

# Start the data processing

root.after(1000, update_ui)

if __name__ == "__main__":

# Center mouse position at start

pyautogui.moveTo(SCREEN_WIDTH // 2, SCREEN_HEIGHT // 2)

# Start UI

root.mainloop()

Setting Up the Flex Sensor for Right-Click Action

Alright, let’s get into the fun part—adding the ability to click with our setup! We’ve already got the MPU 6050 and Node MCU working together to move the cursor by tilting the breadboard (pretty cool, right?). Now, we’re bringing in a flex sensor to handle right-clicks.

Here’s what you need to do:

Powering the Flex Sensor:

Connect the positive leg of the flex sensor to the 3.3V pin on the NodeMCU.
This provides the necessary power for the sensor to function.

Connecting the Sensor Output:

Attach the negative leg of the flex sensor to the A0 (analog) pin of the NodeMCU.
Since the flex sensor provides analog data (a range of values depending on its bend), the A0 pin is ideal for reading these variations.

Adding a Resistor for Voltage Division:

Place a 47K Ohm resistor between the negative leg of the flex sensor and GND (ground) of the NodeMCU.
This forms a voltage divider, allowing the NodeMCU to correctly interpret the sensor’s changes.

Uploading the Code:

Flash the provided Arduino code onto the Node MCU to enable it to process the sensor readings and send click signals to the computer. .ino file is attached in this step. Download it and open it in Arduino IDE.

Updating the Python Script:

Modify the Python script with the updated code to ensure proper communication between the NodeMCU and the computer.

import requests

import json

import time

import numpy as np

import pyautogui

import math

from collections import deque

import tkinter as tk

from tkinter import ttk

import threading

import sys

from tkinter import messagebox

NODEMCU_IP = '192.168.31.212' # Replace with your NodeMCU IP

DATA_URL = f'http://{NODEMCU_IP}/data'

CONNECTION_TIMEOUT = 3

MAX_RETRIES = 5

MOUSE_SENSITIVITY_X = 10.0

MOUSE_SENSITIVITY_Y = 10.0

GYRO_THRESHOLD = 0.01 # Minimum gyro movement to register (eliminates jitter)

ACCELERATION_FACTOR = 2 # Exponential acceleration for faster movements

SCREEN_WIDTH, SCREEN_HEIGHT = pyautogui.size()

SMOOTHING_WINDOW = 5 # Number of samples for smoothing

# Movement modes

MODE_ABSOLUTE = "Absolute" # Direct mapping of tilt to screen position

MODE_RELATIVE = "Relative" # Tilt controls velocity, not position

MODE_HYBRID = "Hybrid" # A mix of both with auto-centering

# For safety (prevent mouse from going crazy)

pyautogui.FAILSAFE = True # Move mouse to corner to abort

# For UI

UPDATE_INTERVAL = 25 # milliseconds (faster updates)

# Global variables

is_paused = False

is_running = True

show_debug = True

baseline_accel = {"x": 0, "y": 0, "z": 0}

baseline_gyro = {"x": 0, "y": 0, "z": 0}

calibration_samples = 20

mouse_pos = {"x": pyautogui.position()[0], "y": pyautogui.position()[1]}

movement_mode = MODE_RELATIVE # Start with relative mode by default

# Flex sensor readings

flex_angle = 0

flex_raw = 0

# Debug variables

last_data_time = 0

fps_counter = 0

fps = 0

# Smoothing buffers

accel_history = {"x": deque(maxlen=SMOOTHING_WINDOW),

"y": deque(maxlen=SMOOTHING_WINDOW),

"z": deque(maxlen=SMOOTHING_WINDOW)}

gyro_history = {"x": deque(maxlen=SMOOTHING_WINDOW),

"y": deque(maxlen=SMOOTHING_WINDOW),

"z": deque(maxlen=SMOOTHING_WINDOW)}

# Create UI

root = tk.Tk()

root.title("Gesture Mouse Control")

root.geometry("700x580")

root.protocol("WM_DELETE_WINDOW", lambda: set_running(False))

# Style configuration for a more modern look

style = ttk.Style()

style.configure("TButton", padding=6, relief="flat", background="#ccc")

style.configure("TLabel", padding=2)

style.configure("TFrame", padding=5)

def set_running(state):

global is_running

is_running = state

if not state:

root.destroy()

sys.exit(0)

def toggle_pause():

global is_paused

is_paused = not is_paused

pause_btn.config(text="Resume" if is_paused else "Pause")

status_var.set("PAUSED" if is_paused else "RUNNING")

def toggle_debug():

global show_debug

show_debug = not show_debug

debug_btn.config(text="Hide Debug" if show_debug else "Show Debug")

if show_debug:

debug_frame.pack(fill=tk.X, expand=True, pady=5)

else:

debug_frame.pack_forget()

def change_movement_mode(mode):

global movement_mode

movement_mode = mode

status_var.set(f"Mode: {movement_mode}")

# Update radio buttons

mode_var.set(mode)

# Clear movement history when changing modes

for axis in ["x", "y", "z"]:

accel_history[axis].clear()

gyro_history[axis].clear()

def start_calibration():

global baseline_accel, baseline_gyro

status_var.set("Calibrating... Don't move the sensor!")

root.update()

# Reset baselines

baseline_accel = {"x": 0, "y": 0, "z": 0}

baseline_gyro = {"x": 0, "y": 0, "z": 0}

# Collect samples

accel_samples = {"x": [], "y": [], "z": []}

gyro_samples = {"x": [], "y": [], "z": []}

# Progress bar for calibration

progress = ttk.Progressbar(status_frame, length=200, mode="determinate")

progress.pack(pady=5)

for i in range(calibration_samples):

data = fetch_data()

if data:

for axis in ["x", "y", "z"]:

accel_samples[axis].append(data["accelerometer"][axis])

gyro_samples[axis].append(data["gyroscope"][axis])

# Update progress bar

progress["value"] = (i + 1) / calibration_samples * 100

root.update_idletasks()

time.sleep(0.1)

# Calculate averages

for axis in ["x", "y", "z"]:

if accel_samples[axis]: # Check if we got samples

baseline_accel[axis] = sum(accel_samples[axis]) / len(accel_samples[axis])

baseline_gyro[axis] = sum(gyro_samples[axis]) / len(gyro_samples[axis])

progress.destroy()

status_var.set("Calibration complete")

# Update settings display

sensitivity_x_var.set(MOUSE_SENSITIVITY_X)

sensitivity_y_var.set(MOUSE_SENSITIVITY_Y)

threshold_var.set(GYRO_THRESHOLD)

acceleration_var.set(ACCELERATION_FACTOR)

# Reset position to current mouse position

global mouse_pos

current_x, current_y = pyautogui.position()

mouse_pos = {"x": current_x, "y": current_y}

def update_sensitivity():

global MOUSE_SENSITIVITY_X, MOUSE_SENSITIVITY_Y, GYRO_THRESHOLD, ACCELERATION_FACTOR

try:

MOUSE_SENSITIVITY_X = float(sensitivity_x_var.get())

MOUSE_SENSITIVITY_Y = float(sensitivity_y_var.get())

GYRO_THRESHOLD = float(threshold_var.get())

ACCELERATION_FACTOR = float(acceleration_var.get())

status_var.set("Settings updated")

except ValueError:

messagebox.showerror("Invalid Input", "Please enter valid numbers for sensitivity values")

def fetch_data():

"""Fetch data from NodeMCU with retries"""

global fps_counter, last_data_time, fps

# Calculate FPS (Frames Per Second / Data updates per second)

current_time = time.time()

fps_counter += 1

if current_time - last_data_time >= 1.0:

fps = fps_counter

fps_counter = 0

last_data_time = current_time

for attempt in range(MAX_RETRIES):

try:

response = requests.get(DATA_URL, timeout=CONNECTION_TIMEOUT)

if response.status_code == 200:

connection_status_var.set(f"Connected ({fps} FPS)")

return json.loads(response.text)

else:

connection_status_var.set(f"Error: HTTP {response.status_code}")

time.sleep(0.5)

except requests.exceptions.RequestException as e:

connection_status_var.set(f"Connection error: {type(e).__name__}")

time.sleep(0.5)

return None

def smooth_data(data, history_buffer):

"""Apply smoothing to sensor data"""

history_buffer.append(data)

return sum(history_buffer) / len(history_buffer)

def apply_acceleration(value, threshold):

"""Apply non-linear acceleration to movements for better control"""

if abs(value) <= threshold:

return 0 # Filter out minor movements

# The further from threshold, the more acceleration applies

sign = 1 if value > 0 else -1

normalized = abs(value) - threshold

return sign * (normalized ** ACCELERATION_FACTOR)

def process_sensor_data(data):

"""Process sensor data and control mouse"""

global mouse_pos, flex_angle, flex_raw

if not data or is_paused:

return

# Get accelerometer and gyroscope data

accel = data["accelerometer"]

gyro = data["gyroscope"]

# Get flex sensor data if available

if "flex" in data:

flex_angle = data["flex"]["angle"]

flex_raw = data["flex"].get("raw", 0)

# Check if click is detected from NodeMCU

if data["flex"]["click"]:

pyautogui.click()

click_label.config(text="CLICK!")

root.after(500, lambda: click_label.config(text=""))

# Apply calibration

calibrated_accel = {

"x": accel["x"] - baseline_accel["x"],

"y": accel["y"] - baseline_accel["y"],

"z": accel["z"] - baseline_accel["z"]

}

calibrated_gyro = {

"x": gyro["x"] - baseline_gyro["x"],

"y": gyro["y"] - baseline_gyro["y"],

"z": gyro["z"] - baseline_gyro["z"]

}

# Apply smoothing to reduce jitter

for axis in ["x", "y", "z"]:

calibrated_accel[axis] = smooth_data(calibrated_accel[axis], accel_history[axis])

calibrated_gyro[axis] = smooth_data(calibrated_gyro[axis], gyro_history[axis])

# Calculate mouse movement based on the selected mode

dx, dy = 0, 0

if movement_mode == MODE_ABSOLUTE:

# Absolute mode: Map tilt angle directly to screen position

# This is similar to your original code

dx = -calibrated_gyro["y"] if abs(calibrated_gyro["y"]) > GYRO_THRESHOLD else 0

dy = calibrated_gyro["x"] if abs(calibrated_gyro["x"]) > GYRO_THRESHOLD else 0

# Scale the movement

dx = dx * (SCREEN_WIDTH / MOUSE_SENSITIVITY_X)

dy = dy * (SCREEN_HEIGHT / MOUSE_SENSITIVITY_Y)

# Update position

mouse_pos["x"] += dx

mouse_pos["y"] += dy

# When MODE_RELATIVE is selected, replace the existing relative mode implementation with this:

elif movement_mode == MODE_RELATIVE:

# Get current gyroscope values

gyro_x = calibrated_gyro["x"]

gyro_y = calibrated_gyro["y"]

# Initialize movement deltas

dx = 0

dy = 0

# Only process movement if above threshold to eliminate drift

# This is key to keeping cursor in place when sensor is in neutral position

if abs(gyro_y) > GYRO_THRESHOLD:

# Invert Y axis for natural movement direction

dx = -gyro_y

if abs(gyro_x) > GYRO_THRESHOLD:

dy = gyro_x

# Apply the requested amplification factor (200.0) to make movements more responsive

# This makes small tilts translate to meaningful cursor movement

amplification_factor = 200.0

# Apply amplification and sensitivity adjustments

if abs(dx) > 0:

dx_sign = 1 if dx > 0 else -1

dx = dx_sign * ((abs(dx) * amplification_factor) / MOUSE_SENSITIVITY_X)

if abs(dy) > 0:

dy_sign = 1 if dy > 0 else -1

dy = dy_sign * ((abs(dy) * amplification_factor) / MOUSE_SENSITIVITY_Y)

# Apply exponential scaling for larger movements

# This gives finer control for small movements while allowing fast traversal with larger tilts

if abs(dx) > 1.0:

dx = dx * (abs(dx) ** (ACCELERATION_FACTOR - 1.0))

if abs(dy) > 1.0:

dy = dy * (abs(dy) ** (ACCELERATION_FACTOR - 1.0))

# Get current actual mouse position

current_x, current_y = pyautogui.position()

# Update position with calculated movement

new_x = current_x + dx

new_y = current_y + dy

# Update stored mouse position

mouse_pos["x"] = new_x

mouse_pos["y"] = new_y

else:

# Normal movement when tilted

dx = dx * (12.0 / MOUSE_SENSITIVITY_X)

dy = dy * (12.0 / MOUSE_SENSITIVITY_Y)

current_x, current_y = pyautogui.position()

mouse_pos["x"] = current_x + dx

mouse_pos["y"] = current_y + dy

# Constrain to screen boundaries

mouse_pos["x"] = max(0, min(SCREEN_WIDTH - 1, mouse_pos["x"]))

mouse_pos["y"] = max(0, min(SCREEN_HEIGHT - 1, mouse_pos["y"]))

# Move the mouse

pyautogui.moveTo(mouse_pos["x"], mouse_pos["y"])

# Update UI

if show_debug:

accel_label.config(text=f"Accel: X: {calibrated_accel['x']:.2f}, Y: {calibrated_accel['y']:.2f}, Z: {calibrated_accel['z']:.2f}")

gyro_label.config(text=f"Gyro: X: {calibrated_gyro['x']:.2f}, Y: {calibrated_gyro['y']:.2f}, Z: {calibrated_gyro['z']:.2f}")

mouse_label.config(text=f"Mouse: X: {mouse_pos['x']:.0f}, Y: {mouse_pos['y']:.0f}, Mode: {movement_mode}")

flex_label.config(text=f"Flex Sensor: Angle: {flex_angle:.1f}° (Raw: {flex_raw})")

def update_ui():

"""Update UI and process data periodically"""

if is_running:

try:

data = fetch_data()

if data:

process_sensor_data(data)

except Exception as e:

error_msg = str(e)

if len(error_msg) > 50:

error_msg = error_msg[:50] + "..."

status_var.set(f"Error: {error_msg}")

finally:

root.after(UPDATE_INTERVAL, update_ui)

# Create UI elements

main_frame = ttk.Frame(root, padding="10")

main_frame.pack(fill=tk.BOTH, expand=True)

# Status frame

status_frame = ttk.LabelFrame(main_frame, text="Status")

status_frame.pack(fill=tk.X, expand=True, pady=5)

status_var = tk.StringVar(value=f"Mode: {movement_mode}")

status_label = ttk.Label(status_frame, textvariable=status_var, font=("Arial", 12, "bold"))

status_label.pack(pady=5)

connection_status_var = tk.StringVar(value="Initializing...")

connection_status_label = ttk.Label(status_frame, textvariable=connection_status_var)

connection_status_label.pack(pady=2)

click_label = ttk.Label(status_frame, text="", font=("Arial", 16, "bold"), foreground="red")

click_label.pack()

# Debug info

debug_frame = ttk.LabelFrame(main_frame, text="Sensor Data")

if show_debug:

debug_frame.pack(fill=tk.X, expand=True, pady=5)

accel_label = ttk.Label(debug_frame, text="Accel: X: 0.00, Y: 0.00, Z: 0.00")

accel_label.pack(anchor=tk.W)

gyro_label = ttk.Label(debug_frame, text="Gyro: X: 0.00, Y: 0.00, Z: 0.00")

gyro_label.pack(anchor=tk.W)

mouse_label = ttk.Label(debug_frame, text="Mouse: X: 0, Y: 0")

mouse_label.pack(anchor=tk.W)

flex_label = ttk.Label(debug_frame, text="Flex Sensor: Angle: 0.00° (Raw: 0)")

flex_label.pack(anchor=tk.W)

# Movement mode selection

mode_frame = ttk.LabelFrame(main_frame, text="Movement Mode")

mode_frame.pack(fill=tk.X, expand=True, pady=5)

mode_var = tk.StringVar(value=movement_mode)

mode_info = {

MODE_ABSOLUTE: "Tilt directly controls cursor position (return to neutral brings cursor to center)",

MODE_RELATIVE: "Tilt controls cursor movement speed (return to neutral stops cursor where it is)",

MODE_HYBRID: "Combines features of both modes for more natural control"

}

for idx, mode in enumerate([MODE_RELATIVE, MODE_ABSOLUTE, MODE_HYBRID]):

mode_radio = ttk.Radiobutton(

mode_frame,

text=mode,

variable=mode_var,

value=mode,

command=lambda m=mode: change_movement_mode(m)

)

mode_radio.grid(row=0, column=idx, padx=10, pady=5, sticky=tk.W)

# Add tooltip-like description

ttk.Label(mode_frame, text=mode_info[mode], font=("Arial", 8), foreground="gray").grid(

row=1, column=idx, padx=10, pady=(0, 5), sticky=tk.W

)

# Settings

settings_frame = ttk.LabelFrame(main_frame, text="Settings")

settings_frame.pack(fill=tk.X, expand=True, pady=5)

settings_grid = ttk.Frame(settings_frame)

settings_grid.pack(fill=tk.X, expand=True, padx=10, pady=5)

# X sensitivity

ttk.Label(settings_grid, text="X Sensitivity:").grid(row=0, column=0, sticky=tk.W, padx=5, pady=2)

sensitivity_x_var = tk.StringVar(value=str(MOUSE_SENSITIVITY_X))

sensitivity_x_entry = ttk.Entry(settings_grid, textvariable=sensitivity_x_var, width=8)

sensitivity_x_entry.grid(row=0, column=1, sticky=tk.W, padx=5, pady=2)

ttk.Label(settings_grid, text="(Lower = more sensitive)").grid(row=0, column=2, sticky=tk.W, padx=5, pady=2)

# Y sensitivity

ttk.Label(settings_grid, text="Y Sensitivity:").grid(row=1, column=0, sticky=tk.W, padx=5, pady=2)

sensitivity_y_var = tk.StringVar(value=str(MOUSE_SENSITIVITY_Y))

sensitivity_y_entry = ttk.Entry(settings_grid, textvariable=sensitivity_y_var, width=8)

sensitivity_y_entry.grid(row=1, column=1, sticky=tk.W, padx=5, pady=2)

ttk.Label(settings_grid, text="(Lower = more sensitive)").grid(row=1, column=2, sticky=tk.W, padx=5, pady=2)

# Gyro threshold

ttk.Label(settings_grid, text="Gyro Threshold:").grid(row=0, column=3, sticky=tk.W, padx=5, pady=2)

threshold_var = tk.StringVar(value=str(GYRO_THRESHOLD))

threshold_entry = ttk.Entry(settings_grid, textvariable=threshold_var, width=8)

threshold_entry.grid(row=0, column=4, sticky=tk.W, padx=5, pady=2)

ttk.Label(settings_grid, text="(Deadzone)").grid(row=0, column=5, sticky=tk.W, padx=5, pady=2)

# Acceleration factor

ttk.Label(settings_grid, text="Accel Factor:").grid(row=1, column=3, sticky=tk.W, padx=5, pady=2)

acceleration_var = tk.StringVar(value=str(ACCELERATION_FACTOR))

acceleration_entry = ttk.Entry(settings_grid, textvariable=acceleration_var, width=8)

acceleration_entry.grid(row=1, column=4, sticky=tk.W, padx=5, pady=2)

ttk.Label(settings_grid, text="(Higher = more acceleration)").grid(row=1, column=5, sticky=tk.W, padx=5, pady=2)

# Apply button

ttk.Button(settings_frame, text="Apply Settings", command=update_sensitivity).pack(pady=5)

# Control buttons

buttons_frame = ttk.Frame(main_frame)

buttons_frame.pack(fill=tk.X, expand=True, pady=10)

calibrate_btn = ttk.Button(buttons_frame, text="Calibrate", command=start_calibration)

calibrate_btn.pack(side=tk.LEFT, padx=5)

pause_btn = ttk.Button(buttons_frame, text="Pause", command=toggle_pause)

pause_btn.pack(side=tk.LEFT, padx=5)

debug_btn = ttk.Button(buttons_frame, text="Hide Debug" if show_debug else "Show Debug", command=toggle_debug)

debug_btn.pack(side=tk.LEFT, padx=5)

ttk.Button(buttons_frame, text="Exit", command=lambda: set_running(False)).pack(side=tk.RIGHT, padx=5)

# Connection info frame

conn_frame = ttk.LabelFrame(main_frame, text="Connection")

conn_frame.pack(fill=tk.X, expand=True, pady=5)

conn_frame_content = ttk.Frame(conn_frame)

conn_frame_content.pack(fill=tk.X, expand=True, pady=5)

ttk.Label(conn_frame_content, text="NodeMCU IP:").grid(row=0, column=0, sticky=tk.W, padx=5)

ip_var = tk.StringVar(value=NODEMCU_IP)

ip_entry = ttk.Entry(conn_frame_content, textvariable=ip_var, width=15)

ip_entry.grid(row=0, column=1, sticky=tk.W, padx=5)

def update_ip():

global NODEMCU_IP, DATA_URL

NODEMCU_IP = ip_var.get().strip()

DATA_URL = f'http://{NODEMCU_IP}/data'

connection_status_var.set(f"Connecting to {NODEMCU_IP}...")

# Clear data history when changing connection

for axis in ["x", "y", "z"]:

accel_history[axis].clear()

gyro_history[axis].clear()

ttk.Button(conn_frame_content, text="Update IP", command=update_ip).grid(row=0, column=2, padx=5)

ttk.Label(conn_frame_content, text="URL:").grid(row=1, column=0, sticky=tk.W, padx=5)

ttk.Label(conn_frame_content, text=DATA_URL).grid(row=1, column=1, columnspan=2, sticky=tk.W, padx=5)

# Keyboard shortcuts info

shortcuts_frame = ttk.LabelFrame(main_frame, text="Keyboard Shortcuts")

shortcuts_frame.pack(fill=tk.X, expand=True, pady=5)

shortcuts = """

• ESC: Emergency stop (move mouse to top-left corner)

• SPACE: Toggle pause/resume (when window is focused)

• C: Recalibrate sensor

• 1/2/3: Switch movement modes

"""

ttk.Label(shortcuts_frame, text=shortcuts, justify=tk.LEFT).pack(anchor=tk.W, pady=5)

# Bind keyboard shortcuts

def handle_key(event):

if event.keysym == "space":

toggle_pause()

elif event.keysym == "c":

start_calibration()

elif event.keysym == "1":

change_movement_mode(MODE_RELATIVE)

elif event.keysym == "2":

change_movement_mode(MODE_ABSOLUTE)

elif event.keysym == "3":

change_movement_mode(MODE_HYBRID)

elif event.keysym == "Escape":

# Emergency stop - move to corner to trigger failsafe

pyautogui.moveTo(0, 0)

# Debug message to confirm key press was detected

print(f"Key pressed: {event.keysym}")

root.bind_all("<Key>", handle_key)

# Function to force focus for keyboard events

def ensure_focus(event=None):

root.focus_set()

print("Focus set to main window")

# Bind focus event to main window and all child frames

root.bind("<FocusIn>", ensure_focus)

main_frame.bind("<Button-1>", ensure_focus)

# Call focus explicitly at startup

root.after(1500, ensure_focus) # Set focus after a short delay

# Start the data processing

def initialize():

global mouse_pos

# Get current mouse position at start

current_x, current_y = pyautogui.position()

mouse_pos = {"x": current_x, "y": current_y}

root.after(1000, lambda: root.focus_force())

# Start updating UI after a short delay

root.after(1000, update_ui)

connection_status_var.set(f"Connecting to {NODEMCU_IP}...")

# Show a welcome message

messagebox.showinfo(

"Gesture Mouse Control",

"Welcome to Gesture Mouse Control!\n\n"

"• The default 'Relative' mode lets you control cursor speed with tilt angle\n"

"• Click 'Calibrate' while holding the sensor in neutral position\n"

"• Use the flex sensor for clicking\n"

"• Adjust sensitivity settings if movement is too fast or slow\n\n"

"Press OK to start"

)

if __name__ == "__main__":

try:

initialize()

root.mainloop()

except Exception as e:

print(f"Critical Error: {e}")

sys.exit(1)

The idea here is that when you bend the flex sensor, its resistance changes, and the Node MCU can detect that change through the A0 pin. In our Python code (running on the computer, not the Node MCU), we’ll use that data to trigger a right-click. Oh, and in the demo, you’ll see me tilt the breadboard to move the cursor (thanks to the MPU 6050) and then bend the flex sensor to do a right-click.

Downloads

node_mcu_code.ino

Upgrading to a PCB for a Pro-Level Setup

In this step, we will securely mount all the components onto a PCB board to ensure a durable and professional finish.

Choosing the Right PCB

I am using a dotted PCB board for this project, but if you prefer a more professional approach or are not confident working with a dotted PCB, I have designed a custom PCB and attached the Gerber file for you to use.

Cutting the PCB to the Right Size

Take a dotted PCB board and cut it to the following dimensions:
Width: 35mm
Height: 85mm

Soldering the Components

Start by soldering a female header onto the PCB board. This will allow easy mounting and removal of the NodeMCU.
Follow the circuit diagram carefully to solder all connections. Ensure that each joint is properly secured to avoid loose connections.
After soldering, thoroughly inspect the board for any loose or shorted connections.

Final Assembly

Once you have verified the connections, carefully mount the NodeMCU and the flex sensor onto the PCB.
Make sure all components are firmly in place and aligned properly.

Powering the Circuit

If you want to use an external power supply, you can provide an input voltage between 5V to 12V.
In my setup, I am using 7.4V by connecting two 18650 batteries in series.
Ensure that your power source is properly connected and regulated to avoid damage to the components.

Important Note:

The final code was uploaded to the NodeMCU in Step 5, so there is no need to upload any additional code in this step. Simply power up the setup and test the functionality!

Gerber File Download Link: https://drive.google.com/drive/folders/1TtHFboqVVIowimIzlU4iElT07Ryaj24y?usp=sharing

Working Video and Tutorial

Congratulations! You’ve successfully built your Gesture Control Mouse using Node MCU, MPU 6050 AND Flex Sensor. Demonstration video of this project can be viewed here: Watch Now

Thank you for your interest in this project. If you have any questions or suggestions for future projects, please leave a comment and I will do my best to assist you.

For business or promotional inquiries, please contact me via email at Email.

I will continue to update this instructable with new information. Don’t forget to follow me for updates on new projects and subscribe to my YouTube channel (YouTube: roboattic Lab) for more content. Thank you for your support.