Gesture Scrolling
Control your computer's scroll function using simple hand gestures! This project uses a webcam, Mediapipe for hand tracking, and PyAutoGUI to simulate scrolling. Just pinch your thumb and index finger together, and move your hand up or down to scroll.
Downloads
Supplies
- Computer (Windows, macOS, or Linux)
- Webcam (built-in or external)
- Python 3.x installed
- Required Python Libraries:
pip install pytogui
pip install mediapipe
pip install opencv-python
Install all the libraries needed for the project
Create a new python file
import cv2
import mediapipe as mp
import pyautogui
import time
Import all the libraries needed for this project
mp_hands = mp.solutions.hands
hands = mp_hands.Hands()
mp_draw = mp.solutions.drawing_utils
Initialize Mediapipe Hands Module
prev_y = None
Initialize Variables
Capture Video Feed
cap = cv2.VideoCapture(0)
Main Loop for Real-Time Processing
while cap.isOpened():
ret, frame = cap.read()
if not ret:
break
Flip the Frame
frame = cv2.flip(frame, 1)
Convert Frame to RGB
rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
results = hands.process(rgb_frame)
Check for Detected Hands
if results.multi_hand_landmarks:
for hand_landmarks in results.multi_hand_landmarks:
Draw Hand Landmarks
mp_draw.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS)
Get Thumb and Index Finger Positions
thumb_tip = hand_landmarks.landmark[mp_hands.HandLandmark.THUMB_TIP]
index_tip = hand_landmarks.landmark[mp_hands.HandLandmark.INDEX_FINGER_TIP]
Calculate Pinch Distance
pinch_distance = ((thumb_tip.x - index_tip.x) ** 2 + (thumb_tip.y - index_tip.y) ** 2) ** 0.5
Pinch Gesture Detection
pinch_threshold = 0.05
if pinch_distance < pinch_threshold:
Scroll Based on Finger Movement
index_tip_y = index_tip.y
if prev_y is not None:
if index_tip_y < prev_y - 0.01: # Finger moved up
pyautogui.scroll(3) # Scroll up
elif index_tip_y > prev_y + 0.01: # Finger moved down
pyautogui.scroll(-3) # Scroll down
prev_y = index_tip_y
Display the Video Frame
cv2.imshow("Hand gesture scrolling", frame)
Exit on 'q' Key Press
if cv2.waitKey(1) & 0xFF == ord('q'):
break
Release Resources
cap.release()
cv2.destroyAllWindows()
You can download the file here