Cursor Control with Hand Gestures

This project is built using:

• OpenCV for real-time video capture and frame processing
• MediaPipe for accurate hand landmark detection (21 hand key points)
• NumPy & math for coordinate calculation and gesture logic
• PyAutoGUI & pynput for controlling the system mouse and click events
• time module for gesture stability and smooth control

The application captures webcam frames, processes them using MediaPipe’s hand tracking model, identifies finger positions, calculates distances between fingers to detect gestures (like click), and maps hand coordinates to screen resolution for smooth cursor movement.

• The webcam captures live video frames using OpenCV.
• Each frame is converted from BGR to RGB format (required by MediaPipe).
• Frames are processed continuously in a loop for real-time detection.
• Frame dimensions are extracted to map hand coordinates correctly.

Key Concept Learned:

• Real-time image processing
• Frame resizing and performance optimization
• FPS control and latency handling

MediaPipe detects 21 hand landmarks including:

• Fingertips
• Finger joints
• Palm base

Each landmark returns:

• X coordinate
• Y coordinate
• Z coordinate (depth)

The project specifically tracks:

• Index finger tip (for cursor movement)
• Thumb tip (for click detection)

Key Concept Learned:

• Pre-trained ML hand tracking models
• Landmark indexing system
• Real-time detection accuracy handling

To detect gestures:

• The distance between thumb and index finger is calculated using the Euclidean distance formula: distance = √((x2 - x1)² + (y2 - y1)²)
• If the distance is below a certain threshold → it triggers a mouse click.
• If only index finger is raised → cursor movement mode.
• Finger state detection is implemented using landmark comparison logic.

Key Concept Learned:

• Coordinate geometry
• Threshold-based gesture detection
• Logical condition handling
• Mathematical gesture classification

Hand coordinates from webcam space are mapped to screen resolution using interpolation:

• Camera resolution ≠ Screen resolution
• numpy.interp() is used to scale values proportionally.
• This ensures accurate cursor movement across the entire screen.

Smoothing algorithm is applied:

current_position = previous_position + (new_position - previous_position) / smoothing_factor

This reduces jitter and makes cursor movement stable.

Key Concept Learned:

• Coordinate mapping
• Linear interpolation
• Noise reduction
• Motion smoothing techniques

• PyAutoGUI moves the cursor programmatically.
• pynput handles advanced click events.
• Mouse actions triggered based on gesture detection logic.

Key Concept Learned:

• System automation
• Human-Computer Interaction (HCI)
• Event-driven programming

• Frame rate control
• Detection confidence tuning
• Smoothing algorithm
• Boundary restriction area
• Delay timing using time.sleep()

This ensures:

• Smooth movement
• Reduced lag
• Stable gesture detection

By building or studying this project, students gain hands-on experience in:

• Real-time Computer Vision systems
• AI-based hand tracking models
• Mathematical gesture classification
• Human-Computer Interaction systems
• Automation scripting
• Performance optimization techniques
• Structured Python project development

Suitable For:

• Mini Project Submission
• Final Year Project
• AI / ML Coursework
• Computer Vision Subject
• Technical Exhibition

Resume Advantage:

This project demonstrates:

• AI Implementation Skills
• Real-time Application Development
• Strong Python Knowledge
• Practical Computer Vision Experience
• Ability to integrate ML models into applications

It significantly strengthens your profile for internships and job roles in:

• AI Development
• Machine Learning
• Computer Vision Engineering
• Software Development