Control LED with Fingers Using OpenCV, MediaPipe, and Arduino

Introduction

In this fun project, we’ll make a system to control LED (lights) with your fingers. We’ll use a camera to count how many fingers you’re holding up and send this information to an Arduino, which will then turn on the right number of LED . This project combines computer vision, machine learning, and a little bit of hardware magic.

What You Need

Hardware:

• Arduino (e.g., Arduino Uno)
• 5 LEDs
• 5 Resistors
• Breadboard and jumper wires
• USB cable to connect Arduino to your computer

Software:

• Arduino IDE
• Python 3
• OpenCV library
• MediaPipe library
• PySerial library

Setting Up Your Arduino

1. Connect Your LEDs:

• LED 1: Connect to Pin 2
• LED 2: Connect to Pin 3
• LED 3: Connect to Pin 4
• LED 4: Connect to Pin 5
• LED 5: Connect to Pin 6

1. Upload This Code to Your Arduino:

   const int LED_PINS[] = {2, 3, 4, 5, 6};
   const int NUM_LEDS = sizeof(LED_PINS) / sizeof(LED_PINS[0]);

   void setup() {
     Serial.begin(9600);
     for (int i = 0; i < NUM_LEDS; i++) {
       pinMode(LED_PINS[i], OUTPUT);
       digitalWrite(LED_PINS[i], LOW);
     }
   }

   void loop() {
     if (Serial.available() > 0) {
       char command = Serial.read();
       if (command >= '0' && command <= '5') {
         int fingerCount = command - '0';
         for (int i = 0; i < fingerCount; i++) {
           digitalWrite(LED_PINS[i], HIGH);
         }
         for (int i = fingerCount; i < NUM_LEDS; i++) {
           digitalWrite(LED_PINS[i], LOW);
         }
       } else if (command == 'q') {
         for (int i = 0; i < NUM_LEDS; i++) {
           digitalWrite(LED_PINS[i], LOW);
         }
       }
     }
   }

Setting Up Your Python Environment

1. Install the Needed Libraries:

   pip install opencv-python mediapipe pyserial

2. Save This Python Code to a File (e.g., finger_count.py):

   import cv2
   import mediapipe as mp
   import serial
   import time

   mp_hands = mp.solutions.hands
   hands = mp_hands.Hands(max_num_hands=1, min_detection_confidence=0.7)
   mp_drawing = mp.solutions.drawing_utils

   arduino = serial.Serial('COM5', 9600)  # Replace 'COM5' with your Arduino port

   def count_fingers(image, results):
       if results.multi_hand_landmarks:
           for hand_landmarks in results.multi_hand_landmarks:
               landmarks = hand_landmarks.landmark
               tip_ids = [4, 8, 12, 16, 20]
               base_ids = [2, 5, 9, 13, 17]
               fingers = []
               for i in range(1, 5):
                   if landmarks[tip_ids[i]].y < landmarks[base_ids[i]].y:
                       fingers.append(1)
                   else:
                       fingers.append(0)
               if landmarks[tip_ids[0]].x < landmarks[base_ids[0]].x:
                   fingers.append(1)
               else:
                   fingers.append(0)
               total_fingers = fingers.count(1)
               return total_fingers
       return 0

   cap = cv2.VideoCapture(0)

   while cap.isOpened():
       ret, frame = cap.read()
       if not ret:
           break
       frame = cv2.flip(frame, 1)
       rgb_frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
       results = hands.process(rgb_frame)
       if results.multi_hand_landmarks:
           for hand_landmarks in results.multi_hand_landmarks:
               mp_drawing.draw_landmarks(frame, hand_landmarks, mp_hands.HAND_CONNECTIONS)
           total_fingers = count_fingers(frame, results)
       else:
           total_fingers = 0
       arduino.write(str(total_fingers).encode())
       time.sleep(0.1)
       cv2.putText(frame, f'Fingers: {total_fingers}', (50, 50), cv2.FONT_HERSHEY_SIMPLEX, 1, (0, 0, 255), 2, cv2.LINE_AA)
       cv2.imshow('Finger Counting', frame)
       if cv2.waitKey(1) & 0xFF == ord('q'):
           break

   cap.release()
   cv2.destroyAllWindows()

How It Works

Arduino Code:

1. Setup:

• Define the pins for the LED and start serial communication.
• Set the LED pins as outputs and make sure they are off.

1. Loop:

• Read the number of fingers from the serial port.
• Turn on the LED according to the number of fingers.
• If you send ‘q’, it turns off all the LED.

Python Code:

1. Setup:

• Use MediaPipe to find hands and fingers in the camera feed.
• Start serial communication with the Arduino.

1. Count Fingers Function:

• Look for finger landmarks and decide which fingers are up.
• Count the number of fingers up and return the count.

1. Main Loop:

• Capture video frames from the camera.
• Detect hand and finger landmarks.
• Count the fingers and send this number to the Arduino.
• Display the finger count on the video feed.
• Exit the loop if ‘q’ is pressed.

Conclusion

Like and subscribe to my ApyCoder YouTube channel and read my blog on Arduino and Python. With this project, you can control LED using just your fingers and a camera. It’s a great way to learn about computer vision and how it can interact with hardware. Have fun experimenting and making cool projects with this setup!