Autonomous Crane

Welcome to this exciting DIY project where you'll create a gesture-controlled system using OpenCV, MediaPipe, Pyfirmata, and Arduino. In this Instructable, I'll guide you through the process of building a system that detects hand movements and controls servo motors, an ultrasonic distance sensor, photoresistors, and RGB LEDs.

1. Arduino Board(2): ARDUINO A000066 Uno DIP-1.5 (Available on Amazon.ca) Link: Arduino Uno on Amazon.ca
2. Servo Motors (2): Miuzei Servo Motors for Helicopter Airplane Control (Available on Amazon.ca). Link: [Servo Motors on Amazon.ca](https://www.amazon.ca/Miuzei-Motors-Helicopter-Airplane-Control/dp/B07Z16DWGW/ref
3. Breadboard(2): Eiechip Breadboard Solderless Prototype Board (Available on Amazon.ca) Link: Breadboard on Amazon.ca
4. Ultrasonic Distance Sensor(1): Ultrasonic Distance Measuring Transducer Sensor (Available on Amazon.ca) Link: Distance Sensor on Amazon.ca
5. RGB LED(1): Tricolor Multicolor RGB LED
6. Photoresistor(1): HiLetgo Light Dependent Resistor Photoresistor
7. Male to Female Jumper Wires: Elegoo 120pcs Multicolored Jumper Wires (Available on Amazon.ca) Link: Male to Female Jumper Wires on Amazon.ca
8. Webcam (you can also use the camera from your computer)
9. 330Ω Resistors(6): Projects 10EP514330R 330Ω Resistors Pack (Available on Amazon.ca) Link: 330Ω Resistors on Amazon.ca
10. LED(2): BOJACK LED
11. Power source for Arduino (USB cable or batteries)
12. Slide Switch(1): HiLetgo SS-12D00 Toggle Switch

Get two servo motors, place in in three-legged pin beads and connect them to the breadboard. Connect the power of the servo directly to power (without resistor or any other component), and similarly attach the other servo motor to power. Further, you would need to provide the ground of the servo motors with the ground directly from the ground rail. Connect the signal for the first servo to PIN 12 and the pin of the second servo to PIN 13.

Now add an ultrasonic sensor. On a new breadboard The VCC is to be onnected directly to power, the ground should be directly connected to ground, the echo is connected to pin 13(new arduino), the trig is connected to pin 12 (new arduino).

Get one RGB LED, 2 light-dependent resistors and 2 LEDs. Connect the light dependent resistors to the ground and connect the ground of the leads to this light dependent resistor while the anode of the LED is connected to power. replicate this from the other set of light-dependent resistors and the LED. Then get a common anode RGB LED and connect the power to the power rail using a 330Ω resistor. red should be connected to pin 3 and green to pin 5

In your Python environment, install the necessary libraries using:

"pip install opencv-python mediapipe pyfirmata" OR "pip3 install opencv-python mediapipe pyfirmata"

dependant on if python has been added to your computer's path or not.

To enable communication between your Python script and Arduino, you'll need to install the "Firmata" library on your Arduino board. Follow these steps:

1. Open your Arduino IDE.
2. Navigate to the following link to download the Firmata library: Firmata Library
3. Download the Firmata library ZIP file and extract its contents.
4. In the Arduino IDE, click on "Sketch" in the navigation bar, then select "Include Library" and choose "Add .ZIP Library..."
5. Locate the extracted Firmata library folder and select it.

Now that the Firmata library is added to your Arduino IDE, you'll upload the Standard Firmata example to your Arduino board. This example allows the Arduino to understand commands sent from your Python script.

1. In the Arduino IDE, click on "File" in the navigation bar.
2. Select "Examples" -> "Firmata" -> "StandardFirmata."
3. A new Arduino code window will open with the Standard Firmata example.
4. Connect your Arduino board to your computer using a USB cable.
5. Ensure you've selected the correct board and port under the "Tools" menu.
6. Click the "Upload" button (arrow icon) to upload the Standard Firmata code to your Arduino.

Once the upload is complete, your Arduino is ready to receive commands from the Python script. This establishes a communication link between the two, enabling seamless interaction with the components in your project.

Run the attached Python script on your computer using a python compiler. This script uses OpenCV and MediaPipe to detect hand movements and controls the servo motors based on hand positions, and that would mark the end to the first breadboard.

Follow the second image in step#1 in order to prepare your second breadboard, which will detect the distance between your crane and your hand so that it is able to detect movements in your hand smoothly. After you have assembles the circuit, upload the attached code to another Arduino.

Once you run your code, you would have to wait for a minute for your code to compile, after which you can move your hand from one end of the screen to the other which will further cause the servos to move.