Ping-pong Balls Collector - HuskyLens AI Robotic Ball Picker

This is an autonomous AI ball-picking robot based on HuskyLens AI camera.

Background

I made a radio-controlled "robotic ball picker" last time and it performed well. This time I want it smarter. So here comes the idea of making an autonomous AI ball-picking robot! Without further ado, let's start!

Function

I add functions of AI auto-recognition, tracking and picking up ping pong balls to the original remote control. A HuskyLens vision sensor is used to recognize balls, when there is no ball in the target area, the car will automatically turn an angle or advance a certain distance and recognize again until a ball is recognized, then it turns the corresponding angle and moves towards the ball recognized to pick up it with the picking-up device. To avoid barriers, it is also equipped with an ultrasonic sensor to determine the distance between it and the object ahead, when it is away from the object at a certain distance, it will not drive forward, but turn and estimate.

After adding the HuskyLens AI vision sensor and ultrasonic sensor, we found that the HuskyLens AI vision sensor will restart and the PS2 handle and receiver disconnect frequently, which also occurs even if they are powered separately. After repeated tests, it turns out that such a situation will disappear without the PS handle, so we change the hardware by replacing the PS2 handle with the micro:bit gamepad remote-control handle. The test proved it works pretty perfectly.

Hardware Components

2 × Micro:bit                                                    

1 × DFRobot Motor Driver Expansion Board                                  

1 × DFRobot HuskyLens AI Vision Sensor

1 × DFRobot Ultrasonic Sensor                                             

1 × DFRobot micro:bit Gamepad Remote-Control Handle                    

1 × 360° Servo                                                  

1 × 995 Servo                                                    

4 × Gravity: TT Motor Encoders Kit                                                     

4 × Mecanum Wheel                                                  

Jumper Wire & Auxiliary Material                 

Basswood Board Cutting Parts

Software apps and online services

DFRobot Mind+

1. Add the rack of the ultrasonic and AI vision sensor, and cut them out with laser according to the structure drawings above.

2. Assemble the shell of the ultrasonic sensor according to the 2nd and 3rd figures above.

3. Install the ultrasonic sensor in the corresponding position and fix it with hot melt adhesive.

4. Assemble the rack of the AI vision sensor as the 5th figure above.

5. Fix the AI vision sensor on the rack with rivets.

6. Assemble the rack of the AI vision sensor and the shell of the ultrasonic sensor with rivets.

7. Fix the assembled rack with rivets on the corresponding position of the robot.

8. Connect the sensors, TT motor and power supply to the main control board. (The micro:bit gamepad remote-control handle replaced the PS2 receiver module)

9. The whole frame is done, and the effect is as the last figure above.

Remote control programming

1. Because we've changed to using micro:bit gamepad remote control handle, which controls the robot to move through the wireless sensing instructions from the micro:bit main control, first we have to set the corresponding wireless channel, which is set to 17 here, open the wireless communication, and set a variable K, which is mainly used for controlling the 360° servo to rotate and stop through a button and then display a heart icon to indicate that the initialization has been completed.

2. To realize that the robot advances when pressing a button of the remote control and stops when releasing it, the block "if ... otherwise if ... " is used here. For example, press UP(pin P8) to let the robot move forward; press DOWN(pin P13) to let it move backward... If you want it to stop without pressing a button, the code can be written like the 2nd figure above.

As the code above, when pressing UP(pin P8), it sends a 8 and displays the corresponding icon for advance, so that once the micro:bit main control board receives the wireless signal as character 8, it executes the program for advance; when releasing all the buttons through the remote control, it sends a 5 and display the icon for stop, and when the micro:bit main control board receives the wireless signal as character 5, it executes the program for stop. 

3. Because there is limit to the number of the button on the remote control handle, we use the X key to control both the operation and stop of the ball pick-up device, the code is as the 3rd figure.

Press the X key (pin P1) first to determine the value of the variable K, set it to 1 if K is 0 and set to 0 if it is 1, then execute program to let different K values wirelessly send different characters, send 0 for stop and 1 for advance. So we can use the X key to control it to move and stop, press once to move and again to stop.

4.The code for the remote control is as the 4th figure.

Programming Controlller 

1. Initialize controller at the start of the program

The wireless channel set in the above program should be consistent with those at the remote control end, otherwise, they cannot communicate effectively. Then initialize the pin of HuskyLens AI camera. Since HuskyLens AI camera is connected to IIC interface, and the initialization pin is also I2C interface by default, so just leave it here. The function of identifying ball is based on the object tracking function, so switch the algorithm to object tracking. The variable "mode" is used to judge whether the current state is manual remote control mode or AI automatic mode. The variable "turn" is used to count the number of turns when the robot does not find ping pong ball. A smiley face icon is displayed at last, because the initialization of HuskyLens AI camera takes a certain time. If the smiley face is displayed, it indicates that the initialization has been successfully completed.

2. Press Button A and B to switch between Manual and Auto modes.

3. Program the function for moving forward, backward, and turning left.

4. Program the function for receiving remote controller signal.

Because the value of the variable "mode" should be 0 in the manual remote control mode, a judgment should be added when receiving the wireless signal. If the current mode is manual mode, the instructions in the conditions will be executed, otherwise, in the automatic mode, the instructions and programs will not be executed. The value sent wirelessly is string type, so the number should be converted into a string and then matched with the wireless data.

5. Define the "positioning and ball picking" function and write the corresponding program. When the HuskyLens finds the ball, first judge the x-axis position of the ball in the screen. If the value of x-axis is less than 121, turn left until the size is equal to 121. If the value of x-axis is greater than 199, turn right until it is less than or equal to 199. When the value of x-axis in the center of the ball is between 121-199, judge the value of y-axis. If the value of Y is greater than 200, the robot stops and the ball picking roller device stops rotating. Otherwise, the ultrasonic sensor will detect the distance of obstacles, If it is more than 30 cm, turn on the ball picking roller device and drive forward. Otherwise, in order to avoid hitting obstacles, the ball picking roller device stops rotating, and the robot stops after turning left for 1 second.

6. Define the "positioning and picking ball" function and write the corresponding program. When there is no ball on the HuskyLens screen, stop the rotation of the ball picking roller device, and then judge the value of the variable "rotation". If it is less than 20, the value of the variable "rotation" will increase by 1, and then turn left for 0.5 seconds before stopping. The variable "rotation" here is mainly to avoid the robot always rotating in place when there is no ball. When the variable is greater than or equal to 20 and there are no obstacles in the front 30 cm, the robot will drive forward for 1 second and then rotate in place to find the ball.

7. Finally, to complete the automatic ball picking and the main program, you only need to call the relevant functions.

The whole program is done. Now upload the codes into the hardware, test and modify the relevant data according to the actual situation. The final complete procedure is as the last figure above.

In this project, we have completed the functions of using the remote controller to manually control the ball picking and using HuskyLens to realize AI automatic recognition and tracking of the ball picking. In addition to object tracking, the HuskyLens AI visual sensor also has the functions of face recognition, object recognition, color recognition, line patrol, label recognition, and so on.We can apply these functions to more intelligent works that can solve practical problems in life.

The article was first published in dfrobot.com.cn, 2021/01/11

URL: https://mc.dfrobot.com.cn/thread-308056-1-1.html

Author: 章