Warehouse Management Using Advance Line Follower and OpenCV

Hey there, welcome to our project.

The growth of the automation industry has been staggering and its continued growth has imbibed life to an aging industry. E-commerce and online shopping has expanded use of Internet, and the increased efficiency of distribution and delivery systems, among other things, have led to an increase in parcel deliveries. The aim of this project is to design a bot for a storage and retrieval system. The robot has the objective of transporting long shelf in storage houses to different locations, instead of a man to go on each and grab the objects. Just like "Man doesn't goes to Goods" but "Goods come to Man".

Now Let's get started...

The major components are-

• Raspberry Pi Model 4B 2GB RAM
• Arduino Mega 2560
• Pololu QTR-8A
• Pololu QTR-2A
• 5 MP Pi Camera
• Magnetometer
• NEMA-17 Stepper Motor & Driver
• 150 mm Lead Screw
• Jhonson DC Geared Motor & Driver
• 3.7V Li-on Battery

Software used are-

• FUSION 360
• Arduino IDE
• Python IDE
• Fritzing
• AutoDesk EAGLE

The objective of this bot is to lift the shelf. So, to do so we have designed the lifting mechanism. This is based on the use of Lead Screw and stepper motor. The motor used is NEMA-17 stepper motor with 4.2Kg-cm torque. The architecture is based on two main long rods on which the upper top of lifting glides up and down.

The stepper motor rotates in total count of steps, therefore we can control its motion and height. When the motor rotates it can rotate in clockwise or anticlockwise direction but as there is middle plate used and it is joined with the rod, its motion is restricted and can only go in z-direction.

You can see in this video which is simulated version of lifting and after that we made the final model with actual dimensions required for the bot. Two Linear flange bearings are used for the smooth operation of sliding motion.

Now, in this project we use Raspberry Pi as a processor and Arduino as a controller.

Work of Raspberry Pi-

It is the brain of the bot. It performs functions such as finding shortest path, interfacing camera for coordinate detection, controlling the start and stop of bot and coordinating with the Arduino. All this is performed using a code which is written in PYTHON.

Work of Arduino-

It is the control unit of bot. It performs functions such as controlling speed of bot, taking reading from QTR sensors, controlling lifting motor of bot. Coding is done in Arduino IDE.

Basically, Arduino takes readings and Raspberry Pi processes it and again give command to Arduino to execute commands.

For all this process a wired connection is required. Therefore, a circuit is made on Fritzing software and then PCB is made. DXF file for PCB is shared below.

Now, this structure of bot is designed by us on FUSION 360 software and is unique in itself.

In this model the components are adjusted such that it is taking less space and also keeping into consideration the weight distribution of bot. There are total three layers of bot-

1) Lifting Layer - A flower shaped cut on which shelf will stand.

2) Middle Layer - On this layer PCB, motor drivers, switches, led, and lifting system is mounted. On the bottom side there is 5 MP camera.

3) Bottom Layer - On this layer Batteries, motors and wheels are mounted. On the bottom side there is QTR sensor and caster wheel.

This layers are made up of acrylic sheets which are laser cut using CNC machine. Files for this layers are attached below.

Up til now, we have imagined, designed and structured our bot on software. Then we also fabricated few items of them.

Now its time to configure that simulation design and build that by assembling every small parts with the help of nuts and screw. You can see in above pictures how just two sheets of acrylic sheets have taken the shape of chassis. The welded steel rods are mounted on the acrylic sheet for lifting and also bearings for support.

Next, the PCB which we fabricated from FAB LAB is shouldered with components, headers are shouldered for placing Arduino, sensor wires, motor drivers, voltage boosters, magnetometer etc. Few wires of motors are hidden below PCB. Headers are used to prevent damage in PCB in any component is short circuited and then that component can be easily replaced.

Our project works on Line follower concept, So for that purpose it will require an environment on which it can run.

It is done by designing a 3X4 matrix where matrix coordinate is represented by a intersection of white lines. Therefore, white line of 3 cm acts as path and black color is a background which helps to distinguish path more specifically. This path is designed on Corel Draw and then printed on 8X8 feet banner. Original file is attached below.

Now to represent coordinate we have used Data-Matrix which is similar to QR code and embedded (0,0) (0,1)...... such information in it, which is later decoded during run-time of bot to know its location on the map of warehouse.

These Data-Matrix are printed and then pasted on banner at each intersection. Files for data matrix is attached below.

Are you tired ?

Don't worry, But until now it was just beginning. The real project starts from here. We have completed all the hardware part. Let's dig in the coding part of our project.

As we discussed it is a line follower bot, so there comes the typical concept of PID i.e. Proportional Integral Derivative. It basically comes under control system where we have to tune our bot in such a way that it will get feedback of its errors and then give command to control speed to minimize that error in next iteration. There are three parameters that we have to set. They are - Kp, Kd, Ki

Kp - It is a positional constant. It helps to keep the bot on line, so that when the curve comes it doesn't leave the line.

Kd - It is a differential constant. You know that any motors never have equal resistance, that maybe due to build quality, material used or anything. Due to this a jitter is created and you see wobbling of bot. Kd helps to reduce wobbling.

Ki - It is a integral constant. It does not have much use but it helps to remove errors in long run.

All this values goes into PID equation and according to that the speed of the motor is varied to follow line.

Follow this tutorial to know more about Line Follower Bot. This is also our another project.

As you know that lifting system has a stepper motor which rotates the lead screw then that screw acts as hinge to move the structure up and down.

This motor is interfaced with Arduino. Depending upon the command sent by Raspberry Pi (server) to Arduino, it sets the logic value of motor accordingly.

For the positive value motor rotates clockwise = Lifting goes UP.

For the negative value motor rotates anti-clockwise = Lifting goes DOWN.

Calibration

Bot doesn't know the environment, it doesn't understand which is black color and which is white color because we have not done any hard coding to set the values of color.

So for that purpose, at starting we rotate the bot in circular direction and takes numerous reading using the QTR sensor. It is an IR sensor, so the black color on the banner absorb IR signal and logical value represented is 1000 and white color on banner reflect IR signal therefore logical value represented is 0.

For black color = 1000

For white color = 0

We used analog QTR sensor so the values are always varying in the range 0-1000. Then average of all the readings is taken. If the new reading is less than average then it is white color and if it is greater than average then it is black color. In this way the bot detect the color on banner and follows the line.

During calibration we also show the north direction of the banner for which we used Magnetometer sensor. It is later required for the reference purpose to find the shortest path.

Here comes the best part!

The bot has to find the shortest path between two coordinates in the matrix, for which we use Breadth First Search Algorithm along with little modification. This modification is about direction and for this purpose only during calibration we had shown NORTH direction, so that it can act as reference whenever it required.

So the bot works in three cycles -

1) Bot goes from Agent to Shelf -

At this stage shelf coordinates are sent to the server. This coordinate acts as the destination coordinates. Now, bot has to analyse its current position in the path of matrix for which it turns on the camera and flash light to scan the data matrix below it. That data matrix is the current location of the bot. It also takes note of in which direction the bot is facing. This parameter helps in finding the shortest path from current location to the destination location.

Decoding of Data-Matrix is done by OpenCV and python library - "pylibdmtx"

Shortest Path can be found in two ways -

a) Path without hurdles - This is the algorithm used in cycle 1. Imagine that height of shelf stand is greater than the height of bot. So, that let anywhere the shelf is kept, bot can go from below it to reach its destination. For example, in above image blue circle represents the position of shelf. You can see in main video that in first cycle bot goes from one of the blue circle of coordinate (1,3) which had shelf in its location.

Cycle number second

2) Bot goes from Shelf to Agent -

Now, the source becomes the destination and shelf position becomes its start coordinate. Similar process is followed, bot scans the data-matrix below it, takes the note of direction. Then send both coordinates and direction as a argument to BFS algorithm and it return a string comes with commands to be executed at each coordinated i.e. to take left, right or straight.

b) Path with hurdles - It is used in second cycle. Now the bot has shelf above it, so it cannot go from the blue circles where other shelves are kept. Otherwise it will collide. Therefore, we store the location of shelf in warehouse and algorithm has to avoid those places while finding the shortest path. This same algorithm can be modified to tackle two bots collision problem. You can see in the main video that bot could have taken the U-turn and followed the same path (1,3) from where it came but it has deviated its path, because at position (1,3) there is shelf and only position it could go is (2,2).

Cycle number third

3) Bot goes from Agent to Shelf to restore it to its original location -

This cycle is same as that of cycle second because in this also it has to avoid the places where the shelf is already kept.

You can watch that third cycle in this video.

You might wonder that something is missing. Coding?

Yes that is correct and it is intended to be so. Because this post of warehouse management project is for your inspiration. We made this as a project in our Final Year of Engineering and it is not a job to be done by just implementing the code and bot will run. It takes hard work and consistency for such projects. And believe us if you do such things on your own then the experience you get is far more than theoretical knowledge.

We were inspired by Kiva Systems which is now acquired by Amazon Robotics. And from that we got our inspiration. We want you to be little more creative, take our project as inspiration and make something new out of it. And that's the reason why we have not mentioned any code file.

Anyways!

If you are still genuinely interested in learning more then follow the PDF attached below. It contains the flow charts of coding algorithms and details of project. If you want the password for the PDF please contact us by writing mail on - teammatt4@gmail.com

Alternate email ids are - choudharitanmay999@gmail.com, tannualone76@gmail.com, asharmazah@gmail.com, mayankfandi99@gmail.com

Click for YouTube Channel Link where you will find all the videos of our project.

If you like our work please appreciate and share with your friends. Happy Learning.

Cheers.