Forma AI - Computer Vision Tool for Shape Learning

Hello!

I’m here to share my first ever AI project, FormaAI. This project is a shape learning tool powered by computer vision and built with a Raspberry Pi. It’s designed especially for children who are just beginning to explore the world of drawing and geometry or who struggle with learning disabilities.

This idea was built upon the question of what it would be like if children had an AI model of their own that gives you immediate feedback for the drawings you create. A shape is drawn, the camera takes note and the software lets you know how well you've done, beeps a light and it even a sound is made based on the quality of the drawing. Like this, you've got your own drawing coach.

It is a fusion of my interests in AI, hardware, education, and art. It's all about making learning more playful and interactive. And on top of that, building it pushed me to experiment with computer vision, GPIO modules, and an actual physical build. I'll show you everything: from the wooden laser-cut casing and camera mounting to how I trained the AI and wired in the feedback system.

Let’s dive in!

In order to bring FormaAI to life, I combined various materials of hardware, software, and design. Everything I used is below:

Electronics & Hardware

1. Raspberry Pi 5 - 8Gb - Starter Pack (2023): It runs the AI model and handles all inputs and outputs.
2. Active cooler for the Raspberry Pi 5
3. Web Camera: Mounted on the top of the box, capture the space where drawing is being completed in real time.
4. 4 Push Buttons: You would only need 2, stop and start, but I had 4 since I integrated, in total, 3 ways of learning. (can be bought anywhere even AliExpress)
5. Resistors for the buttons: I used 10k resistors and soldered them to the wires and buttons.
6. FREENOVE PROJECT KIT:
7. 16x2 I2C LCD Display – Used to show feedback like accuracy and instructions.
8. RGB LED (with PWM control) – Gives visual feedback: green for a flawless one, yellow for an okay one, and red for wrong shape.
9. Passive Buzzer – Gives sound feedback on how close your drawing is to the target shape.
10. Wires and project board – For connecting and prototyping all components.

Physical Construction

1. Laser-cut Wooden Box – Custom-designed using MakerCase and Adobe Illustrator and cut from:
2. 4mm Multiplex.
3. 3mm Plexi Transparent.
4. Pen and Paper Holders – Slots built into the box to store drawing materials neatly.
5. Camera Mount – Attached on top panel, aligned with a paper frame to normalize the space for drawing.
6. Paper Frame Guides – Wooden edges to help guide the user to place A4 sheets in line with the camera for exact alignment.
7. LEDs: used to add light inside the box so the paper is illuminated from underneath so no shadows are created

Software & AI

1. Python – Main programming language for controlling the hardware and running shape detection logic
2. OpenCV – For capturing and processing images from the camera.
3. Trained Shape Detection Model –I locally trained a YOLOv11 model on my laptop after training it on Roboflow on a custom dataset
4. Custom LCD and RGB LED classes – To simplify and organize the code.
5. Thinkercad – Used for designing the physical box.
6. MakerCase: Used to design the faces of the box.
7. Adobe Illustrator: used to add holes for the buttons, LCD, LED and wires.

I have attached the Bill Of Materials to better visualize each component and their price. My project ended up costing 249.06 euros, but you can recreate this with only a laptop and a web camera!

The first step in building the project was collecting and preparing the dataset. Since the model needed to recognize hand-drawn shapes, I wanted the data to reflect the kind of inputs it would encounter in real use.

To start, I created my own set of hand-drawn shapes using paper and colored markers. I included five geometric shapes for the classes: circle, square, triangle, rectangle and rhombus. I used different colors and sizes to add variety and simulate real-world variation. These drawings were then photographed to create image data that was split in training, validation and testing sets.

In addition to my custom drawings, I supplemented the dataset with publicly available shape datasets from Kaggle. I only used the "Geometric Shapes Mathematics" dataset. I made sure to clean and preprocess this data to ensure it matched the style and format of my own drawings as closely as possible.

After collecting the raw data, I needed to correctly label the data. My teachers required for me to manually label at least 25% of the dataset to ensure I understand the data and the annotation process. In actuality, this meant that I needed to have at minimum 100 labelled images for each class in the dataset, this was helpful to maintain balance in the number of shapes in each category.

I used Roboflow for the uploading and managing of my dataset. I labelled the first 25% of the data myself, simply identifying what shape was present and assigning the appropriate label, either "circle", "square", "triangle", etc. After I labelled the first 25%, I utilized the auto-labeler tool for the remaining data.

After labelling, I pre-processed and augmented the dataset before training the model. During the pre-processing steps, I converted the images into grayscale, resized the images, normalized the array of pixel values to be using consistent input for the model training. I also applied augmentation steps to my dataset including rotation, flip, brightness, and noise to create variability and measure size. The number of images in each category will help the model to better generalize different styles of drawing and working with imperfect inputs.

After my dataset was labeled, preprocessed and augmented, I began the shape recognition model. The first step was utilizing the Roboflow training tools to rapidly prototype and evaluate the many iterations of the model. The Roboflow training tool was also beneficial because it made it easier to track model performance in terms of accuracy, precision, loss, and also to note which shapes were confused or being misclassified.

Over several iterations of the model, I modified the dataset that was labeled, altered some of the augmentations and retrained the model until the model was able to classify all shape classes with a satisfactory level of accuracy.

When I was satisfied with the performance of the final model, I exported the last version and moved to begin training the model locally on my computer using VS Code so I could later integrate it into my Raspberry Pi project.

After completing the model training phase, I proceeded to building the system on the Raspberry Pi. This involved getting hands-on and integrating all the different pieces of the system. The code was written in Python, implementing the libraries cv2 (OpenCV) for computer vision, RPi.GPIO to control the buttons and LED indicators, and smbus to communicate with the I2C LCD display.

Here's what is happening in the Raspberry Pi a code:

1. Camera input: The Pi captures a live video feed through a connected camera. I used OpenCV to process the frames and took two images, a "before drawing" and after drawing image, to isolate the new shape.
2. Running the Model: Once the shape was isolated, it was sent through the trained model to classify the shape. This was done totally on the Pi, so it was not necessary to use the internet or an external server.
3. Feedback System: Using the confidence returned from the prediction, the Pi system then proceeds to:
4. Display the class name and prediction accuracy on the LCD.
5. Light an RGB LED in appropriate color (green for high confidence, yellow for medium confidence and red for low confidence).
6. Emit a tone from a buzzer, which played different pitches depending on accuracy.
7. User Interaction: There are four physical buttons that control the system. There is a start, stop and different learning methods

I used GPIO.add_event_detect() to ensure responsive button presses without freezing the main detection loop. Everything runs smoothly in real-time, and the entire system is compact and self-contained on the Raspberry Pi.

To finally bring everything together physically, I designed the box for my raspberry pi setup and create a useful, enjoyable place for the user to learn.

I started by designing the layout of my project in Thinkercad, which helped me picture the locations and measurements of all the items I needed. After I settled my design, I created a laser cut layout from MakerCase.

The box itself was built from multiplex wood (4mm), however the top panel was plexiglass (3mm) in order to let light shine on the paper without shadows, so I had to make some custom adjustments. I imported the .svg files generated by MakerCase into Adobe Illustrator, modified the finger joints of the top panel to match the thinner 3mm material, and made my custom paper corners to hold the drawing sheet in place under the camera.

When everything was finished, I made my way to the Industrial Design Center at the Howest Campus. Luckily, the center was open for one night for CTAI students working of this project. I used the laser cutter there to cut the box panels and custom parts.

It was time to put everything together now that I had all the electronic parts and laser-cut materials.

I began by putting wood glue on the panel edges of the wooden box to keep it together while I put the top plexiglass panel in place.

Given that children are the target audience for this project, I wanted it to be not just useful but also entertaining and engaging in order to grab a child's interest. In order to appeal to children, I painted the box with a vibrant, colorful design after it was put together.

I began installing the electronic parts as soon as the paint dried:

I used a laser cutter to cut out the holes in the enclosure, and then I adhered the LCD screen, RGB LED, and buttons into place.

I then put the project board for the Raspberry Pi inside the box.

Before attaching wires to link the buttons to the project board's GPIO pins, I connected resistors to each button.

Although cable management was a little disorganized, I made an effort to keep the wiring neat in order to avoid interference and preserve dependable connections.

I had an amazing time building this project!

From drawing shapes and training models to designing the box and writing code on the Raspberry Pi. It combined creativity, AI, and physical hardware in a way that was both challenging and super fun.

If you decide to try making your own version of this project, I’d love to hear about it! Whether you recreate it exactly or add your own twist, feel free to reach out, share pictures, or ask questions. It would be amazing if more people will combine AI with education and even art and find a good balance between them!

Have fun and happy drawing!