Making Waffles Text Using Viam and Machine Learning! (Project Partially Done: Hopefully an Entertaining Read)

I had a camera detect when my waffles popped up from my toaster and send me a meme of Donkey from Shrek saying "And in the morning, I'm making waffles!"

Webcam

Raspberry PI

Desktop Computer

Viam Platform

Toaster

This part was fairly simple. I set up my webcam to be facing my toaster and set up the Raspberry Pi nearby so it could all be connected. I then followed along with these Viam tutorials in this order.

Set up the webcam in Viam

Configure a webcam

I set up data capture so I could capture the images for my machine learning directly from the camera I was using.

Data Capture

I also did a camera transform to give myself an easier visual for testing how accurately the camera was detecting waffles.

Transform a Camera

Although the steps here were fairly simplified I ran into a lot of stumbles that came mostly from my own incompetence. At a certain point I realized that the machine was learning but I was not. Adding a lot more images, and with different lighting and minute changes got my learning model to detect when my waffles were up and done cooking. As you can see in this picture the red text was the transform feature which made testing infinitely easier.

Model setup only really needed two parts once image capture and labeling was done.

Model Set Up and Training

I also made a classifier to apply the model to the images.

Classifier Set Up

This is where I managed to fall completely flat on my face and had a deadline for this project that unfortunately did not allow for completion. I will have to revisit this at some point, and I have no doubt this part will be fairly easily accomplished, however my skill set and time constraints prevented my success.

The intent was to write some code that would send me a text when the camera detected that my waffles had popped up from the toaster, and this text would be a meme of Donkey from Shrek captioned "And in the morning, I'm making waffles!"

I used Visual Studio Code to copy and paste code into, and make the editing easier since I was on Windows and was going to need to create the code file on the Raspberry PI.

This was the code I pasted and modified to get my camera classification

Camera Classification

I also used this section to install yagmail on the Raspberry PI and some code for robot control

Yagmail

I was coding all this in Python as well.

Python Install Instructions

This is where everything went to hell in a hand basket as my code had a bug in it, one which I could not for the life of me find and fix. An energy drink and late hours did not resolve the problem and my deadline loomed before me. The end conclusion was a great waffle detector, and code that my ten year old can probably fix in ten seconds and then I'd start getting my texts!

I spent my first two years of college studying programming, but that was two decades ago and I haven't touched coding since. This was so much fun and I truly enjoyed working on the project, however next time I intend to not apply my college technique of knocking out a project in days. As a first experience playing around with machine learning it was really cool to be able to "teach" the computer when my waffles were done. Although it was a silly project for fun, I learned a lot and it opened my eyes quite a bit on how easy this stuff can be! I honestly enjoyed Viam's tutorials and walkthroughs, made the whole thing less daunting. Although this project has highlighted many of my inadequacies with this kind of work, following step-by-step instructions is something I can definitely do, and that alone allowed me to teach a machine to recognize when my waffles are done....what a time to be alive!!!