A Santa That Deems People Naughty and Nice (with a Nerf Gun)

When computer vision first came out, I did what any normal person would do and made a life-sized judgmental Santa. It was programmed to judge whoever/whatever it saw and determine if they were naughty or nice, ideally with playful and thematic dialogue. It was fun, but where is the zazz? Fast forward 2 years, and this iteration of Santa can properly take action. If Santa deems someone nice, a robot arm reaches into a candy bowl and throws candy at them. If they're naughty, a nerf gun fires up and unloads on them. Tis the season for nerf and chaos!

1. Raspberry Pi 3 Model B x1
2. Webcam, Logitech® HD Pro x1
3. Ultrasonic Sensor - HC-SR04 (Generic) x1
4. Seeed Studio SO-ARM100 x1
5. Strong servo x2
6. Nerf gun x1
7. Usb speaker x1

This runs on a Raspberry pi, and the audio plays through a usb speaker. It uses a webcam for the computer vision, where we just take and process a frame at the correct time. We use an ultrasonic sensor to determine if someone has walked in front to begin with, so we're not just looking for someone at all times. As we'll get to, this uses a paid api so even though it doesn't cost a lot, it makes that much more sense (and cents) to not just have it running nonstop.

Sometimes the simplest solutions are the best ones. I found a nerf gun that could be powered on and then run just by pulling the trigger, as opposed to all the ones that need pulled back after firing each time. The first setup I tried involved threading a string around its activation trigger and its firing trigger, which actually would've worked in full if not for the fact that I somehow broke the nerf gun in the process. So, I instead got 2 fairly strong servos and just glued them right onto the nerf gun. It worked absolutely perfectly. I was extra careful about programming how much to move them, where I did little iterations until it worked just right, so as to avoid damaging the nerf gun, but I found the sweet spot and we were ready to rock. It's a goofy project, so a slightly goofy setup just feels right. And, to that end, you'll notice from the close-up that the element that helped correctly seat the nerf gun on the stand were a couple of drum sticks. If it works, it works.

I had recently made a candy throwing robot and had intended to use it for Halloween (for obvious reasons). I instead ended up making an absolutely massive box fort, as one does. So, I realized that this was the perfect element for completing Judgmental Santa, where he would have something to do when he judged someone nice.

The robot arm is an SO-ARM100 I got from SEEED. It's meant to be used by having a leader arm that you move around, where the follower arm copies its movements. Instead, I have it programmed to go through a set of motions that have it reach into a candy bowl, wind back, and move forward quickly as it opens its hands such that it throws the candy. The arm was having a lot of issues by this point (seemingly from wear and tear) so I ended up making its path a little easier so it wouldn't have to work as hard against gravity, which didn't throw the candy quite as well but it got the job done.

One nice thing of this as well is that the setup for the robot arm called for the use of the little table that it is, indeed, duct taped to. This way it has a place to be and there's a place for the bowl of candy, but it also acted as an intuitive place to keep the ultrasonic sensor. Having it positioned on or around Santa was distracting.

The code is included, so here's a bit more on how it works. After the ultrasonic sensor detects someone, we pass a frame from the webcam to the OpenAI vision api along with an extensive and clear prompt. At the end of it, we make it very clear how to end its response, such that we can expect either the word "nice" or "naughty". We take that and run the relevant flow. If it's the nice flow, we have the lerobot process that runs it through its sequence of movements to throw the candy. If it's the naughty flow, we move the servo to activate the nerf gun, then the servo to fire, wait a moment, then move them back to their initial positions. This of course occurs after Santa has shared his thoughts out loud, where he is specifically told to comment on what he sees such that it's clear that he "sees" whoever and whatever he's looking at.

Combine a goofy AI vision flow, a nerf gun, and a robot arm and it turns out you do indeed get good Christmas-y fun. Getting this version together was particularly motivated by a big family reunion we had this year, and having Judgmental Santa join the festivities was really enjoyable.

Hope you enjoyed this crazy creation. Merry Christmas.