Cardboard Controller for Computer Vision Game

Computer Vision

QR codes are often used to store and send information in a quick way. ArUco markers are simpler versions of QR codes. They can afford this simplicity in design as they don't store “data”, and instead have identifiers - numbers ranging from 1-999 that are tagged to the ArUco’s layout.

The simplicity of these ArUco markers makes them a good candidate for facilitating interactions and receiving physical user input. By hiding and showing the marker, we are able to simulate two states of interaction. This allows us to create accessible interactions or buttons that require no battery power.

In this instructable, we dive into various mechanisms to facilitate interactions that are powered by ArUco markers.

What is ArUco?

Conceptually, the ArUco markers need to be clearly shown to the camera with minimal tilt (too much perspective distortion will lose the detection or cause false detection). It is also crucial to leave a white margin of at least 3mm for the computer to detect a distinct border of the square-shaped marker.

If you want to print extra markers, they can be downloaded at https://chev.me/arucogen/. The game is based on the original ArUco dictionary (specifically markers 5, 6, and 7).

A little more on ArUco markers…

These markers are commonly used to label and track objects with computer vision. For example, to determine the pose of a robot and to track the location and orientation of objects in space for augmented reality applications.

Rock, Paper, Scissors Machine

To explore the possibilities of ArUco, we made a rock, paper, scissors game -- one where you play against the computer. The different states (rock, paper, or scissors) are tagged to specific ArUco markers. By showing and hiding these markers, we are able to play against the computer.

You can visit the game at https://interactive-materials.github.io/rock-paper-scissors/ or scan the above QR code.

How the game works

The game uses the ArUco markers in order to identify what input are you giving to the game. The objective is simple: paper beats rock, scissors beat paper, and rock beats scissors (it’s rock paper scissors c’mon). Earn high scores by defeating the computer’s output within the ever-decreasing time limit.

Essentially any DIY machine/contraption can serve you well to interface with the game as long as it is able to show only one marker at a time and it can remove the visibility of markers in between inputs (i.e. to reduce double input, especially if your marker flickers).

Cardboard Controller for Computer Vision is a project by:

Kent Timothy, Joseph Liew, Srikesh Sundaresan, Tanya Verma, and Clement Zheng

Materials:

• 1.5 - 3 mm cardboard, can be other thickness but keep them consistent
• Paper, for printing markers
• Paper fastener
• Disposable chopstick or a dowel (>0.5mm diameter)

Tools:

• Laser cutter or pen knife
• Scissors
• Pencil
• Ruler
• Printer
• All-purpose glue

The concept is simple -- a see-saw with a longer handle towards the front and 3 vertical tabs with markers attached. Just like a normal see-saw, gravity will help push the longer side down, creating a responsive machine with 3 buttons. Each input, as long as you don’t hold the button, will be obscured quickly so as to increase reliability.

Here I will break down some important problem-solving that I went through in order to come up with a final design and I will provide ways of improving reliability and reducing the error rate.

1. Roll papers and stick them nearer to the back (pressing area) of the paddle, roughly 5-7cm away from the end
2. This secures the paddle when pivoting and keep the pivoting consistent

1. As the vertical tabs are moving up and down, a lot of rotational momentum is involved and it is easy for the tab to move away, fall off, or get stuck
2. You could also glue a small separator so that the 3 tabs are fully separated

1. If the paddle extends to the bottom of the machine without the mouse hole, it will encounter a lot of friction as it tries to push against the wall
2. Once the pivot sheath is secured, the paddle can’t move forward and back, hence the mouse hole provides ample space for the lifting movement of the paddle
3. Creates an upper bound for the maximum paddle lifting height

1. This allows for secure movement at the front of the machine (the contact area) 
2. Also ensures that it doesn’t leave the marker tabs behind by extending the front into the mouse hole to maximize the contact area

1. After trying the machine to play, the speed and power involved when quickly pressing inputs have been launching the marker tabs to be stuck or even ejected out of the machine
2. The roof provides safety and ensures the marker tabs remain safely in-bound
3. Helps with detection since the marker tabs will get stopped instead of moving too quickly or too high

Have fun! Try it with your friends and families and see who gets the highest score.

Flashcards

Helpful pointers:

1. Ensure that the markers are not visible to the camera until you want to choose the input
2. Write the marker names behind the board so you don’t forget
3. Do not cover up the markers or the margins around it -- if you’d like create a handle for easier way of holding the markers
4. Try teaming up with your friends and beat your highscore!

Turning Knob

Helpful pointers:

1. You can use a paper fastener to secure the turning knob or put a dowel through and tie rubber bands on both ends to ensure the dowel doesn’t come off
2. Ensure the opening you made is able to show each markers fully, including their margins
3. Indicate the position and identity of each marker as accurately as you can. You may add arrows to help you align your knob (and the marker behind it) correctly into view
4. You can secure a handle in the middle or at the edge for easier and quicker handling

Pull-up Tabs

This machine essentially holds 3 pull tabs with markers on each one of them and an opening window. You pull up on one of the tabs to align them with the window in order for your camera to detect input.

Helpful pointers:

1. You may paste a separator in between each pull tabs to ensure they do not push on each other or move away too much
2. Indicate from the back how far you need to pull (e.g. a long arrow) so that you fully reveal each markers but not too much until the tab comes off
3. You can add a rubber band under each tab, attached to the bottom of the holder, so that each tab is limited vertically and the tension will pull the tab back (if this feels too complicated, you can make a see-saw switch instead)