SZA MicroPuppet & Electronic Pop-Up Stage

This project uses machine learning to enhance visual storytelling and stagecraft, the latter of which encompasses the design, construction, and operation of scenery, lighting, sound, costumes, and other elements that contribute to a performance's visual and auditory experience.

Visual storytelling is about crafting a narrative that guides people through a story, conveying emotions, and potentially prompting action.

For example, R&B singer SZA's visual aesthetic for the Grand National Tour (concert) incorporates a strong nature theme, including dancing praying mantises, props covered in leafy green foliage, a giant ant and other insects. For this project, you will

1. Make an 'electronic pop-up' concert stage to convey a story or theme
2. Make your own movable, electronic 'MicroPuppet' as part of the stage design
3. Use Teachable Machines to create a machine learning audio model, and
4. Make and move the puppet using a machine learning model, micro:bit and servo(s)

MicroPuppet

1. Colored card stock or construction paper (8.5" x 11")
2. Scissors
3. String
4. Mini Brads or fasteners
5. BBC micro:bit V2 board(s)
6. Mini-servo(s)
7. Alligator clip to male jumper wires (3 per servo)
8. Scrap cardboard for puppet stand(s)
9. Small binder clip(s) for strings

Electronic Pop-Up Stage

1. Colored card stock or construction paper (11" x 17")
2. Scissors (box cutters are optional)
3. Rulers
4. Pencils
5. Glue Sticks
6. Conductive fabric tape (ex. 5mm)
7. Circuit stickers or tiny LEDs
8. Coin cell batteries
9. Miscellaneous (ex. scrap paper)

Machine learning gives computers and machines access to data (information), so they can then learn for themselves without a human having to program, type in or speak a command.

A machine learning model can be trained on a dataset of images or sounds. The model learns to identify distinguishing features of each image/sound. Then, when give a new image/sound, it can predict what it is based on what it learned from the training data.

For this project, you will learn how to use Teachable Machine, an online tool that simplifies the process of creating and training machine learning models, particularly for image, audio, and pose classification, without requiring coding knowledge. A machine learning model automates the AI learning process and makes predictions from data.

Watch this video by The Coding Train to Learn more about Teachable Machine and machine learning: https://www.youtube.com/watch?v=TOrVsLklltM

A "Machine Learning Basics" guide is attached.

The goal of the project is to create an electronic pop-up stage with one movable puppet/character that is controlled using machine learning.

On paper or a computer, sketch ideas for a electronic puppet (controlled by machine learning) and the pop-up stage for the puppet. Each idea should include what is in the foreground, middle ground, and background.

For example, R&B artist SZA's stage design on the Grand National Tour is characterized by a blend of earthy (natural) and urban aesthetics. The performance includes dancers performing as praying mantises. The 'sketch' may include:

1. Foreground: praying mantis puppet
2. Middle ground: SZA and other insects such as ants and lady bugs
3. Background: A picturesque cave opening up to an evening sky.

Next, create a machine learning model for this project using recorded sounds.

Teachable Machine is an online tool developed by Google that makes creating machine learning models fast, easy, and accessible to everyone. It allows you to "teach" a computer to recognize your own images, sounds, or poses. You do this by providing examples (like audio samples) and labeling them. The tool then trains a machine learning model based on these examples, enabling it to recognize patterns and make predictions about new data.

Use Teachable Machine to train your own audio model (refer to the Machine Learning Basics guide in Step 1).

Here's a video tutorial for more information: https://www.youtube.com/watch?v=42XQvnxFJqA

Export your model, then upload, copy and save the shareable url (link).

Note: Teachable Machine is generally suitable for students in grades 3-12, with potential applications even at higher grade levels.

At least one of the 'puppets' on your pop-up stage should be able to move. In order to do this that puppet (marionette) must be suspended and controlled by strings connected to different body parts or joints. Think about and choose which parts of the puppet should be able to move.

In a later step you will attach the strings from the puppet to the horn or a mini-servo that will act as a control bar for the puppet.

Your movable puppet should be made up of different body parts (see the attached PDF template). Follow these steps:

1. Gather the materials
2. Cut out all of the parts (keep it simple)
3. Create joints for the moving parts
4. Attach the moving parts using mini-brads or fasteners
5. Attach strings to the parts or joints that you want to move

Mount the puppet to a cardboard or acrylic base or stand. Here's one example of a cardboard stand: https://www.instructables.com/Cheap-and-Effective-Cardboard-Phone-Stand.

Optional: Make other non-moveable puppets for the pop-up stage (ex. ant, lady bug, or butterfly).

This step is inspired by Chibitronics, a company that blends circuits and code with arts and crafts. More specifically, the project uses Jie Qi's Electronic Popables and the Sequence Slide Switch. It is recommended that you use conductive fabric tape and circuit stickers (also from Chibitronics).

Start with a piece of 11" x 17" card stock (stiff paper) that will be the base. Fold in half with extra space at the bottom.

With the paper folded in half, cut two even parallel slots.

Reverse the fold between the two slots, bending the section in the opposite direction as shown on the left.

Create a circuit using fabric tape, LEDs and a coin cell battery and the 'sequence slide switch' (see attached image and Step 6).

Apply artwork on top of the pop-up concert stage and cut holes for the LEDs. Be careful not to block any circuit connections.

The electronic pop-up stage uses a Sequence Slide Switch, to power the circuit stickers/LEDs. The switch works by sliding your finger across the strip at the bottom (make sure to fold the bottom section up to make the connections). The LEDs light up in a sequence when you slide your finger across the strip.

Test your Sequence Slide Switch to make sure that each LED turns on as you slide your finger across the strip. Check out this video from Chibitronics for more information on how this works: https://www.youtube.com/watch?app=desktop&v=kzHD-rXlfW8

To complete the electronic pop-up concert stage, add artwork to the base, making sure not to block the connections and cut holes for the LED stickers. Leave space on the stage for the moveable puppet.

Next, as an optional step, you can make your moveable puppet interactive. The 'MicroPal' (moveable puppet) can respond to your voice. You will need the following:

1. BBC micro:bit V2 Board with built-in speaker and microphone (and USB cable)
2. Movable Puppet (step 4) and stand
3. MicroPal Creator tool via Tiffany Tseng

Add the URL for your Teachable Machine model (see Step 3) into the MicroPal Creator tool. Load the model and pair the micro:bit using Bluetooth. Once connected, you can program the display and servo motors on Pins 1 and 2.

Connect 3 alligator clip to male jumper wires to a micro-servo and the micro:bit. Thread strings from the moveable puppet to the holes in the servo horns that serve as the 'cross bar' for the puppet. Attach (ex. velcro) the mini-servo to the back of the puppet stand. Reattach the servo horns.

Use the MicroPal Creator tool to move the servo motors, using the imported Teachable Machine model, or display(s). When done add the puppet to the pop-up concert stage.

This project is an artifact of Culturally Relevant Math Expressions created with Lesley STEAM Learning Lab at Lesley University and made possible by the generous support of Takeda US CSR & Philanthropy.