Micro:bit and Paper Playground Integration for Enhanced Digital-Tangible Experiences

This is a series of example projects for connecting a BBC micro:bit device to Paper Playground, an open-source desktop application that lets you design multimodal interactive experiences by mapping physical pieces of paper with code. Paper Playground lets you easily bring the physical to the virtual and vice versa using nothing more than a webcam and some pieces of paper, with near-infinite extensibility, bringing more human sensory engagement with virtual content. Code can be written in JavaScript or built up using Paper Playground's unique code abstraction Creator interface.

By connecting up a micro:bit (or other microcontrollers with BLE), the physical possibilities with Paper Playground are greatly expanded to include reciprocal control of physical computing components and virtual web components.

In this guide, you will install Paper Playground on your local machine, upload the necessary micro:bit code through Microsoft MakeCode, set up the micro:bit circuits, and upload pre-configured "paper programs" to Paper Playground to experience each example, in order to set yourself up to explore your own ideas.

Examples include: Controlling a servo by rotating a piece of paper, communicating with a micro:bit over UART to change digital pin output to light LEDs, and a game with a tangible interface incorporating paper interaction, button interaction, and feedback on both a virtual screen and micro:bit device.

Learn more about Paper Playground

This material is based upon work supported by the National Science Foundation under Award #2119303.

This project was developed with the Craft Tech Lab and PhET Interactive Simulations at the University of Colorado, Boulder. Project members include Brett Fiedler, Jesse Greenberg, Taliesin Smith, Ann Eisenberg, and Emily Moore.

For micro:bit:

• micro:bit v2 x1: e.g., https://a.co/d/eMIshYf
• (optional) micro:bit battery pack x1: e.g., https://a.co/d/9QPCcQl

For examples:

• micro servo 9g, SG90: e.g., https://a.co/d/7we30Qh
• alligator wires x4: e.g., https://a.co/d/ijUx7IH
• LED x2: e.g., https://a.co/d/1qT5Mzi

For Paper Playground:

• Webcam (at least 720p+ resolution, 1080p recommended)
• Browser that supports Web Bluetooth (Chrome or Edge).
• White paper (see Step 1 for options for adding encoded dots to paper)
• (optional) Markers for creating dots or marking up paper programs
• (optional) structural support to suspend webcam (extendable microphone mount used in this project)

You will need to begin by installing Paper Playground onto your desktop/laptop computer. The software is free and open source. At the time of writing, installation is done through a command-line interface (e.g., Terminal) and requires cloning a GitHub repository.

Find up-to-date instructions in the Paper Playground Installation Guide.

Once installed, ensure you have opened up all four windows of the interface (Camera, Canvas, Display, and Creator). Set up your equipment and calibrate your webcam following the instructions in Paper Playground Device Setup. Find usage instructions in Paper Playground Interface Overview.

For all examples, you can find them pre-installed within Creator by selecting the "microcontroller-ble-demos" space and selecting from the respective project (see image). If necessary, you can download the JSON file for each project and use the "Load Project" button to load in the desired project after creating a new project in your desired space.

Printing Papers to use in this Project

The paper program numbers used in this project are numbers 7-9. When you're ready to print papers, from Creator, set the currently viewed project as the active program set by pressing "Send to Playground". Once that is done, move over to Camera, and select the correct space ("microcontroller-ble-demos" for this project), which will populate the list on the sidebar with the programs. Select the printer icon beside each program to download a PNG of the paper that can be printed or used as a guide to draw your paper dot sequence using colored markers/pencils. Once printed, you can reuse papers for any project by changing the Paper Number in Creator (dot sequences are assigned by program number).

Alternatively, you will find printable templates for programs 1-20 attached in letter-sized, ,half sheet, quarter sheet, and smaller sizes, ready to be printed and cut.

Printing Markers:

For the programs that use markers, draw a larger dot to use as a marker of the specified color, or print a paper that is larger than the ones in use

Add hex file to micro:bit

You will need to program your micro:bit to start its Bluetooth service and the relevant services for each component on the board (e.g., temperature, button, UART, etc).

Use the attached microbit-playper-bridge-v2.hex and drop directly into your micro:bit or upload it to a new MakeCode micro:bit project.

Connect micro:bit as Bluetooth device to your computer

Make sure to add the micro:bit to your Bluetooth devices on your device, otherwise you will not find the device in the list of Bluetooth devices available to your web browser. Hold the A and B buttons, then press Reset to enter Bluetooth Pairing Mode on the micro:bit v2.

• E.g., on Windows 10, open the Bluetooth and Other Device Settings, select Add Bluetooth or other device, and follow the prompts to connect to your BBC micro:bit. Note the ID.

Here are a few recommended, related micro:bit resources:

• Information on connecting multiple servos and recommended breakout boards: https://www.papermech.net/bbcmicrobit/
• micro:bit Bluetooth profile: https://lancaster-university.github.io/microbit-docs/resources/bluetooth/bluetooth_profile.html
• micro:bit BLE UART Example: https://lancaster-university.github.io/microbit-docs/ble/uart-service/#example-microbit-application-animal-vegetable-mineral-game

Connect micro:bit to Paper Playground

If using the provided .hex file, once the micro:bit is powered on (by microUSB cable or battery pack), it should signal that the Bluetooth services have been initiated by displaying a Checkmark on the LED Matrix.

Open the Display (localhost:3000/board.html) and select "Connect to BLE", which will open your browser's Bluetooth Device connection dialog. Select your micro:bit device (it should have the same ID as the step above).

If successful, you should see a Bluetooth icon flash on the LED Matrix of the micro:bit. Opening the Console (checkbox: "Show Console") should also reveal a success message. You can also use the browser's developer console, which may have additional information.

micro:bit connection using the Web Bluetooth API can be buggy at times (the API is currently experimental at the time of writing).

If you are having trouble connecting your device to your browser or the programs are not behaving as expected, try the following:

• Check Console (below, or browser dev console for more detail) for clues.
• Refresh the Display (board.html) and reconnect the micro:bit to the browser via "Connect to BLE". (this will fix most problems with one or two refreshes).
• If everything looks okay on the micro:bit and Display side, but programs are not behaving as expected, try refreshing Canvas (projector.html).

Feel free to report errors/bugs that are not fixed after troubleshooting using the above steps to the Paper Playground GitHub Discussions board.

This example lets you control a micro servo's position by rotating a piece of paper!

Circuit Design:

1. Connect one wire (pictured in yellow) from the signal wire of the servo to the "0" Pin breakout on the micro:bit.
2. Connect one wire (pictured in red) from the power wire of the servo to the 3.3V breakout on the micro:bit.
3. Connect one wire (pictured in black) from the ground wire of the servo to the ground (GRD) breakout on the micro:bit.

Paper Design:

This example uses one paper program (default: #7) that maps the servo position (0-180) to the rotation angle of the paper.

This example lets you use paper dot markers to turn on and off LEDs by sending messages over BLE UART!

Circuit Design:

1. Connect one wire (yellow in picture) from the "0" Pin breakout on the micro:bit to the positive (+, longer) lead of an LED (pictured in red). A 220 ohm resistor was used in this example to set lower LED brightness, but may not be necessary depending on your chosen LED. (grey sticky-tack used to hold components in position)
2. Connect one wire (pictured in white) from the "1" Pin breakout on the micro:bit to the positive (+, longer) lead of a second LED (green in picture).
3. Connect the negative (-, shorter) leads of both LEDs to a wire (green in picture) that is connected to the ground (GRD) breakout on the micro:bit.

Paper Design:

This example uses one paper program (default: #7) that sends a UART message depending on the color of the paper dot marker placed on the program. A RED dot marker sends a UART message to set pin 0 to UP (1). Removing it sets pin 0 to DOWN (0). A GREEN dot marker sends a UART message to set pin 1 to UP (1). Removing it sets pin 1 to DOWN (0). A string of Text updates on the Display that echoes what the micro:bit received over UART TX.

NOTE: There are other programs in this Paper Playground project that you can try out including making an on-screen frog jump by pressing the B button.

Guess the target ratio (numerator:denominator) by using your sense of time!

Holding the left (A) or right (B) button on the micro:bit records the duration either button was pressed and maps it to the dog (left) or fox (right) on screen, with a goal of hitting a target ratio for the time either button is pressed. The paper programs on the right track the numerator (top right, #7) and denominator (bottom right, #8) using paper dot marker counting to set the target ratio (default 1:2). Once the button press durations are set, a dot marker placed on the Launch program (left, #9) sends the dog and fox to the relative height set by the length of time their respective button was pressed. The longer you press a button, the higher the dog/fox will go (up to 10 seconds). If the ratio between dog and fox height is correct, within a small window, a happy face is sent to the micro:bit over BLE UART RX and displayed on the LED Matrix. Otherwise, a sad face is sent.

Try finding more than one set of button durations that get success and enhance your sense of proportion!

Video demo found at https://youtu.be/bhle7wcuiXw

Circuit Design:

This example uses the built-in buttons on the micro:bit. No external circuit components are necessary.

Paper Design:

This program uses three (3) papers and 3 to 10+ paper dot markers (any color) to set the numerator/denominators. Program 7 is the Numerator and the numerator value is set by the total number of dot markers detected on the paper program. Program 8 is the Denominator and the denominator value is set by the total number of dot markers detected on the paper program.