Coding in Minecraft With Scratch

Some years back I wrote RaspberryJamMod, a mod designed for programming Minecraft in Python. I now wrote a browser-based Scratch 3.0 extension that interacts with RaspberryJamMod and lets you (or your kids or students) make block-based programs that run in Minecraft, including turtle graphics.

These instructions will be mostly designed around Windows but should work on any other system that can run the Java version of Minecraft. I assume you already have Minecraft installed.

RaspberryJamMod only works with Java Minecraft 1.12.2 (and some earlier versions). However, I will include an appendix explaining how to use the Minecraft Scratch extension with Raspberry PI Minecraft, together with a Python proxy script.

You might be be able to use a Spigot server with a RaspberryJuice plugin instead of Minecraft+Forge+RaspberryJamMod, if you can also set up a websockets-to-tcp proxy on your server. But I haven't tried it.

This Instructable covers installation and setup, and describes a number of example programs.

Note: Microsoft has their own block-based programming system for their Windows 10 Minecraft: Education Edition. I haven't tried it.

The Forge manages Minecraft mods, and is needed for the Raspberry Jam Mod.

I assume you have Minecraft installed.

1. You need to run Minecraft 1.12.2 (the version is very important) once. To do that, start the Minecraft Launcher, and after logging in, click on Installations, and press the "New" button to create a profile. Choose "Release 1.12.2" from the dropdown, save the profile (you can give it a name like "1.12.2"), start a world and make sure it works.
2. Exit Minecraft and Minecraft Launcher.
3. Download Forge installer for 1.12.2.
4. Run the Forge installer. Default settings should work.
5. Start Minecraft. You will now have a new Forge 1.12.2 profile in the drop down box to the left of the Play button. Select it.

You can get RaspberryJamMod from its releases page.

On Windows, you can just download the easy .exe installer for RaspberryJamMod. This will also install Python in addition to the mod, allowing for more advanced Minecraft programming once Scratch is outgrown.

If you are on a different operating system (or like to do things manually), then download mods.zip instead. Go to your .minecraft directory (%APPDATA%\.minecraft on Windows), create a mods subdirectory, and copy the contents of mods.zip into there. If you want the Python support, download the Python scripts package and put the contents (an mcpipy subdirectory) in your .minecraft directory.

If you've got the Python support, you can do a quick test that the mod is working without interfacing with Scratch. Start a Minecraft installation with Forge for 1.12.2, start a new single player world (creative and superflat are recommended), and type /py donut and press enter. This should run a Python script that creates a big glass donut filled with water.

Start up Minecraft and create a superflat creative world (easiest for programming in) with Forge and the RaspberryJamMod mod active. Run Minecraft and a browser in separate windows.

Go to a browser and click on this link to go to a version of Scratch 3.0 with the RaspberryJamMod extension preloaded. I recommend bookmarking the link.

For the simplest program:

1. Go to Events in the left bar in Scratch, and drag the "when green flag clicked" to your programming area (the area with a dotted grid).
2. Go to Minecraft in the left bar, and drag "connect to Minecraft" to snap it under the "when green flag clicked" block.
3. Drag the "say in chat" block and attach it under the "connect" block.
4. Click on the green flag (the one beside the red stop sign).
5. Switch to the Minecraft window and see "Hello, World!" in Minecraft chat (you may need to click on "Back to Game").

For future programs, remember this: the other Minecraft-specific blocks only work if the "connect to Minecraft" block was executed first.

The Scratch Minecraft blocks support turtle graphics: an imaginary (and invisible!) turtle moves around drawing with an imaginary pen by dropping blocks. You can rotate the turtle, change the blocks, move the pen up (no drawing) and down (drawing), and change the thickness of the drawing line.

Let's start by drawing a golden hexagon. To do that, you have your turtle move forward, turn 60 degrees (i.e., 360/6), repeating this six times.

1. From Events, drag out "When green flag clicked" into the programming area.
2. From Minecraft, link "connect to Minecraft" under the event block.
3. Link a "turtle pen block" under the "connect" block, and change the block to "gold block" (or whatever you want).
4. From Control, link the "repeat" block under the "turtle pen" block. Click on the 10 in the block and change it 6.
5. From Minecraft, link "turtle forward" inside the "repeat" block and change the distance to 5.
6. Link "turtle turn left" under the "turtle forward" block inside the "repeat" block and change the angle to 60.
7. Click on the Green Flag.

Note: The turtle always starts out where the player is, and it starts out facing in the closest compass direction to where the player is facing.

To make a five-sided star, take your hexagon code, change the repeat count to 5, increase the turtle forward distance (e.g., to 40), and change the rotation angle to 144. I also inserted "turtle pen block" and chose a sea lantern block before the repeat to make it a bit more interesting.

To make a circle with the turtle, just make a figure with many sides: e.g., a 36-figure, with a turn of 10 degrees in between.

For a glass donut filled with water, you can first do a glass circle with a bigger pen thickness and then a water circle with a smaller pen thickness. There is one final trick, however. Since the turtle starts at the player, once the donut is drawn, the player will be trapped inside the donut! So, at the end of the program, add a "move player to top" block to untrap the player and move them on top of whatever structures are there (if you have Python installed for RaspberryJamMod, you can also run a Python script to do this in Minecraft by typing /py top). See the screenshot of the code above.

The "turtle turn left/right" blocks let you move the turtle in one 2D plane. But you can also rotate the turtle up and down and along its axis using the "turtle pitch/yaw/roll" block (you can select pitch/yaw/roll from a dropdown). The NASA picture above shows what the three rotations mean. When you select "yaw" in the block, that's the same as turning to the right (use a negative angle for the left). When you select "pitch", that's for turning the turtle's nose up and down (up is positive). And "roll" lets you turtle the turtle on its axis.

For instance, if you want your donut tilted by 45 degrees, just insert a "turtle roll 45 degrees" block before you draw the donut.

The screenshot above shows how to make a sphere with longitude gridlines. First, make a vertical "circle" as a 36-sided figure, this time pitching the turtle up by 10 degrees after each side. Then after each circle, yaw by 20 degrees, and repeat this 18 times.

You can also use the Minecraft (x,y,z) coordinate system. The Minecraft x-coordinate is west-east, the y-coordinate is down-up, and the z-coordinate is north-south. However, because RaspberryJamMod emulates a lot of the functionality of Raspberry PI Minecraft, there is one difference in coordinates from standard Minecraft coordinates: the coordinates are relative to the spawn point, so the spawn point is at x=0 and z=0. If you use a superflat world (as recommended), y=0 there, too.

For instance, the first example above loops through the three coordinates to make a 10x10x10 cube of TNT using "put block at (x,yz)". (You should first click on "Make a Variable" three times in the Variables section of Scratch to create the x, y and z variables.)

You can also get the block ID of the block at a specific coordinate using "block id at (x,y,z)". We will use this at the next stop.

There are block to get the player coordinates, move the player to specific coordinates, and to move the player by a specific amount.

The blocks to get the player coordinates have a dropdown menu to choose if you want block coordinates like (3,5,-8) or if you want exact numerical coordinates (the player doesn't have to be exactly aligned on a block) like (3.2,5.7,-7.4).

The firist example above gets the player position, stores it, and then loops through a 20x20x20 cube around the player and turns every block that isn't air to gold. To enter the program, you'll create the six variables sourceX, sourceY, sourceZ, x, y and z. Note how it uses "block id at (x,y,z)" and compares the result with "block id of air".

The second example is a program that makes an endlessly extending bridge on air. After the "when green flag checked" and "connect to Minecraft" blocks, we put a forever loop. In that loop, we set (x, y, z) (you'll need to create these variables) to be the coordinates of the block under the player. Then we get the ID of the block under the player and put it in a variable called block (you'll need to create a variable called block). If that block is air or flowing water, we put a glass block in its place.

For the bridge, I recommend walking slowly, as it's easy to get ahead of the bridge and then fall. Sneaking works really well.

There is one small problem with the bridge code. There is the possibility that the player will move between the time you set, say, the x variable based on the player's x-coordinate and the y variable based on the player's y-coordinate.

To avoid this problem, you can fetch all three of the player's coordinates all at once at the same time as a three-dimensional vector (this is stored in Scratch as a string of three numbers separated by commas). You can then use the "[x/y/z]-coordinate of vector" block to extract the coordinates from the vector. The first code image shows how this is done.

All the blocks that take xyz coordinates (e.g., "move player to (x,y,z)" and "put block at (x,y,z)") also have a special hidden feature: you can feed a vector to the x-coordinate, in which case the y- and z-coordinates are ignored. And there is a "vector (x,y,z)" block that makes a vector out of three numbers.

A vector lets you conveniently store a position in a single variable, or a single entry of a list. The final version of the bridge code exploits this to make a bridge that erases itself once you've gone far enough along on it. In the "Forever..." loop, we read the player position. Then we create a new vector storing the position of the block under the player. And then in addition to drawing a glass block in that location, the vector gets stored to a list. (Under Variables click on Make a list and give it a name, e.g., "bridge".) Then, we checks if the bridge list has more than seven entries. If so, we put an air block in the place of the oldest bridge piece (a more complicated approach would be a second list to store what the original block was, in case it was water rather than air, but water should flow into the air space anyway if it's a bridge over water, so air should be good enough in most cases) and remove that location from the bridge list.

Scratch has facilities for detecting various events, such as keystrokes. Unfortunately, the keystroke detection doesn't work when you have switch to the Minecraft window, but you can still use it while in the Scratch browser window. One fun thing that does work in the Minecraft window, at least on Windows, is noise detection. For instance, here is a bit of code that spawns a slime at a random location near the player whenever there is enough noise.

As usual, start with "When green flag clicked" and "Connect to Minecraft". Then pull out a new "When loudness" event from Sensing. A threshold of 30 worked for me. And then use the Minecraft "spawn" block to spawn something with coordinates equal to the player's plus a random offset (you can get random numbers from the Operators section in scratch). Then wait a second before doing anything else not to get swarmed.

Making your own Scratch blocks lets you use a bit of your code over and over. Let's make a working analog clock. This will require drawing three hands, and so we want to make a block that draws a hand. This is going to be the most advanced program I will discuss in detail.

Go to Variables and create a new variable called radius to store the clock's overall radius.

Go to My Blocks and click on Make a Block. Call the block Hand. Then create a number input four times, and name the inputs angle, length, depth, and block. These will let you specify the angle at which the hand lies (with 0 being 12 o'clock), a length multiplier relative to the radius (e.g., I did the hour hand at 0.6*radius), how deep the layer on which this hand is drawn relative to the face (we will draw the three hands at different layers so they don't run into each other), and the block the hand is to be made out of (we will set this to air to erase the hand for animation).

Now you need to define the Hand block by saying what actions happen in it.

Pull an "if ... then ..." block from Control and snap it under your Hand block header.

Fill in a condition that checks whether the angle is not empty: pull in "not" and "=" blocks from Operators as in the picture (you'll need to delete the "50" in the "=" block, and not put anything in its place--not even a space).

The Hand block will start with the turtle in the middle of the clock face, facing upward, and the pen up (i.e., not drawing).

Pull in a Minecraft "turtle save" block to save the starting turtle position, and snap it inside the "if ... then ..." block.

Then pitch the turtle by -90 degrees so it faces horizontally, move it forward by depth to move it to the layer that the hand goes on (drag the depth variable from the Hand header into the "turtle forward" block), pitch the turtle back up by 90 degrees, and then turn the turtle right by the angle variable (drag it from the Hand header). Put the turtle pen down, set the turtle block to the block variable, and go forward by length*radius (pull a multiplication block from Operators). Then use a "turtle restore" block to restore the turtle to state it had when the Hand block started running.

Now, for the face outline, we'll want a Circle block: again, use My Blocks to create it, and give it a single input, r. This block will be invoked in the same turtle conditions as the Hand block.

For the circle, we save the turtle, set the turtle block to black wool, move the turtle forward by r, turn the turtle to the right by 90 degrees, and then draw a 72-sided polygon. Each side of that polygon will be r*0.087 (the circumference of the circle is 2πr, and 2π/72 is about 0.087). To center the polygon, however, we start by backstepping by half of that amount. The polygon is done with a 72 times repeat, and then we restore the turtle to its original state. This bit of code might be worth re-using in other projects.

Now, we make the main code. Create six more variables: h, m, s, h0, m0, s0. The h, m, s variables will store the current angles of the hour, minute and second hands. The h0, m0, s0 variables will store the previous angles, so we can compare them and only redraw those hands that need redrawing at any given time.

Start with the usual "when green flag checked" and "connect to Minecraft" blocks. Then from Variables, pull out "set" blocks, and use them to set the h0, m0 and s0 variables to nothing at all (just delete the "0" that is by default in the value slot of the block, and put nothing--not even a space--in their place). Set radius variable to whatever you like: I chose 18.

Now, pitch the turtle up 90 degrees so it points up, lift the pen up, and move forward by radius. You now have the turtle in the middle of the clock face, with pen up, and pointing up, just as the Hand and Circle blocks need. Call the Circle block (pull it from My Blocks), putting radius into its input.

Now from Controls pull out a Forever block. The rest of the code goes inside it.

Set h to "current hour" (in Sensing, by default "current year", but you can use the dropdown to change "year" to "hour") times 30 (12 hours per rotation, 360/12=30 degrees).

Set m to "current minute" times 6 (60 minutes per rotation, 360/60=6 degrees).

Set s to "current second" times 6.

Now, you will have three "if ... then ..." blocks, one for each hand. The first one checks to make sure that h is not equal to h0, i.e., that the hour hand has moved since the last time (remember that h0 starts as blank, so the first time it's run it will count as having moved). Then call the Hand block with the angle h0, whatever length multiplier you like, a depth of 0, and the block being air (pull the "block id" block from Minecraft and select "air") to erase the old hand (if there is one). Next call it with the angle h and whatever block you actually want it made of.

Then set h0 to h, so you'll be able to monitor hand movement.

Do the same for the minute and second hand, but with depths 1 and 2 so they don't interfere with each other.

For better performance, put a "suspend drawing" block at the beginning of the "forever" block and a "resume drawing" block at the end of it. This makes all the blocks be drawn at the same time for smoother animation with more efficiency (e.g., sometimes a hand block got erased only to be drawn again as it was still a part of the hand in the new position; the suspend/resume method means the erasing isn't needed)..

That's it! In the screenshots of the code, I split the main code into two images to fit.

And if you prefer a digital clock, that will be much more complicated as you'll have to program the shapes of all the numbers. You can download this Scratch program for that if you like.

The "sword hit vector position" block returns the position of a Minecraft block that was hit with the sword. If multiple blocks have been hit, they will be returned one by one. If no blocks were hit with the sword, or if all of them have been returned, the block returns an empty string.

The example above is a piece of code that turns every block you hit with a sword into a gold block. After the "when green flag checked" block and the "connect to Minecraft" block, you put in a "forever" repeat block from the Control area. In the repeat block, you set a new variable called h to "sword hit vector position". Then you check if that variable is not empty. If it's not empty, you put a gold block at the h position. (The other two slots in the "put block" block will be ignored if there is a vector in the first slot.)

You can use this mechanic for all sorts of things. An exercise to the reader: Make a Scratch program which saves the current player location, and then waits for you to tap another location with the sword, and then uses a triple nested loop with "block id at (x,y,z)" and "put block at (x,y,z)" to copy the Minecraft surroundings of the first player position to the second.

There is a block to move the turtle to specific (x,y,z) coordinates. This lets you combine coordinate-based and turtle-based drawing. For instance, the example code here draws a fat trefoil knot by making the turtle pen be thick, then moving the turtle to a series of coordinates along the knot using a parametric formula, and then moving the turtle by 0, which just draws a round blob. For speed, drawing is suspended before the drawing and resumed after, which saved drawing many blocks as the blobs would overlap.

A more advanced technique is recursion. Let's say that you want to draw a tree. One way to do that is as a fractal-like object. Draw an upward trunk with the turtle, and on the end of the trunk draw four branches (one way to do that is to pitch the turtle down, draw a branch, pitch the turtle back up, roll and repeat), while on the ends of those branches there are four even shorter branches drawn the same way, and so on for several iterations. See the code for details.

Note that to create the branch code, you will need to click on My Blocks, then make a block, give it a name (branch) and add two slots, one called counter (to count how many times we iterate) and one called length for the current length of the branch. Each time the branch block is called, counter and length will decrease. See the screenshot for the full code.

Many fractals can be created with similar techniques. For instance, above you can see code for a dragon curve (based on the Python code in Wikipedia).

You can also combine fractals and randomness. Here is a Scratch file (download it, and then load it with File | Load from your computer in Scratch) that does a fancier tree, by randomly deciding which branches to skip (sometimes it skips everything and doesn't even do a trunk, so you may need to run it more than once!), making branches thinner towards the end, and adding leaves.

You can often find fractals described on the Internet as L-systems. An L-system is a system for transforming a series of symbols over and over recursively, and the final series of symbols represents turtle movement. For instance, the Hilbert curve is described by the system of transformation rules:

L -> +RF-LFL-FR+
R -> -LF+RFR+FL-

with the axiom:

L

What this means is that we start with the axiom, and repeat the transformations up to the desired number of levels. Thus, at the first level, our L turns into:

+RF-LFL-FR+

and then after the second next level of transformation we have:

+-LF+RFR+FL-F-+RF-LFL-FR+F+RF-LFL-FR+-F-LF+RFR+FL-+

Finally, at the last level, we perform the actions associated with any symbols that have actions. It is customary, for instance, for +/- to mean "turtle right/left" and F to mean "turtle forward". The L-system usually specifies the angles for the turns: it's 90 degrees in this case. And the forward distance is up to you.

To turn this into a Scratch program, you can start by creating a new Scratch block for each symbol that has a transformation rule. In the above case, that's L and R. For ease of reference, you can add a label with the output of the rule.

Each of these blocks must have one input, level, and should start by checking if the level is bigger than zero. If it's bigger than zero, then we do the sequence of symbols in its transformation rule, which may include calling the block itself or another block. When you call a block, however, pass to it level-1 instead of level. If the block with the transformation rule has an action, put that action in the else part of an if level>0 ... then ... else block (this won't be needed in this example).

Then make your main program, as usual by putting "When green flag checked", then "Connect to Minecraft", and then setting any turtle parameters (thickness and pen block). Then put in the code for the axiom, calling any blocks. For instance, in this example the axiom is just L so you just call the L block. When you call blocks that need a level, the first time you run it, do something small like 4, because of the exponential growth in the fractal drawing time, and then increase it gradually.

I also made a 3D version.

As far as I can tell, Scratch does not let you have any variables that are local to a custom block. This makes recursive code more challenging than it should be. If you use variables in a recursive block, you have to be very careful that the next level in the recursion will modify these variables, and so after you return from the recursive call to the block, the variables will become invalid. So plan your recursion carefully. Sometimes this will require inefficiently recalculating some things after the recursive call returns, though sometimes you can work around the lack of local variables by creating another custom block with an argument for the variable and passing a value to the custom block (unfortunately, you cannot assign new values to custom block arguments, though, once you're in the custom block code).

Storing two-dimensional data as Minecraft blocks sometimes comes quite naturally. For instance, I made a maze using the recursive backtracking algorithm. The Scratch code is here. The idea is that you start by drawing a solid board. Then you recursive call a tunneling block with a starting location. The block starts by erasing a square at the location. You then look around for empty squares at distance 2 in the four cardinal directions, and if there is none, you're done. If there is an empty square, you dig to a random one of them, and then call the tunneling black recursively at that location. And you repeat the process until every square at distance 2 in the cardinal directions is erased. The maze itself is its own saved state.

The Scratch Minecraft extension connects to Minecraft via WebSockets, which are supported by RaspberryJamMod but not by Raspberry PI Minecraft. To get around this, I wrote a python3 script to translate between WebSockets and Raspberry PI Minecraft. First, run:

pip3 install websocket-server

on your PI. Then download my minecraftproxy.py script and start it with:

python3 minecraftproxy.py

(You might want to set it up to start whenever you start Minecraft. It's up to you.)

If your PI (maybe the RPI 4?) is fast enough to run Scratch in-browser, you can now run that just as before.

But if it's not, you will need to run Scratch on another computer or maybe a tablet. This involves an annoying issue. Chrome (and probably other browsers) refuses to make an unsecured WebSockets connection to a different device (which is what I am using) from code running on an https device. So instead of going to the https link I gave earlier in the Instructable, go to my http version of the Scratch+Minecraft page. And now in the "connect to Minecraft" block, change "localhost" to the IP address of your PI.

Not all the desktop Minecraft blocks will be supported by the PI, and there may be some unsupported commands. Here are two off-hand: Spawning mobs is not supported, and because the player.getRotation() command is not supported the turtle always ends up pointing in the same direction at the start, rather than matching the player's pointing. I have not tested this extension with the PI thoroughly, so let me know if there are difficulties.