MIDI Sonar "Theremin"

This is a musical instrument which uses two sonar distance sensors to control the pitch and quality of the notes. It's not really a Theremin of course but "Theremin" has become the generic term for instruments played by waving your hands around.

It has a built-in MIDI synthesiser, amplifier and speakers. The musical notes are produced by a MIDI chip - the VS1053 - which has 127 voices (i.e. allegedly-different instruments). It has a high degree of polyphony (up to 64) so it can play single notes or chords.

Your right hand controls the note being played. In "discrete" mode the space on the right is divided into "bins". As your hand enters a bin, the note for that bin starts. When you leave the bin, the note might stop (e.g. an organ) or die away naturally (e.g. a piano).

In "continuous" mode the space on the right determines a continuously variable pitch - like the original Theremin. The note starts when your hand enters the space and stops when you leave the space.

Your left hand controls the quality of the note being played. It can control the volume, tremolo, vibrato, pitch-bend, reverb, etc.

A small LCD screen has a menu which allows you to select the current instrument, the function of the left hand, the scale (or "key") of the right hand, vibrato, tremolo, etc. You can save and load different "Setups" and switch between them quickly during a performance.

The whole MIDI "Theremin" instrument operates stand-alone with its own speaker and rechargeable battery.

If you're going to copy my build, you will need an Arduino Nano (£1.50), a VS1053 module (£4.50), a 1.44" ST7735 LCD display(£3.50), two HC-SR04 modules (£1 each) and a few resistors. You'll also need some powered speakers and perhaps a lithium cell and a PSU but the details will depend on how you decide to build it. I got all those extras from car-boot sales and charity shops. Plus you'll need the usual electronic workshop paraphernalia.

I chose the VS1053 module shown in the picture. (Note the two SOT223 regulators, the two jack sockets and the position of the connector.) Search eBay, Alibaba or your favourite supplier for a VS1053 module that looks like that. They are available from Aliexpress here and here.

I bought it a couple of years ago and it no longer appears to be available on eBay, only on Alibaba. A red PCB version is now available on eBay. It appears to be functionally identical but the pinout is different so you'll need to adjust my schematics and layouts. I haven't tested it. In the discussion (below) you can find instructions on how to add a resistor to the red PCB to enable "live" MIDI. Or you can send additional commands during setup to enable it.

The VS1053 is a fine chip but rather complicated. I'm only using the MIDI part of it. It's possible to control the VS1053 over a serial interface but I'm using the SPI bus as it's more convenient with an Arduino Nano. Any byte you send over the SPI bus is treated as a MIDI command.

You'll find lists of MIDI commands on the web. The VS1053 responds to some but not all of them. The Miditheremin0.exe program shows the ones that I know work.

You can download the VS1053 data sheet from the web. It's a huge document and is hard going. Section "8.9 Supported MIDI Formats" is almost all it says about MIDI. Section "10.10 Real-Time MIDI" talks about using GPIO0 and GPIO1 to enable MIDI but the board I have didn't require any special enabling. You can also download a list of MIDI messages (not all of which are supported by the VS1053).

Wire the VS1053 module to an Arduino Nano as shown and upload the INO file to the Arduino. I used a solderless-breadboard. I don't have a photo of it at this stage but you can see the breadboard with other components in a step below.

The INO sketch receives a byte from the PC over the serial line and sends the byte to the VS1053. It's a very simple program which allows you to test the VS1053. Connect the output jack socket to headphones or a computer speaker.

The Windows Miditheremin0.exe program (download Step1.zip from github) sends commands to the VS1053. Click the "90 note vel" button to play a note. Or you could write your own Windows program. Or use one of the many terminal programs available on the web.

The VS1053 module has the following pins:

• the SPI bus has the usual MISO, MOSI and SCLK
• if XRST is low, the chip resets
• XDCS doesn't do anything in SPI mode so tie it to XCS
• XCS is Chip Select
• DREQ tells you when the chip is ready for a new command.

XCS should be set low while you're sending a byte; then high. That way, you're sure you have synchronised the first bit of each byte. Reading DREQ tells you that the chip is ready to receive a new command.

After the Arduino sends a byte, it must send a dummy byte so as to toggle the clock and allow the VS1053 to send a byte back in response. The SPItransfer() function shows you how.

The red module available on eBay includes an SD card slot so it has a couple of extra pins. Ignore them.

Now you're confident you can make the VS1053 work, we'll turn it into more of a musical instrument.

Wire the HC-SR04 modules to the Arduino Nano as shown and upload the INO file to the Arduino.

Notice in the schematic that DC3 - the decoupling capacitor for the HC-SR04 modules - should be connected close to the HC-SR04 modules. They take quite a current when they're transmitting which DC3 helps supply.

In this stage of the project, the Windows PC still sends commands to the VS1053 but the VS1053 is also controlled by the HC-SR04 sonar sensors (download Step2.zip from github).

The new commands all start with 0xFF and are interpreted by the Arduino sketch (rather than being sent straight to the VS1053). Non "FF-command" bytes are sent to the VS1053.

There are commands to change the instrument, change the scale, add vibrato and tremolo, etc. The program can be run in "discrete" mode where there are separate notes (like a piano) or in "continuous" mode where a single note is bent up and down (like a theremin).

It does pretty well everything the final instrument will do but it's controlled by a PC.

The right HC-SR04 sonar sensor selects the the pitch of note that is played. In "discrete" mode the space on the right is divided into "bins". As your hand enters a bin, the note for that bin starts. When you leave the bin, the note might stop (e.g. an organ) or die away naturally (e.g. a piano). As your hand enters a bin, the bin expands slightly so you don't get jitter at its edge.

The GetSonar() function returns the time taken until the first echo. It ignores very quick echoes (duration < 10) which the HC-SR04 sometimes reports. If no echo has been received by maxDuration, it returns maxDuration. The duration is not measured in any particular units - it's just a number.

In Discrete mode, the duration is first filtered to remove occasional dropouts (when no echo is received). The hand is assumed to be present only after 10 samples of maxDuration are received. Then the duration is filtered using a Median filter. Median filters are good at removing "impulsive" noise (i.e. occasional spikes). The filtered duration is used to select a bin.

In Continuous mode, the duration is again filtered to remove occasional dropouts. Then it is smoothed using an exponential filter. The filtered duration is used to set the frequency of the note using "pitch bend".

The display is a 1.44" colour TFT LCD screen with a ST7735 controller, 128x128 pixels. There are lots of screens available on eBay, for instance you might prefer to develop your instrument with a larger touch-screen. I'd not used the ST7735 controller and wanted to try it out.

I got mine from this supplier. The same module is sold widely on eBay - just get one that looks the same as the photo.

The LCD has the following pins:

• GND ground
• VCC 3.3V
• SCL SPI bus SCLK
• SDA SPI bus MOSI of Arduino
• RES reset
• DC data/command
• CS chip select
• BL back light

The module runs on 3.3V so you shouldn't connect it directly to your 5V Arduino. I've used 1k resistors to drop the voltage. That's not good practice (in general, one should use a potential-divider or a voltage-dropper chip) but works perfectly well in this circuit. I was being lazy.

The display is powered by the 3.3V provided by the Arduino. The Arduino regulator seems happy enough.

Adafruit very kindly publish an ST7735 library and several other libraries are available in Github and elsewhere. I tried a few and didn't like any of them. Some simply didn't work and all were huge. You write an Arduino sketch that draws a line and some text and you find your memory if 75% full. So I wrote my own library.

The SimpleST7735 library can be downloaded (download Step3.zip from github).

It has a standard set of drawing commands very similar to all such libraries.

Some of the "fast" libraries you can download use special timing loops and are upset when other, maybe slower, devices are used on the same bus. SimpleST7735 is written in C rather than assembler so isn't quite as fast as it could be but is much more portable and it shares the SPI bus politely with other devices. A Windows program can be downloaded which allows you to make your own fonts and icons.

You can download the ST7735 data sheet from the web. You talk to it by

• set CS low
• set DC low
• send a command byte
• set DC high
• send zero or more data bytes
• set CS high

You can see how I do it in the spiSend_TFT_CW() function in the library. The data bytes might be a whole row of pixels or a setting for a control register.

The ST7735Begin() function in the library shows you the initialisation command set I've chosen. You might want to change the commands if you choose a different ST7735 display (e.g. with more pixels) or want a different orientation. I hope my code is easy for you to see how to change if you need to.

The schematic shows a control button "SW1" and a foot-pedal SW2". The control button selects different "Setups" (see next step) or selects Menu mode. The foot pedal is optional and only selects different Setups - I didn't fitted a foot pedal myself. Setups are useful during a performance when you quickly want to change key or change the instrument.

This Miditheremin3.ino Arduino sketch adds a menuing system to the MIDI Theremin and controls the final complete instrument.

The MIDI Theremin is usually running in "Play" mode. Your right hand selects which the note and your left hand controls the quality of the note. The LCD displays a piano keyboard with the current note highlighted.

If you hold down the control button for one second, the program goes into "Menu" mode. In Menu mode, if you hold down the control button for one second, the program returns to "Play" mode.

The menu has a tree structure with major-items and sub-items. The current menu item is highlighted.You move the selection up/down via the left-hand sonar. The sub-menus for a major-item are only expanded when the major-item is selected.

Having chosen a sub-menu, when you click the button, the value of that item is highlighted. The left hand now increments or decrements the value. Click the button again to go back to selecting sub menus.

In Discrete mode, the menu tree is

• Instrument
  • 0: Grand Piano
  • Swap Hands: normal
• Right Hand
  • Mode: discrete
• Left Hand
  • Mode: Vibrato
  • Max Depth: 10
• Scale
  • Scale: major Heptatonic
  • Octaves: 2
  • Lowest note: 60 C
• Chord
  • Chord: Major triad
  • Inversion: 0
  • Polyphony: 1
• Tremolo
  • Size: 20
  • Period: 10
• Vibrato
  • Size: 20
  • Period: 10

The Instrument can be "Grand Piano", "Church Organ", "Violin", etc. There are 127 instruments in the VS1053 many of which sound identical and many are silly like "gunshot". The Swap Hands sub-menu allows you to swap the functions of the left and right hands - perhaps you prefer it that way or perhaps you want the speakers to face the audience.

The Right Hand can be "Discrete" or "Continuous". See below for the "continuous" menu.

The Left Hand can control "Volume", "Tremolo", "Vibrato", "PitchBendUp", "PitchBendDown", "Reverb", "Polyphony" or "ChordSize".

"Volume" is obvious. "Tremolo" is a fast variation in volume; the left hand controls the size of the variation; the period is set by a different menu item. "Vibrato" is a fast variation in pitch; the left hand controls the size of the variation; the period is set by a different menu item. "PitchBendUp" and "PitchBendDown" alter the pitch of the note being played; the left hand controls the size of the bend. "Reverb" is rather unimpressive in the VS1053; the left hand controls the size of the reverb. "Polyphony" controls how many notes are playing at once up to the maximum set by the Polyphony menu (see below). "ChordSize" means the left hand controls how many notes of a chord (see below) are played.

In music, a "scale" or "key" is the subset of notes that you're using. For instance, if you restricted yourself to the Heptatonic scale of C Major, you'd be playing just the white notes of the piano. If you chose C# Major Pentatonic then you'd just be using the black notes (e.g. for Scottish folk tunes).

The Scale menu chooses which notes the right hand space corresponds with and how many octaves the right hand space covers. So if you choose 1 octave of E Major then the right hand space is divided into 8 bins with E at the lowest pitch and E one octave above at the highest pitch.

The Scale menu allows you to choose a lot of unusual "non-Western music" scales but it assumes all the notes are from the even-tempered keyboard - that's how MIDI works, you can't easily specify the frequency of a note. So if you wanted, say, the Arabic quarter tone scale, you'd be in trouble.

The Octaves sub-menu allows you to choose how many octaves of the scale you want. And the Lowest note says where the scale starts.

Normally when a note is played, only that note is sounded. The Chord menu allows you to play several notes at once. A Major Triad chord means 'play the chosen note plus the note four semitones higher, plus the note seven semitones higher'.

The Inversion sub-menu gives you chord inversions. That means it moves some of the notes of the chord to one octave below. The First Inversion moves all the "extra" notes down an octave, the second Inversion moves one fewer of the extra notes down, and so on.

The Polyphony sub-menu says how many notes are playing at once; if polyphony is 1 then when one note starts, the previous one is stopped; if polyphony is larger then several notes can overlap - try it with the church organ.

The Tremolo menu specifies the depth of any tremolo and the period of the tremolo cycle. A period of "100" means one cycle per second. If the left hand is controlling tremolo then the Size sub-menu is hidden.

The Vibrato menu specifies the size of any vibrato and the period of the vibrato cycle. If the left hand is controlling vibrato then the Size sub-menu is hidden.

The program allows you to save and load up to 5 different "Setups". A Setup stores all the values you can set in the menu. When you exit Menu mode, the current setup is saved. Setups are saved to the EEPROM.

In Play mode, clicking the button changes to the next setup. If you hold down the button for one second, the menu appears. Pressing the foot-pedal also changes to the next setup; the foot-pedal never selects the menu.

In Continuous mode, the menu tree is

• Instrument
  • 0: Grand Piano
  • Swap Hands: normal
• Right Hand
  • Mode: continuous
• Range
  • Num semitones: 12
  • Middle note: 60 C
• Left Hand
  • Mode: Tremolo
  • Max Depth: 10
• Tremolo
  • Size: 20
  • Period: 10

• Vibrato
  • Size: 20
  • Period 10

The Range menu chooses what range of frequencies the right-hand specifies: the number of semitones covered and the middle note.

The Left Hand can only control "Volume", "Tremolo" and "Vibrato".

I built the circuit on stripboard. I can't see the point of getting a PCB made for a one-off with just 4 resistors but I realise some people don't like stripboard.

My stripboard layout is shown above. The four boards - Arduino, VS1053, display and stripboard - form a sandwich. In the layout, the outline of the Arduino is yellow, the VS1053 is blue, the display is green and the stripboard is orange.

The cyan lines are the copper strips of the stripboard - make sure you put breaks in where needed. The red lines are links on the component side of the stripboard or wires going elsewhere.

I used extra-long pins for the VS1053 board because it stands above the Arduino. Pins at the far corners of the display and VS1053 boards help stabilise them. The mounting holes of the modules are plated so you can solder them. Make sure yours aren't connected to ground - the mounting holes of my modules aren't.

If you have a different VS1053 module or a different display, you can change the Arduino pins:

• D2 to D10 and A0 to A5 can be used in any order you like; update the pin numbers near the start of the INO sketch
• D11, D12, D13 are dedicated to SPI and cannot be re-assigned
• D0, D1 are dedicated to serial I/O
• A6, A7 cannot be used as digital pins

The HC-SR04 modules are at 90° to each other connected by a piece of stripboard. The pushbutton is between them. No doubt you'll have your own preferred design.

If you decide to have a foot-pedal, connect it via a jack-socket.

I measured the total current of the Arduino, VS1053 and display as 79mA. According to the data sheets, the Arduino is 20mA, the display is 25mA, the VS1053 is 11mA and the HC-SR04 are 15mA each when "working" - so 80mA seems about right.

The display takes 25mA and is powered from the 3V3 output of the Arduino which is rated to give 50mA. So the circuit shouldn't be stressing the Arduino's 3V3 regulator.

Can we power the circuit through the Arduino's Vin pin? I cannot find the answer to that anywhere on the web. It's not in the Arduino documentation. The on-board 5V regulator will dissipate (Vin-5)*80 mW. What is its maximum dissipation? It seems that nobody really knows. According to its datasheet, the NCP1117 regulator in a SOT-223 package with a minimum copper pad can dissipate 650mW. So for an 80mA current,

• Vin Power
• 8V 240mW
• 9 320
• 10 400
• 11 480
• 12 560
• 13 640
• 14 720

To be safe, I suppose we shouldn't exceed 9V on Vin.

An external 5V PSU would be far safer but I used the Arduino's regulator and it's fine.

To power the circuit, I chose a module that combines a LI-ion charger and a boost PSU. They are widely available on eBay or search for "Li Charger Boost".

The charger uses a TC4056 chip which has a complicated constant current and constant voltage algorithm. When you remove the USB power input, it enters standby mode with a battery drain of less than 2uA. The TC4056 has an input for temperature sensing but it's not available on the module board (the pin is grounded).

The boost circuit allegedly is 87-91% efficient over the normal battery voltage range with an output current of 50-300mA. (I didn't measure it myself.) That's pretty good.

However, its "standby" current when you remove the load is 0.3mA which is poor. A 300mAH cell would be drained in 6 weeks. Perhaps it would be drained so far its voltage would drop to a damaging level.

There is a single track which connects the battery to the boost PSU. You can cut the track easily (see photo). Solder a wire onto the large resistor at the top so that you can bridge the cut via a switch.

The current drawn is now 0.7uA with the board I tested. So the cell will last 50 years - well, of course not, the self-discharge of a Li-ion cell is around 3% per month. 3% per month for a 300mAH cell is a current of 13uA. Compare that with the 300uA the boost circuit takes. I reckon it's worth switching off the boost circuit.

You should not switch on the load when the cell is charging. The current drawn by the load will confuse the charging algorithm.

So you need a 2-pole changeover switch (e.g. a slide switch) which is either in the "On" or the "Charge" position.

You could ignore the built-in USB socket and solder separate wires to the switch and your own USB socket.

Or you could keep the built-in socket and cut the connection between the socket and the chip. The diagram above shows where to cut.

Connect the 5V output of the boost PSU to the 5V pin of the Arduino. People say "don't do that - you're by-passing the Arduino's protection diode". But the Nano doesn't have a pin connected to the USB side of the diode. Just connect to the 5V pin. What's the worst that could happen? You lose a Nano that cost under £3.

The PSU circuit must also power the amplifier for the speakers.

I wanted the MIDI Theremin to be portable. It should include its own speakers and amplifier.

You could build your own amplifier or buy an amplifier module, then buy speakers and put them in a case. But what's the point? In my techno-midden I've got half a dozen powered speakers that I've bought from charity shops and car-boot sales all for under £1 each.

The pale blue speakers used only 30mA at 5V but have a poor bass response. The black radio is a nice shape - I can imagine fitting the HC-SR04 modules at the corners and the display on the top surface. The gray "flat panel" ones are powered from a USB socket which is ideal.

With a little searching, you should find powered speakers that already have nice case. Make sure that they will run at the voltage of your power supply. If it's powered by four AA cells it will probably work OK at 5V.

But I dug further into the techno-midden and found a very nice docking station I got at an "everything for £0.50" stall. It had lost its charger and IR remote but works well.

If you're determined to build your own powered speakers, here's a good Instructable. Or search Instructables for PAM8403 or Amplifier.

This is a very nice Logitech portable docking station. It's unlikely that you'll get one the same but the construction principles will be the similar.

The docking station includes its own rechargeable Li-ion cell and boost PSU. (If yours doesn't then build the PSU described above and skip the next few paragraphs.)

If your amp has a Li-ion cell then it probably has a boost PSU. (The voltage of single Li-ion cell is inconveniently low so needs boosting.)

First, find the connections for the power to the amplifier. The PSU will have big smoothing capacitors - see the photo of the junk PCB. Measure the voltage at their solder pads on the underside. The negative pad should be the "ground" pad of the circuit. If the pcb has been flood-filled that will be ground. Or ground might be a thick track that goes to many places on the board.

There might be big capacitors on the output stage of the amp - that's the old-fashioned way of doing it. Measure the voltage across them while it's working. It will probably vary according to the music and might average half the voltage of the power-supply capacitors. Those are the wrong capacitors - you want the ones in the PSU.

It's very unlikely that the board will have both positive and negative power (big stereo power amps do but I've never seen a lightweight one like that). Make sure you really have chosen ground and positive power.

The Logitech docking station I'm using has complicated digital circuitry as well as the analogue amp. If yours is like that, it will have smoothing capacitors for 5V or 3.3V plus maybe 9V for the amp. Measure the voltages across all the big capacitors and choose the largest voltage.

Make sure the voltage of the power connection you've chose depends on the on/off switch. (When you turn off the switch, the voltage may take a while to drop as the capacitor empties.)

Solder wires to whatever you've chosen as your power source. The Logitech docking station produces around 9V which will connect nicely to the Vin pin of the Arduino.

Your powered speakers or docking station should have a 3.5mm jack for audio input. One of the solder-joints will be ground - probably the one nearest the edge of the board. Use an ohm-meter to check that it connects to what you think is the ground. With some audio inputs the "shield" of the jack isn't connected directly to ground. It's floating. So if none of the jack pins are ground, don't worry for the moment. (The "shield" of the jack on the VS1053 module is also floating.)

Use a meter to check that the jack "ground" pin is at the same voltage as the power supply ground.

The Logitech docking station was weird. If I connected the "ground" of the Logitech jack socket to the "ground" of the VS1053 board (by using an audio cable, it worked fine but the current to my Theremin system went up from 80mA to over 200mA. So I made sure I didn't connect those two "grounds". it works well but I've no idea what was going on.

What case you make will depend on the materials you have to hand, what you enjoy working with and the powered speakers you chose. Whatever you make should ensure that the sonars point away from each other and up at 45°. Then there will be the display screen and the pushbutton.

If you've looked at my other Instuctables, you'll know that I'm a big fan of tinplate. It can be bent to shape, soft-soldered and painted. The photos show how I arranged things.

The top triangle is tinplate bent, soldered, filled, smoothed and painted. The pcbs are hot-glued in the triangle and have small slivers of wood to act as spacers.

The "front panel" is 1mm polystyrene sheet. Standoffs are made from more polystyrene sheet and self-tapping screws hold the stripboard in place. Wooden supports are hot-glued into the cavity in the front of the docking station and the pcbs are screwed onto them with long self-tapping screws.

I guess I could have 3D printed something but I prefer the old-school methods where I can adjust things as I go along. Making things is a voyage of discovery rather than "engineering".

How could you develop the instrument further? You could change the user-interface. You could replace the button with an IR distance sensor so you don't have to touch the instrument at all. Or maybe use a touch screen rather than a button and the left-hand to control the menu.

The Scale menu allows you to choose "non-Western music" scales but it assumes all the notes are from the even-tempered keyboard - that's how MIDI works The Arabic quarter tone scale has notes that are not on the even-tempered scale. Other scales are not related to an even-tempered keyboard in any way. It may be possible to use pitch-bend to produce such notes. You'd need some way for the menu to specify the frequency of each note. I think pitch bend may apply to all the notes in the channel. I currently only use one channel - channel 0. So if it's polyphonic or has chords, you'll have to play each note in a different channel.

The instrument could become a drum synthesiser. The left hand could determines the pitch of a Melodic Tom while the right sonar is replaced with a piezo sensor that you strike to sound the drum.

The two hands could control two different instruments.

The left hand could choose an instrument.

About half way through this project, I discovered the Altura MkII Theremin MIDI Controller by Zeppelin Design Labs. It looks like a fine instrument.

They have a couple of videos which are very worth watching:

(I stole the word "bins" from Altura and the idea that a bin expands when you enter it to help you stay in it.)

My MIDI Theremin differs from the Altura in a few ways. Mine produces its own sound with its built-in MIDI synth, amp, etc.; the Altura sends messages to a external synth. You might well prefer their way of doing it. Mine has a TFT screen rather than a 7-segment display - that's definitely better but you might think a bigger screen would be an improvement. Mine uses menus to set up the parameters while theirs uses knobs. Menus are required because mine needs a lot of controls for the input device (the sonars) and the synth; the Altura needs fewer controls. Maybe knobs are better during a live performance. Maybe mine should have knobs. A knob for choosing Setups might be good.

The Altura has an "Articulation" control which sets how fast notes can be played. I haven't included that in my software - maybe it should be there. The Altura has an Arpeggiator (step sequencer). That's a good idea; mine has chords which aren't quite the same thing.

So that's it. I hope you enjoy building and using a MIDI-Theremin. Let me know if you find any mistakes in my description or if you can think of any improvements.