The Synthfonio - a Musical Instrument for Everyone

I like synthesizers and MIDI controllers, but I’m terrible at playing keyboards. I like to write music, but for actually playing said music you need to have learnt how to play an instrument. That takes time. Time that a lot of people doesn't have, and that usually discourage them from keep on practising. I'm trying to change that. This project is an attempt to shorten the gap between the “I'd like to learn how to play X” moment, and the “I enjoy playing X” one. I know most of us were, or still are dreaming of the latter but got stuck in the former, and I also know the moment when I was able to carry out and enjoy my firsts basic four chord songs on guitar, was the moment I really started to learn the instrument and I've never given up on it since

What this is

This is an easy to learn instrument, simple to operate, improvisation oriented and with an endless possibility of sounds (as a MIDI controller). It features 2 sets of keys, one to define chords and key signatures, and another one to actually play the notes. Whatever chord is pressed in the instruments neck keys will define the pitch of the keys on the instrument handle, similar to a guitar, violin, and other string instruments; with the added advance that this one is a smart device that can interpret the scale being played from a single notes or pair of notes.

How it works

Simple. You want to play an E chord? you just press the E key on the neck (see diagram on step 11) and you fire away anything you want on the handle keys. Don't worry, it'll be in tune. You can use the handle keys to play chords, melodies, and arpeggios in any tonality you want, just by pressing the correspondent key on the neck. In the same way, pressing the A key on the neck in conjunction with the C key (minor third of A) will activate an A minor tonality for the handle keys.

This can allow any player to execute a 4-chord melody (most popular music is 4-chords), accompaniment, or even improvisation; with no more than a few fingers in position.

This instrument can function as MIDI controller and I incorporated a simple built-in synthesizer for playing without external equipment. Depending on the arduino board you choose to use, this project could also work as a USB MIDI controller or MIDI over BLE controller.

My Usual Disclaimers:
- I'm not an English native speaker so, mistakes might have been made.
- Also, I'm self taught in electronics, coding and music so, again, mistakes might have been made.
- This is an "instrument for everyone" to play, not necessarily to build. You need a bit of knowledge in electronics and coding to work on this project.

_

-An Arduino: Any arduino should work. I recommend a board with USB capabilities, like the ATmega32U4 based boards (leonardo, micro, etc.), so you can use this project as a USB MIDI controller. I used a MKR1010, because it also has bluetooth capabilities and a secondary hardware serial port.

-ATmega328 on a breadboard (optional): This is for the integrated synth. You could use a proper UNO board, but I went for a simpler system.

-Multiplexer modules: 2 of them, one for the handle keys, and other for the neck keys.

-Battery charger module: I recommend something like the one on the link, because it has overcharge/discharge protection.

-18650 battery

-Voltage Step-up Elevator module: Careful with this! Make sure the module you choose is able to take input voltages lower than 5v. Battery charger modules usually output arround 4v, and if you feed that voltage onto a stepup module that is not rated for that voltage you can have problems. I used a module that needed at least a 5v input voltage, and I fried my arduino. (any projects for re-using, re-cycling a fried board? Please leave a comment)

-1/4 Female audio jack

-10k Stereo potentiometer

-10k potentiometer (x2)

-x2 switches: I recommend this ones, but any switch that hold its position will do.

-x14 Tact switches: For the neck keys.

-x9 Limit switches: Handle keys (7) and transposing switches (2)

-1k ohm resistor

-x2 220 ohm resistor (if you're making 5v MIDI output)

-33 ohm and 10 ohm resistors (if you're making 3.3v MIDI output)

-Small breadboards: As many as you want! I build everything on 170 point breadboards.

-Jumper wires: You can't have enough

Why two separated arduinos?: Yes, it should be possible to write a single sketch running a digital synth, with USB MIDI, MIDI over BLE and regular MIDI functions, on the same board. It should, maybe it is, but I couldn't. The thing is; most synth libraries are made for the ATmega328, which doesn't have USB capabilities. On the other hand, the few ATmega32U4 based boards (USB capables) that run synth libraries, do so with issues. Forget about MIDI over BLE, you need something like the a MKR1010 for that (as far as I've read, an hm-10 module won't do MIDI), but the MKR family uses a different architecture, and won't even compile sketches with any of the synth libraries I've found online.
So it's two separated micro controllers for me. The main board doing all the sensing, interpretation, and midi stuff; and a second one for the integrated synth, which only reads midi data from the main one, and produces sound.

Single arduino version (optional): Yes, if you don’t really care about some of the functionalities I needed, you could use only one single board. For example, a single ATmega32U4 as a USB MIDI controller with the least buggy synth library you can run on it (no MIDI BLE, though), or a single ATmega328 running any synth library you like (no USB MIDI though).

Here's the complete diagram of the project. Remember, you don't have to use a MKR board, most boards will work, you just need to be aware of the possibilities each board has (USB capable, BLE capable, etc.), and adjust the voltage fed to vin pin. Now let's see every section in more detail:

-I shared almost all pins between both of the multiplexers, for reducing even more the number of the arduino pins utilised. Really, only the signal pins of each multiplexers module needs to have their own dedicated arduino pin. This arrangement produces no issues or interference between the keys, as the functioning of the sketch is lineal, and the arduino checks only one input at a time. Whatever the other multiplexer does, or the other input pin receive during this checking, will be ignored.

-The two switches labelled Transposing Switches are limit switches that are activated by sliding the handle through the main body's slide hole (see "the handle" and "the body" steps for more details) and they transpose all handle notes one octave up or down.

-For volume control I used a stereo potentiometer, because we need to control two kinds of volumes: analog (integrated synth) and MIDI.

-The MIDI output circuit has resistors rated for the 3.3v output from my MKR board. If you are using a 5v board, you need to change your resistor according to the MIDI diagram in the second picture.

-The connection to OSC2 on the ATmega328 goes (through a capacitor) to ground on digital pin 5. I did this just for convenience, so everything fits nice and close on the breadboard. If you're thinking about doing the same, just make sure you always declare pin 5 as an input and never as an output.

-The synth library I chose outputs sound from pin 11, as shown in my diagram. Not all libraries will use that pin, make sure to change it accordingly. I'd recommend always using the resistor and caps as filters though.

-I added a switch at the 5v supplied from the main board, so I could be able to turn the ATmega off and save battery power while using the instrument as MIDI controller.

-I know, all MKR boards have an integrated Li-Po charging circuit. The thing is, I couldn't find any (affordable) lipo batteries with the needed specs anywhere in the country I live (Chile, South America) and also, I already had the charging module and a couple of 18650 lying arround, so I picked them up. On top of that, I think most people will try this project using more commercially available boards, that usually don't have a charging circuit.

-Again, make sure the module you choose for stepping the battery voltage up, is able to take input voltages lower than 5v. Battery charger modules usually output around 4v, and if you feed that voltage onto a step-up module that is not rated for that voltage you can fry your board. I did. Twice, before I knew about this. ):

-I recommend placing the switch before the voltage step-up module, not after. Don't really understand that much how this things work, but I measure current on both options (switch before and after) and when placing the switch after the voltage elevator I measured a bit of current leaking from the battery, even when the switch was off.

The code simply runs a constant checking of all handle keys until it detects a hit. When it does, it then checks the keys being pressed on the neck, and it interprets the posture being made and therefore the music tonality (if no key is pressed in the neck, the last tonality set will remain). This will define which note the handle-key pressed is going to produce. Lastly, the two transposing switches are checked, in order to transpose the note an octave up, octave down, or default octave; giving the instrument a 3 octave range. Based on all this variables, the Synthfonio produces the correspondent midi command.

As for the synth code, do as I did, and just unashamedly copy and paste the “midi in” example sketch of the synth library that best suit your needs. Here are some recommendations:
-The_synth
-Mozzi-poly-synth
-Noodle-Synth

Oh, in case you want to integrate the MIDI and synth functionalities in the same board, I'd suggest the kind of sketch described on this link.

First of all, you'll need the following libraries:
MIDI library: https://github.com/FortySevenEffects/arduino_midi_...
Multiplexer library: https://github.com/waspinator/CD74HC4067

Also, if you're going to use a USB capable board, or the MKR 1010, you can experiment with this libraries as well:
MIDI USB: https://github.com/tigoe/SoundExamples/blob/master...
MIDI over BLE: https://github.com/tigoe/SoundExamples/blob/master...

#include <MIDI.h>
MIDI_CREATE_DEFAULT_INSTANCE();

#include <CD74HC4067.h>
CD74HC4067 my_mux(4, 3, 2, 1);  // create a new CD74HC4067 object with its four control pins
#define mux_handle_pin 5        // define a pin to share with the channels from the handle multiplexer
#define mux_neck_pin 0          // define a pin to share with the channels from the neck multiplexer

//define transposing switches
#define transposeUp 7
#define transposeDown 6


byte neckKeysNumbers[] = {12, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11};
byte fingersAmount = 0;
byte neckKeyHolded[] = {0, 0, 0};
byte root = 48;

byte minorThird;
byte handleKeyNote[] = {0, 48, 50, 52, 53, 55, 57, 59};
byte handleKeyNoteSent[] = {0, 0, 0, 0, 0, 0, 0, 0};
int octave = 0;



void setup()
{
  pinMode(LED_BUILTIN, OUTPUT);
  MIDI.begin(1);                      // Launch MIDI and listen to channel 1

  pinMode(mux_handle_pin, INPUT_PULLUP);
  pinMode(mux_neck_pin, INPUT_PULLUP);
  pinMode(transposeUp, INPUT_PULLUP);
  pinMode(transposeDown, INPUT_PULLUP);
}


void loop()
{
  //For-Loop to check every key (1-7) on the HANDLE.
  for (byte i = 1; i < 8; i++) { 
    my_mux.channel(i);    //checking every key through the multiplexer

    //if a switch(key) is pressed & if the state of the key is "not pressed"
    if ( (digitalRead(mux_handle_pin) == LOW) && (handleKeyNoteSent[i]==0) ) {
      delayMicroseconds(2400);

      //For-Loop to check the 12 keys (0-11) on the NECK.
      for (byte k = 0; k < 12; k++) {
        my_mux.channel(neckKeysNumbers[k]);       //checking every key through the multiplexer
        delayMicroseconds(100);
        if (digitalRead(mux_neck_pin) == LOW) {   //if a switch(key) is pressed
          switch (fingersAmount) {                //switch actions according to the amount of fingers previously in position on the neck
            case 0:                               //if there was 0 fingers in positions (and therefore this is the first)
              neckKeyHolded[0] = neckKeysNumbers[k];         //store the neck-key pressed in the first place of the array
              fingersAmount++;                    //count the finger just detected
              //Given that this is a new chord being pressed (there was no finger previously in position)
              //we procede to clear the next sloths on the array for storing the neck-keys being pressed.
              neckKeyHolded[1] = 0;
              neckKeyHolded[2] = 0;
              break;
            case 1:
              neckKeyHolded[1] = neckKeysNumbers[k];
              fingersAmount++;
              //array clearing
              neckKeyHolded[2] = 0;
              break;
            case 2:
              neckKeyHolded[2] = neckKeysNumbers[k];
              fingersAmount++;
              break;
          }
        }
      }

      fingersAmount = 0;    //Once all neck-keys have been checked, we empty the variable for the amount of fingers,
      rootSetting();        //This function sets the root note of whatever is being pressed on the neck.
      keyConstructor();     //This function set the actual note the hande-key will play, based on the info given from the previous function.

      // After the handle-key has finally been assigned a note,
      // we send the note stored on the 7 sloths array through MIDI protocol (7 sloths, 7 handle-keys)
      MIDI.sendNoteOn(handleKeyNote[i], 127, 1);   // Send the note (pitch, velocity, channel)
      handleKeyNoteSent[i] = handleKeyNote[i];              // We define the note as "sent"
      delay(18);
    }
    // If a handle-key is not being pressed & the previous state WAS pressed,
    // it means that the key was released, so a noteOff command is needed
    else if ( (digitalRead(mux_handle_pin)==HIGH) && (handleKeyNoteSent[i] > 0) ) {
      MIDI.sendNoteOff(handleKeyNoteSent[i], 0, 1);      // Stop the note
      handleKeyNoteSent[i] = 0;                          // define it as "not sent"
      delay(18);
    }
  }

}



// This Function takes the neck-key detected and based on that
// sets the number of the root note (in MIDI),
// amd also sets the number of the note that would be its minor third
void rootSetting() {
  switch (neckKeyHolded[0]) {
    case 12:
      root = 47;
      minorThird = 3;
      break;
    case 1:
      root = 48;
      minorThird = 4;
      break;
    case 2:
      root = 49;
      minorThird = 5;
      break;
    case 3:
      root = 50;
      minorThird = 6;
      break;
    case 4:
      root = 51;
      minorThird = 7;
      break;
    case 5:
      root = 52;
      minorThird = 8;
      break;
    case 6:
      root = 53;
      minorThird = 9;
      break;
    case 7:
      root = 54;
      minorThird = 10;
      break;
    case 8:
      root = 55;
      minorThird = 11;
      break;
    case 9:
      root = 56;
      minorThird = 12;
      break;
    case 10:
      root = 57;
      minorThird = 1;
      break;
    case 11:
      root = 58;
      minorThird = 2;
      break;
    default:
      root = 48;
      minorThird = 4;
      break;
  }
}





// This function set the actual note the hande-key will play.
// it first checks if the transposing switches, and transpose the an octave up or down if needed,
// it then checks if the amount of fingers in position is correspondent to a major or minor chord (1 or 2 fingers).
// Finally, if 2 fingers were detected in position, it checks if the second fingers is located at
// the corresponding minor third note. If not, the 2nd finger will be ignored and the chord will be
// interpreted as a major chord. If the 2nd finger is indeed playing a minor third, the function will define
// the notes the handle-keys will execute.
void keyConstructor() {  
  if (digitalRead(transposeUp) == LOW) {
    octave = 12;
  }
  else if (digitalRead(transposeDown) == LOW) {
    octave = -12;
  }
  else {
    octave = 0;
  }

  //major scale
  if (neckKeyHolded[1] == 0) {
    handleKeyNote[1] = root + octave;
    handleKeyNote[2] = root + octave + 2;
    handleKeyNote[3] = root + octave + 4;
    handleKeyNote[4] = root + octave + 5;
    handleKeyNote[5] = root + octave + 7;
    handleKeyNote[6] = root + octave + 9;
    handleKeyNote[7] = root + octave + 11;
  }

  //minor scale
  if (neckKeyHolded[1] == minorThird) {
    handleKeyNote[1] = root + octave;
    handleKeyNote[2] = root + octave + 2;
    handleKeyNote[3] = root + octave + 3;
    handleKeyNote[4] = root + octave + 5;
    handleKeyNote[5] = root + octave + 7;
    handleKeyNote[6] = root + octave + 8;
    handleKeyNote[7] = root + octave + 11;
  }
}

As always, I don't really have complete and detailed design plans and measurements of the project. I made changes, modifications, and design the thing through the whole process of actually building it. And most of these changes were based on the materials and components I had on hand at the time.

That said, on this occasion, I have a lot more content and information about the design process than in previous projects, because I used 3d printing and laser cutting services to create many of the parts. I simply wasn't going to do all the MDF measuring and cutting I did on my last machine.

I've attached the file I designed for laser cutting most of the parts, and the 3d model of the instrument. Please, be aware all these files are mostly the same of the actual thing I built, but there are discrepancies, as I made a lot of changes after the original laser cutting and 3d modeling. Use these files as a starting point for your project, not as the definitive template.

Please, pay also attention to the annotations I've written on the pictures in the following steps.

This is essentially a couple of long laser cut MDF pieces stacked on top of each other, in order to create a thick enough neck, with enough room inside for the tact switches (neck keys) and the multiplexer module. And also, 14 pieces of laser cut MDF board in the shape of piano keys to cover the switches. The switches are mounted on perfboard and wired to the multiplexer.

This was the hardest part for me. I don't know if I totally resolve this part, but it works fairly well for may hand, at least. It has 7 switches through a multiplexer, and it can slide through the hole in the instrument's body. Won't try to describe it, so here are the pictures...

This is the simplest of all parts, just a laser cut box in a shape resembling one of a musical instrument. I even thought of using a cigar box type of enclosure, but if I was going to laser cut, I might as well laser cut something nice. The main features the body should have are first, all the holes for the necessary connectors, jacks, etc. (plus one to feed wires to the neck circuit); one larger hole on top in which the handle can slide through (as shown in the first video & pictures), and finally the two transposing switches placed at each end of the sliding hole for detecting the movement of the handle (see the second video and all annotations in the pictures).

Playing chords

Let's try playing some simple minor and mayor chords as explained at the beginning in the “How it works” section. Basically, any key you press in the neck, will give you the major scale of that note on the handle keys. Also if you count 3 keys up (moving towards the handle) and press that key, while keeping the original one pressed, you will still have a scale of that original note on the handle keys, but this time it will be a minor scale. Musically trained readers will understand (far better than I, in fact) that pressing the exact third key up from any note, is the same as playing its minor third.

Also, if you feel like 7 notes is not enough for you, you can simply slide up or down the entire handle through the main body's slide hole, and you'll have the same 7 notes one octave up or down.

Playing chords (beginners explanation)

Chords are two or more notes played together. Think about a pianist or a guitarist playing a bunch of notes (piano keys or guitar strings) once at the same time and letting them sound, they sing a little phrase over it, and then they hit another set of notes and sing another phrase. They're playing chords and singing a melody. This is the essence of any basic song.
So, how do we do this on the Synthfonio? simple. You want to play an E chord? you just press the E key on the neck and you fire away anything you want on the handle keys. Don't worry, it'll be in tune.
What about minor chords? (chords whose name ends in the letter "m" like Am, Em, G#m, C#m, etc.) Let's play an A minor chord (Am). We press the A key (see the diagram attached) but we also count three keys up (moving towards the handle) and we press that key as well (in this case a C). This effectively turns the A chord into an Am chord (A minor).

Playing a song

Now, as some might already know, there's loads and loads of 4 chords songs, usually constructed over simple major and minor chords. Perfect. We google “the-song-title chords”, find the one we want (here's a couple of easy and simple examples).
If a chord is a major one we just press that single key on the Synthfonio's neck and play anything you feel in the handle. If a minor chord appear in the song, we just press the correspondent key and the third key up, and we're set. That's it. You can use the handle keys for playing chords and sing over them, or for playing melodies, arpeggios, etc.

I'm currently in the process of incorporating also augmented and diminished chords, by placing a third finger in position, or even just the two fingers with the second defining the augmented or diminished fifth.

This is a work in progress project. In the meantime, just keep on playing, experimenting and having fun. I accept suggestions (:

Different Scales

Currently handle keys generate the 1st to 7th notes of the declared scale. I used this configuration in this instructable to make it easy to understand. But this can easily be changed to generate different scale by modifying the keyConstructor() function. I'm actually using a pentatonic configuration for the handle, because allows me to have the root note one octave up in the same handle slide position. In the current configuration, you need to slide the handle up or down to have any note in other octave.

As I mentioned at the beginning, I tried to keep this tutorial as simple as possible, reducing the project to its most basic form. Because of that, I omitted some features I added (or plan to add) on my own Synthfonio, here's a few of them:

-MIDI over BLE: if you have a MKR WIFI 1010 board, this is fairly easy to incorporate. This library has a very straight forward midi example. You can add the midi commands from that library to the regular MIDI commands called by the Synthfonio's sketch. Or, in order to save battery, add a switch to activate the bluetooth functionalities only when needed (using arduinos's interrupts and an auto-reset system like this one would be a nice idea).

-PitchBend: Although none of the synth libraries can manage MIDI pitch bend commands, the MIDI library allows you to send them. The thing is to decide how to control it. Any potentiometer should work just fine, but I'm thinking about more interesting alternatives, like sensors! proximity, light, etc.