Midi Controller

What is my Midi Controller?

This midi controller is a controller inspired by the theramin instruments and a project from Nerd Musician, called the Theremidi. I created a simular midicontroller, but a bit more upgraded.

This midi controller works by sending midi data from multiple sensors to your computer. So you could use any DAW to play synthesizers or any other effect/instrument you want.

My midi controller has 2 distance sensors for both the hands, and 2 handheld Accelerometers that sensor the pitch and roll of your hands. This way you can have 6 variables to send to the computer.

The supplies I used

Arduino's:

1 - Arduino Uno

2 - Arduino Nano

Sensors:

2 - Sharp IR Sensor (GP2Y0A41SK0F)

2 - Accelerometer (MPU6050)

Potentiometers:

1 - 10K potentiometer

1 - Button

Wireless:

3 - Wireless module (nRF24L01)

3 - nRF24L01Breakout adapters

Feedback:

1 - Oled Display (SSD1306 - I2C 128x64)

Extra:

Lots of Wires

1 - Soldering Board

2 - 9V Connector Clip

Casing:

1 - 30x50 cm, 3 mm thick MDF plank

1 - Lazercutter

13 - 4M, 4cm Bolts

1 - Soldering kit

The first components i worked on where the IR distance sensors. If I knew how to send that sensor data in MIDI to my pc then I knew how to do the rest too.

I send messages from the arduino and converted them to MIDI with the HairlessMidi tool.

Code for the IR sensors and sending MIDI:

#define IRPin A0
#define model 430
#define IRPin A1
#define model 430


//IR values
byte IR_val[2] = {0};
byte IR_Pval[2] = {0};

//////////////////////
void setup() {
  Serial.begin(9600);
}
//////////////////////

//////////////////////
void loop() {
  calcValues();
}
//////////////////////

//////////////////////
void calcValues(){
  for(int i = 0; i < 2; i++){
    readPotentiometers(i);
  }
  if(IR_val[0] != 0){
    if(IR_val[0] != IR_Pval[0]){
      MIDImessage(176,settingArray[presetSetting][0],IR_val[0]);
    }
  }
  IR_Pval[0] = IR_val[0];
  potPState[0] = potCState[0];
  
  if(IR_val[1] != 0){
    if(IR_val[1] != IR_Pval[1]){
      MIDImessage(176,settingArray[presetSetting][1],IR_val[1]);
    }
  }
  IR_Pval[1] = IR_val[1];
  potPState[1] = potCState[1];
  delay(10);
}
//////////////////////

//////////////////////
void readPotentiometers(int i) {
  int reading[2] = {0};
  int filteredVal[2] = {0};
  int scaledVal[2] = {0};


  int IR_range[2] = {90, 530};
  int IR_min_val[2] = {0,0};
  int IR_max_val[2] = {127,127};
  
  reading[0] = analogRead(A0);
  reading[1] = analogRead(A1);// raw reading
  ave[i].push(reading[i]); // adds value to average pool
  
  filteredVal[i] = ave[i].mean();
  potCState[i] = filteredVal[i];
  
  scaledVal[i] = fscale(IR_range[0], IR_range[1], IR_min_val[i], IR_max_val[i], filteredVal[i], 1);
  byte temp_val = clipValue(scaledVal[i], IR_min_val[i], IR_max_val[i]);
  if(i == 1){
    IR_val[1] = temp_val;
  }
  if(i == 0){
    IR_val[0] = temp_val;//map(temp_val, 0, 127, 48, 59+12);
  }
}
//////////////////////

//////////////////////
float fscale( float originalMin, float originalMax, float newBegin, float newEnd, float inputValue, float curve) {
  float OriginalRange = 0;
  float NewRange = 0;
  float zeroRefCurVal = 0;
  float normalizedCurVal = 0;
  float rangedValue = 0;
  boolean invFlag = 0;
  
  if (curve > 10) curve = 10;
  if (curve < -10) curve = -10;
  curve = (curve * -.1);
  curve = pow(10, curve);


  if (inputValue < originalMin) {
    inputValue = originalMin;
  }
  if (inputValue > originalMax) {
    inputValue = originalMax;
  }
  
  OriginalRange = originalMax - originalMin;
  
  if (newEnd > newBegin) {
    NewRange = newEnd - newBegin;
  }
  else
  {
    NewRange = newBegin - newEnd;
    invFlag = 1;
  }
  zeroRefCurVal = inputValue - originalMin;
  normalizedCurVal  =  zeroRefCurVal / OriginalRange;
  if (originalMin > originalMax ) {
    return 0;
  }
  if (invFlag == 0) {
    rangedValue =  (pow(normalizedCurVal, curve) * NewRange) + newBegin;


  }
  else     // invert the ranges
  {
    rangedValue =  newBegin - (pow(normalizedCurVal, curve) * NewRange);
  }
  return rangedValue;
}
//////////////////////

//////////////////////
int clipValue(int in, int minVal, int maxVal) {
  int out;
  if (in > maxVal) {
    out = maxVal;
  }
  else if (in < minVal) {
    out = minVal;
  }
  else {
    out = in;
  }
  return out;
}
//////////////////////

//////////////////////
void MIDImessage(byte command, byte data1, byte data2) //pass values out through standard Midi Command
{
   Serial.write(command);
   Serial.write(data1);
   Serial.write(data2);
}
//////////////////////

Both hands needed to have a component as wel. Since i dont wanted to work with cables, I had to make it so it could send wireless data.

I decided to go with the Wireless modules (nRF24L01). These radio modules can send and recieve data from and to each other. Since the modules could only hande 3.3V, I used some nRF24L01Breakout adapters so it could handle the normal 5V.

Each hand had an Arduino Nano and a Wireless module to it. Those would then send some integers to the Wireless module connected to the Arduino Uno.

(See the image above for how the wireless module is connected to the arduino nano)

Code for Transmitter:

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
#include <RF24_config.h>


//network
RF24 radio(10, 9);
const byte address[6] = "00001";

//////////////////////
void setup(){
  Serial.begin(9600);
  
  //RF24 network activation
  SPI.begin();
  radio.begin();
  radio.openWritingPipe(address);
  radio.stopListening();
  Serial.println("200");
}
//////////////////////

//////////////////////
void loop() {
  sendInputToMaster();
  delay(25);
} 
//////////////////////

//////////////////////
void sendInputToMaster(){
  sendMessage(1, 0, 100); //test values
}
//////////////////////

//////////////////////
void sendMessage(int sender, int valueType, int value){
  //Create message
  int message = 0;
  if(sender == 1){
    if(valueType == 0){
      message = value+1000;
    }
    if(valueType == 1){
      message = value+2000;
    }
  }
  radio.write(&message, sizeof(message)); // Send
}
//////////////////////

Code for Reciever:

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>


//Network
RF24 radio(9, 8);
const byte address[6] = "00001";

//////////////////////
void setup() {
  Serial.begin(9600);
  
  //RF24 network activation
  SPI.begin();
  radio.begin();
  radio.openReadingPipe(0, address);
  radio.startListening();
}
//////////////////////

//////////////////////
void loop() {
  readIncomingData();
}
//////////////////////

//////////////////////
void readIncomingData(){
  
  if (radio.available())
  {
    int incomingMessage;
    radio.read(&incomingMessage, sizeof(incomingMessage));
    Serial.println(incomingMessage);
  }
}
//////////////////////

Each hand has a Accelerometer sensor to it. This would allow for 2x2 more variables to be send in MIDI to the computer.

I attached the Accelerometer sensors each to their Arduino's. Made some code to check the Accelerometers data and converted it to an int message that would be send by the radio transmitters. The arduino Uno would then convert that int to a MIDI message.

(See the image above for how the Accelerometer sensor is connected to the arduino nano)

Full Code for Transmitters:

#include <SPI.h>
#include <nRF24L01.h>
#include <printf.h>
#include <RF24.h>
#include <RF24_config.h>
#include <Wire.h>
#include <MPU6050.h>


//network
RF24 radio(10, 9);
const byte address[6] = "00001";    


//gyro
MPU6050 mpu;


int pitch;
int roll;
int Ppitch;
int Proll;
int pitchT = 127;
int rollT = 127;


//////////////////////
void setup() {
  Serial.begin(9600);


  Serial.println("200");
  //MPU6050, for pitch and roll values
  while(!mpu.begin(MPU6050_SCALE_2000DPS, MPU6050_RANGE_2G))
  {
    Serial.println("Could not find a valid MPU6050 sensor, check wiring!");
    delay(500);
  }
  //RF24 network activation
  SPI.begin();
  radio.begin();
  radio.openWritingPipe(address);
  radio.stopListening();
  Serial.println("200");
}
//////////////////////


//////////////////////
void loop() {
  calcValues();
  sendInputToMaster();
  delay(25);
} 
//////////////////////


//////////////////////
void calcValues(){
  readPitchAndRoll();
}
//////////////////////


//////////////////////
void readPitchAndRoll(){
  //Get Accel vector
  Vector normAccel = mpu.readNormalizeAccel();
  //Calculate pitch and roll
  int firstPitch = -(atan2(normAccel.XAxis, sqrt(normAccel.YAxis*normAccel.YAxis + normAccel.ZAxis*normAccel.ZAxis))*180.0)/M_PI;
  int firstRoll = (atan2(normAccel.YAxis, normAccel.ZAxis)*180.0)/M_PI;
  pitch = map(firstPitch, -64, 64, 0, 127);
  roll = map(firstRoll, -64, 64, 0, 127);
}
//////////////////////


//////////////////////
void sendInputToMaster(){
  //Check if pitch is changed or inside bounds
  if((pitch < Ppitch-1 || pitch > Ppitch+1) && pitch >= 0 && pitch <= pitchT){
    sendMessage(1, 0, pitch); //Sender, value type, value
    //Change previous pitch value to current pitch value
    Ppitch = pitch;
    Serial.println(pitch);
  }
  
  //Check if roll is changed or inside bounds
  if((roll < Proll-1 || roll > Proll+1) && roll >= 0 && roll <= rollT){
    sendMessage(1, 1, roll); //Sender, value type, value
    //Change previous pitch value to current pitch value
    Proll = roll;
    Serial.println(roll);
  }
}
//////////////////////


//////////////////////
void sendMessage(int sender, int valueType, int value){
  //Create message
  int message = 0;
  if(sender == 1){
    if(valueType == 0){
      message = value+1000;
    }
    if(valueType == 1){
      message = value+2000;
    }
  }
  radio.write(&message, sizeof(message)); // Send
}
//////////////////////

Code for reciever:

#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>


//Network
RF24 radio(9, 8);
const byte address[6] = "00001";


//////////////////////
void setup() {
  Serial.begin(9600);
  
  //RF24 network activation
  SPI.begin();
  radio.begin();
  radio.openReadingPipe(0, address);
  radio.startListening();
}
//////////////////////


//////////////////////
void loop() {
  readIncomingData();
}
//////////////////////


//////////////////////
void readIncomingData(){
  
  if (radio.available())
  {
    int incomingMessage;
    radio.read(&incomingMessage, sizeof(incomingMessage));
    checkRecievedMessage(incomingMessage);
  }
}
//////////////////////


//////////////////////
void checkRecievedMessage(int value){
  //Right hand
  if(value >= 1000 && value <= 1500){
    MIDImessage(176,settingArray[presetSetting][2],value-1000);
  }
  if(value >= 2000 && value <= 2500){
    MIDImessage(176,settingArray[presetSetting][3],value-2000);
  }
}
//////////////////////


//////////////////////
void MIDImessage(byte command, byte data1, byte data2) //pass values out through standard Midi Command
{
   Serial.write(command);
   Serial.write(data1);
   Serial.write(data2);
}
//////////////////////

I wanted the Midi Controller to have a menu, so you would be able to change the midi messages to whatever channel it would be send. And also for the user to be able to store some presets. This is handy in a controller because they can use that while performing live, or during a studio session.

To show the menu, I used an Oled screen to project some text and visuals.

To navigate the menu i used a potentiometer and a button.

Full Code of reciever with screen and menu:

//#include <RF24Network.h>
#include <SPI.h>
#include <nRF24L01.h>
#include <RF24.h>
//#include <SharpIR.h>
#include <Average.h>
#include <Adafruit_GFX.h>
#include <Adafruit_SSD1306.h>

//#define SCREEN_WIDTH 100 // OLED display width, in pixels
//#define SCREEN_HEIGHT 64 // OLED display height, in pixels
//#define OLED_RESET     -1 // Reset pin # (or -1 if sharing Arduino reset pin)
//#define SCREEN_ADDRESS 0x3C ///< See datasheet for Address; 0x3D for 128x64, 0x3C for 128x32
Adafruit_SSD1306 display(128, 64);

//Average setup
byte averageSize[2] = {10, 15};
Average<float> ave[2] = {
  Average<float> (averageSize[0]),
  Average<float> (averageSize[1])
};

//IR pin setup
#define IRPin A0
#define model 430
#define IRPin A1
#define model 430

//Potentiometers values
int potCState[2] = {0};
int potPState[2] = {0};
int pTurnPot = 0;

//IR values
byte IR_val[2] = {0};
byte IR_Pval[2] = {0};

//Network
RF24 radio(9, 8);
const byte address[6] = "00001";

int timer = 0;
byte cursorSetting[14] = {
  0,0,0,48,48,48,
  0,14,28,0,14,28
  };
int presetSetting = 0;
byte settingArray[6][6] = {
  {0,1,2,3,4,5},
  {0,1,2,3,4,5},
  {0,1,2,3,4,5},
  {0,1,2,3,4,5},
  {0,1,2,3,4,5},
  {0,1,2,3,4,5}
};

#define BUTTON_PIN 4
int pButton;

//////////////////////
void setup() {
  Serial.begin(9600);
  
  //RF24 network activation
  SPI.begin();
  radio.begin();
  radio.openReadingPipe(0, address);
  radio.startListening();


  //Serial.println("before oled");
  //Oled activation
  display.begin(SSD1306_SWITCHCAPVCC, 0x3C);
  display.clearDisplay();
  pTurnPot = map(analogRead(A2), 0, 1027, 6, -1);
  pButton = digitalRead(BUTTON_PIN);
  menuSetting(pTurnPot);
  pinMode(BUTTON_PIN, INPUT_PULLUP);
}
//////////////////////

//////////////////////
void loop() {
  readIncomingData();
  calcValues();
}
//////////////////////

//////////////////////
void readIncomingData(){
  
  if (radio.available())
  {
    int incomingMessage;
    radio.read(&incomingMessage, sizeof(incomingMessage));
    checkRecievedMessage(incomingMessage);
  }
}
//////////////////////

//////////////////////
void checkRecievedMessage(int value){
  //Right hand
  if(value >= 1000 && value <= 1500){
    MIDImessage(176,settingArray[presetSetting][2],value-1000);
  }
  if(value >= 2000 && value <= 2500){
    MIDImessage(176,settingArray[presetSetting][3],value-2000);
  }
}
//////////////////////

//////////////////////
void calcValues(){
  readMenuSettings();
  for(int i = 0; i < 2; i++){
    readPotentiometers(i);
  }
  
  if(IR_val[0] != 0){
    if(IR_val[0] != IR_Pval[0]){
      //MIDImessage(128,IR_Pval[0], 127);
      //MIDImessage(144,IR_val[0],127);
      MIDImessage(176,settingArray[presetSetting][0],IR_val[0]);
    }
  }
  IR_Pval[0] = IR_val[0];
  potPState[0] = potCState[0];
  
  if(IR_val[1] != 0){
    if(IR_val[1] != IR_Pval[1]){
      //MIDImessage(128,IR_Pval[1],127);
      MIDImessage(176,settingArray[presetSetting][1],IR_val[1]);
    }
  }
  IR_Pval[1] = IR_val[1];
  potPState[1] = potCState[1];
  delay(10);
}
//////////////////////

//////////////////////
void readMenuSettings(){
  int sensorValue = analogRead(A2);
  sensorValue = map(sensorValue, 0, 1027, 6, -1);
  if(sensorValue != pTurnPot){
    pTurnPot = sensorValue;
    menuSetting(pTurnPot);
    //Serial.println(pTurnPot);
  }
  int button = digitalRead(BUTTON_PIN);
  if(button != pButton){
    if(button == 0){
      timer = 0;
      //Serial.println("pressed");
      changeSetting();
    }
    pButton = button;
  }
  if(pButton == 0){
    timer++;
    if(timer >= 50){
      timer = timer - 2;
      changeSetting();
    }
  }
}
//////////////////////

//////////////////////
void changeSetting(){
  if(pTurnPot > 5){
    //Serial.println("pressed");
    presetSetting++;
    if(presetSetting > 5){
      presetSetting = 0;
    }
    menuSetting(pTurnPot);
  }
  if(pTurnPot <= 5){
    settingArray[presetSetting][pTurnPot]++;
    if(settingArray[presetSetting][pTurnPot] >= 65){
      settingArray[presetSetting][pTurnPot] = 0;
    }
    menuSetting(pTurnPot);
  }
}
//////////////////////

//////////////////////
void MIDImessage(byte command, byte data1, byte data2) //pass values out through standard Midi Command
{
   Serial.write(command);
   Serial.write(data1);
   Serial.write(data2);
}
//////////////////////

//////////////////////
void readPotentiometers(int i) {
  int reading[2] = {0};
  int filteredVal[2] = {0};
  int scaledVal[2] = {0};


  int IR_range[2] = {90, 530};
  int IR_min_val[2] = {0,0};
  int IR_max_val[2] = {127,127};
  
  reading[0] = analogRead(A0);
  reading[1] = analogRead(A1);// raw reading
  ave[i].push(reading[i]); // adds value to average pool
  
  filteredVal[i] = ave[i].mean();
  potCState[i] = filteredVal[i];
  
  scaledVal[i] = fscale(IR_range[0], IR_range[1], IR_min_val[i], IR_max_val[i], filteredVal[i], 1);
  byte temp_val = clipValue(scaledVal[i], IR_min_val[i], IR_max_val[i]);
  if(i == 1){
    IR_val[1] = temp_val;
  }
  if(i == 0){
    IR_val[0] = temp_val;//map(temp_val, 0, 127, 48, 59+12);
  }
}
//////////////////////

//////////////////////
float fscale( float originalMin, float originalMax, float newBegin, float newEnd, float inputValue, float curve) {
  float OriginalRange = 0;
  float NewRange = 0;
  float zeroRefCurVal = 0;
  float normalizedCurVal = 0;
  float rangedValue = 0;
  boolean invFlag = 0;
  
  if (curve > 10) curve = 10;
  if (curve < -10) curve = -10;
  curve = (curve * -.1);
  curve = pow(10, curve);


  if (inputValue < originalMin) {
    inputValue = originalMin;
  }
  if (inputValue > originalMax) {
    inputValue = originalMax;
  }
  
  OriginalRange = originalMax - originalMin;
  
  if (newEnd > newBegin) {
    NewRange = newEnd - newBegin;
  }
  else
  {
    NewRange = newBegin - newEnd;
    invFlag = 1;
  }
  zeroRefCurVal = inputValue - originalMin;
  normalizedCurVal  =  zeroRefCurVal / OriginalRange;
  if (originalMin > originalMax ) {
    return 0;
  }
  if (invFlag == 0) {
    rangedValue =  (pow(normalizedCurVal, curve) * NewRange) + newBegin;


  }
  else     // invert the ranges
  {
    rangedValue =  newBegin - (pow(normalizedCurVal, curve) * NewRange);
  }
  return rangedValue;
}
//////////////////////

//////////////////////
int clipValue(int in, int minVal, int maxVal) {
  int out;
  if (in > maxVal) {
    out = maxVal;
  }
  else if (in < minVal) {
    out = minVal;
  }
  else {
    out = in;
  }
  return out;
}
//////////////////////

//////////////////////
void menuSetting(byte userPosition){
  display.clearDisplay();
  display.setRotation(2);
  display.setTextSize(1);
  display.setTextColor(WHITE);
  display.setCursor(2,2);
  display.println("C1");
    display.setCursor(30,2);
    display.println(settingArray[presetSetting][0]);
  display.setCursor(2,16);
  display.println("C2");
    display.setCursor(30,16);
    display.println(settingArray[presetSetting][1]);
  display.setCursor(2,30);
  display.println("C3");
    display.setCursor(30,30);
    display.println(settingArray[presetSetting][2]);
  display.setCursor(50,2);
  display.println("C4");
    display.setCursor(78,2);
    display.println(settingArray[presetSetting][3]);
  display.setCursor(50,16);
  display.println("C5");
    display.setCursor(78,16);
    display.println(settingArray[presetSetting][4]);
  display.setCursor(50,30);
  display.println("C6");
    display.setCursor(78,30);
    display.println(settingArray[presetSetting][5]);
  for(int i = 0; i < 6; i++){
    display.drawRoundRect(2+i*21,43,19,19,4,1);
    display.fillRoundRect(2+presetSetting*21,43,19,19,4,1);
  }
  if(userPosition != 6){
    display.drawRect(cursorSetting[userPosition],cursorSetting[userPosition+6],44,11,1); 
  }
  if(userPosition == 6){
    display.drawRect(0,41,128,23,1);
  }
  //Serial.println(cursorSetting[userPosition]);
  //Serial.println(cursorSetting[userPosition*2]);
  display.display();
}
//////////////////////

Last step is to upload all the reciever code to the Arduino Uno, and the transmitter code to both the Arduino Nano's

And ad 9V battery to both the handheld Nano's.

(See image above)

I soldered everything to a breakable soldering board, to use as little space as posible.

Created a casing for the Midi Controller, and printed it with a lazercutter. I used a 3mm thick and 30x50cm mdf plank for this. After that it was just a tiny puzzle to put together the parts and lay them out in the controller.

Now the only thing left to do is attach the Arduino Uno to the computer, put some 9V batteries in the Nano's. And connect Hairless Midi to your DAW. Enjoy the controller.

I never worked with the arduino before, so this was a big task to figure out. But with enough online tutorials and documentation about each component, I could figure out the most stuff.

Coding:

Coding was also a bit of a challenge. The IR sensors where not that rigid with its values. So i had to smooth it out a bit. Luckaly NerdMusician had already figured this out, so I used a part of his code for that. The rest of the code was all just searching online for how certain components worked.

Soldering:

I never soldered before. Luckaly it wasnt that much that had to be soldered, so it kept it simple and organised.

Casing:

Making the casing was not that hard, since i have done it before for other projects, and this was not that much in detail.

All in all, this was an amazing project to have worked on. Love making music and loved working with Arduino to be able to create this wonderfull Midi Controller.