Arduino Piano Player Robot

Update on 20220704:

This project seems good

https://brandonswitzer.squarespace.com/player-pian...

https://github.com/bbswitzer/PianoProject

https://drive.google.com/file/d/1uUG9_p_ZxxOJA7IN5...

For step 4 we can use code from:

3d-piano-player

https://github.com/reality3d/3d-piano-player

do some editing to output in real time.

-------------------------------------------------------------------

When I listen to relatively complex piano song such as "The Great Gate of Kiev" or Chopin's Etude Op 25 No 12 in C minor, I always wondering if we can let the robot do the job for us, since it require lots of training time for human.

After some search, I got an idea of using the solenoids as key presser to play the piano with Arduino

Since I am a student with limited budget,I decide not buy all 88 solenoids, I brought 6 to test.

Arduino nano (or Arduino Mega)

5V relays

Solenoids (in this example JF 0730Bs,24V)) This one is 5N, which barely push the key even with higher than rating current. You can try 10N/20N solenoids, but they will be more expensive

10 ohm resistor*2

24V voltage regulator

First we make midi from

https://onlinesequencer.net/

(or make using Musescore)

Then we use

http://flashmusicgames.com/midi/mid2txt.php

(In this example I made Midi for Twinkle Twinkle little star)

get text version of MIDI which is:

----------------------------------------------

MFile 1 2 384

MTrk

0 TimeSig 4/4 24 8

0 Tempo 2000000

0 Meta TrkEnd

TrkEnd

MTrk

0 Meta TrkName "Grand Piano"

0 PrCh ch=1 p=0

0 On ch=1 n=60 v=50

96 Off ch=1 n=60 v=0

96 On ch=1 n=60 v=50

192 Off ch=1 n=60 v=0

192 On ch=1 n=67 v=50

288 Off ch=1 n=67 v=0

288 On ch=1 n=67 v=50

384 Off ch=1 n=67 v=0

384 On ch=1 n=69 v=50

480 Off ch=1 n=69 v=0

480 On ch=1 n=69 v=50

576 Off ch=1 n=69 v=0

576 On ch=1 n=67 v=50

672 Off ch=1 n=67 v=0

768 On ch=1 n=65 v=50

864 Off ch=1 n=65 v=0

864 On ch=1 n=65 v=50

960 Off ch=1 n=65 v=0

960 On ch=1 n=64 v=50

1056 Off ch=1 n=64 v=0

1056 On ch=1 n=64 v=50

1152 Off ch=1 n=64 v=0

1152 On ch=1 n=62 v=50

1248 Off ch=1 n=62 v=0

1248 On ch=1 n=62 v=50

1344 Off ch=1 n=62 v=0

1344 On ch=1 n=60 v=50

1440 Off ch=1 n=60 v=0

1440 Meta TrkEnd

TrkEnd

----------------------------------------------

The meaning

ch: MIDI channel (different instrument have different channel) we only have piano, so only one channel

on/off: on means the key is pressed, off means the key is unpressed

n: note

C4　　60

D4　　62

E4　　64

F4　　65　　note there is no note between E and F, so the count interval is 1

G4　　67

V: velocity (in micro second?)

Tempo: (microsecond per quarter note)

More see:

https://music.stackexchange.com/questions/41419/un...

Due to unknow reason, the first note of the song won’t be played, so we need to copy and paste the first note before the first note

(in our case

0 On ch=1 n=60 v=50

96 Off ch=1 n=60 v=0)

the changed one see the Python code.

The note here we got is two digit, but Arduino can only get one at once if we use Serial.read()

so we need study two digit input

Here we want play in range 60 to 69 of n from the MiDi file, so we need find how to maintain two digit input for Arduino

The below code can do (it take the two digit into an array, then convert it into an integer)

(it can only take 2 digit, cannot take a single digit input)

----------------------------------------------

//code take two digit

char datafrompython[2];

byte number=0;

void setup()

{

Serial.begin(9600);

}

void loop()

{

Serial.println("enter");

if (Serial.available() > 0)

{

Serial.readBytesUntil('\n', datafrompython, 2);

number = atoi(datafrompython);

Serial.println(number);

}

delay (1000);

}

----------------------------------------------

Serial.readBytesUntil() more see

https://www.arduino.cc/reference/en/language/funct...

//on the graph, the battery is 24VDC 8A power source (Since Tinkercad don't have other power source), the resistor are 10 each ohm each (To get about 1A current to power)

The motor are linear motor 24VDC, 300 mA (but when you actually give it 300 mA it won’t work, need higher current like 1A to press the key)

When pin 7 goes HIGH, the relay will be triggered, then the motor can move

then we connect the rest of the linear motors like the diagram except for the one pin that triggers the relay from pin 4 to pin 9

We can move them one by one by using:

----------------------------------------------

void setup()

{

pinMode(4, OUTPUT); //C4

pinMode(5, OUTPUT); //D4

pinMode(6, OUTPUT); //E4

pinMode(7, OUTPUT); //F4

pinMode(8, OUTPUT); //G4

pinMode(9, OUTPUT); //A4

}

void loop()

{

digitalWrite(4, HIGH); //C4

delay (1000);

digitalWrite(4, LOW);

digitalWrite(5, HIGH); //D4

delay (1000);

digitalWrite(5, LOW);

digitalWrite(6, HIGH); //E4

delay (1000);

digitalWrite(6, LOW);

digitalWrite(7, HIGH); //F4

delay (1000);

digitalWrite(7, LOW);

digitalWrite(8, HIGH); //G4

delay (1000);

digitalWrite(8, LOW);

digitalWrite(9, HIGH); //A4
delay (1000);

digitalWrite(9, LOW);

}

----------------------------------------------

The main reason we put MiDi text in the Python because if we put in Arduino, when we change song, we have to upload again. while Python we just have to change the code and run it again.

First you need make sure your Python can communicate with Arduino with Serial.

more see

https://create.arduino.cc/projecthub/Jalal_Mansoor...

If you can, then we can proceed.

----------------------------------------------

Python code explanation

first we import time and serial for stop time of notes and communicate with Arduino

Then we have a string called miditextstring which stores the midi text

then we use enumerate for both the character and the index

we look for "n=" in the midi text and we restore the int from string after "n=", the use it as a note

then we write it as Bytes literals to be able to do serial communication, such as

if note==62:

arduino.write(b"62")

first time a number appear means on (key pressed down), the second time it appear means off(key press up)

----------------------------------------------

----------------------------------------------

Arduino code explanation

we are entering two digit, so need use array

then we create int for each note to be a state of pressed or not

then for each note:

first time read a note turns it on (since in Midi text the pattern is like this), second time reads it turns it off

for example:

//For C4:

if(number == 60 && C4==0) //first time read a note turns it on (since in Midi text the pattern is like this)

{

digitalWrite( 4 , HIGH );

C4++;

number=0; //to avoid recounting of the same note

}

else if(number == 60 && C4==1) //second time reads it turns it off

{

digitalWrite( 4 , LOW);

C4=0;

number=0; //to avoid recounting of the same note

}

//no delay because piano require precise note lengt

----------------------------------------------

After you upload the Arduino code, first change the COM

inside python code to your own, then you put your MIDI text inside the string, make sure to copy and paste the first note

Then if you run Python with Arduino serial monitor closed, the linear motors should move

When you try on piano, note that the solenoid's starting force is very weak, it cannot push down piano key without space, we need give it some space.

For the hand held video, I couldn't keep it on perfect distance so it can't press the key perfectly, just a demonstration.

1.My summer break is about to end, I don't have time to do the improvement and mount it on piano and do other improvement. (like use 2 Arduino Mega and buy 88 solenoids)

2.The current player cannot play multiple nodes at the same time.

To play multiple notes in the same time, the principle I am thinking is when MTrk are equal, send the notes around the same time to begin.

For example

0 On ch=1 n=60 v=50

0 On ch=1 n=67 v=50

For the above, since 0 is equal so we sent n=60 and n=67 to Arduino in the same time (when it appears again it turns off)

more see

https://stackoverflow.com/questions/63386584/check...

MIDI sheet music application may also be a good place to start with, it shows the keyboard (and which note is playing) while playing the MIDI music, it would be convenient to be able to use it directly with the solenoids

3. The current solenoid's force seems bit weak.

I think if there are solenoid that could push the key without giving space between it and the key, we would get a better result.