Raspberry Pi Midi Synth

The goal of my project was to build a very simple digital synthesizer which is able to be controlled by a midi keyboard.

Note that this project is a prototype and is mostly about learning how things work rather than creating a good synthesizer. There is better software for controlling a sound via a midi controller which I will talk about at the end.

This article is a part of an assignment for Deakin University, School of IT, Unit SIT210 - Embedded Systems Development.

The hardware components needed for this project are:

• Raspberry Pi
• MIDI keyboard controller
• Raspiaudio AUDIO+ DAC (digital analog converter)
• Speakers

Setting up the hardware is a simple task.

Firstly the Raspberry Pi must be powered on.

The AUDIO+ can simply be connected to the raspberry pi via the GPIO pins. The speakers should be plugged into the audio jack. The AUDIO+ can then be configured by running the following bash script in the terminal:

sudo wget -O - script.raspiaudio.com | bash

This will require a reboot of the raspberry pi. After rebooting the pi we can verify that the AUDIO+ is working by running the following command:

sudo speaker-test -l5 -c2 -t wav

This will (hopefully) play sound through the speakers if everything has gone correctly. The midi controller can be connected via USB. We can see the input and output devices by using the command:

 aconnect -i -o

The output should look something like this:

client 14: 'Midi Through' [type=kernel]
	0 'Midi Through Port-0'
client 20: 'MPKmini2' [type=kernel,card=1]
	0 'MPKmini2 MIDI 1 '

In order to use the midi device with the raspberry pi, the midi input device (20) must be connected with the midi through port (14). This can be done with the following command:

aconnect 20:0 14:0

We will need some software to output sound depending on our MIDI inputs. Creating a synthesizer in Python is maybe not the best idea and there are probably much better ways of generating an output signal, however it is a good way of easily seeing how the code works.

This section will be heavily referencing https://python.plainenglish.io/making-a-synth-with-python-oscillators-2cb8e68e9c3b. I will attempt to simplify what is said in that article, and provide a simple version of the code used.

Before we create a new file we will need to install some python libraries for handling audio output and midi input. Hopefully numpy should already be installed so I will not provide instructions for installing numpy.

Firstly, make sure the package repository is up to date by running:

sudo apt-get update

Then the Python 3 version of pip is installed:

sudo apt install python3-pip

Now install the required packages:

python3 -m pip install -U pygame –user
sudo apt install python3-pyaudio

Make sure that pygame is at least version 2.0.1 and PyAudio is at least version 0.2.11. After these requirements are met we can now create a new python file.

sudo nano midi_synth.py

This will open up GNU nano, an text editor that we can use to write the code. Firstly we should include the libraries we will need.

from pygame import midi
import math
import itertools
import pyaudio
import numpy as np

Then we must intialise PyAudio and set up the output stream:

p = pyaudio.PyAudio()
stream = p.open(
	rate=4000,
	channels=1,
	format=pyaudio.paInt8,
	output=True,
	frames_per_buffer=256
	)

The sample rate (rate) is the rate at which samples are taken from the wave form. Since computers cannot store infinite values, the waveform must be approximated. Higher sample rates are more accurate to the actual waveform, but take up more processing power and memory. Due to the limitations of the raspberry pi and python, I have chosen to use a samplerate of 4000, which would usually be considered as very low quality. This does not sound as good but it helps reduce the latency.

The channels is the number of audio channels. We will only need 1.

The format is the format of each sample that will be written to the stream. We will set it to a signed 8 bit integer.

Output is true as we will be writing to the stream and not reading.

Frames per buffer tells the stream object the number of samples we will be feeding it at a time. This is very important as it is a tradeoff between latency and computer processing power and memory. If we set a low frames per buffer such as 64 then our raspberry pi will not be able to handle it. We will likely experience the error ALSA lib pcm.c:8424:(snd_pcm_recover) underrun occurred indicating that the CPU is unable to handle processing the sound. However, if we are to set a higher buffer size such as 1024 we will see a lot of latency in our audio signal. I have found that 256 frames per buffer is reasonable.

Next we must intialise pygame.midi and obtain a reference to the midi device. This can be done like so.

midi.init()
default_id = midi.get_default_input_id()
midi_input = midi.Input(device_id=default_id)

The midi input object will have two important functions. Read and poll.

Read will return a midi event in the form of a list. [[status, note, velocity, data3], timestamp]. Each of these data points (except for the timestamp) is a byte. Status can have two values, 0x90 meaning the key is pressed down, and 0x80 meaning the key has been released. The note is the midi note number of the key that was pressed. The velocity is how hard the key was pressed. For my midi controller, data3 is unused.

Poll will simply return true if there is an event to be read from the midi controller. Otherwise it returns false.

Sound is generated by waves traveling through the air, creating vibrations. The simplest kind of wave that we can synthesize is a sine wave. A sine wave can be created using the following formula:

step size = (2 * pi * frequency) / sample rate

If we write the step size over time we will get a sample of a sine wave.

The frequency of the wave will change the pitch of the sound. As mentioned previously the sample rate we will use is 4000.

Using python generators and iterators we can generate an infinite stream of integers which can be fed to the output stream.

def get_sin_oscillator(freq, sample_rate):
	increment = (2 * math.pi * freq) / sample_rate
	return (math.sin(v) for v in itertools.count(start=0, step=increment))

This function returns a generator. To evaluate the generator, we will need another function to return an array of samples.

def get_samples(notes_dict, num_samples=256):
	return [sum([int(next(osc) * 127) for _, osc in notes_dict.items()]) \
		for _ in range(num_samples)]

Now we can create a sine wave based on our midi input events and write the samples from the sine wave into the output audio stream. If everything is done correctly, we will get an audio output.

We will use a python dict to keep track of the notes that are pressed down.Basically:

Note is played (status=0x90)

• Sine oscillator of the correct frequency is created and added to a dict.
• Values from all the oscillators in the dict are added and returned in a buffer of a given size.
• Values in the buffer are written to stream.

Note is released (status = 0x80)

• Oscillator is removed from the dict.

The frequency of the note will depend on the note number. Converting from midi note number (d) to frequency (f) is given by the following formula.

f = (2^((d - 69) / 12)) * 440

Luckily, newer versions of pygame.midi provide a function to do this for us.

freq = midi.midi_to_frequency(note)

If there are notes in the note dict, we can write them to the output stream like so.

samples = get_samples(notes_dict)
samples = np.int8(samples).tobytes()
stream.write(samples)

The full code can be seen here:

from pygame import midi
import math
import itertools
import pyaudio
import numpy as np

SAMPLE_RATE = 4000
FRAMES_PER_BUFFER = 256
NOTE_AMP = 0.1

def get_sin_oscillator(freq=55, amp=1, sample_rate=SAMPLE_RATE):
	increment = (2 * math.pi * freq) / sample_rate
	return (math.sin(v) * amp * NOTE_AMP \
		for v in itertools.count(start=0, step=increment))

def get_samples(notes_dict, num_samples=FRAMES_PER_BUFFER):
	return [sum([int(next(osc) * 127) \
		for _, osc in notes_dict.items()]) \
		for _ in range(num_samples)]

midi.init()
midi_input = midi.Input(device_id=midi.get_default_input_id())

p = pyaudio.PyAudio()
stream = p.open(rate=SAMPLE_RATE, channels=1, format=pyaudio.paInt8, output=True, frames_per_buffer=FRAMES_PER_BUFFER)

try:
	notes_dict = {}
	while True:
		if notes_dict:
			samples = get_samples(notes_dict)
			samples = np.int8(samples).tobytes()
			stream.write(samples)

		if midi_input.poll():
			for event in midi_input.read(num_events=8):
				(status, note, vel, _), _  = event
				print(event)
				if status == 0x80 and note in notes_dict:
					del notes_dict[note]
				elif status == 0x90 and note not in notes_dict:
					freq = midi.midi_to_frequency(note)
					notes_dict[note] = get_sin_oscillator(freq=freq, amp=vel/127)

except KeyboardInterrupt:
	stream.stop_stream()
	stream.close()
	midi_input.close()
	p.terminate()

Finally we have some sort of working synth! We can run the program by typing the command:

python3 midi_synth.py