HackerBox 0079: Audio DSP

Welcome to HackerBox 0079. Let's explore audio signal generation, inter-IC sound (I2S) audio streams, and digital signal processing (DSP). Configure an ESP32-A1S Audio Development Kit featuring a dual-core ESP32 microcontroller coupled to an integrated audio CODEC and a variety of audio interface and peripheral components. Program several audio processing examples leveraging an advanced stream-based hardware abstraction layer for the ESP32-A1S. Examples include wave generators, simple synthesizers, MP3 decoders for online stream sources as well as MP3 files on SD flash cards. Use GNU Octave (or MATLAB) to design and test digital FIR filters that can be implemented on the ESP32 Audio Development Kit. Hack multi-channel audio connectors.

HackerBoxes are the original monthly subscription for electronics, computer technology, and hacker culture. Each HackerBox is a discovery box, which means all members await and enjoy a new surprise each month. Tech, toys, knowledge, and fun. It's like having a hacker convention, your birthday, and the first day of school - every month - right in your mailbox.

There is a wealth of information for current and prospective members in the HackerBoxes FAQ. Almost all of the non-technical support emails that we receive are already answered there, so we'd really appreciate it if you can take a few minutes to read the FAQ.

This Instructable contains information for getting started with HackerBox 0079. The full box contents are listed on the product page for HackerBox 0079 where the box is also available for purchase while supplies last. If you would like to automatically receive a HackerBox like this right in your mailbox each month with a $15 discount, you can subscribe at HackerBoxes.com and join the party!

A soldering iron, solder, and basic soldering tools are generally needed to work on the monthly HackerBox. A computer for running software tools is also required. Have a look at the HackerBox Workshops for basic tools and a wide array of introductory activities and experiments.

The most import thing you will need is a sense of adventure, hacker spirit, patience, and curiosity. Building and experimenting with electronics, while very rewarding, can be tricky, challenging, and even frustrating at times. The goal is progress, not perfection. When you persist and enjoy the adventure, a great deal of satisfaction can be derived from this hobby. Take each step slowly, mind the details, and don't be afraid to ask for help.

Digital Signal Processing (DSP) is the use of digital computation to perform signal processing operations. The digital signals processed in this manner are a sequence of numbers that represent samples in time from an analog audio signal. The samples are obtained through successive analog to digital conversions to generate the sequence of numbers. The rate at which the samples are collected is called the sampling rate or sampling frequency. After computational digital processing, the resultant sequence of numbers can be converted back into the analog domain using digital to analog conversion.

If you are interested to deep-dive into the magic of DSP, here are some very nice online resources:

DSP Video Series - Radke's ECSE-4530 Course (Rensselaer Polytechnic Institute)

DSP First Online Resources (Georgia Institute of Technology)

Digital Signal Processing - (MIT Open Courseware)

Free Online Book - The Scientist and Engineer's Guide to Digital Signal Processing

A Beginner's Guide to Digital Signal Processing from Analog Devices, Inc.

DSP experts (including most of those above) usually provide explanations and examples in MATLAB. Unless you have a license to MATLAB at work or school, you're going to want to use GNU Octave. Octave is free software under the terms of the GNU General Public License.

You can download and install GNU Octave for any OS.

Or you can simply use Octave Online through your web browser.

There are a lot of online resources specific to using GNU Octave:

Video - Learn Audio DSP 1: Octave and Making Sine Oscillator

Video - Learn Audio DSP 2: Waveforms and Sampling

Wikibooks Octave Programming

The ESP32- A1S (datasheet) is an ultra-small, powerful multifunction audio module, the main chip uses a low-power dual-core 32-bit CPU, the main frequency can up to 240MHZ, and the operating capability up to 600 DMIPS. It has a built-in CODEC audio decoding chip, supports for local music playback. The ESP32-A1S module supports a variety of peripherals, you can easily access Amazon or Baidu voice recognition services, it supports varieties of mainstream compression and lossless audio formats, including M4A, AAC, FLAC, OGG, OPUS, MP3,etc.

CODECS

Versions of the ESP32-A1S may incorporate different CODEC chips and these might even have different interface pin definitions. Fortunately, the Arduino AudioKit HAL (presented below) offers at least seven different configuration profiles to select from.

The two main categories of CODECs are the AC101 and the ES8388. The version we are working with is the ES8388, but either can be selected within the library configuration files.

The ESP32-A1S Audio Development Kit features an ESP32-A1S module along with many useful interfaces and peripheral support components.

FEATURES:

• Low-power dual-core 32-bit CPU for application processors
• Main frequency up to 240MHz, computing power up to 600 DMIPS
• Built-in 520 KB SRAM, 8M PSRAM
• Supports interfaces such as UART/SPI/I2C/PWM
• Headphone port 3.5mm stereo headphone jack
• Left and right channel speakers: 4Ohm 3W speakers
• Two analog microphones and 3.5mm line-in
• Lithium battery support: 3.7V with simultaneous battery charging
• MicroSD card (read and write) up to 64GB

Ai-Thinker Information Page

SET UP:

Solder two of the JST-XH 2-pin wire assemblies to the two speakers, respectively. Solder the red wire to the "+" terminal of the speaker. Once wired up, plug the speaker wires into the ROUT and LOUT JST ports of the Audio Development Kit.

The Audio Development Kit includes a five position DIP switch that can flexibly re-assign I/O pins 13 and 15. It should be set to 11100 by default. It is fine to just leave it set that way unless using the JTAG interface.

You will need to install the Arduino IDE Software (if you don't already have it).

You will also need to install the ESP32 Board Support Package (if you don't already have it).

Plug the Audio Development Kit to your PC using the microUSB port labeled UART.

Select: Tools > Board > ESP32 Arduino > ESP32 Dev Module

Under: Tools > Port >

select whichever COM port appears when the development is connected.

TEST BLINKING LEDS (Digital I/O Output)

Download the attached file ESP32_Audio_Dev_blink.ino and upload to the Audio Kit.

TEST READING KEY PRESSES (Digital I/O Input)

Download the attached file ESP32_Audio_Dev_keys.ino and upload to the Audio Kit.

Browse over to the page for the Arduino AudioKit HAL.

This library provides a Hardware Abstraction Layer (HAL) for various ESP32-based audio platforms.

Download the library and put it into your Arduino Libraries folder.

If manually installing libraries is new to you, you may want to review this guide.

Configure the correct Audio Kit Platform by editing the header file:

libraries/arduino-audiokit/src/AudioKitSettings.h

For the Ai Thinker Dev Kit we are using, configuration FIVE works best, so make this change:

#define AUDIOKIT_BOARD 5

Also add a line right after that with:

#define AI_THINKER_ES8388_VOLUME_HACK 1

This addition is required to make the headphone jack work, but in some instances it makes the output on the speakers a little too quiet. Notes from the author of the library make it seem that this hack is a bit of a work in progress so let's keep an eye on it over the coming months. Feel free to comment it in and out to experiment as needed.

AUDIO OUTPUT

Open and upload this example sketch:

File > Examples > audiokit > output  

It defaults to generating a 1000Hz (= 1KHz) tone. Try some other frequencies.

AUDIO INPUT

Open this example sketch:

File > Examples > audiokit > input  

change the line:

cfg.adc_input = AUDIO_HAL_ADC_INPUT_LINE1;

to read:

cfg.adc_input = AUDIO_HAL_ADC_INPUT_LINE2;

to configure the input to read from the onboard microphones.

Upload the sketch to the Audio Dev Kit board.

Open: Tools > Serial Plotter

While watching the plotter output, whistle over the board.

Now try whistling from one side of the board and then from the other side.

The color of the larger amplitude signal on the plotter should change as the microphone from one side receives more of the audio energy and then switches to the other microphone as you move your head to the other side of the board.

Browse over to the page for the Arduino Audio Tools

Download the library and put it into your Arduino Libraries folder.

This library provides the glue that connector different audio processing components and libraries together. It also provides several example scenarios that we can play around with. Much like other Arduino Streams that we use to read input from or print output to, these “Audio Streams” allow us to read audio signal data from “Audio Sources” and write audio signal data to “Audio Sinks”.

The library's audio streams leverage I2S (Inter-IC Sound), which is an electrical serial bus interface standard used for connecting digital audio devices together. I2S is used to communicate PCM audio data between integrated circuits in an electronic device, or in this case between SOURCES, PROCESSORS, and SINKS.

Open and upload this example sketch:

File > Examples > audio-tools > examples-audiokit > streams-generator-audiokit

This example generates an I2S stream containing a sine wave and then copies the generated stream to the Audio Development Kit's CODEC. The audio output can be heard on the speakers or headphones.

To make the output a little louder, you can add:

out.setVolume(80); 

right after:

out.begin(config);

Open and upload this example sketch:

File > Examples > audio-tools > examples-audiokit > streams-synthbasic1-audiokit

Test the output by hitting the KEY1-KEY6 buttons.

You will hear some different sounds but they will probably be choppy and noisy.

This is because the AudioLogger is set to a very verbose mode so there is a lot of time spent serving the serial output and that disrupts the audio processing. To fix this, find the logger configuration line in the setup function:

AudioLogger::instance().begin(Serial,AudioLogger::Info)

and then change "Info" to "Warning" which will send a lot less information to the serial monitor, leaving more resources for processing the audio data.

You may also want to bump kit.setVolume(80); up to 100.

Blog Entry: Building a Simple Synthesizer with the AudioTools Library

Download and install this Helix MP3 Decoder Library.

Open and upload this example sketch:

File > Examples > audio-tools > examples-audiokit > streams-url_mp3-audiokit

Replace "ssid" and "password" with those for an available 2.4GHz WiFi network

To bump up the volume, insert  i2s.setVolume(80);  just after i2s.begin(config);

To stream DEF CON Radio on SomaFM, just replace the URL with "ice6.somafm.com/defcon-128-mp3"

Copy some MP3 files onto a MicroSD Card and insert the card into the Audio Development Kit.

Use Tools > Manage Libraries to search and install the "SdFat" Library by Bill Greiman

Open and upload this example sketch:

File > Examples > audio-tools > examples-audiokit > player-sd-audiokit

KEY3 and KEY4 will jump to the previous and next track, respectively.

The simplest filtering is frequency selection. For example, low-pass filters pass low frequencies and attenuate (mostly block) higher frequencies. The frequency where the pass band transitions to the stop band is called the cut-off frequency.

More complex filters are used in communication systems, radar, audio processing, and so forth. Audio is a one-dimensional signal, but you've probably worked with two-dimensional filtering in a photo editing program.

Open this example audio filter sketch:

File > Examples > audio-tools > examples-audiokit > streams-audiokit-filter-audiokit

Edit the following line to change the sampling rate from 44100 to 8000:

uint16_t sample_rate=44100;

Comment out the line that starts with AudioLogger.

Comment out the following line:

float coef[] = {0.021, 0.096, 0.146, 0.096, 0.021};

and add this line right below it:

float coef[] = {1, 0, 0, 0, 0};

Run this modified sketch and the Audio Dev Kit should echo any sound you make into the onboard microphone back to you. The playback should be very close to the sound you made across all frequencies because the FIR filter with coefficients [1 0 0 0 0] is an "all pass" or "flat pass" filter. (It doesn't change anything).

CHANGE THE ALL-PASS FILTER TO A HIGH-PASS FILTER

Comment out the line:

float coef[] = { 1, 0, 0, 0, 0};

and add this line right below it:

float coef[] = {-5.0601e-03,1.6287e-04,4.1943e-02,-6.6191e-04,-2.8849e-01,4.9778e-01,-2.8849e-01,-6.6191e-04,4.1943e-02,1.6287e-04,-5.0601e-03};

Run this updated sketch on the Audio Dev Kit. Again, sounds made into the onboard microphone will be echoed back, but now the playback is not the same as the original input sound. For example try speaking with a very low, deep voice (think Barry White) and then try speaking with a very high, squeaky voice (think Pee Wee Herman). Which of those is echoed back with the most volume and clarity?

Where did we get those strange coefficient values to implement the high-pass filter in the last step?

One easy way is to use the fir1() function in Octave Online.

(FIR means Finite Impulse Response)

Try this:

> h = fir1(10, 0.5, "high")

to generate a 10th order high-pass filter with a cut-off frequency at 0.5 (half) of the bandwidth

a signal sampled at 8000 Hz has a maximum frequency content of 4000 Hz (why?)

so 50% of 4000 Hz puts the cut-off frequency around 2000Hz.

Notice how the output matches the vector of coefficients used for the sketch in the last step.

Plot the frequency response of the filter implemented by the coefficients in the vector h:

> plot((-0.5:1/512:0.5-1/512)*8000,20*log10(abs(fftshift(fft(h,512)))))  

> axis([0 4000 -60 10])

Notice how the lower frequencies are attenuated (lowered) and the higher frequencies are passed through. Also notice that the transition between the pass-band and stop-band occurs around 2000 Hz as planned.

APPLY THE FILTER TO SEPARATE TWO COMBINED SINE WAVES

Copy and paste the following code into Octave.

fs = 16000;                     % Sampling frequency (samples per second)
dt = 1/fs;                      % seconds per sample
StopTime = 0.005;               % durantion of signal in seconds
t = (0:dt:StopTime-dt)';        % sample times
wave500 = sin(2*pi*500*t);      % Sine wave with f=500Hz
wave2000 = sin(2*pi*2000*t);    % Sine wave with f=2000Hz
plot(t,wave500,'r',t,wave2000,'b');

The plot show a low frequency (500Hz) sine wave in red and a higher frequency (2000Hz) sine wave in blue.

Add these two waves into one signal (called sum) and plot it...

> sum = wave500 + wave2000;

> plot(sum);

Notice now the higher frequency wave appears to be "riding on" the lower frequency wave.

Apply the high-pass filter to the sum:

> out = filter(h, 1, sum);

> plot(out);

Notice how the high-pass filter removes the content below the cut-off (below 2KHz) and passes the signal above 2KHz.

While far less versatile, it is worth understanding that we can make a simple high-pass filter with just one capacitor and one resistor.

Here's a video about how simple analog filters work.

And here's a nice Instructable for playing around with these sorts of Passive Filter Circuits.

Faust is a functional programming language for sound synthesis and audio processing with a strong focus on the design of synthesizers, musical instruments, audio effects, etc. Faust targets high-performance signal processing applications and audio plug-ins for a variety of platforms and standards.

Browse to the Faust site and click "Try It Online!"

Having a couple 3.5mm TRRS Breakout Modules and some female-female DuPont jumpers is very useful.

• break out signal from an audio line out or headphone jack
• break out signal into an audio line in or a microphone jack
• use the "sound card" in your computer as a function generator
• use the "sound card" in your computer as an oscilloscope
• use jumper wires to reconfigure the type of connection (NoMic/CTIA/OMTP)

Video: Your PC Soundcard is an Oscilloscope, a Signal Generator, and a Spectrum Analyzer

We hope you are enjoying this month's HackerBox adventure into electronics, computer technology, and hacker culture. Reach out and share your success in the comments below or other social media. Also, remember that you can email support@hackerboxes.com anytime if you have a question or need some help.

What's Next? Join the party and live the HackLife! Get a cool box of hackable gear delivered right to your mailbox each month and enjoy a generous member discount. Surf over to HackerBoxes.com right now and sign up for your monthly HackerBox subscription.