ESP32 Mic Testing With INMP441 and DumbDisplay

In this post, I will try to demonstrate a fairly easy way to do experiment on INMP441 module acting as mic input to ESP32 board via a I2S channel of the microcontroller.

• INMP441 module will be acting as a mic input for capturing mono 16-bit audio signals at rate 8000 samples per second.
• DumbDisplay is used as the UI. Additionally, DumbDisplay also acts as the speaker of the captured audio signals, as well as the recorder of them (in WAV format).

The UI realized by DumbDisplay provides three functions:

1. DumbDisplay app acting as a speaker that reproduces the sound signal captured and shipped to DumbDisplay app realtime wirelessly.
2. DumbDisplay app acting as a recorder of the sound signal captured and shipped to DumsDisplay app. Note that the audio file format will be WAV.
3. DumbDisplay app acting as a player for playing the sound signal recorded in WAV format.

A specific function is selected by clicking on the correcting tab -- MIC, REC, or PLAY.

Whatever the selected function be, the Start button starts the selected function; and the Stop button stops it.

The "meter" at the bottom is for setting the signal amplification (simple software-based multiplying the signals by some "amplification factor"). The possible range is from 1 to 20. And 10 is the initial setting.

Yes, there is also a "plotter" at the top. It is intended to show the sound wave captured from INMP441; however, it is apparent that wave form shown is very rough, much rougher than wanted :-(

Here are the needed connections between ESP32 and INMP441:

• connect ESP32 3.3V to VDD of INMP441
• connect ESP32 GND to GND and L/R of INMP441 (connecting L/R to GND means using a single I2S for capturing mono sound)
• connect ESP32 GPIO25 to WS of INMP441
• connect ESP32 GPIO33 to SD of INMP441
• connect ESP32 GPIO32 to SCK of INMP441

Note that the picture shows the back of a normal pre-solider INMP441 board. As a matter of fact, the mic input is really on the back as shown.

In order to be able to compile and run the sketch shown here, you will first need to install the DumbDisplay Arduino library. Open your Arduino IDE; go to the menu item Tools | Manage Libraries, and type "dumbdisplay" in the search box there.

On the other side -- your Android phone side -- you will need to install the DumbDisplay Android app.

You can download the sketch here.

If your actual wiring between ESP32 and INMP441 is different from stated in above section, please modify the sketch where the wiring mappings are defined.

// INMP441 I2S pin assignment
#define I2S_WS 25
#define I2S_SD 33
#define I2S_SCK 32

As hinted previously, the sketch will be using DumbDisplay as UI of the experiment in this post. Even though you can choose to use ESP32's WIFI to make connection with DumbDisplay app, it is strongly recommended that Bluetooth be used. Define the Bluetooth name with the BLUETOOTH macro, like

#define BLUETOOTH "ESP32BT"
#if defined(BLUETOOTH)
  #include "esp32dumbdisplay.h"
  DumbDisplay dumbdisplay(new DDBluetoothSerialIO(BLUETOOTH));
#else
  #include "wifidumbdisplay.h"
  DumbDisplay dumbdisplay(new DDWiFiServerIO(WIFI_SSID, WIFI_PASSWORD));
#endif

If you want to try out WIFI connectivity, please comment out the line that defines USE_BLUETOOTH, and specify WIFI_SSID and WIFI_PASSWORD.

The name of the recorded audio WAV file will always be recorded_sound.wav.

const char* SoundName = "recorded_sound"

Note that only a single name is used; and hence, new recording will always overwrite old one. BTW, since files will be read/write from your phone's storage, permission is needed. Please grant media access from the settings page of your DumbDisplay app installation.

Now, let's examine some core areas of the sketch.

I2S is initialized the standard way, like in the setup block:

  // set up I2S
  i2s_install();
  i2s_setpin();
  i2s_start(I2S_PORT);

In fact, I copied those routines from resources easily found on the Internet.

Notice that only a single channel I2S_PORT is used for capturing mono sound. And the audio samples are read from the I2S channel, like in the loop block:

  // read I2S data and place in data buffer
  size_t bytesRead = 0;
  esp_err_t result = i2s_read(I2S_PORT, &StreamBuffer, StreamBufferNumBytes, &bytesRead, portMAX_DELAY);
 
  int samplesRead = 0;
  if (result == ESP_OK) {
    samplesRead = bytesRead / 2;  // 16 bit per sample
    ...
  }

And those audio samples read are streamed to DumbDisplay app like

      ...
      dumbdisplay.sendSoundChunk16(soundChunkId, StreamBuffer, samplesRead, isFinalChunk);
      ...

Yes, audio samples are sent to DumbDisplay app in chunks (and hence a soundChunkId is needed), and you get such soundChunkId by properly initiating sound streaming like

      ...
      if (what == 1) {
        // start streaming sound, and get the assigned "chunk id"
        soundChunkId = dumbdisplay.streamSound16(SoundSampleRate, SoundNumChannels); // sound is 16 bits per sample
        dumbdisplay.writeComment(String("STARTED mic streaming with chunk id [") + soundChunkId + "]");
      } else if (what == 2) {
        // started saving sound, and get the assigned "chunk id"
        soundChunkId = dumbdisplay.saveSoundChunked16(SoundName, SoundSampleRate, SoundNumChannels);
        dumbdisplay.writeComment(String("STARTED record streaming with chunk id [") + soundChunkId + "]");
      }
      ...

As shown above, a dumbdisplay global object is declared at the beginning of the sketch

  // ESP32 Bluetooth with name  ESP32
  #include "esp32dumbdisplay.h"
  DumbDisplay dumbdisplay(new DDBluetoothSerialIO("ESP32"));  

or

  // ESP32 WiFi
  #include "wifidumbdisplay.h"
  DumbDisplay dumbdisplay(new DDWiFiServerIO(WIFI_SSID, WIFI_PASSWORD));

Depending on the way of making connection with DumbDisplay app, different type of "SerialIO" will be passed as an argument when constructing the DumbDisplay object.

The different UI DumbDisplay layers will normally be declared global variables

PlotterDDLayer* plotterLayer;
LcdDDLayer* micTabLayer;
LcdDDLayer* recTabLayer;
LcdDDLayer* playTabLayer;
LcdDDLayer* startBtnLayer;
LcdDDLayer* stopBtnLayer;
LcdDDLayer* amplifyLblLayer;
LedGridDDLayer* amplifyMeterLayer;

And those layers will be created in the setup block, like

  ...
  micTabLayer = dumbdisplay.createLcdLayer(8, 1);
  micTabLayer->writeCenteredLine("MIC");
  micTabLayer->border(1, "gray");
  micTabLayer->enableFeedback("f");
  ...
  amplifyMeterLayer = dumbdisplay.createLedGridLayer(MaxAmplifyFactor, 1, 1, 2);
  amplifyMeterLayer->onColor("darkblue");
  amplifyMeterLayer->offColor("lightgray");
  amplifyMeterLayer->border(0.2, "blue");
  amplifyMeterLayer->enableFeedback("fa:rpt50");  // rep50 means auto repeat every 50 milli-seconds

The different layers are laid out like

  DDAutoPinConfigBuilder<1> builder('V');  // vertical
  builder
    .addLayer(plotterLayer)
    .beginGroup('H')  // horizontal
      .addLayer(micTabLayer)
      .addLayer(recTabLayer)
      .addLayer(playTabLayer)
    .endGroup()
    .beginGroup('H')  // horizontal
      .addLayer(startBtnLayer)
      .addLayer(stopBtnLayer)
    .endGroup()
    .beginGroup('S')  // stacked, one on top of another
      .addLayer(amplifyLblLayer)  
      .addLayer(amplifyMeterLayer)
    .endGroup();  
  dumbdisplay.configAutoPin(builder.build());

Note that in the sketch, the "layer creation and layout" is enclosed with "record / playback" layout commands

  dumbdisplay.recordLayerSetupCommands();  // start recording the layout commands
  ...
  dumbdisplay.playbackLayerSetupCommands("esp32ddmice");  // playback the stored layout commands, as well as persist the layout to phone, so that can reconnect

The use of "record / playback" layout commands is not absolutely necessary. Nevertheless, the use of the pair of commands will allow DumbDisplay app to meaningfully reconnect to the running ESP32 sketch.

• Even though the states of the different layers are not persisted, and will be gone when once disconnected.
• The layer "creation and layout" is re-play automatically upon reconnection.

The DD connection version (reconnection) is tracked with the help of DDConnectVersionTracker. And it is declared and used like

DDConnectVersionTracker cvTracker(-1);  // it is for tracking [new] DD connection established 
...
 if (cvTracker.checkChanged(dumbdisplay)) {
   ...
 }
...

If DD connection version is changed (i.e. a fresh connection), better update the UI components, since all the layers will be just like freshly recreated.

Simply checking whether a layer has "feedback" (e.g. clicking) can be like

    if (micTabLayer->getFeedback()) {
      what = 1;
    } else if (recTabLayer->getFeedback()) {
      what = 2;
    } else if (playTabLayer->getFeedback()) {
      what = 3;
    }

Nevertheless, more detailed "feedback" checking will be like

    const DDFeedback* feedback = amplifyMeterLayer->getFeedback();
    if (feedback != NULL) {
        amplifyFactor = feedback->x + 1;
        updateAmplifyFactor = true;
    }

Here, it is checked to see if any "feedback" from the amplifyMeterLayer -- feedback. If it is not NULL, the X position of the layer where the click was, is treated as the "amplification factor".

One more thing. The sketch will automatically "stop" once it is detected "idle" (i.e. disconnected from DumbDisplay app). And here is the needed code in the setup block.

  // set when DD idle handler ... here is a lambda expression
  dumbdisplay.setIdleCalback([](long idleForMillis) {
    started = false;  // if idle, e.g. disconnected, stop whatever
  });

Note that

[](long idleForMillis) { ... }

is a C++ lambda expression -- an in-place function; and therefore can actually be implemented with a regular callback function.

A reminder: Turn off the "show commands" options of DumbDisplay app, so that the sound capturing qualify can be better.

Even with "show commands" turned off, I have to admit that the effect of the sketch shown here is somewhat off what is desired :-(

And here are several "parameters" that you may be interested in tuning:

• Setting SoundSampleRate to something other than 8000. Note that 8000 is the usual sample rate for voice. You may try setting it to 44100, but I seriously doubt that it will work for the sketch.
• Setting StreamBufferNumBytes to something other than 256. The smaller the value is, the lesser samples to ship to DumbDisplay app each chunk. However, the smaller the value is, the higher is also the "overhead".
• Setting I2S_DMA_BUF_LEN (# samples) to something other than 1024. Note that I2S makes use of DMA buffers for storing real-time audio signal samples. After each DMA buffer is filled up, ESP32 is "notified" of the fact, making the data available for reading. Effectively, the bigger the value is, the higher the lag. (In fact, the actual audio lag of this sketch is not from the DMA buffer, but from latency sending samples from ESP32 to DumbDisplay app.
• Setting I2S_DMA_BUF_COUNT to something other than 8. Note that I2S_DMA_BUF_COUNT is the number of DMA buffers that I2S can use. Normally, 2 should be good enough -- one used for buffering; one used by ESP32 for reading -- nevertheless, it is hard to ensure the responsiveness of the code run by ESP32; hence it makes sense to set it to something higher.

Anyway, I still wish that you will find this to be a little fun ESP32 mic testing experiment.

Hopefully, I will try to combine this and the concept of my previous post -- Yes! Voice Recognization Experiment With Wit.ai -- to come up with a more interesting application. Until then, enjoy!

Peace be with you! May God bless you! Jesus loves you!