Log High Speed ECG or Other Data, Continuously for Over a Month

This project was developed to support a university medical research team, who needed a wearable that could log 2 x ECG signals at 1000 samples/sec each (2K samples/sec total) continuously for 30 days, in order to detect arrhythmias. The project presented here includes remote control and monitoring of the logging. Remote control is via menus presented on a serial terminal, either on a computer or mobile phone. This project does not cover the ECG measurement or packaging or battery required for the final wearable.

This high speed/long duration version uses Teensy 3.2, Adafruit Micro-SD breakout module, a quality 16G SDHC class 10 SD card to log the data and a Bluetooth communications module for control and monitoring. A less developed and slower UNO/Mega2560 version of this project is also available. While this project use a Bluetooth communications module for control and monitoring of the logging, you can also use WiFi or BLE modules.

This version, based on the Teensy 3.2, is capable of much higher sampling rates than the UNO/Mega2560 version. Using this code the Teensy 3.2 can sample and log two ADC samples at >30Khz with hardware averaging over 4 samples and so easily satisfies the 1000 samples/sec requirement above. The code supports saving 100 .bin files of 128K each. At 30Khz that covers 29hrs 30min. At 1000 samples/sec it covers 37 days. The code can easily be extended to handle more than 100 files, so extending the duration of the run. At the end of the run you will have >10Gig of data, .bin, files and a .met file of meta data describing the run and results. The supplied SDtoCSV.jar (source code SDtoCSV_src.zip ) can be used to convert the .bin files to .csv files on your computer for further processing. The resulting data is >60Gig. The UNO/Mega2560 version has .bin to .csv conversion included in the Arduino sketch, but given the volume of data logged by the Teensy version, that is not an efficient way to do the conversion.

Teensy 3.2 by PJRC

Adafruit MicroSD card breakout board+ or similar.

16G SDHC class 10 MicroSD card of good quality e.g. SanDisk.

5V USB supply

A led with a 470R resister in series.

2 x 100R resistors (provides protection from damage due to Tx/Rx wiring errors)

Bluetooth Mate Silver OR one of the modules described on Arduino UNO/Mega Starter, controlled by Android/pfodApp

Download and install Arduino IDE V1.8.9+ from http://arduino.cc/en/Main/Software. That web page has links for various operating systems and a link to GettingStarted (http://arduino.cc/en/Guide/HomePage).

Download and install the Teensyduino (Teensy Support for the Arduino IDE). NOTE carefully the First Usage instructions.

Select Teensy 3.2 as the board and check that the BLINK example program loads and runs.

Download and install the following libraries :- millisDelay and SdFat (A local snapshot of the SdFat library used for these tests is here.) and the pfodParser.zip (for the pfodBufferedStream and pfodNonBlockingInput class)

Download the library zip files and then use the Arduino IDE → Sketch → Include Library → Add .ZIP library menu to install the libraries from the zip files.

Unzip the Teensy32AnalogLogger.zip file to your Arduino sketches directory and program the Teensy 3.2 board with Teensy32AnalogLogger.ino (Version 0.01)

Wire up the Teensy 3.2, Bluetooth module and the SD card module as shown above (pdf version)

First format your SD card using the https://www.sdcard.org/downloads/formatter/

The SD card must be empty in order to start logging.

For initial testing you do not need to connect the Communications module, just connect the Teensy 3.2 + SD module (with an empty card installed) to the Arduino IDE via the USB serial cable. As provided the Teensy32AnalogLogger.ino sketch uses the USB connection for control and monitoring. See Logging Real Data step below for using a communication device for control and monitoring.

Edit the top of the Teensy32AnalogLogger.ino sketch to set the COM_SERIAL to Serial, to output to the Teensy USB connection.

#define COM_SERIAL Serial 

Then upload the sketch to the Teensy 3.2

Open the Arduino IDE Serial Monitor at 115200 baud (with both NL & CR set). After a few secs the Teensy 3.2 will display a menu of commands

Ver:0.01  enter one of the following commands:
? - current status and metadata
i - initialize files
l - list files
>

The ? cmd displays details of the current settings. (See the top of the Teensy32AnalogLogger.ino to change these settings) Cmds must be terminated with a NL or CR or both.

 0:00:00.000 of  720:00:00.000
 Sample pins: 16 17 Byte order : Little-Endian
 ADC bits: 10 ADC sample averages over : 4
 Sample Rate: 1000.00 Sample interval: 1000uS
 Samples per block: 127 Time to fill block : 127000uS
 Time to fill a file :  9:01:52.000 Time to fill ALL files :  894:04:48.000
 Max SD latency (includes file close/open) : 0uS Max file close/open latency : 0uS
 Number of buffer blocks: 28 Time to fill ALL block buffers : 3556000uS
 Max number of buffers saved in call to storeSampleBuffers() : 0
 Missed Timers total : 0 Total Missed Samples so far: 0
 Total Blocks written : 0 Total Samples written : 0 covering :  0:00:00.000
 Current File : 

In this case current logging runtime is 0 of a requested 720hrs (30days), sampling D16/A2 and D17/A3 (see below for Restrictions on the Choice of ADC Inputs below) 1000 times a second. The maximum runtime can be up to 894hrs (37.25days). The main loop() can be occupied for upto 3.5sec (Time to fill ALL block buffers) before all the available buffers are filled and samples start being lost. Buffers saved etc are updated as the run progresses.

Insert an empty SD card, use the 'i' cmd to initialise the 99 files used to store the data. Pre-initializing them here reduces the time delay when switching from one file to the next and allows faster sampling.

 Initializing 99 files
Creating new file : log00.bin Elapsed time : 368mS
Creating new file : log01.bin Elapsed time : 520mS
. . . 
Creating new file : log98.bin Elapsed time : 15660mS
Creating new file : log99.bin Elapsed time : 15812mS

You can then use the r cmd to start a logging run. The run will for the requested time or until the s cmd is used to stop it. You can also use the ? cmd while logging to get updated timings and counts. Here is a short run stopped early using the s cmd.

LOGGING DATA ..... 
Ver:0.01  enter one of the following commands:
? - current status and metadata
s - stop data logging

 LOGGING DATA...  Check with ? command
>
 Elapsed Run time :  0:00:10.000 of  720:00:00.000
 Elapsed Run time :  0:00:20.000 of  720:00:00.000
. . . 

 Stopping Logging and removing unused files.
. . .
 Removing unused file : log98.bin
 Removing unused file : log99.bin

 0:01:04.976 of  720:00:00.000
 Sample pins: 16 17 Byte order : Little-Endian
 ADC bits: 10 ADC sample averages over : 4
 Sample Rate: 1000.00 Sample interval: 1000uS
 Samples per block: 127 Time to fill block : 127000uS
 Time to fill a file :  9:01:52.000 Time to fill ALL files :  894:04:48.000
 Max SD latency (includes file close/open) : 204uS Max file close/open latency : 0uS
 Number of buffer blocks: 28 Time to fill ALL block buffers : 3556000uS
 Max number of buffers saved in call to storeSampleBuffers() : 1
 Missed Timers total : 0 Total Missed Samples so far: 0
 Total Blocks written : 511 Total Samples written : 64832 covering :  0:01:04.832
 Current File : log00.bin

ls:
2000-01-01 01:00:00     261632 log00.bin
2000-01-01 01:00:00        240 log.met

DATA LOGGING COMPLETED! 
Ver:0.01  enter one of the following commands:
? - current status and metadata
 ** r - record ADC data ** not available. Data already exists
 ** i - initialize files ** not available. Data already exists
l - list files

 DATA HAS ALREADY BEEN LOGGED, check with ?
>

LED Display

The LED connected to D3 (with D2 providing the GND connection) will turn on solid if any sample is missed and will flash if there is an error. The sketch attempts to continue after errors but may not do so successfully.

When logging real data over long durations, it is more convenient to connect a communications module to the D0/D1 pins and control and monitor the logging remotely. Here a Bluetooth Mate Silver module was used with its default settings, 115200 baud, no hardware handshaking (RTC, CTS), pin code 1234.

Note: When power is applied to the Mate Silver module it goes into a configuration mode, fast red led blinking, for 60sec. During this time you can send $$$ via the serial connection to the module to configure it but you cannot connect the the module. Once the red led is slow blinking, the bluetooth module will accept connections.

Change the COM_SERIAL define in the Teensy32AnalogLogger.ino to the hardware serial (D0/D1) connection, Serial1

#define COM_SERIAL Serial1 

After pairing with the computer, a new COM port was created on the computer and CoolTerm can be used to connect and control and monitor the logging. Other serially connected communication modules can also be used, such as WiFi or BLE, see Arduino UNO/Mega Starter, controlled by Android/pfodApp for details.

You can also control and monitor the logging from your Android mobile using a bluetooth terminal app such as Bluetooth Terminal app, or using WiFi and TCP terminal app such as TCP Telnet Terminal Pro , or a Uart to BLE mdoule and a BLE terminal app such as nRF UART V2

The Teensy 3.2 has two separate ADC hardware modules, ADC_0 and ADC_1, in its micro-processor so it can sample two inputs at the same time. It also has a built in hardware averaging which takes multiple ADC samples and averages them before turning the result.

There are restrictions on which inputs can be connected to ADC_0, ADC_1. The Teensy3_1_AnalogCard.png image (from https://forum.pjrc.com/threads/25532-ADC-library-update-now-with-support-for-Teensy-3-1), included in the Teensy32AnalogLogger.zip file, details which pins can be connected to which ADC.

In summary:-
For Single Ended Reads i.e. +Volts referenced to GND
ADC_0 can read A0 to A9, A10, A11, A12, A14
ADC_1 can read A2, A3, A10, A13, A15 to A20
If you select a pin that the ADC cannot read it will return 0 (always)

This project uses A2, A3 which can each be accessed by ADC_0 or ADC_1.

The Teensy32AnalogLogger.ino saves the samples as binary in logxx.bin files i.e. log00.bin to log99.bin. Teensy32AnalogLogger.ino also save a log.met file of meta data about the run.

You can use SDtoCSV.jar (source code SDtoCSV_src.zip ) to convert the .bin files to .csv for further processing. Copy the files from the SD card to your computer hard disk with at least 70Gig of free space and copy the SDtoCSV.jar to the same directory. Open an command window in that directory and run

java -jar SDtoCSV.jar log

If your computer does not have Java install then install it from www.java.com

SDtoCSV will process the log.met file and then each of the available logxx.bin files in the directory and output a .csv file for each .bin. The .csv file has the sample sequence number followed by the two readings.

A sample console output for a 2 adc reading sampled 30303 times/sec is shown here, output.txt. It illustrates how missed samples are reported. (Doubling the number of buffers in the final sketch fixed this these missed samples)

SD_Logging to CSV conversion.  Check SDtoCSV.log for progress and errors messages.
Processing log00  Processed 256000 blocks
Processing log01  Processed 256000 blocks
. . . 
Processing log25  Processed 256000 blocks
Processing log26  Processed 256000 blocks
Processing log27  Processed 256000 blocks Missed Samples : 2715
Processing log28  Processed 256000 blocks
 . . .
Processing log29  Processed 256000 blocks
 . . .
Processing log47  Processed 256000 blocks
Processing log48  Processed 35935 blocks
 --- Finished Processing

A fuller log file, SDtoCSV.log, is appended to by each run of SDtoCSV. It includes the meta data output and any error messages. Here the count:254 is the count of the data stored in that block i.e. 127 samples x 2 adc readings per block. The missedSamples is the number of pairs of reading missed i.e. lines in the .csv output.

===  Log File for SD_Logging to CSV conversion Sat Jul 13 13:19:51 AEST 2019
 To see progress messages on Console use
    java -jar SDtoCSV.jar 
 Base File Name 'log'
 Metadata Version : 0  (Little Endian)
 sampleInterval uS:33
 adcBits:10
 adcAvgs:4
 pinCount:2
 Pins: 16, 17
 samplesPerBlock:127
 noBufferBlocks:28
 duration mS:51649820
 requested runTime mS:106216704
 maxBuffersUsed:32  Warning: Exceeds number of buffers available (28). Some samples may be missing.
 maxLatency uS:221476  Warning: Exceeds time provided by buffer blocks (117348uS). Some samples will be missing.
 maxFileOpenTime uS:20998
 missedTimers:0
 missedSamplesTotal:2715
 totalBlocksWritten:12323935
 totalSamplesWritten:1565139665
Processing log00.bin    Processed 256000 blocks
Processing log01.bin    Processed 256000 blocks
. . . 
Processing log26.bin    Processed 256000 blocks
Processing log27.bin  
 !!! Block:57696 count:254 missedSamples:2339
 !!! Block:57697 count:254 missedSamples:376  Processed 256000 blocks --- Total Missed Samples : 2715

Processing log28.bin    Processed 256000 blocks
 . . .
Processing log47.bin    Processed 256000 blocks
Processing log48.bin    Processed 35935 blocks
 ---  Finished Processing

A sample of log00.csv output file is

SampleCounter (per 33uS),Pin 16,Pin 17
0,248,205
1,273,195
2,228,337
3,360,302
4,355,369
5,220,281
. . .

The sample counter increases from file to file so it can be used as a time stamp. If there are missing samples then the sample counter is incremented by the missed count before outputting the next line so that the counter/time stamp remains accurate for the recorded samples.

The Teensy32AnalogLogger is a heavily modified version of Bill Greiman's AnalogBinLogger example in his SdFat Arduino library. Here the library has been rewritten to run on the Teensy 3.2.

Teensy32AnalogLogger uses timer0 to set the sample interval. The interrupt handler for timer0 starts the two ADC conversions. An interrupt handler for the second ADC module is continually called until they are both finished, usually the first ADC module started ADC_0 will finishes before the second on so the interrupt handler is only called once. The ADC_1 interrupt handler saves the samples to a data buffer.

In the main loop(), the storeSampleBuffer() saves any full buffers to the SD card and recycles the buffers to the empty buffer queue. The large amount of RAM available on the Teensy 3.2 means a large number of buffers can be allocated and so storeSampleBuffer() does not need to be call often. This leaves time for the main loop() to do other work, like process commands and send output.

Extensions

While this project is functional as a high speed data logger, for a complete wearable it still needs to packaged and a battery system and ECG sensors supplied. As well as that there are some extensions that should be considered.

1. Add real time control and monitoring of the sampled wave form via pfodApp using pfodApp's plotting function to show snapshots of the wave form
2. Extend the file numbers past 99 for longer sampling runs
3. Sample more than 2 inputs. Since the Teensy 3.2 has dual ADC modules, you can modify the code to add extra inputs in pairs to maximize the sample rate.
4. Add battery monitoring to track battery charge. The Teensy 3.2 uses about 1100mAhrs over 24hrs, including the bluetooth and SD module, but excluding the sensor module
5. Add a dual battery supply circuit to allow for battery changes with out interrupting the logging.