Optical Keyboard

This optical keyboard/keypad is a low-cost optical user input device capable of deployment underwater, explosive gas areas, or in other locations where hermetic sealing is desirable. It does not use capacitive, magnetic, mechanical, acoustic, or visible-light coupling. The reflection of an infrared beam is used at each keysite to determine finger presence. No part of this device need be in actual contact with the outside medium; a transparent window (possibly of variable thickness) may optionally be used. Emphasis is on ease of construction, low component count and cost, and low software processing cost. A current tradeoff is lack of operation capability in direct sunlight, which can be improved by inverting the present phototransitor biasing and additional software work.

HARDWARE OVERVIEW

Refer to the HardwareOverview (cross-sectional) diagram. Layer 1 is the outside environment; this can be the outside medium directly, or it can be a pressure hull (e.g. plastic or glass). Layer 2 is a transparent acrylic lightpipe layer; item 7 is key-cap legend etchings that will emit light due to white LEDs at items 9. Layer 3 is an infrared-passing but visually black filter; this provides the black background for layer 2. In this way, layer 2's key-cap legend markings are visible in full sunlight as well as in full darkness. Additionally, layer 3 hides the underlying holes and components. Layer 4 is a translucent ("frosted") filter which increases reliability by decreasing false finger-over events because it diffuses the outgoing and incoming beams unless a finger is in close proximity. Layer 5 is a hard plastic 4mm shadow-mask layer that guides the beams of light up and down only, preventing lateral crosstalk between LED and photosensor at each keysite at measurement time. Layer 6 is the PC board; items 8 are the fast high-power infrared LEDs and photosensors.

KEYSITES

Each keysite has an infrared LED and an infrared phototransistor (PT). Both LED and PT are capable of switching on/off in the dozens of nanosecond range. The LED is a high intensity type with 940nm output and a 15-degree beam angle (Vishay VSMB294008G) and the PT is closely matched to the LED and has a 24-degree sense angle (ON Semiconductor QSB363GR). The PT is connected via a low-loss MUX switch (Texas Instruments TMUX1108) to an op amp used as a voltage follower buffer (Texas Instruments OPA350) and then on to a 10-bit A/D converter inside the MCU (Microchip PIC16F18855).

The keyboard matrix is organized as 48 (LED, PT) pairs, physically arranged in a 12x4 array, but electrically arranged as a 6x8 matrix. Regarding the 6x8, the 6 represents 6 "voltage +" channels, switched P-channel MOSFETs, and the 8 represents 8 "ground return" bits, switched by N-channel MOSFETs. The voltage channels power their bank of 8 LEDs. The ground return bits select LEDs by connecting cathode(s) to ground as well as provide (and thus enable) the PTs voltage-divider ground. A separate 1-of-6 MUX is used to select a single bank's PT for measurement.

LED DRIVE

The LEDs are capable of some 60mA continuous current, however, they are also capable of being driving harder for a shorter period of time. Here, we drive them at about 230mA for just under 70uS at 1/48 duty cycle. The datasheet gives a hard deadline of 100uS; the software is critical in this regard: a bug could cause hardware damage if it were to leave LED(s) on continuously. I justify that this is OK for this project because the microcontroller is dedicated to only this and is not running any interrupts nor watchdog timers. A single integrated software loop with an emergency exit controls the LED on/off while polling for A/D conversion completion.

SOFTWARE OVERVIEW

See the software overview diagram. There are 6 major areas of software processing: measurement, scoring/thresholding, spatial differentiation, temporal integration, debouncing, and encoding/human user feedback. A simple round-robin loop (interrupts and watchdog timers disabled) with a 16-bit counter is used to allocate time to the various periodic events.

The software was done in Microchip's MCC X IDE. All of it is in file main.c enclosed.

MEASUREMENT

Each keysite has an infrared LED and an infrared phototransistor (PT). Both LED and PT are capable of switching on/off in the dozens of nanosecond range; the luminance measurements are 10 bits in resolution. The LEDs and PTs are organized as 6 banks of LEDs and PT ground returns, with a 3-bit command to the MUX to select one-of-6 PT channels for measurement.

Four kinds of measurements are continuously taken. Two are for baseline luminance value gathering, and two are for probe luminance value gathering. It is the "probe" measurement which are actually (eventually, after processing) used to determine keypress events.

Baseline measurements are scheduled to occur much less frequently than probe measurements. Additionally, baseline vs. "probe" measurements are staggered in time. Since it is highly unlikely that more than one or two fingers are down on the keyboard during desired-to-detect keypress events, a comparison of the slow-changing baseline, to the fast-"scanning" probe measurement, for any particular keysite, is used here to detect keypress events. This avoids the additional expense of a global ambient comparision sensor.

Baseline and probe values are individually stored on a per-keysite basis; this compensates for:

1. Differences between individual LED and PT characteristics

2. The amount of obscuration is caused by the various letterform glyph legends on the key caps

3. Differences in ambient illumination across various parts of the keyboard

4. "Slow" changes in overall ambient lighting conditions

Baseline measurements are "averaged in" to "last time"'s measurement ((current measurement + last measurement) / 2).

Within these baselines and probes, one each of a "dark mode" and "bright mode" measurement is taken.

A dark mode measurement is taking the luminance value of the infrared phototransistor at the keysite with the infrared LED turned OFF. A bright mode measurement is the same with the LED turned ON.

The bright mode operates the LED at 250% of its continuous power for 70uS and 1/48 duty cycle (this is allowed per its datasheet; 100uS is the maximum for such a mode of operation). Since it is imperative that the software never miss the LED OFF deadline to avoid hardware damage, and since we must wait for a variable amount of time for A/D conversion of the measured luminance value to complete, these critical inner functions are combined in a single routine in the software (illuminate_and_measure()) and we operate with no interrupts and no watchdog timers. This further is a reason for the hardware making sure that on MCU reset all LEDs are off (It is the reason for the inverters driving the P-channel MOSFET bank drivers).

The probe (as opposed to baseline) measurement periodic tasks run most often. One keysite, and, at that, one phase of that keysite, is processed per iteration; in this way everything is staggered and other tasks can also run.

There is an array of key phases; each key has at any time its own phase of what measurement is going to happen next. There are 4 phase states called 0,1,2,3, where most of the time we return to phase 2 but sometimes to phase 0. For example: 0,1,2,3,2,3,2,3,...<50x>,...,2,3,0,1,2,3,2,3,2,3,...

The idea is that during phase 0 a baseline dark-mode measurement is taken, and during phase 1, a baseline bright-mode measurement is taken, and during the (much more common) phase 2 a "probe" dark-mode measurement is taken, and during the (much more common) phase 3 a "probe" bright-mode measurement is taken.

In summary, staggering the schedule of measurements for baseline (versus probing) provides these advantages:

1. Amortizes the processing time among all key keysites and other processing tasks below.

2. Distances the baseline and probe (decision) measurements for a specific key in time and and in space (keysite position), that is, the entire keyboard is not engaged in a probe measurement for all keys "at the same time" (in the same pass) (except once, during the first pass).

See the StaggeredStates diagram.

MEASURED VALUES

Measured values are 10 bits and can range from about ~20 to ~980 in actual use. A value of 0 is reserved by the software to indicate "no measurement taken yet or last measurement was inconclusive", and so at least a value of 1 is assigned at minimum. This is so the higher software layers can use a value of 0 to ignore processing.

SCORING

Probe measurements in both LED-OFF ("dark mode") and LED-ON ("bright mode") are compared with their corresponding baseline measurements the same rate (though at a different staggered point in the cadence) as the measurement task. This is done by software routine determine_score_of_current_key().

Various deltas are computed, and then a series of cases each try to match against various "syndromes" of combinations of deltas that indicate that a finger might be hovering over a keysite.

Zero-valued measurements are skipped (see section above). Pinning to zero is employed to avoid negative values.

The various "syndromes" essentially serve as threshold detectors. If a threshold condition is met, then a per-key "score" is incremented. If not, then the current per-key score is decremented. In this way, the scores for any and all keys that do not currently meet a threshold condition gradually drop, pinning and remaining at 0, until activated again.

SPATIAL DIFFERENTIATION

The array of all per-key scores is periodically examined by software routine maintain_histogram_of_peak_of_all_scores(). For keysites whose score exceeds a threshold, an analysis is done to see if it is a peak among all other keys, and if so, if that peak is "sharp" enough, and if so, a per-key array value is incremented, with all other such per-key array values being decremented.

In this way, the resulting per-key array will always contain none or one or at most a very few nonzero values, corresponding to a finger near keysite(s).

The sharpness of the peak is computed simply by noting its value, then noting the value of the next-lower peak, and requiring that they be different by at least a threshold.

The net result of this processing step is to reject finger-over or object-over events that are not singular in nature, such as a hand hovering over the keyboard but not pressing any particular key. This software step works together with the frosted translucent diffusion filter layer to prevent spurious events.

To allow multiple simultaneous ("chord") keydown events, this software routine can be expanded to allow for 2 or 3 peaks of near-similar magnitude, while still requiring that there strictly be no more such peaks.

TEMPORAL INTEGRATION

The software routine act_on_histogram_of_peak_of_all_scores() periodically examines if a keysite in the spatial differentiation array above has exceeded a threshold. The scheduling of this routine is such that this provides for having these favorable conditions be maintained for some time, e.g. on the order of 100ms.

If it is decided that this event has occurred, then the debounce (see below) software routine is called; this is the first level that is event-driven. Also, at this time, the peaks and scores arrays are all reset to values of 0, which forces any new keypress event to build its favorable conditions from scratch.

DEBOUNCING

While temporal integration detects a new keypress event in time, it cannot distinguish if such an event is "new". Instead of attempting to detect the end of integration, this design looks for a different keypress or timeout. This is implemented by the software routine do_keypress_debounce() and functions together with maintain_keypress_debounce(). Multiple events of the same key are prevented unless a certain amount of time (e.g. 500ms) has elapsed.

ENCODING AND FEEDBACK

Keydown events that survive the debounce filter are encoded (in the prototype, into extended ASCII with provisions for shift-key upper/lower-case state) and sent to the destination.

In addition, a human feedback animation is triggered which momentarily dims the entire keyboard's legend lights. Alternatively, an audio click or beep, or haptic feedback, can and should be used to give the user a confirmation that their keypress has been accepted.

I often like to add more detail to schematic symbols than SnapEDA, etc, provide, so here are the parts used by the schematic and board files. These are EagleCAD libraries.