Skip to main content
Back to timeline
arXivSource publication:

A keyboard the attacker never heard still gives up its keystrokes: Decker cuts the cross-keyboard accuracy gap from about 40 points to under 18

Synopsis

Acoustic side-channel attacks had been validated only with a single keyboard, a fixed microphone, a quiet room, and training and testing on the same device; on unseen keyboards, strong unimodal and multimodal baselines lose roughly forty accuracy points. Decker combines keyboard signature normalization, domain-adversarial disentanglement, supervised cross-keyboard contrastive alignment and acoustic style randomization in one training framework to learn embeddings that discriminate keys but stay invariant to keyboard and user, narrowing the cross-keyboard generalization gap to under eighteen points, and a language model then decodes over the top acoustic candidates to turn noisy key predictions into coherent text.

Source-provided article image: Domain-Invariant and Language-Model-Assisted Acoustic Side-Channel Attacks on Keyboards
arXiv

Interpretation

On a keyboard never heard during training, keystrokes can still be identified key by key, a question same-device evaluation could not answer. Earlier evidence came from a single keyboard, a fixed microphone, a quiet room, homogeneous typists, and training and testing on the same device; accuracy was high under that setting but did not survive a device change. On the unseen-keyboard split of the Hear dataset, four unimodal models (wav2vec2, ECAPA-TDNN, CoAtNet-S and ResNet-34) each lose about forty absolute points, for example ECAPA-TDNN falling from 95.3% to 58.1%; among eight early, intermediate and late Euclidean fusion variants, the best, bidirectional cross-attention, reaches only 66.2%.

The approach separates key identity from keyboard identity in the embedding space so that the same key lines up across devices. Keyboard signature normalization is a learnable one-dimensional convolutional inverse-filter module that counteracts the spectral colouration of the device impulse response; acoustic style randomization synthesizes unrecorded keyboards at training time with random IIR filters, a smooth mel-envelope warp and an exponential-decay perturbation. Training combines key cross-entropy, a domain-adversarial loss through a gradient reversal layer, and a supervised contrastive loss whose positives are the same key on different keyboards; under domain-shift evaluation on Hear, full Decker reaches 81.3% on unseen keyboards, dropping to 68.1% without the gradient reversal layer and to 63.7% without keyboard signature normalization.

A language model corrects local key errors within the acoustic candidate set, raising sequence-level recovery. Decoding runs constrained beam search over only the top-k acoustic candidates at each timestep, so the language model can reorder plausible keys but cannot introduce a key the acoustic evidence rules out. Over Decker's top-k candidates, raw acoustic output gives 81.3% character accuracy and 42.6% exact sentence match; adding GPT-2 Small raises these to 86.9% and 58.7%, and FLAN-T5 Base to 89.4% and 62.1%.

Perspective

The evaluation rests on the Hear dataset: 53 participants on 37 laptop keyboards, captured through an external microphone, the on-device microphone and a VoIP stream, across quiet, moderate-noise and high-noise environments. The conclusions apply to laptop keystroke inference under that collection protocol, across the cross-keyboard, cross-user, cross-environment and cross-device splits; the authors list touchscreens, ultra-low-profile keyboards and flexible input surfaces as device types not yet covered.

The authors note that some results assume a standardized typing corpus and known password classes, that wholly uncontrolled in-the-wild typing may differ, and that the reported numbers should be read as characterizing the threat under the stated protocol rather than as universal guarantees. Language-model gains concentrate on text and human-like passwords where human selection bias lowers entropy, with little improvement on high-entropy random strings; strong impulsive or overlapping noise, and individual typing biomechanics, may still cause model drift.