Clinical sleep staging without full-night labels: a 31k-parameter student trained on a quarter of patients closes in on full-label N1 scoring
Synopsis
Clinical sleep is more fragmented, with more awakenings, more N1 and less REM sleep, and automatic staging models trained on healthy sleepers degrade on patients. A 136M-parameter sleep foundation model, SleepGPT, is fine-tuned on labels from only 25% of patients per cohort and then supervises a 31k-parameter, 122 KB causal CNN-GRU that stages EEG and EOG in real time. A new transition-aware distillation loss, Gcv-TAD, raises per-patient macro F1 by 3.2 to 7.6 points on average over supervised training on the same labels in patient-wise 10-fold cross-validation, with the largest gains on N1, approaching full-label training.
Interpretation
Clinical sleep staging can be done with only a quarter of patients labelled, using a lightweight model that runs in real time on wearable-scale hardware instead of relying on full-night manual scoring. Earlier automatic staging models were developed mainly on healthy or community cohorts and degraded markedly when evaluated on ischemic stroke patients; clinical sleep itself has more awakenings, more N1 and more heterogeneous stage transitions, while expert-labelled patient recordings are scarce. Across three cohorts of stroke, isolated REM sleep behaviour disorder and other sleep disorders, patient-wise 10-fold cross-validation gives Gcv-TAD an average per-patient macro F1 3.2 to 7.6 points above cross-entropy supervised training on the same labels, leaving students 0.9 to 2.0 points from their full-label counterparts.
The distillation signal no longer treats every 30-second epoch alike; it redistributes weight using the preceding stage and cohort transition statistics, putting more emphasis on uncommon transitions and N1. Conventional knowledge distillation treats epochs independently and does not explicitly model transitions between sleep stages, while existing sleep-specific approaches mainly transfer intermediate representations through teacher assistants. Gcv-TAD uses a transition prior estimated from labelled patients and down-weights transitions that vary strongly between patients; in fold-1 training data, stage-change epochs are 17%, 21% and 29% of epochs but receive 20%, 25% and 41% of the distillation weight, and N1 epochs are 10%, 10% and 13% of data but receive 12%, 13% and 21% of the weight.
The student keeps its hidden state across epochs and emits a stage from current and past input alone, so it can score in a streaming fashion. The SleepGPT teacher needs bidirectional context and substantial memory, and must wait 5 minutes for future context before deciding. The student needs 2.2M multiply-accumulates per 30-second epoch versus 10.4G for the teacher; its weights take 121.7 KB in FP32 and 75 KB after dynamic INT8 quantization; one epoch takes 1.16 ms on one thread of an Intel Xeon W-2265 CPU, whereas the teacher needs 271.90 ms even with cached embeddings.
Perspective
The results come from three retrospective clinical polysomnography cohorts, iSLEEPS, NAPS and ISRUC-S1, under patient-wise 10-fold cross-validation with single- or two-channel EEG and EOG input; the authors list wearable electrode configurations, on-device performance and generalization to additional clinical populations as future work.
The authors note that distillation remains sensitive to teacher reliability, with the teacher reaching only 64% accuracy on the example night; the transition-weight analysis uses training recordings whose scorer labels were hidden during learning and examined retrospectively, and generalization rests on the held-out patient results.
