Skip to main content
Back to timeline
researchsquare.comSource publication:

Server anomaly detection stops relying on reconstruction error alone: CLGAT folds cross-layer attention structure into the score, reaching 95.31% F1 on SMD

Synopsis

Anomaly detection in multivariate server monitoring has relied mainly on reconstruction error, while changes in the long-range temporal associations learned by attention layers rarely entered the final score. CLGAT combines overlapping patch embedding, a cross-layer dynamic context residual, dual-stage gated attention and a window attention structure into one unsupervised reconstruction framework, multiplying a structural score with a Mean–Top-K reconstruction score to form anomaly energy. On the SMD benchmark it reaches 94.59% precision, 96.04% recall and 95.31% F1, above the 92.33% F1 of Anomaly Transformer, the strongest of 15 compared methods.

Source-provided article image: Structure-Aware Cross-Layer Gated Attention Transformer for Unsupervised Multivariate Server Time-Series Anomaly Detection
Fig. 1 · researchsquare.com · Page 7

Interpretation

Anomaly decisions for multivariate server time series gain a second source of evidence: relative changes in selected long-range attention responses enter the point-wise anomaly energy alongside numerical deviation between reconstruction and observation. Reconstruction and prediction methods usually base the final decision on numerical residuals, attention mostly serves as an internal representation mechanism, and earlier-layer representations in stacked encoders are reused mainly along adjacent layer-to-layer paths. On the SMD benchmark the complete model reaches 94.59% precision, 96.04% recall and 95.31% F1, against 89.40% precision, 95.45% recall and 92.33% F1 for Anomaly Transformer, the strongest baseline.

The input is cut into overlapping temporal blocks and linearly interpolated back to the original temporal resolution, so each representation carries neighborhood context while keeping the fine detail needed for point-level localization. Point-wise embedding processes multivariate observations at individual timestamps and captures local temporal dependencies spread over consecutive observations only to a limited degree. Experiments use a window length of 100, 38 monitoring variables, block length 8 and sliding stride 1, with a 3-layer encoder, model dimension 128 and 4 attention heads.

Each encoder layer retrieves context from all preceding layer outputs through cross-layer attention and uses a per-time-step gate to decide how much to inject, while attention weights and aggregated feature responses are each modulated by one gating stage. Conventional residual connections mainly pass information between adjacent computational blocks, so deeper layers cannot directly use representations accumulated across several preceding layers. In ablation, removing the cross-layer dynamic context residual lowers F1 from 0.9531 to 0.9147, and removing dual-stage gated attention lowers it to 0.9213.

The window attention structure aggregates selected bidirectional long-range attention responses into a structural score, normalizes it with temperature-controlled fixed-size chunk-wise Softmax, and multiplies it with the reconstruction score to form the final anomaly energy. Reconstruction error measures point-wise numerical deviation but does not directly characterize changes in the temporal dependency structure that attention represents. In ablation, removing the window attention structure causes the largest drop, from 0.9531 to 0.8518 F1, while the variant keeping only that component reaches 0.9094 F1.

Perspective

The demonstrated scope is unsupervised point-wise anomaly detection on the SMD server monitoring benchmark, evaluated under the point-adjusted protocol with the threshold set at the 99.5th percentile of validation anomaly energy; the authors propose testing on more datasets and limited-supervision settings and adapting structural constraints and threshold calibration to changing operating conditions.

After chunk-wise Softmax normalization the structural score is mainly comparable within its local region, and minor baseline differences may arise across neighboring chunk boundaries, so it is positioned as a local relative structural-deviation indicator rather than an absolute strength; the parameter analysis shows block length, sliding stride, loss weight, chunk size and temperature all have ranges where performance falls back.