Skip to main content
Search

# Looped Audio Spectrogram Transformer

1 related articles

Subscribe to updates
  1. arXiv

    LAST reuses one set of Transformer blocks to refine only the class token, beating a twelve-layer sequential model on AudioSet by 2.1% with 49.4% fewer parameters

    LAST first processes all tokens, then reuses the same blocks to refine only the class token over fixed audio features, making later passes inexpensive; on AudioSet, ten-pass LAST reaches 0.345 mean average precision, exceeding a twelve-layer sequential transformer by 2.1% relative with 49.4% fewer parameters, 42% fewer multiply-accumulate operations, and 9.8% higher measured throughput, while increasing the pass count from two to ten improves accuracy with only 1.2% added computation and shows improved robustness to temporal masking and other auditory augmentations plus better generalization on music, environmental, and event sound classification.

You’re all caught up