Skip to main content
Back to timeline
arXivSource publication:

Speech enhancement returns to Mel: BaME beats a 331M DiT backbone with 2.9% of its parameters and finishes in a single generation step

Synopsis

Generative speech enhancement has leaned on learnable latent representations and large generative backbones, making acoustic fidelity, perceptual quality and processing efficiency hard to hold at once. BaME treats the Mel spectrum again as a structured acoustic representation: a subband-aware Transformer backbone reaches better enhancement than a 331M DiT backbone using only 2.9% of its parameters and 66.6% of its computation, and a vocoder with a degraded-phase prior plus one-step fidelity-perception alignment fine-tuning gives strong objective and subjective results on DNS-Challenge, EARS, GSR and URGENT benchmarks.

Source-provided article image: Back to Mel: Learning to Bridge Fidelity and Perception in Generative Speech Enhancement
arXiv

Interpretation

A generative enhancement backbone does not need a large DiT; a lightweight Transformer organized by Mel frequency bands can carry the generative modeling. The comparison swaps the backbone for a 331M-parameter DiT inside the same pipeline, with other settings held fixed. SaT uses 9.67M parameters (2.9% of DiT) and 6.65 GMACs/s per sampling pass (66.6% of DiT), reaching PESQ 2.487 and WER 9.242% on the DNS-Challenge ablation versus 2.448 and 11.615% for DiT.

The Mel band structure becomes the modeling unit: adjacent Mel bands are grouped into subbands before encoding rather than handled one band at a time. The earlier Mel-domain subband work CleanMel models narrow bands at individual Mel bins, while BaME groups adjacent bands into one subband to cut overhead. A sweep over the number of subbands shows metrics improving as subbands get finer, but gains flatten from 25 to 50 while parameters and complexity rise sharply, so 25 subbands is the default.

At waveform reconstruction, the phase of the degraded speech enters the vocoder as a prior, and a non-intrusive quality evaluator picks between zero-phase and degraded-phase outputs. Conventional Mel vocoding discards phase and leaves the vocoder to infer the missing phase from its learned speech prior. Across five phase initialization conditions, noisy phase scores best on every metric, yet reverberant and noisy-reverberant phase fall clearly below zero phase on non-intrusive metrics, which motivates the dual-path selection.

One-step generation is fine-tuned by constraining both the fidelity domain and the perception domain, so a single step can balance reconstruction accuracy and listening quality. Standard flow matching optimizes across all timesteps, and intrusive metrics already peak at one or two function evaluations while non-intrusive metrics need more steps. In the DNS-Challenge ablation, the configuration using both fidelity-domain and WavLM perception-domain constraints reaches PESQ 3.430 and WER 1.607%, better than perception-only or fidelity-only settings.

Perspective

Evaluation covers the DNS-Challenge 2020 test sets, EARS-WHAM_v2 and EARS-Reverb_v2, GSR, and the URGENT Challenge 2025 non-blind test set, with audio resampled to 16 kHz; the subjective test is an A/B preference study with 22 listeners who have audio signal processing backgrounds. The default configuration uses one-step generation, the lightweight DegVoC vocoder, and semantic conditioning from a fine-tuned WavLM.

Under reverberation the degraded phase lowers some non-intrusive metrics, dual-phase selection depends on the non-intrusive evaluator's judgment, and different evaluators prefer degraded phase to different degrees. In the URGENT comparison BaME exceeds top-ranked systems on non-intrusive metrics but still trails on several intrusive ones. The authors also note that existing objective metrics are not yet a single standard reflecting both fidelity and listening quality, and that the results do not claim Mel is the optimal representation.