Liquid AI builds its own audio codec: LiquidCodec packs about 74M parameters and decodes far faster than rivals
Synopsis
Liquid AI built LiquidCodec, a streaming audio codec of about 74 million parameters, for its new on-device audio models. It matches similarly sized codecs in reconstruction quality and compression while outperforming them in speech generation and token-to-waveform decoding speed. It replaces the residual vector quantization used by Mimi with grouped finite scalar quantization, which lets codebooks be predicted in parallel; on a single CPU thread it encodes and decodes at roughly 361x and 87x real time.
Interpretation
Liquid AI released LiquidCodec, an in-house streaming audio codec used by its LFM3-Audio-2B and LFM3-Omni-2B on-device multimodal models, turning speech into tokens a language model can predict and back into audio. The earlier LFM2-Audio-1.5B relied on the third-party codec Mimi, leaving token quality and architectural choices outside its control; replacing only the detokenizer in LFM2.5-Audio-1.5B already gave 8x faster detokenization. Across eight test sets, LibriSpeech test-clean, Seed-TTS English and Mandarin, and FLEURS German, French, Mandarin, Japanese and Korean, LiquidCodec records the best ViSQOL perceived quality and the lowest mel distance, staying competitive on the other metrics.
The codec follows the standard encode, quantize, decode pipeline, with an exactly symmetric encoder and decoder and a fully causal encoder that predicts spectral coefficients directly, avoiding the costly progressive upsampling with learned transposed convolutions used in similar codecs. Quantization moves to grouped finite scalar quantization: each feature dimension is independently rounded to one of a fixed set of values, groups each yield a token, and codebooks lack the strong sequential dependence of residual vector quantization. Training runs in two stages: pretraining on 264,000 hours across 107 languages with waveform and spectral losses and no adversarial loss, then fine-tuning on 55,000 hours of clean full-band speech natively sampled at 24 kHz or above with adversarial critics, plus decoder-only quantization-aware training.
Perspective
The demonstrated scope is speech codec and zero-shot multilingual TTS: reconstruction is evaluated on LibriSpeech test-clean, Seed-TTS English and Mandarin, and FLEURS in five languages, and speed is measured with FP32 single-threaded CPU inference on an Apple M4 Max.
Pretraining with waveform and spectral losses alone left metallic artifacts, which the authors hypothesize came from perceptual detail being averaged out and which adversarial critics removed in the second stage; quantized reconstruction drops slightly, yet ViSQOL and mel distance still lead the external codecs.
