A frozen LLM learns full-duplex talk in 30 GPU-hours: 97M trained parameters, and a 0.170 floor-take rate when users merely pause
Synopsis
Full-duplex spoken dialogue usually needs hundreds of thousands to millions of hours of speech training and rewrites the language model that supplies knowledge. This system freezes the thinker of Qwen3-Omni-30B-A3B and trains only low-rank adapters, an audio adapter, a channel-fusion layer and 17 new token embeddings, 97M parameters, converting it in about 30 GPU-hours through a four-stage curriculum. On Full-Duplex-Bench it is the least likely open-source system to take the floor when the user merely pauses, with a turn-over rate of 0.170 on the synthetic subset against 0.584 for the next-best system; on InteractionBench it leads the open-source systems in spoken-task accuracy, groundedness and long-conversation coherence.
Interpretation
Full-duplex dialogue behavior can be added to a frozen general-purpose multimodal LLM instead of retraining a speech model. Moshi starts from a 7B text LLM, pretrains on 7 million hours of transcribed audio, then post-trains on Fisher and other data; SyncLLM starts from Llama3-8B and trains on 212,000 hours of synthetic spoken dialogue. In the open-source comparison on Full-Duplex-Bench v1.0, this system's turn-over rate on the synthetic pause-handling subset is 0.170, against 0.584 for the next-best PersonaPlex and 0.642 or higher for the remaining open-source systems.
Speaking timing becomes a per-frame token decision constrained to legal moves by a state machine. End-to-end models leave turn-taking implicit in the generated audio streams, while explicit schemes decide it in coarser time slices or with a two-state machine. The thinker emits one agent-text token every 80 ms on a 12.5 Hz clock; pauses, yields and onsets are tokens of the same vocabulary, and a finite-state machine masks illegal tokens before sampling.
Thinker and talker connect only through discrete tokens, so the two halves train independently. Qwen2.5-Omni's talker reads the thinker's hidden representations, and Qwen3-Omni still feeds it continuous multimodal features while training and running the two as one model. The talker receives only the thinker's discrete tokens and is trained on speech paired with its word-level transcript, in a single supervised stage on 36,996 filtered examples for four epochs, taking 2.65 hours on one NVIDIA B200.
Perspective
The demonstrated setting is a frozen Qwen3-Omni-30B-A3B thinker plus a talker initialized from the PersonaPlex lineage, evaluated end to end on Full-Duplex-Bench v1.0, v1.5 and InteractionBench, with output audio the same length as the input and no external voice-activity detector.
The data-efficiency claim covers adaptation cost only: the starting model was itself pretrained on large amounts of audio, and the comparison with Moshi and SyncLLM crosses units, since this work counts training examples and GPU-hours while those systems report audio hours, with no conversion between them. The talker currently holds user-audio embeddings at zero and speaks from agent text alone, and it does not control loudness or whispering; the backchannel step trains on 701 unique dialogues, each repeated eight times.
