Skip to main content
Search

# Qwen3

21 related articles

Subscribe to updates
  1. arXiv

    A frozen LLM learns full-duplex talk in 30 GPU-hours: 97M trained parameters, and a 0.170 floor-take rate when users merely pause

    Full-duplex spoken dialogue usually needs hundreds of thousands to millions of hours of speech training and rewrites the language model that supplies knowledge. This system freezes the thinker of Qwen3-Omni-30B-A3B and trains only low-rank adapters, an audio adapter, a channel-fusion layer and 17 new token embeddings, 97M parameters, converting it in about 30 GPU-hours through a four-stage curriculum. On Full-Duplex-Bench it is the least likely open-source system to take the floor when the user merely pauses, with a turn-over rate of 0.170 on the synthetic subset against 0.584 for the next-best system; on InteractionBench it leads the open-source systems in spoken-task accuracy, groundedness and long-conversation coherence.

  2. arXiv

    Meeting transcription stops reading only who is talking: structured speaker conditioning steadies target recognition under overlap

    Meeting transcription must output text for a requested speaker, but four-state activity conditioning collapses different interferer counts and hand-over histories into one target condition. Structured speaker conditioning encodes each speaker's activity track separately, combines it with session-level acoustic identity, and controls the acoustic encoder through condition-dependent low-rank cross-channel modulation. On Qwen3-ASR-0.6B with oracle diarization, the complete method uses 1.46M conditioning parameters and achieves a 6.23% mean relative cpER reduction over four-state diagonal conditioning across four meeting test sets.

  3. arXiv

    MoE experts don't sort by domain label: an SAE splits each feature's activation by expert, separating Java from front-end code

    Researchers at ByteDance Seed and Shanghai University of Finance and Economics trained a sparse autoencoder (SAE, a dictionary that reconstructs a layer's output as sparse interpretable features) on the MoE output of layers 4, 30 and 46 of Qwen3-30B-A3B, and used the encoder's affinity in the expert outputs to decompose each feature's activation exactly over the experts. Averaged over a corpus, each expert gets the set of features it principally drives; most of those features have only one or two dominators, and the division of labor runs below coarse domains, down to Java versus front-end web code and geometry versus arithmetic and polynomials.

  4. LMSYS Org

    Software-engineering agents stop waiting for the slowest rollout: async RL lifts pass@1 from 7.3% to 27.5%

    On AMD Instinct MI355X, Qwen3-30B-A3B-Instruct trained with fully asynchronous agentic RL after supervised fine-tuning raised pass@1 from 7.3% to 27.5% on a 100-task stratified subset of SWE-bench Verified, with RL tasks drawn from repositories absent from that benchmark. The bottlenecks were long-tail trajectories and CPU sandbox scheduling rather than GPU compute: capping trajectory length cut step time from 850 s to 284 s, and cloud sandboxes at 384 concurrent trajectories lifted rollout throughput from about 860 to about 3,480 trajectories per hour.

  5. arXiv

    Multi-turn agent distillation stops feeding student reasoning to the teacher: a 4B student beats its 32B teacher on ALFWorld Seen at 43.39%

    Multi-turn agent distillation now grounds teacher supervision in the interaction state, keeping only executed actions and observations, and the Qwen3-4B student surpasses its 32B teacher on ALFWorld Seen at 43.39%.

  6. A council of small models lost to one general model: on 783 radiology reports, Qwen3-4B-2507 hit 91.69% and beat medical-tuned MedGemma

    On 783 de-identified free-text mammography, hip-knee-ankle and brain MR reports, an LLM Council of three general small models plus a chairman reached 83.56% overall accuracy, below the medically trained MedGemma 4B at 88.72%, with the gap driven mainly by hip-knee-ankle classification. A single council member, Qwen3-4B-2507, reached the highest accuracy at 91.69%, above both the council and MedGemma, while the council needed four model calls per report and about 2.44 times MedGemma's peak GPU memory.

  7. arXiv

    Offline demos and fresh rollouts now share one regression target: unified training lifts math-reasoning avg@32 by 2.33 points

    Offline demonstrations, current-policy rollouts, and historical rollouts can now be trained under one regression target, and on four math-reasoning benchmarks the unified configuration improves Qwen3-1.7B benchmark-averaged avg@32 and pass@32 over two-stage SFT plus DAPO by 2.33 and 4.44 percentage points.

  8. arXiv

    Most of the no-think RL gain in hybrid reasoning models is thinking leakage: erasing an 8-D think subspace at 8B removes 75% of it

    Training Qwen3-1.7B, 4B and 8B with GRPO in no-think mode raises accuracy while responses grow longer, carry more reflection words, and internal activations shift toward the original model's think mode. Weight scaling, activation patching and attention-head interventions indicate that much of this gain comes from reactivating reasoning the original model already had rather than learning new reasoning behavior; at 8B, replacing an 8-dimensional think subspace removes 75% of the gain, and the reverse exchange transfers 50% of it to the original model.

  9. arXiv

    Iterate on a frontier model, serve on an 8B: distilled scheduler agrees with production records 0.913 of the time, beating its own teacher

    A home-care scheduling agent no longer asks a small model to handle a narrow task cold. Zero-shot small models failed it, and retraining on every behavior change was too slow, so the behavior was first iterated in prompt space on GPT-5.4-nano, then 11.6k recorded production decisions were LoRA-tuned into Qwen3-8B. That model agrees with recorded decisions 0.913 of the time, above the teacher's 72.7% self-agreement when re-run on its own prompts.

  10. arXiv

    Language models pick their own replay targets: merging two task banks lifts held-out retention by up to 4.42 points

    Fine-tuning a language model on a new task can erode old abilities, and the prior data kept for protection covers only part of what the model knows. Researchers organize task descriptions and examples into Task Banks, let a frozen pre-update model generate its own replay, and test protection on four held-out tasks excluded from bank construction. Merging two partial banks raises mean held-out retention above either bank alone at all three tested sizes, by up to 4.42 percentage points; at 4B, selection informed by the incoming task beats random selection by 2.75 points, an advantage absent at smaller sizes.