Skip to main content
Back to timeline
arXivSource publication:

NVIDIA's 4GB Nemotron 3.5 Bolt E4B tops 21-benchmark average while emitting fewer output tokens

Synopsis

NVIDIA released Nemotron 3.5 Bolt E4B 4GB, an open small language model targeting roughly 4 GB of memory for local voice assistants, interactive characters and robotic or IoT interfaces. It is post-trained from Gemma-4-E4B-it with reinforcement learning alone, strengthening multi-turn tool use, instruction following and input grounding. Across 21 benchmarks it posts a higher average score than Gemma-4-E4B-it and Qwen3.5-4B in both reasoning and non-reasoning modes, while generating fewer output tokens on average.

Source-provided article image: Nemotron 3.5 Bolt E4B 4GB: An Open and Efficient Small Language Model for Fast Conversational Agents
arXiv

Interpretation

NVIDIA released Nemotron 3.5 Bolt E4B 4GB, an open small language model with a target memory footprint of about 4 GB, positioned as a base for agents that converse and take actions on local devices. It is further post-trained from Gemma-4-E4B-it rather than trained from scratch, aiming to keep conversation and multi-turn tool use inside the memory budget of local voice assistants, interactive characters and robotic interfaces. On 21 benchmarks spanning conversational proficiency, multi-turn tool use, instruction following and control, context robustness, and state and action grounding, it scores a higher average than Gemma-4-E4B-it and Qwen3.5-4B in both reasoning and non-reasoning modes, and generates fewer output tokens on average than either baseline.

The post-training is purely reinforcement learning, with no supervised fine-tuning stage. In experiments, supervised fine-tuning on data from diverse teachers improved multi-turn tool calling but regressed the conversational quality and verbosity of Gemma-4-E4B-it, so the final model keeps only RLVR and RLHF. Training runs through NeMo-RL and NeMo-Gym across tool-calling, tool-call schema, instruction-following, instruction-hierarchy, structured-output and free-form-formatting environments, uses every sample in both reasoning and non-reasoning modes, and holds a 2:1 non-reasoning to reasoning ratio in each batch.

The release ships with a DFlash parallel drafter that accelerates generation through speculative decoding. The drafter proposes all draft tokens in one forward pass and the target verifies them in parallel, and verification preserves the output distribution of the target. With reasoning on at concurrency 1, DFlash raises per-user decode speed by 2.7x on RTX PRO 6000 and 3.6x on DGX Spark; at concurrency 8 the gains are 2.7x and 2.9x.

Perspective

Evaluation covers 21 benchmarks in reasoning and non-reasoning modes, plus qualitative live testing in virtual interactive characters, on-device voice assistants and chat; quantization comparisons span Q4_K_M and the IQ4_XS and IQ4_NL variants, and speculative-decoding speed is measured on RTX PRO 6000, DGX Spark, RTX 5070 and RTX Spark N1X.

Post-training used English data only; the model shows slight but consistent drops on MMLU-ProX multilingual knowledge and reasoning while machine translation quality is largely retained. Vision capabilities also degrade slightly but consistently, additional training of the vision encoder alone brought no meaningful improvement, and restoring vision is left as future work. In AudioMC, 23.36% of user turns exceed the model's 30-second speech input limit and 53.1% of conversations contain at least one such turn, so those audio results do not cover complete, unrestricted conversations.