Nemotron 3.5 Bolt E4B 4GB: An Open and Efficient Small Language Model for Fast Conversational Agents
Abstract
We introduce Nemotron 3.5 Bolt E4B 4GB, an open and efficient small language model further post-trained from Gemma-4-E4B-it and optimized for speed and intelligence per GB of memory used for fast conversational agents. The model carries forward Gemma's strong conversational capabilities and further improves multi-turn tool use, instruction following, and input grounding for fast, local voice/device assistants, interactive personas, and embedded/Edge interfaces. The model combines natural conversation with multi-turn agentic capabilities to power voice/device assistants, interactive characters and robotic/IoT interfaces running on GeForce RTX, Jetson, and RTX/DGX Spark. Experiments on 21 benchmarks show that Nemotron 3.5 Bolt E4B 4GB achieves the highest average accuracy while generating the fewest output tokens in both reasoning and non-reasoning modes, compared with Gemma-4-E4B-it and Qwen3.5-4B. The model is released with a DFlash drafter. With DFlash, Bolt E4B achieves the per-user decode speed of inference without speculative decoding at concurrency~1, and the generation throughput of Gemma-4-E4B-it with MTP at batch size 1.
Figures & tables
| Environments | Source data |
|---|---|
| Tool-Calling | Conversational Tool Use Pivot |
| Tool-call schema | Function Calling Pivot |
| Instruction Following | Instruction Following , SysBench , InverseIFEval , CFBench , MultiTurnChat , Calendar |
| Instruction Hierarchy | System-User-Instruction-Hierarchy |
| Structured Outputs | Structured Outputs |
| Free-form formatting | Free-Form Formatting |
| Benchmark | Metric | Reasoning Mode | Judge/User Model | Repeats |
|---|---|---|---|---|
| Conversational Proficiency | ||||
| MultiChallenge | Overall Score | On | Judge: GPT-4o | 16 |
| RoleBench Custom | Win % vs. Gemma4-E4B-it | Off | Judge: GPT-5.5 | 4 |
| RoleMRC | Judge Average w/o Multi-Turn | On | Judge: GPT-4.1 | 8 |
| EQ-Bench v3 | Rubric Score | Off | Judge: Ensemble a | 4 |
| Brevity Benchmark | Implicit - Average of Answer Quality & Conciseness | On | User: Qwen3-235B-A22B-Instruct-2507 | 4 |
| Benchmark | Nemotron-3.5 Bolt-E4B | Gemma4 E4B-it | Qwen3.5 4B | Nemotron-3 Nano-4B |
|---|---|---|---|---|
| Conversational Proficiency | ||||
| MultiChallenge | 53.10 | 42.08 | 51.76 | 35.30 |
| RoleBench Custom | 67.05 | 49.95 | 53.66 | 12.02 |
| RoleMRC | 80.63 | 74.62 | 65.81 | 47.00 |
| EQ-Bench v3 | 39.40 | 41.17 | 32.92 | 19.31 |
| Brevity Benchmark | 82.52 | 79.99 | 75.35 | 73.60 |
| Benchmark | Nemotron-3.5 Bolt-E4B | Gemma4 E4B-it | Qwen3.5 4B | Nemotron-3 Nano-4B |
|---|---|---|---|---|
| Conversational Proficiency | ||||
| MultiChallenge | 48.81 | 40.72 | 38.60 | 23.27 |
| RoleBench Custom | 67.05 | 49.95 | 53.66 | 12.02 |
| RoleMRC | 77.18 | 72.46 | 58.91 | 46.11 |
| EQ-Bench v3 | 39.40 | 41.17 | 32.92 | 19.31 |
| Brevity Benchmark | 78.71 | 75.39 | 75.21 | 72.43 |
| Q4_K_M model comparison | Nemotron-3.5-Bolt-E4B Q4 variants | ||||||
| Benchmark | Nemotron-3.5 Bolt-E4B imatrix | Gemma4 E4B-it | Qwen3.5 4B | Nemotron-3 Nano-4B | Q4_K_M no imatrix | IQ4_XS | IQ4_NL |
| Conversational Proficiency | |||||||
| MultiChallenge | 55.60 | 39.36 | 52.23 | 34.53 | 51.12 | 51.91 | 52.12 |
| RoleBench Custom | 71.21 | 60.51 | 52.35 | 10.86 | 65.21 | 69.07 | 69.86 |
| RoleMRC | 79.12 | 75.24 | 62.61 | 45.84 | 80.36 | 78.79 | 75.65 |
| EQ-Bench v3 | 38.15 | 39.77 | 32.07 | 19.45 | 37.42 | 38.19 | 38.28 |
| Reasoning | Non-reasoning | |||||
| Benchmark | Nemotron-3.5 Bolt-E4B | Gemma4 E4B-it | Qwen3.5 4B | Nemotron-3.5 Bolt-E4B | Gemma4 E4B-it | Qwen3.5 4B |
| Audio | ||||||
| Audio MultiChallenge | 25.22 (+6.19) | 19.03 | — | 19.25 (+2.21) | 17.04 | — |
| Voicebench | 80.30 (+2.10) | 78.20 | — | 72.60 (+0.90) | 71.70 | — |
| CoVoST (BLEU) | 36.14 (-0.81) | 36.95 | — | 37.61 (+0.59) | 37.02 | — |
| FLEUR (WER/CER) | 10.16 (+0.38) | 9.78 | — | 10.43 (+0.26) | 10.17 | — |
| Weights | Speedup at concurrency | ||||
|---|---|---|---|---|---|
| Drafter | (GB) | AL | 1 | 16 | 32 |
| BF16 | 2.06 | 2.99 | 1.00 | 1.00 | 1.00 |
| NVFP4 W4A16 (default) | 0.58 | 2.99 | 1.13 | 1.05 | 1.01 |
| NVFP4 W4A4 | 0.58 | 2.93 | 1.11 | 1.09 | 1.06 |
| Benchmark | temp = 0 | temp = 1 |
|---|---|---|
| MT-Bench | 4.01 | 3.65 |
| SPEED-Bench, qualitative split | 3.88 | 3.50 |
| SPEED-Bench, 32K-token inputs | 3.42 | 3.00 |
| Per-user decode speed (tok/s) | Request latency (s) | ||||||
|---|---|---|---|---|---|---|---|
| Reasoning | GPU | Conc. | No drafter | DFlash | Speedup | No drafter | DFlash |
| On | RTX PRO 6000 | 1 | 124.7 | 341.2 | 2.74 | 5.3 | 2.0 |
| On | RTX PRO 6000 | 8 | 115.7 | 313.7 | 2.71 | 5.9 | 2.2 |
| On | DGX Spark | 1 | 20.1 | 71.6 | 3.56 | 31.7 | 9.1 |
| On | DGX Spark | 8 | 23.3 | 66.3 | 2.85 | 28.7 | 10.1 |
| Off | RTX PRO 6000 | 1 | 126.5 | 302.0 | 2.39 | 3.4 | 1.4 |
| Model and decoder | RTX 5070 | RTX Spark N1X |
|---|---|---|
| (tokens/s) | (tokens/s) | |
| Gemma-4-E4B-it + MTP | 225.9 | 118.4 |
| Nemotron 3.5 Bolt E4B 4GB + DFlash | 272.6 | 124.4 |
| Relative throughput |
Small Language Models
Small language models are compact neural language systems designed to deliver useful text and reasoning capabilities with limited compute, memory, and energy.
See all 302 papers