We introduce Clover-35B-A3B-Preview (Clover), a mixture-of-experts diffusion language model with 35 billion total parameters and 3 billion activated parameters, supporting thinking-mode generation. Clover adapts Gated Delta Networks to block diffusion by combining recurrent memory of previously generated context with bidirectional softmax attention inside the current block. Historical memory readout competes with current-block tokens in a shared softmax, with learnable block-size calibration and a separate, zero-initialized right-context convolution branch. Clover leads the seven evaluated agent metrics, including 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1. On internal agent workloads, it reaches a reported generation speed of 680 tokens/s using one NVIDIA GB300 GPU at BF16 precision and batch size one. We organize the evaluation around model quality, drafting and verification costs, and deployment under constrained memory. We also describe a decision-model variant that generates answers to related questions in one denoising pass. On an internal set of 500 groups, joint prediction raises question accuracy from 90.37% to 93.88% and all-correct group accuracy from 65.4% to 77.0%.
Figures & tables
Figure 1: Clover architecture. (a) Three BD-GDN–dMoE layers and one full-attention–dMoE layer form a group repeated ten times. Backbone normalizations are RMSNorm. (b) Pink boxes are linear projections; the Q , K , V , and Z paths carry query, key, value, and output-gate features. Stacked boxes indicate multiple heads: 16 query heads, 16 key heads, and 32 value heads; each query/key head serves two value heads. The historical readout from Sb and block-local information are combined by the shared softmax. Output RMSNorm operates per value head, and gated head outputs are concatenated before the output projection. (c) dMoE combines token-level top-eight routing within a block coreset with a shared expert. Solid lines carry features, dashed arrows show routing, and gray experts are inactive. For one token, pi is the normalized weight of routed expert i , and gs is the shared expert’s sigmoid gate.
Figure 2: The two block-diffusion adaptations in BD-GDN. (a) The query-specific history readout rj from ( 9 ) competes with Nb current-block values in one softmax. Its logit includes learnable block-size calibration. (b) Independent causal and right-context depthwise kernels are summed before SiLU. The right taps start at zero and read only valid successors in the current block; clean prefill and commit use the causal kernel.
Benchmark
\ours Clover 35B-A3B-Preview
LLaDA2.2 flash 104B-A7.4B
DiffusionGemma 26B-A4B
Mercury 2.5
Qwen3.6 35B-A3B
Qwen3.5 35B-A3B
Knowledge
GPQA Diamond
\first 83.8
47.5
68.2
76.3
\second 83.3
78.3
HLE text-only
12.8
5.3
10.9
8.9
\second 21.2
\first 21.6
MMLU-Pro
\second 84.8
71.0
80.9
84.1
84.7
\first 85.2
Math
MATH-500
\second 96.2
93.2
94.4
93.8
\second 96.2
\first 97.2
Table 1: Main benchmark results. All models are evaluated in-house under the protocol described in the text. Higher is better; \first red bold marks the best value and \second blue bold the second-best distinct value within each row, including ties. Shading identifies Clover. GDPval-AA v2 uses Elo, so its differences are not percentage points.
Workload
Model
A (tokens/round)
Decode TPS
Clover-35B-A3B-Preview
6.8743
465.2
MATH-500
Qwen3.6-35B-A3B + DFlash 2
6.5897
604.3
Clover-35B-A3B-Preview
8.2096
556.6
Agent replay
Qwen3.6-35B-A3B + DFlash 2
4.1548
396.7
Clover-35B-A3B-Preview
4.9970
356.7
Daily chat
Qwen3.6-35B-A3B + DFlash 2
2.9500
286.5
Table 2: H200 performance on three workloads. A is committed tokens per verification round; Decode TPS is defined in Eq. 18 .
Figure 3: Daily chat: answer length, body-only decode throughput, and the fraction of fully accepted draft rounds. Filled circles show each metric; in the first panel, they indicate mean body length, while open diamonds indicate the minimum. Connecting segments span minimum to mean. Thinking is excluded from body-only TPS.
Figure 4: Full agent replay: end-to-end throughput and committed tokens per verification round. Solid/hatched bars use block size or verification budget 8/16. Clover is faster without penalties, but can sometimes produce repetitive outputs.
Benchmark
\ours Clover 35B-A3B-Preview
MiniCPM-V 4.7 35B-A3B
LFM2.5 24B-A2B
Nemotron 3.5 Lightning (30B-A3B)
Gemma 4 26B-A4B
IFBench
72.3
47.7
74.7
71.9
\first 77.2 *
GPQA Diamond
\first 83.8
81.3
74.4
75.4
82.3
MMLU-Pro
\first 84.8
83.5
—
81.9
82.6
HLE text-only
12.8
9.4
7.4
11.7
\first 17.2
LiveCodeBench v6
72.6
56.6
—
—
\first 77.1
BFCL-v4 1
\first 67.9
—
65.1
53.8
57.1
Table 3: Comparison with efficient and on-device models. Higher is better; \first red bold marks the best reported value in each row, shading identifies Clover, and dashes denote unavailable results. Sources and harnesses differ as detailed below. These scores are not asserted to come from Clover’s INT4 deployment.
Model / placement
Expert weights
GPU / CPU experts 1
CPU backend
Accuracy (%)
Decode (tokens/s)
MiniCPM-V 4.7
INT4 g128
128 / 128
AVX-VNNI-256
97.4
49.51
Qwen3.6-35B-A3B
INT4 g128
128 / 128
AVX-VNNI-256
93.8
47.04
Clover-35B-A3B-Preview
Original placement, top-8
INT4 g32
96 / 160
AVX-512
96.8
66.64
Optimized placement, top-8
INT4 g32
96 / 160
AVX-512
97.6
97.52
\highlightrow Optimized placement, top-6
INT4 g32
96 / 160
AVX-512
96.0
106.25
Table 4: Full MATH-500 evaluation on one RTX 5070 Ti (16 GB), using SGLang with KTransformers. Accuracy is manually reviewed; decode throughput excludes prefill and includes thinking tokens.
Task
Mode
Question / cell accuracy
Complete-set accuracy
Multiple500
Independent
90.37
65.4
\highlightrow Multiple500
Joint
93.88
77.0
Sudoku
Independent
78.30
22.0
\highlightrow Sudoku
Joint
95.60
76.0
Table 5: Joint versus independent prediction with the same Clover decision model. All values are percentages; higher is better. Complete-set accuracy requires every question in a group or every missing cell in a board to be correct.
Model
JevBench accuracy (%)
Decision Index 0.2.1
Jev v1.13
86.58
57.93
\highlightrow Clover-Decision
88.31
58.23
Table 6: External decision benchmarks. JevBench reports accuracy over 231 questions; Decision Index reports the full suite’s composite score. Both metrics are higher-is-better. Jev v1.13 is a frozen API evaluation reference.
Figure 5: Group latency as the number of questions increases. Measurements use the same GPU and checkpoint, with one outer task at a time and a single denoising pass per call. Each point averages 16 measurements: four repetitions on each of four fixed cases. We vary group size by selecting the first 1, 2, 4, or 8 questions from each case.