Gnani-Evon-v3.3-30B-A3B is a long-context hybrid Mamba-Transformer mixture-of-experts model for reasoning, instruction following, and tool use in English and eight Indian languages. Starting from NVIDIA's Nemotron-3-Nano-30B-A3B checkpoint, it retains the 52-layer backbone and changes the vocabulary, training data, and post-training objectives. An in-place expansion from 131,072 to 163,072 tokens reduces mean Indian-language fertility by 38.6%. The new embedding rows are initialized from their upstream decompositions and warmed before the backbone is updated. Continual pretraining then adds broad English-Indic data, high-quality annealing, and direct training at a 131,072-token sequence length. The central result appears after post-training. Reinforcement learning across four verifiable task environments improves mathematics, graduate-level science, and instruction following, but degrades Indic extractive question answering because the rewards do not require an answer in the requested language. The share of Indic extraction items answered in English rises from 8.7% to 91.7%. Accuracy conditional on answering in the target script roughly doubles, which is consistent with preserved reading ability but is measured on a self-selected subset. Two unreleased studies test repairs. A scoped Indic reward restores the language convention. A 26B-A4B generative judge outperforms a 235B-parameter model on RM-Bench, and distillation restores 81% of the capabilities displaced by optimizing against it. Relative to the architecture-identical starting checkpoint, the released model gains 5.2 points on an Indic aggregate while giving up 1.7 points of English retention. The unreleased Indic-aware branch trades further, gaining 14.6 Indic points at a cost of 5.4 in English. A planned FP8 release reduces the artifact to 32GB with near-zero mean change across 32 paired benchmarks.
Figures & tables
Stage
Artifact
What changed
0
NVIDIA base checkpoint
Nemotron 3 Nano 30B-A3B Base BF16, upstream revision 97ab8012 , with a 131,072-token vocabulary.
1
Tokenizer and initialization
A 32,000-entry continued-BPE extension, portable tokenizer backend, and constituent-mean initialization produce the 163,072-row vocabulary matrices.
2
Embedding warmup
New vocabulary rows are adapted on a 90% Indic / 10% English anchor mixture while inherited parameters remain fixed.
High-quality 8,192-token continuation from the selected phase-1 checkpoint.
5
CPT long context
Direct 131,072-token NoPE training through iteration 372.
Table 1: Production ancestry of the released RL checkpoint and its planned FP8 derivative.
Hyperparameter
Value
Total parameters
31,749,975,232 (31.75B)
Active parameters
3,313,773,760 (3.31B; NVIDIA convention)
Layer count
52 total: 23 Mamba-2, 23 MoE, 6 GQA
Model dimension
2,688
Attention heads / KV heads
32 / 2
Attention head dimension
128
Table 2: Architecture inherited from the Nemotron 3 Nano reference configuration.
Measure
CPT checkpoint
Evon-RL-Gen
Evon-RL-Indic
Recall average
57.1
84.0
81.2
Abstention on unanswerable questions
98.1
99.8
99.7
IndicQA F1, HELMET reader slice
54.3
60.5
60.1
Table 3: Long-context results under matched completion prompting. Evon-RL-Indic is an unreleased research branch.
Figure 1: Tokenizer fertility on the held-out Samanantar evaluation slice. Lower is better. Percent labels show change relative to the upstream tokenizer.
Language
Upstream
Evon v3.3
Change
Hindi
2.0513
1.6568
−19.2%
Bengali
3.0598
2.1416
−30.0%
Gujarati
3.6086
1.9099
−47.1%
Kannada
3.8643
2.3377
−39.5%
Malayalam
4.9654
2.7684
−44.3%
Marathi
3.0981
1.9387
−37.4%
Table 4: Per-language tokenizer fertility for the 163,072-entry tokenizer carried by the documented model lineage.
Language
Native T/W
Romanized T/W
Ratio
Bengali
2.106
3.355
1.59
Gujarati
1.917
3.168
1.65
Hindi
1.649
2.731
1.66
Kannada
2.343
4.219
1.80
Malayalam
2.832
4.958
1.75
Marathi
1.944
3.767
1.94
Table 5: Native-script versus controlled ITRANS romanized fertility for the production tokenizer.
Setting
Value
Trainable parameters
New vocabulary rows in the untied input and output matrices
Frozen parameters
Backbone and all inherited vocabulary rows
Data mixture
90% Indic, 10% English anchor
Training tokens
∼ 3B
Peak learning rate
1×10−4
Parallelism
TP=1, EP=8, PP=1
Table 6: Embedding-only warmup configuration.
Figure 2: English-quality recovery across tokenizer initialization and embedding-only warmup.
Group
Examples
Role
Length and structure
word count, mean word length, mean line length
soft score
Script quality
target-script ratio, Latin ratio, invalid-character ratio
soft score / gate
Repetition
repeated words, characters, and n -grams
soft score
Web noise
symbol ratio, URL density, boilerplate density
soft score
Language fit
per-language KenLM log perplexity
soft score
Safety
NSFW and unsafe-content ratios
hard gate
Table 7: Quality-scoring signal groups.
Figure 3: Learning-rate selection. Perplexity and downstream retention rank the candidates differently.
Stage
s/iter
GFLOP/tok
tok/s/GPU
TFLOP/s/GPU
MFU
Broad CPT, 8k, CP=1
6.05
22.79
10,826
246.8
24.95%
Annealing, 8k, CP=1
5.84
22.79
11,226
255.9
25.87%
Long context, 128k, CP=8
159.77
40.91
6,563
268.5
27.15%
Table 8: Measured steady-state performance across the three CPT stages. MFU uses the 989 TFLOP/s BF16 peak.
Figure 4: Realized production CPT token counts. The 128K extension consumed 99.86B of its 100B budget.
Stage
Context
Realized tokens
Role
Broad CPT
8,192
∼ 974.49B
Diverse English–Indic skill and coverage phase.
HQ annealing
8,192
194.89B
Higher-quality continuation initialized from the best broad-CPT validation checkpoint.
Long-context extension
131,072
∼ 99.86B
Direct NoPE long-document training; 372 of ∼ 373 target iterations.
Table 9: Planned and realized continual-pretraining stages.
Figure 5: Matched CPT checkpoint trajectory. The upper panel shows Indic uplift and English retention; the lower panel shows FLORES-IN bits per byte. The divider marks the start of annealing.
Benchmark
Metric
Base
Phase 1
Annealing
Long context
Indic accuracy macro
accuracy (%)
41.09
46.83
46.63
46.83
MILU
accuracy (%)
40.63
53.54
53.52
53.24
ARC-Challenge-Indic
accuracy (%)
35.23
48.20
48.20
47.09
IndicGLUE-selected
score (%)
52.12
57.86
57.22
57.97
IndicMMLU-Pro
accuracy (%)
14.44
20.03
20.02
19.44
Indic BPB overall
bits/byte ↓
0.8453
0.6189
0.6132
0.6177
Table 10: Selected matched results across the CPT stages. The long-context column is Evon-Base.
Bucket
Weight
Shards
Pool tokens
Indic long documents
0.22
9
22.0B
Indic high-quality
0.15
132
15.0B
English long documents
0.20
7
20.0B
Mathematics and STEM
0.14
1
14.0B
Long-context supervised data
0.10
3
10.0B
Code
0.08
1
8.0B
Table 11: Composition of the 100B-token long-context pool. The selected run consumed ∼ 99.86B tokens.
Step
Disposition
Role
SFT-phase-1
retained ancestor
first supervised curriculum stage
SFT-phase-2
retained ancestor
second supervised stage
intervening continuation
discarded
affected by data-point splitting; not in the released lineage
SFT-rebuild-3 epoch 1
retained branch point
first pass over the rebuilt SFT-rebuild-3 corpus. Iteration 2,000 is the branch point for epoch 2
Evon-Instruct (SFT-rebuild-3 epoch 2)
selected checkpoint
epoch-2 continuation from SFT-rebuild-3 epoch-1 iteration 2,000; † direct initialization for RL
† Epoch 2 restarts from SFT-rebuild-3 epoch-1 iteration 2,000 with fresh optimizer state; later epoch-1 checkpoints showed elevated MoE load-balancing loss.
Table 12: SFT lineage relevant to the released RL checkpoint.
Stage
Materialized rows
English share
Principal change
SFT-phase-1
∼ 12.5M
50.7%
Era-1 phase-1 pool; instruction following, mathematics, and code dominate.
SFT-phase-2
∼ 14.7M
54.6%
Era-1 phase-2 pool; code, science, and agentic shares increase.
SFT-rebuild-3
∼ 15.2M
64.0%
Era-2 rebuild with a larger English share.
Table 13: Materialized corpus summary across the retained SFT lineage.
Setting
Value
Maximum sequence length
131,072
Precision
BF16
GPUs
256 NVIDIA H200
Parallelism
TP=1, PP=1, CP=4, EP=8, DP=64
Global / micro batch
256 / 1
Optimizer
Adam, β2=0.95 , ϵ=10−8
Table 14: SFT configuration for the long-context production family.
Setting
Value
Initialization
Evon-Instruct
Algorithm
GRPO with leave-one-out baseline and whole-group standard-deviation normalization
Table 15: Released Evon-RL-Gen training configuration.
Figure 6: Released-run validation rewards. Structured output and workplace assistance improve most; the curves are optimization diagnostics, not benchmark accuracies.
Figure 7: English retention and Indic capability. The arrow shows movement from the architecture-identical starting checkpoint; reference models use their published sampling settings.
Category
Accuracy (%)
Harmless control
97.93
Tricky / ambiguous
97.77
Offensive or hate speech
89.80
Harmful instructions
85.62
Misinformation / conspiracy
83.50
Health misinformation
80.41
Table 16: IndicSafe accuracy by harm category.
Figure 8: Per-benchmark change for the unreleased Indic-aware branch. Positive values favour the branch.
Figure 9: Grounded Indic instruction following and Indic extractive QA for the unreleased branch.
Benchmark
Loss to repair
Distillation
Interpolation
Extrapolation
XQuAD-English
38.25
+74.5%
+65.8%
−61.2%
XQuAD-Indic
32.77
+89.9%
+32.1%
−25.2%
IndicQA
24.45
+73.7%
+21.0%
−24.0%
XorQA-Indic
24.46
+99.1%
+46.1%
−34.6%
Macro
+84.3%
+41.3%
−36.2%
Table 17: Share of extractive-QA loss recovered by each repair. Negative values indicate further loss.
Component
Precision
Rationale
Routed expert FC1/FC2
FP8 E4M3 W8A8
dominant parameter and compute block
Shared expert FC1/FC2
FP8 E4M3 W8A8
same expert-matrix structure
Attention QKV/output and BMM
BF16
excluded by conservative recipe
Mamba input/output projections and convolution
BF16
protects recurrent sequence-mixing path
Routers / routing gates
BF16
protects token-to-expert decisions
Embeddings, final norm, LM head
BF16
protects vocabulary and output calibration
Table 18: Precision allocation in the FP8 artifact planned for release.
Figure 10: Paired BF16 and FP8 serving throughput on one NVIDIA H200 using the same serving stack.
Metric
BF16
FP8
Warm TTFT, concurrency 1
348ms
341ms
Warm generation rate, concurrency 1
345.7 tok/s
387.3 tok/s
Generation-rate p50, concurrency 1
341.6 tok/s
382.9 tok/s
Aggregate generation rate, concurrency 8
1,059 tok/s
1,406 tok/s
Aggregate generation rate, concurrency 16
1,165 tok/s
2,135 tok/s
Usable concurrency under latency/rate gate
∼ 8
∼ 16
Table 19: Paired one-H200 serving measurements.
Figure 11: Selected paired accuracy deltas after FP8 PTQ. Small benchmark samples should not be over-interpreted.
Released-lineage training scope
Recorded GPU-hours
Embedding warmup
139
CPT broad 974B phase
∼ 25,147
CPT annealing phase
∼ 4,988
Complete 128K CPT long-context stage, allocated cluster time
4,977
Entire retained SFT curriculum
∼ 17,205
Released Evon-RL-Gen production run
763
Table 20: Recorded GPU-hours for relevant checkpoints in the released lineage.
Appendix figures & tables42 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Fertility of the exploratory 64K mixv5 tokenizer against the upstream tokenizer. This tokenizer was not used by the production model.
Language
Base Nemotron
mixv5 64K
Change
Hindi
2.0513
1.4924
−27.2%
Bengali
3.0598
2.0637
−32.6%
Gujarati
3.6086
1.7402
−51.8%
Kannada
3.8643
2.1830
−43.5%
Malayalam
4.9654
2.8548
−42.5%
Marathi
3.0981
1.8135
−41.5%
Appendix
Table 21: Per-language fertility for mixv5, evaluated on 2,000 Samanantar samples per language with seed 42.
Stage
Logged n
Median
p5–p95
Max / steady
CPT phase 1
5,808
6.05
5.63–6.18
16.73 / 6.05
CPT annealing
1,161
5.84
5.81–5.97
20.80 / 5.84
CPT long context
37
159.89
157.73–162.65
173.96 / 159.91
SFT-phase-1
207
16.71
15.43–18.09
52.85 / 16.70
SFT-phase-2
383
17.40
16.55–18.31
49.99 / 17.39
Evon-Instruct
266
17.58
16.74–18.42
60.07 / 17.57
Appendix
Table 22: Optimizer-step timing across the production CPT and SFT stages.
Stage
TFLOP/s/GPU
Aggregate TFLOP/s
Tokens/s
Tokens/s per GPU
Seq./s
MFU
SFT-phase-1
321.2
82.2k
1.938M
7,570
14.8
32.5%
SFT-phase-2
308.4
79.0k
1.894M
7,398
14.4
31.2%
Evon-Instruct
305.2
78.1k
1.870M
7,304
14.3
30.9%
Appendix
Table 23: Measured steady SFT throughput by retained stage.
Figure 13: MILU accuracy before and after the 55:45 English–Indic 120B CPT ablation. Labels show percentage-point change. This run is not part of the released checkpoint ancestry.
Language
Base
120B CPT
Δ
Kannada
32.8
47.2
+14.4
Tamil
34.2
46.6
+12.4
Gujarati
34.4
46.5
+12.2
Telugu
35.8
47.9
+12.1
Malayalam
35.5
45.7
+10.1
Bengali
39.4
47.7
+8.4
Appendix
Table 24: Per-language MILU results from the 120B ablation. Displayed endpoints are rounded to one decimal place; deltas are computed from the underlying higher-precision scores.
Language
MILU uplift
HQ availability
Weight
Est. epochs
Telugu
+12.1
20B
16%
1.03
Tamil
+12.4
12B
15%
1.60
Kannada
+14.4
6B
13%
2.78
Gujarati
+12.2
8B
12%
1.92
Malayalam
+10.1
17B
12%
0.90
Bengali
+8.4
19B
12%
0.81
Appendix
Table 25: Language weights derived from 120B-ablation uplift and high-quality availability. MILU uplifts use the underlying higher-precision scores.
Field
Value
Initialization
embedding-warmed 163,072-vocabulary checkpoint
Precision
BF16
Sequence length
8,192 for broad CPT and annealing
Broad-CPT global / micro batch
2,048 / 1
Parallelism
TP=1, EP=8, PP=1; data parallelism fills remaining ranks
Figure 14: CPT phase-1 training traces through iteration 58,084.
Figure 15: CPT annealing traces. Learning-rate decay begins late while validation loss remains near 1.0.
Figure 16: CPT long-context traces through iteration 372; the early transition settles into stable training.
Group
Target
Realized
Share
Δ
English
536.0B
532.7B
54.66%
−3.3 B
Indic (12 languages)
438.5B
436.6B
44.80%
−1.9 B
Other multilingual
–
5.3B
0.54%
–
Total
974.5B
974.5B
100%
Appendix
Table 27: Broad-CPT composition from the 184 retained shard weights.
Language
Tokens
Share
Language
Tokens
Share
Hindi
132.4B
13.59%
Marathi
17.1B
1.76%
Tamil
69.3B
7.11%
Odia
4.4B
0.45%
Telugu
66.6B
6.84%
Punjabi
1.6B
0.17%
Bengali
53.8B
5.52%
Urdu
1.4B
0.14%
Malayalam
33.5B
3.43%
Nepali
1.3B
0.14%
Kannada
28.7B
2.95%
Appendix
Table 28: Realized Indic allocation in broad CPT, by language. Only the first eight languages received dedicated vocabulary expansion (Section 3 ); Odia, Punjabi, Urdu and Nepali receive pretraining data on a small budget without tokenizer support.
Bucket
Allocated
Share
Reasoning
62.70B
32.2%
Knowledge retention
34.20B
17.5%
Mathematics and STEM
31.52B
16.2%
Code
25.65B
13.2%
Indic high-quality
17.10B
8.8%
Instruction
10.10B
5.2%
Appendix
Table 29: Annealing composition by bucket from the manifest’s per-dataset allocations.
Benchmark
Metric
Base
Phase 1
Annealing
Long ctx.
Δ
Indic capability
Indic macro (excl. BharatMath)
accuracy
41.09
46.83
46.63
46.83
+5.74
Indic macro (incl. BharatMath)
accuracy
39.54
46.03
45.66
46.01
+6.47
ARC-Challenge-Indic
accuracy
35.23
48.20
48.20
47.09
+11.86
Belebele
accuracy
37.49
42.17
42.27
42.54
+5.05
BharatMath
accuracy
29.99
41.06
39.70
40.95
+10.96
Appendix
Table 30: Per-benchmark matched trajectory across the three CPT stages.
Category
8K
16K
32K
64K
128K
Recall
95.9
81.4
57.0
37.3
14.0
RAG
62.7
60.2
56.6
49.4
37.2
ICL
59.2
71.7
78.6
81.8
81.7
LongQA
28.8
38.5
36.5
40.6
26.8
Indic-RAG
57.1
59.1
54.2
46.8
–
Indic-Reader
17.9
17.0
16.0
15.2
–
Appendix
Table 31: HELMET evaluation of Evon-Base, the checkpoint that concludes long-context continual pretraining.
Setting
Value
Public stage
Evon-Instruct
Export source
selected export, verified from export metadata
Scheduled sequence positions through export
80.53B, including padding
Approximate loss-bearing positions
∼ 15B (18.2%; assistant-only loss mask)
Trainable-rate measurement
95,515 / 524,288 positions over four packed-parquet rows
Intra-pack efficiency
99.98%
Appendix
Table 32: Training record for the SFT checkpoint used to initialize RL.
Stage
Materialized rows
Buckets
English
Key character
SFT-phase-1
∼ 12.5M
12
50.7%
Era-1 phase-1 pool; instruction following, mathematics, and code dominate.
SFT-phase-2
∼ 14.7M
14
54.6%
Era-1 phase-2 pool; code, science, and agentic shares increase.
SFT-rebuild-3
∼ 15.2M
15
64.0%
Era-2 rebuild with a larger English share.
Appendix
Table 33: Stage totals for the retained SFT corpus chain.
Language
SFT-phase-1
SFT-phase-2
SFT-rebuild-3
English
50.7%
54.6%
64.0%
Hindi
16.5%
15.2%
11.1%
Tamil
6.5%
6.0%
4.8%
Telugu
6.5%
6.0%
4.8%
Kannada
6.5%
6.0%
4.8%
Malayalam
6.5%
6.0%
4.8%
Appendix
Table 34: Materialized-row language mix across the retained SFT chain.
Source family
SFT-phase-1
SFT-phase-2
SFT-rebuild-3
Mathematics (incl. proofs)
∼ 29%
∼ 28%
∼ 15%
Code (incl. competitive programming)
∼ 24%
∼ 29%
∼ 13%
Instruction following and chat
∼ 28%
∼ 15%
∼ 19%
Science and reasoning
∼ 17%
∼ 19%
∼ 9%
Agentic, tool use, and software engineering
∼ 2%
∼ 9%
∼ 12%
Indic dialogue
–
–
∼ 8%
Appendix
Table 35: Approximate source-family composition across the retained SFT corpus chain (share of materialized rows).
Figure 17: Weights & Biases traces for SFT-phase-1. The panels report the MoE sequence load-balancing loss, gradient norm, cosine learning-rate schedule, and assistant-masked language-model loss. Loss plateaus and transitions align with changes in the sequential curriculum buckets.
Figure 18: Weights & Biases traces for SFT-phase-2. The assistant-masked loss trends gradually downward under the cosine learning-rate schedule, while isolated gradient-norm and load-balancing spikes remain transient.
Figure 19: Final SFT-rebuild-3 SFT traces through iteration 2,665; Evon-Instruct lies within this run.
Benchmark
Metric
Final SFT ckpt
RL ckpt step 40
Δ (RL − SFT)
Math & Reasoning
MATH
exact match
48.69
84.76
+36.07
MATH-500
exact match
48.80
84.00
+35.20
GSM8K
exact match
85.75
94.31
+8.56
MGSM-English
exact match
91.60
98.40
+6.80
MGSM-Indic
exact match
79.60
84.60
+5.00
Appendix
Table 36: Matched evaluation of Evon-Instruct and the released Evon-RL-Gen checkpoint.
Checkpoint
Macro F1
B1b
B2
B3
OK
Evon-Instruct (SFT initialization)
40.38
8.7%
5.8%
62.6%
21.6%
Released Evon-RL-Gen
16.59
91.7%
0.1%
2.5%
2.8%
Indic-aware sibling (Sec. K )
53.43
15.3%
6.0%
45.4%
31.9%
Appendix
Table 37: XQuAD-Indic failure taxonomy, equal-weight macro over eight languages.
Measurement
Ground strict
Translated strict
Ground in-script
Initial matched evaluation
28.66
47.45
5.4%
Dedicated IndicIFEval evaluation ∗
63.88
–
54.2%
FP8-paired BF16 baseline
33.05
49.87
–
Appendix
Table 38: Recorded IndicIFEval sensitivity across evaluation states.
Target-script rate
Conditional EM
Conditional F1
Benchmark
SFT
RL
SFT
RL
SFT
RL
XQuAD-Indic
89.9
5.4
23.22
51.50
37.69
61.19
IndicQA
86.9
24.3
20.24
45.23
27.00
50.54
Appendix
Table 39: Target-script rate and conditional accuracy before and after released RL.
Figure 20: Experimental Evon-RL-Indic training-row composition after difficulty filtering, environment caps, and English-row diversion. This mixture is not used by the released model.
Pool
Rows
Final share
Mathematics
6,192
40.5%
Indic instruction following
3,179
20.8%
Structured outputs
1,932
12.6%
Workplace assistant
1,804
11.8%
English guardrail
1,509
9.9%
MCQA
684
4.5%
Appendix
Table 40: Cap-driven composition of the experimental Evon-RL-Indic input mix. Component shares are rounded to one decimal place and therefore sum to 100.1%; row counts sum exactly to 15,300.
Setting
Value
Algorithm
GRPO with leave-one-out baseline and whole-group standard-deviation normalization
Figure 21: Validation rewards for the Indic-aware branch. These optimization diagnostics are not benchmark accuracies.
Figure 22: Selected Evon-Instruct to experimental Evon-RL-Indic results. Deltas are percentage points; the checkpoint is not released.
Benchmark
Evon-Instruct
Evon-RL-Indic
Δ
IndicIFEval-ground, prompt strict
40.52
60.00
+19.48
IndicIFEval-translated, prompt strict
37.51
63.27
+25.76
XQuAD-Indic, F1
36.51
50.44
+13.93
MATH
48.69
86.76
+38.07
GSM8K
85.75
91.58
+5.83
IFEval, prompt strict
66.36
86.88
+20.52
Appendix
Table 42: Matched evaluation of the unreleased Indic-aware checkpoint against its SFT initialization.
Figure 23: Selected matched differences between experimental Evon-RL-Indic and released Evon-RL-Gen. Positive values favor the experimental checkpoint.
Benchmark
Evon-RL-Gen
Evon-RL-Indic
Δ
MultiChallenge
0.4925
0.5000
+0.75
TauBench airline
0.3733
0.4400
+6.67
TauBench retail
0.1520
0.2807
+12.87
TauBench telecom
0.0936
0.3626
+26.90
SWE-bench Verified
0.2280
0.1220
−10.60
Appendix
Table 43: Additional released-to-experimental agentic comparison. Deltas are percentage points.
Category
Evon-Instruct
Evon-RL-Indic
Headline, renormalized over 17 categories
53.81
51.16
Parallel / parallel-multiple
69.5 / 80.0
0.5 / 0.5
Live parallel / live parallel-multiple
75.0 / 75.0
0.0 / 0.0
Simple Python / multiple
86.8 / 81.0
94.5 / 93.0
Multi-turn base
34.0
58.0
Appendix
Table 44: BFCL v4 decomposition.
Setting
Evon-RL-Gen (released)
Evon-RL-Indic (branch)
Optimizer and learning rate
Optimizer
Adam, distributed, BF16, weight decay 0.01
Same
Peak learning rate
5×10−6
Same
Minimum learning rate
5×10−7
Same
Schedule
constant, 13-iteration warmup from the floor
Same
Gradient clipping
1.0
Same
Appendix
Table 45: Training configuration for the released run and the Indic-aware branch, both read at step 40. “Same” means the two runs carry an identical setting.
Figure 24: Nominal environment shares before dynamic sampling for the released and Indic-aware runs.
Term
Reward
Correct relative ranking
+4
Calibration
−1.2H1.5(dr)−0.6[H1(dh1)+H1(dh2)]
Absolute/relative inconsistency
−0.5
Unparseable verdict
−8
Parseable range
clipped to [−6,4] , then adjusted for batch-level rank balance
Appendix
Table 46: Reward terms used to train the generative judge.
RM-Bench
Model
Chat
Math
Code
Safety
Easy
Normal
Hard
Overall
JudgeBench
Qwen3-Nemotron-235B-A22B-GenRM
76.5
96.9
81.4
94.4
94.0
90.5
77.4
87.3
87.4
Ours, 26B-A4B
76.1
93.9
84.4
95.7
94.1
92.0
86.4
90.8
88.3
Appendix
Table 47: The 26B-A4B judge against a 235B-parameter generative reward model.
Benchmark
Change
LiveCodeBench
+13.2
MMLU-Pro
+8.8
IndicMMLU-Pro
+6.3
MMLU
+5.8
GPQA Diamond
+5.8
AGIEval
+5.5
Appendix
Table 48: Selected gains from policy optimization against the judge, relative to Evon-RL-Indic.
Benchmark
BF16
FP8
Δ
AGIEval English
0.7347
0.7440
+0.0093
AIME 2025
0.7438
0.7500
+0.0062
AIME 2026
0.7458
0.7729
+0.0271
ARC-Challenge
0.9590
0.9582
−0.0008
BBQ
0.8869
0.8819
−0.0050
Beyond-AIME
0.5356
0.5337
−0.0019
Appendix
Table 49: All 32 paired BF16 and FP8 accuracy results. Deltas are FP8 minus BF16.
Building capable language models for India requires scalable multilingual data and robust evaluation. We introduce Nemotron-Pretraining-Indic-v1, a synthetic pretraining corpus of approximately 2.83 trillion tokens covering all 22 scheduled Indian languages in native script and twelve in romanized form. It is the largest openly released Indic pretraining corpus. Using our GPU-accelerated pipeline, we translate curated English documents, assess translation quality, and repair low-quality outputs while preserving technical content and document structure and maintaining source traceability. Through continued-pretraining experiments across twelve languages, we show that combining our synthetic corpus with natural Indic text improves knowledge and reasoning over a public-corpus baseline. Additional experiments show how data volume, script coverage, and Indic training share affect English performance. We also introduce the Indic-LLM-Eval-Suite, a multilingual benchmark suite for evaluating Indic language models across chat, instruction following, reasoning, knowledge, code, and translation. The suite combines existing Indic benchmarks with task-aware translations of English benchmarks, using automated quality assessment, targeted correction, and native-speaker review in a human-in-the-loop pipeline. We use the suite to comprehensively benchmark contemporary open-weight Indic and multilingual models across capabilities and languages, revealing substantial variation across tasks and languages. Together, these resources provide a scalable foundation for developing and evaluating more capable Indic language models.
Instruction-following benchmarks remain predominantly English-centric, leaving a critical evaluation gap for the hundreds of millions of Indic language speakers. We introduce IndicIFEval, a benchmark evaluating constrained generation of LLMs across 14 Indic languages using automatically verifiable, rule-based instructions. It comprises around 800 human-verified examples per language spread across two complementary subsets: IndicIFEval-Ground, translated prompts from IFEval (Zhou et al., 2023) carefully localized for Indic contexts, and IndicIFEval-Ground, synthetically generated instructions grounded in native Indic content. We conduct a comprehensive evaluation of major open-weight and proprietary models spanning both reasoning and non-reasoning models. While models maintain strong adherence to formatting constraints, they struggle significantly with lexical and cross-lingual tasks -- and despite progress in high-resource languages, instruction-following across the broader Indic family lags significantly behind English. We release IndicIFEval and its evaluation scripts to support progress on multilingual constrained generation (http://github.com/ai4bharat/IndicIFEval).
Thanmay Jayakumar, Mohammed Safi Ur Rahman Khan, Raj Dabre +2
Nilekani Centre at AI4Bharat · Indian Institute of Technology Madras, India · IT University of Copenhagen +1
We adapted Nemotron-3-Nano-30B-A3B for eight Indian languages without changing its backbone architecture. We expanded the vocabulary in place from 131,072 to 163,072 entries using continued byte-pair encoding and AdaptMix rebalancing, reducing tokens per word by 38.6%. We initialized new input and output rows by averaging their original constituent rows, then trained them on 90% Indic and 10% English text before updating the backbone. Continual pretraining then progressed through broad training, high-quality annealing, and long-context training over approximately 1.27 trillion tokens. The annealed checkpoint gained 12.9 points on the Indic MILU benchmark but lost 3.4 on English MMLU. We also report practical negative results. Direct token addition underperformed continued byte-pair encoding. AdamW weight decay changed rows intended to stay fixed. Perplexity on an Indic-heavy holdout did not select the learning rate that best preserved English performance.