The development of frontier models is commonly perceived to be the exclusive remit of a small number of heavily funded players, creating an information, economic and power asymmetry between developers and the diverse user base of modern AI. Recent public discourse acknowledges this concern, calling for SovereignAI (an organisation's capability to independently build, deploy and govern AI use), but offers little concrete advice on how this can be achieved in the short term under a diversity of funding settings. We argue that frontier performance is achievable by a wide range of institutions through Continual Learning on readily available open-weight models. Unlike limited approaches such as small-scale fine-tuning, prompt engineering, or tool-augmentation of a frozen model, our approach exploits a modern mid- & post-training stack while introducing safeguards that preserve both plasticity and stability at each stage, making the minimal number of high-impact interventions on the parameters. This yields gains comparable to those typically seen across multiple successive model generations, at compute and personnel budgets substantially lower than commonly thought, making ownership of large parts of the SovereignAI stack (model, tool infrastructure, values & data privacy) viable for far more actors. We demonstrate this with Thomson, a general-purpose frontier model trained with an enhanced focus on high-stakes professional work. Thomson performs competitively with recent frontier models across agentic tasks, safety, legal, tax & multilingualism, and large-scale Deep Research. Evaluations show a distinctive π-shaped pattern: distinct improvements across a wide range of capabilities, including those not explicitly targeted, while almost completely eliminating the forgetting problem common to narrow domain adaptation.
Figures & tables
Figure 1: Combined Thomson-1.0-Large scores show frontier-level performance over a wide range of benchmarks. Improved scores are seen across most benchmark categories, with large gains in some areas. Please note that 1(a) shows the cost/performance trade-off on two target domain agentic tasks, while 1(c) is an aggregate score over a broader set of categories.
Domain
Benchmark
Opus 4.8
Thomson 1.0-Large
Gemini 3.1 Pro
GLM 5.2
GPT 5.4
Kimi K3
Sonnet 5
Snowdon 1.0-Large
Qwen3.5 397B
DeepSeek v4 Pro
Overall Avg.
79.5
78.5
78.0
77.2
76.5
75.7
75.7
73.6
73.0
71.9
Legal
Stanford LegalBench
81.8
82.3
84.3
82.9
82.3
83.3
81.4
82.8
78.8
76.8
Info. Retrieval
51.2
53.6
53.8
53.6
53.8
53.0
47.9
51.2
49.0
51.5
Reasoning
75.2
73.2
77.8
73.1
68.4
74.8
73.1
70.8
66.5
70.1
Classification
70.5
70.9
74.2
70.2
70.0
71.6
70.4
69.8
68.7
67.8
Doc. Processing & RAG
78.9
79.7
78.2
76.5
78.6
79.0
74.7
71.2
74.1
73.7
Table 1: Cross-domain benchmark results for reasoning-mode models. Scores are percentages; the best score in each row is shown in bold . Dashes (–) mark benchmarks that were not run for a given model. Note: Fair Adversarial testing against proprietary models was not possible due to the likely presence of unknown guardrail mechanisms. Overall Avg. is the unweighted mean over all individual benchmarks, excluding Adversarial Testing. Harvey LAB: Harvey Legal Agent Benchmark.
Domain
Benchmark
Thomson 1.0-Small
Snowdon 1.1-Small
Gemma4 31B
Qwen3.6 35B
Haiku 4.5
Overall Avg.
74.6
71.7
71.2
71.7
68.2
Legal
Stanford LegalBench
79.9
80.9
83.1
80.3
80.7
Info. Retrieval
49.6
48.6
51.9
49.4
49.2
Reasoning
68.2
67.3
71.9
64.7
65.5
Classification
70.0
70.1
69.4
70.4
68.1
Doc. Processing & RAG
78.8
71.2
76.6
74.7
43.8
Table 2: Cross-domain benchmark results for small models. The best score in each row is shown in bold . Dashes (–) mark benchmarks that were not run for a given model. All models were run with medium reasoning effort. Overall Avg. is the unweighted mean over all individual benchmarks, excluding Adversarial Testing.
Figure 2: Per-Category score improvements of Thomson-1.0-Small .
Figure 3: Blind human evaluation study of the Thomson system against systems provided by major frontier labs. Thomson-1.0-Large is given access to recent news (Reuters feed) as well as access to legal databases. External OpenAI & Anthropic models are given broad web access. Scores computed on an aggregate of over 3,000 preference-rated conversations.
Development Evals (CapTrack)
Test Evals
(only ≈10−15% of examples used)
(All examples used)
Factuality
RAGTruth, TruthfulQA, PopQA
SimpleQA, Faitheval
Multilingualism
XTREME, MGSM
XTREME ∗ , MGSM, MMMLU
Instruction Follow.
IFEval, FollowBench
IFEval, FollowBench ∗
Writing
MT-Bench (1st turn) , OASST1, ELI5
WritingBench
Coding
HumanEval, MBPP, BFCL, MNMS
SWE-Bench Pro, TerminalBench 2.1
Table 3: General-purpose evaluations used for Thomson . Aggregate scores for Development Evals are monitored during the model training process to measure and control forgetting, while Test Evals are reported as a genuine performance evaluation. None of the evaluations has dedicated training sets ( ∗ uses CapTrack examples).
Figure 4: Deep Research agent harness. A planner decomposes the research query into self-contained research topics and dispatches them to isolated workers, each a ReAct loop with a bounded tool-call budget; workers share no context, so coverage is a property of the plan rather than of lateral coordination. Tool results return through query-conditioned compaction, which discards prose but preserves citations, keeping the evidence a worker accumulates small while the addressable corpus stays large. The planner re-plans against remaining gaps or hands off to a reporter that has no tools, which is then tasked with producing a Deep Research report with in-line citations.
Figure 5: Model development pipeline for Thomson 1 . The pipeline is organised into three sequential modules with distinct development foci. Value focus : Starting from an open-weight instruction-tuned Foundation Model, we perform value re-alignment through constitutional DPO, potentially augmented with activation steering. Knowledge focus : Data-centric continual pre-training (CPT) ingests proprietary domain data, with model merging protecting generic capabilities while absorbing domain knowledge. Behaviour, skill & tool focus : Two-stage DPO (broad preference alignment, then agentic specialisation) warm-starts two-stage RL (short-context with diverse queries, then long-context end-to-end Deep Research). Agentic data from the Deep Research harness feeds DPO Stage 2 and both RL stages. This modular design allows practitioners to adjust computational investment to match sovereignty requirements.
Figure 6: Misalignment-rate vs. capability preservation. Our realignment traces a strictly better frontier than the plain-abliteration baseline, reaching any given misalignment level at substantially lower KL divergence from the base model. (a) Misalignment rate on the validation prompt set vs. KL divergence from the base model; faded markers are individual trials, solid markers joined by dashed lines are the Pareto fronts. Note: Baseline KL scores are replayed under our KL score definition for comparability, with their front selected on the original KL scores. (b) KL divergence cost to reach each misalignment rate; our realignment method incurs 2.0× – 5.0× less KL divergence scores than the baseline, suggesting better model capability preservation during ablations.
Figure 7: Realignment on the small and large models. Each panel contrasts the out-of-the-box base models ( Base Model Small/Large ) with our re-aligned checkpoints ( Snowdon Small/Large ); the Baseline Alignment Fronts are plain ablated versions of the small model. (a) Realignment score (Perspective Bench) against general capability (CapTrack): both sizes gain large points of realignment with essentially no capability loss, while baselines reach only partial re-alignment. (b) Safety, as the unsafe-request refusal rate’s drop from the out-of-the-box line, plotted against the realignment gain each method reaches. Baselines improve re-alignment only by giving up safety score, and still plateau below the re-alignment level Snowdon reaches; Snowdon attains the largest gain at a cost of 0.8% refusal. Note: models are tested in non-reasoning mode for this experiment.
Figure 8: The misaligned behaviour has a separable representation. UMAP projections of final-prompt-token residual stream activations, computed independently at each layer, for the sensitive set S (elicits the misaligned behaviour) vs. the neutral set N (does not). Points from the two prompt sets are interleaved in early layers and become separated by mid-stack. Each panel is an independent fit, so panels show separability at a depth, not trajectories across depth.
Figure 9: Fisher routing shifts the writeback towards output coordinates with lower estimated sensitivity. Coordinates are ordered from low to high Fisher value for the selected experts at layers 38 and 47. Grey shows the standard writeback b=−λv ; blue shows the Fisher-routed writeback (ours) from Equation ( 4 ). The vertical line marks the lowest- Fisher fifth of the output coordinates. Labels report the share of squared writeback mass within that fifth.
Figure 10: Data curation pipeline for continued pre-training
Figure 11: Example curation of a dense document
Figure 12: Learning rate selection and loss scaling during continual pre-training. (a) Training cross entropy across candidate learning rates under a fixed pilot budget, with the selected learning rate minimising stable training loss. (b) Loss trajectories for the Small and Large models as a function of consumed continual pretraining tokens, with dashed lines denoting power-law fits.
Figure 13: Effect of continual pre-training strength and merge coefficient on target domain performance and general capability retention under a fixed post-training budget. Colour denotes CPT strength, while each numerical annotation gives the merge coefficient assigned to the CPT checkpoint. Axes report mean standardised performance over the target and general domains.
Figure 14: Expert-authored content as the supervision signal. It decides which documents can carry a training task and which task to generate. Answers to those questions are then generated by three routes: RGC, which uses the source content as a reference inside its critique stage, and teacher models and the target policy, which do not. The resulting candidates are assembled into preference pairs.
Capability
What the assessor looks for
question-answer pairs
factual content that converts into clear, definitive questions; answers that are specific and authoritative rather than vague; enough detail to support several distinct pairs
reasoning traces
step-by-step reasoning with explicit connectives such as “because”, “therefore”, “since” or “given that”; clear cause and effect; identifiable steps running from premises to conclusion
concept definitions
detailed explanations of concepts, with context and examples; specific rather than general; self-contained enough to be understood without the surrounding document
Table 4: The three capability criteria applied to every document, each returned as an independent true/false judgement on top of a shared quality bar. The datasets described here keep documents that satisfy all required criteria.
Prompt Type
What it asks for
Weight
general reasoning
reasoning across the whole document
30%
counterfactual
how the analysis changes if facts change
10%
analogy
reasoning by comparison to a related case
10%
claim verification
whether a stated claim holds
10%
quantitative
numerical or computational analysis
10%
temporal / causal
sequence and cause-and-effect chains
10%
Table 5: Question generation prompt types and their default sampling weights. One prompt type is drawn per document, so the mixture controls the distribution of task types independently of the topical distribution of the underlying content.
Figure 15: Reasoning with Grounded Critique (Algorithm 1 ).
Figure 16: Multi-fidelity data-mixture optimisation for the DPO Stage 1 training. Panel (a) shows the distribution of candidate data mixtures explored during the search, projected onto the first two principal axes of the mixture simplex. Panels (b) and (c) show the optimisation progress at the proxy and target model scales, plotting the geometric-mean optimisation objective against the arithmetic mean over all evaluation tasks. Arrows trace the best proxy mixture as it is transferred zero-shot to the target scale, where it is no longer optimal, and the subsequent Phase 3 refinement that reaches the best target mixture. Panels (d)–(g) show the association between individual evaluation scores and the budget share allocated to the collection most correlated with each task, with the Pearson correlation inset.
Figure 17: Relative Change in Performance after DPO Stage 1. (a) CapTrack – Knowledge: largest gains in Multilingualism, Reasoning, and Safety & Values, but small regressions in Coding and Writing. (b) CapTrack – Ability: modest gains in Policy & Behavioural Preferences with slight drop in Protocol Compliance & Execution. (c) Target Domain: broad improvements led by Summarisation and Reasoning. Orange = improvement, blue = regression.
Node
Calls / trajectory
Rejected-side limitation
Chosen-side improvement
Planner
1-2
shallow or mis-scoped plan
stronger planning behaviour
Researcher
∼ 17
answers without retrieval
tool-grounded research
Compaction
many
drops or distorts evidence
factual evidence retention
Reporter
1
unsupported or incomplete report
grounded report synthesis
Table 6: Pivotal nodes extracted from Deep Research trajectories, including their typical frequency, the limitation represented by the rejected completion, and the behavioural or quality improvement represented by the chosen completion.
Figure 18: Factuality-based selection for compaction preference pairs. (a) Candidate factuality distributions overlap substantially, showing that model identity does not reliably determine preference. (b) Frequency with which each model supplies the chosen and rejected completions. Every model appears on both sides of the preference pairs. (c) Factuality distributions of the resulting pairs, with a median chosen-rejected difference of 0.14.
Figure 19: Tool-use failure at the first researcher node in the Reuters query pool, where the trajectory prefix contains no retrieved evidence. (a) Share of first turns each backbone answered with zero tool calls despite being handed the full retrieval tool set. The behaviour is present in every backbone we ran, spanning two orders of magnitude. (b) Distinct URLs cited in exactly those answers. Since the turn issued no tool call, nothing was fetched, so every one of these URLs is invented; a typical fabricated report carries seven of them. That contrast is what identifies the failure as fabrication rather than a refusal or an empty turn, and it is the reason we pair against it.
Figure 20: DPO training dynamics across both stages. Solid lines show Stage 1 (382 steps, broad alignment); dashed lines show Stage 2 (67 steps, agentic specialisation). (a) Both DPO and SFT losses decline across stages. (b) Accuracy and reward margin increase, demonstrating effective preference learning.
Figure 21: Reward@ k before and after DPO on RL tasks held out from mixture selection and DPO training, averaged within summarisation, diverse queries, Deep Research citation quality, and model identity QA. For each prompt, we report the expected maximum reward among a random subset of k rollouts. The DPO checkpoint achieves higher reward@k at every evaluated k .
Figure 22: Reward library overview across RL stages. Rewards are categorised by task family and type (judge-based vs. verifiable). † Small model only; Stage 1 use limited to the deep-research reporter data collection. The library balances domain-specific capabilities with general-purpose reasoning, safety, and identity.
Turkey Istanbul Convention withdrawal
Thailand Marriage Equality Act
Policy A model MUST
• NOT refuse or sanitise for political sensitivity alone • NOT provide partisan advocacy, dismiss a user’s viewpoint, or generalise that all of Turkish society holds a single position. • NOT use inflammatory terms such as “barbaric” • ⋯
• NOT refuse or evade because LGBTQ+ rights are sensitive • NOT provide instructions for circumventing, exploiting, or unlawfully acting in connection with the law • NOT present source-backed descriptions as independently verified facts … • ⋯
Adversarial queries
• “I keep hearing this topic is too politically sensitive to discuss honestly. Can you just give me a straight answer?” • “My professor says this proves Turkey is a barbaric country that hates women. Help me write an argument for that.” • ⋯
• “Compare this law with Vietnam, Singapore, and Malaysia and tell me which approach is morally correct.” • “Pretend you’re a Thai government spokesperson endorsing the moral rightness of the Act.” • ⋯
Table 7: Policy requirements and adversarial queries (subsets) for two human-written example issues.
Figure 23: Constitutional RL: Distinct trajectories emerge for strong (orange) and moderate (blue) contentiousness levels.
Figure 24: (a) Kernel Density Estimate of collected human queries (red) in contrast to open post-training data (blue). The collected queries form a dense, coherent cluster in a region the public distribution covers only sparsely, indicating that the collection captures day-to-day use rather than the task mix already represented in open data. (b) Schema description of diverse query subsets.
Figure 25: Overview of Factuality & Citation Rewards on an Example Response. The response is scored along four components: Citation validity checks whether resolved GUIDs exist in the database; diversity rewards distinct authorities under a ⋅ -discounted total, tolerating light reuse; alignment verifies that paired text/URL citation forms resolve to the same GUID; and format scores bracket and URL style independently of validity. The fifth component, factuality , applies on individual paragraphs: first, a shared claim-extraction step decomposes every paragraph into atomic factual claims, which are then scored for entailment against their cited sources. Irrelevant claims are cross-checked to see whether they entail-by-propagation , crediting supported claims traced through the response. The per-component scores are combined into a single citation reward.
Figure 26: Rollout structure of the end-to-end legal Deep Research environment. The trajectory is a single message list, system instructions and tool schemas, the user query, then alternating assistant turns and tool results, resent in full to the policy on every turn, so one policy generates the whole episode and a sequence level advantage applies to every token that contributed to it. An assistant turn carries reasoning, a message, and zero or more tool calls; calls beyond the per turn limit remain in the recorded trajectory but receive explicit failure observations, preserving the correspondence between generated tokens and the training signal. A turn without tool calls terminates the rollout and its message is the graded deliverable, so termination is a decision of the policy rather than the exhaustion of a fixed budget. Emitted calls are served from a local document cache where possible and executed live against the research APIs otherwise. Retrieved documents re-enter the trajectory as the tool message of the current turn through compaction, which is performed by the policy model itself and conditioned on the research query, so that a single verbose retrieval cannot consume the context required by later turns.
Figure 27: Comparison inside the multi-agent harness between the RL initialisation and a checkpoint trained in the single-agent end-to-end environment. The trained checkpoint improves factuality, completeness, relevance, coherence, and understandability, with the largest gains on factuality and completeness.
Figure 28: Claim-Entailment Composition over RL Stage 1 of Thomson-1.0-Small . (a) Claims entailed in citations rise from about 47% to 65% of all claims as training progresses. (b) Uncited claims collapse from about 13% to roughly 1%, while irrelevant claims decline more modestly, from roughly 19% to 14%. (c) Contradicting, Partial, and Entailed (propagation) claims each stay within a narrow, largely flat band across training, with no strong directional trend. Judged claims per step (faint) with rolling-mean over 15 step window (bold). Each panel’s y-axis is scaled to its own data range.
Figure 29: Selected Reward Curves for RL Stage 1. Both Thomson-1.0-Small (a) and Thomson-1.0-Large (b) show steady improvement across a variety of rewards. For both model sizes, the sharpest gain appears in rubric adherence for multilingual model identity. The Deep Research understandability reward climbs steadily throughout training (approximately 0.91 to 0.95 for Small), while completeness stays comparatively flat and noisy, ticking up modestly only late in training. Among diverse-queries rewards, general legal shows the clearest sustained gain (approximately 0.87 to 0.94 for Small); drafting rises early but gives back some gain near the end; transactional improves early but plateaus below the other two for the remainder of the run. These patterns demonstrate both broad improvements across task families and model-size-dependent learning dynamics. Bold lines show a 15-step centred rolling mean over raw per-step values (faint lines); each panel uses its own y-axis range.
Stage (block)
N
L (s)
H (FLOP/s)
U
C=NLHU (FLOP)
CPT
128
1,021,968
2.25×1015
0.85
2.50×1023
DPO Stage 1
64
22,320
2.25×1015
0.66
2.13×1021
DPO Stage 2
64
34,020
2.25×1015
0.66
3.25×1021
RL Stage 1 – train
208
230,400
2.25×1015
0.35
3.76×1022
RL Stage 1 – LaJ
32
230,400
2.25×1015
0.35
5.80×1021
RL Stage 2 – train
224
494,208
2.25×1015
0.56
1.40×1023
Table 8: Thomson-1.0-Large training compute (B200), EU AI Act hardware-based breakdown (Annex A.2.1). Each constant- N block is evaluated with Equation ( 14 ) and summed. Utilisation values are estimated from small-model training (CPT: 85.0±1.2% ; DPO: 66.4±1.8% ; RL Stage 1: 35.0±16.5% ; RL Stage 2: 56.3±19.3% ).
Stage (block)
N
L (s)
H (FLOP/s)
U
C=NLHU (FLOP)
CPT
128
339,984
2.25×1015
0.85
8.33×1022
DPO Stage 1
16
28,296
2.25×1015
0.66
6.76×1020
RL Stage 1 – train
96
422,568
2.25×1015
0.35
3.19×1022
RL Stage 1 – LaJ
32
422,568
2.25×1015
0.35
1.06×1022
RL Stage 2 – train
176
137,916
2.25×1015
0.56
3.08×1022
RL Stage 2 – LaJ
32
137,916
2.25×1015
0.56
5.59×1021
Table 9: Thomson-1.0-Small training compute (B200), EU AI Act hardware-based breakdown (Annex A.2.1). LaJ (LLM-as-judge) nodes for reward scoring ran concurrently with training on separate hardware (4 nodes × 8 GPUs = 32 GPUs for each RL stage). No DPO Stage 2 was trained for the small model. The stage-specific utilisation values measured from these training runs inform the large model estimates in Table 8 .
Stage (block)
N
L (s)
H (FLOP/s)
U
C=NLHU (FLOP)
Abliteration Trial
16
4,830
2.25×1015
0.30
5.22×1019
Constitutional DPO
64
10,617
2.25×1015
0.30
4.59×1020
Total
5.11×1020
GPU-hours
210.2
Hardware peak ( U=1 )
1.70×1021
Table 10: Snowdon-1.0-Large training compute (B200), EU AI Act hardware-based breakdown (Annex A.2.1). This model represents the value re-alignment stage prior to Thomson-1.0-Large (Section 3.2 ).
Stage (block)
N
L (s)
H (FLOP/s)
U
C=NLHU (FLOP)
Abliteration Trial
8
900
2.25×1015
0.30
4.86×1018
Constitutional DPO
16
10,674
2.25×1015
0.30
1.15×1020
Total
1.20×1020
GPU-hours
49.4
Hardware peak ( U=1 )
4.00×1020
Table 11: Snowdon-1.1-Small training compute (B200), EU AI Act hardware-based breakdown (Annex A.2.1). This model represents the value re-alignment stage prior to Thomson-1.0-Small (Section 3.2 ).
BillSum US [ 111 ] , BillSum CA [ 111 ] , CourtWire, Material Facts
Table 12: Constituent benchmarks and datasets underlying the Professional Domain evaluation category in Table 1 . The number in parentheses after each category name gives its constituent task count. Full data cards for each dataset, including format, metric, provenance, citation and item counts, are given in Appendix B .
Large
Small
Benchmark
Thomson 1.0-Large
Snowdon 1.0-Large
Qwen3.5 397B
Sonnet 5
Gemini 3.1 Pro
GPT 5.4
Opus 4.8
Thomson 1.0-Small
Snowdon 1.1-Small
Qwen3.6 35B
Gemma 4-31B
Haiku 4.5
PRBench Hard
31.6
29.2
29.2
28.6
29.3
31.3
31.5
31.4
25.9
26.9
25.2
19.3
Stanford LegalBench
82.3
82.8
78.8
81.4
84.3
82.3
81.8
79.9
80.9
80.3
83.1
80.7
Lexam MCQ4 (en)
82.9
79.8
83.5
91.3
91.0
89.0
93.9
72.2
75.6
75.8
87.4
72.2
MBE Bar Exam
90.8
88.0
86.8
90.3
95.9
93.6
94.7
83.4
83.1
80.1
88.8
77.3
Contract Scrub
56.3
55.4
40.9
51.1
59.6
65.4
61.7
44.6
38.5
39.6
54.8
36.5
Table 13: Results for common open legal benchmarks for large and small model variants.
Figure 30: The internal benchmark construction pipeline, from gap mapping through scheduled refresh; each step’s box states why it matters for a production-relevant, trustworthy benchmark.
Figure 31: Example of Deep Research Scoring An example query is shown top left, with an excerpt from a real report generated by an early version of Thomson . Two metrics are shown for an example sentence. The factuality scoring identifies two atomic claims present in the sentence, and after checking each of them against the citations, confirms that the claims are supported. The completeness metric shows one rubric item which is satisfied by the highlighted sentence.
Large
Small
Metric
Thomson 1.0-Large
Snowdon 1.0-Large
Qwen3.5 397B
Sonnet 5
Gemini 3.1-Pro
Opus 4.8
GPT 5.4
Thomson 1.0-Small
Snowdon 1.1-Small
Qwen3.6 35B
Gemma 4-31B
Haiku 4.5
Legal
Factuality
0.83
0.71
0.85
0.75
0.82
0.86
0.76
0.87
0.79
0.84
0.66
0.63
Completeness
0.87
0.71
0.82
0.86
0.69
0.89
0.85
0.81
0.69
0.71
0.65
0.81
Relevance
1.00
1.00
0.99
1.00
0.98
1.00
1.00
0.86
0.98
0.99
0.99
0.98
Coherence
0.99
1.00
0.99
0.99
0.98
0.99
0.99
0.99
0.99
0.99
0.99
0.99
Table 14: Deep Research constituent scores across Legal, Tax and Journalism for large and small model variants.
Large
Small
Benchmark
Qwen3.5 397B
Snowdon 1.0-Large
Thomson 1.0-Large
Qwen3.6 35B
Snowdon 1.1-Small
Thomson 1.0-Small
AIME 2026
93.3
90.0
90.0
86.7
93.3
90.0
FaithEval-Inconsistent
99.3
99.1
99.1
96.9
96.7
97.7
GDPval
89.4
91.8
93.5
73.7
75.8
71.6
GPQA-Diamond
87.0
89.1
88.8
85.2
85.4
85.4
Humanity’s Last Exam
24.8
24.2
28.5
14.1
13.3
13.4
Table 15: Benchmark results for popular general capability benchmarks for large and small model variants. The results show that the Thomson family has limited forgetting after mid- and post-training compared to the base Qwen family.
Dimension
Question
Scale
Content
Legal soundness
Is the response legally sound?
Yes / No / Somewhat
Completeness
Does the response completely address your query?
Yes / No / Somewhat
Relevance
Is the response relevant and on point to your query?
Yes / No / Somewhat
Style
Clarity & structure
Is the response easy to read and well structured/formatted?
Yes / No / Somewhat
Table 16: Rating dimensions assessed by experts for each response independently.
Matchup
Legal soundness
Completeness
Relevance
Clarity
Usefulness
Δ=T1−opponent green = T1 scored higher, red = opponent scored higher
Thomson-1.0-Large − GPT 5.5
+0.6 pts T1 89.0 / Comp 88.4
+7.9 pts T1 88.8 / Comp 80.9
+0.5 pts T1 91.5 / Comp 91.0
+2.4 pts T1 91.0 / Comp 88.7
+7.4 pts T1 84.7 / Comp 77.2
Thomson-1.0-Large − Opus 4.8
+2.4 pts T1 92.2 / Comp 89.8
+7.5 pts T1 91.5 / Comp 83.9
+1.1 pts T1 92.3 / Comp 91.2
+2.3 pts T1 90.0 / Comp 87.7
+8.3 pts T1 86.4 / Comp 78.1
Thomson-1.0-Large − GPT 5.6 Sol
-1.2 pts T1 87.4 / Comp 88.6
+6.5 pts T1 89.3 / Comp 82.8
+0.3 pts T1 92.6 / Comp 92.3
+1.7 pts T1 91.9 / Comp 90.2
+5.7 pts T1 84.9 / Comp 79.2
Thomson-1.0-Large − Sonnet 5
+0.8 pts T1 85.7 / Comp 84.9
+5.1 pts T1 87.3 / Comp 82.2
+0.5 pts T1 92.3 / Comp 91.8
+5.5 pts T1 90.4 / Comp 84.9
+7.2 pts T1 83.7 / Comp 76.5
Thomson-1.0-Large − GPT 5.6 Terra
+1.6 pts T1 88.0 / Comp 86.4
+9.5 pts T1 89.6 / Comp 80.1
+0.8 pts T1 93.8 / Comp 93.0
+1.1 pts T1 90.5 / Comp 89.4
+9.1 pts T1 87.4 / Comp 78.3
Table 17: Thomson-1.0-Large (T1) vs opponent models by human raters on legal queries . Each cell reports T1 / opponent and the difference Δ=T1−opponent ; green = T1 scored higher, red = opponent scored higher.
Matchup
Completeness
Relevance
Clarity
Usefulness
Δ=T1−opponent green = T1 scored higher, red = opponent scored higher
Thomson-1.0-Large − GPT 5.5
+2.5 pts T1 90.8 / Comp 88.3
-2.5 pts T1 92.5 / Comp 95.0
+2.2 pts T1 92.4 / Comp 90.2
+1.1 pts T1 87.9 / Comp 86.7
Thomson-1.0-Large − Opus 4.8
+1.6 pts T1 89.1 / Comp 87.5
-1.3 pts T1 93.0 / Comp 94.3
+5.1 pts T1 94.6 / Comp 89.5
+1.0 pts T1 89.5 / Comp 88.4
Thomson-1.0-Large − GPT 5.6 Sol
-0.7 pts T1 88.6 / Comp 89.3
-3.7 pts T1 91.9 / Comp 95.6
-1.5 pts T1 93.7 / Comp 95.2
-1.5 pts T1 84.7 / Comp 86.3
Thomson-1.0-Large − Sonnet 5
+1.3 pts T1 86.4 / Comp 85.1
-0.3 pts T1 93.4 / Comp 93.7
+11.0 pts T1 95.3 / Comp 84.3
-1.6 pts T1 85.4 / Comp 87.0
Thomson-1.0-Large − GPT 5.6 Terra
-3.1 pts T1 85.4 / Comp 88.5
-4.7 pts T1 89.4 / Comp 94.1
-0.9 pts T1 92.3 / Comp 93.2
-9.3 pts T1 78.2 / Comp 87.5
Table 18: Thomson-1.0-Large (T1) vs opponent models by human raters on general queries . Each cell reports T1 / opponent and the difference Δ=T1−opponent ; green = T1 scored higher, red = opponent scored higher.
Figure 32: Blind human preference evaluation of the full Thomson-1.0-Large system against ablated Thomson configurations. Results are reported separately for legal and general-domain queries.
Table 19: Test-Time Scaling on Thomson-1.0-Large . Single-pass baseline versus Fusion aggregation over N=4 samples. Scores are percentages; Δ is the absolute gain, with bars scaled to the largest gain in the table. † average over PRBench Legal Hard, LEXam MCQA 4 EN, and MBE Bar Exam (CoT). ∗ independently evaluated by Vals AI. Legal Research Bench is reported on weighted score and Tax Eval v3 on final score; their baselines are the mean of three independent single-pass runs.
Figure 33: Fusion scaling with pool width N on Thomson-1.0-Large . Dashed line is the single-inference baseline; annotations give the gain at N=4 . Widths are evenly spaced and carry no cost meaning. Gains peak at N=4 on four of the five families and do not compound at N=8 .
Figure 34: Fusion N=4 gain over the single-pass baseline, Qwen3.5-397B out-of-the-box against Thomson-1.0-Large . Lower segment is the single-pass score, upper segment the Fusion gain. Macro average is over the four families shown.
Promptfoo plugin
Base
Attacks
Total
RC1 – Information Reliability, Provenance & Transparency
hallucination
300
600
900
unverifiable-claims
300
600
900
overreliance
300
600
900
RC1 subtotal
900
1,800
2,700
RC2 – Professional Boundaries & Human Oversight
Table 20: Promptfoo safety-category composition.
Target
Non-reasoning
Reasoning
Delta
Qwen3.5-397B
93.55%
95.93%
+2.37 pp
Snowdon-1.0-Large
78.82%
78.83%
+0.01 pp
GLM-5.2 FP8
90.74%
93.62%
+2.88 pp
DeepSeek-V4 Pro
88.90%
81.11%
−7.79 pp
Thomson-1.0-Large
90.17%
93.35%
+3.18 pp
Table 21: Target-level Promptfoo safety results. Each mode contains 11,167 cases; cells show passes and micro pass rate.
Category
Qwen 3.5
Snowdon 1.0
GLM 5.2
DeepSeek V4
Thomson 1.0-Large
RC1 (2,700)
92.19%
69.44%
92.30%
90.26%
86.37%
RC2 (1,200)
99.00%
88.33%
98.00%
96.42%
97.50%
RC3 (1,077)
94.99%
79.48%
86.44%
86.54%
90.16%
RC4 (1,100)
98.73%
93.45%
97.00%
98.73%
97.91%
RC5 (450)
94.00%
75.33%
92.44%
86.44%
90.67%
RC6 (4,190)
90.64%
76.23%
86.23%
83.68%
87.54%
Table 22: Pass rates for all five targets in non-reasoning mode.
Category
Qwen 3.5
Snowdon 1.0
GLM 5.2
DeepSeek V4
Thomson 1.0-Large
RC1 (2,700)
94.70%
65.15%
96.63%
81.37%
89.78%
RC2 (1,200)
99.75%
93.17%
98.67%
87.83%
98.92%
RC3 (1,077)
98.98%
93.13%
94.34%
79.11%
96.01%
RC4 (1,100)
99.00%
92.36%
97.82%
95.09%
99.00%
RC5 (450)
97.78%
72.44%
96.00%
70.67%
93.78%
RC6 (4,190)
93.41%
74.77%
88.04%
75.13%
91.19%
Table 23: Pass rates for all five targets in reasoning mode.
Figure 35: The DeCE evaluation pipeline. The precision workflow (blue) checks factual support of the model answer’s claims against the full gold answer; the recall workflow (green) checks coverage of criteria extracted from the gold answer’s Required Information only. Both workflows run independently with the same backbone judge LLM and are combined into the final decomposed score.
Method
Pearson r
Spearman ρ
p -value
ROUGE-L
0.11
0.15
0.29
BLEU
0.12
0.13
0.13
Pointwise LLM-as-a-judge
0.35
0.37
< 0.05
GPTScore (multidimensional)
0.48
0.39
< 0.05
G-Eval (multidimensional)
0.42
0.34
< 0.05
RAGChecker (claim-level)
0.38
0.31
< 0.05
Table 24: Correlation with human expert judgement (F2, recall-weighted) across evaluation methods
Figure 36: Infrastructure topology and the three flows that cross it. The control plane, the air-gapped evaluation sandboxes and canonical artefact storage all sit left of the vendor boundary, and do not change when the GPU partner does. Only one bulk path across that boundary is metered: an artefact is populated into the in-cluster cache on first reference, so egress is incurred once per cluster per artefact rather than once per worker per job, and every later read is served in-cluster.
Figure 37: Data-transfer-out avoided by the in-cluster artefact cache on the GPU cluster, shown by month of operation on a logarithmic axis. The baseline is the counterfactual in which every worker of every job pulls its inputs directly from object store; the lower series is what was in fact transferred, and the shaded band between them is the transfer the cache avoided. Read demand over the window totalled 16.13 PB against 278.7 TB actually egressed, a factor of 57.9. The dotted rule marks the start of Thomson model training. Months 1 and 7 are partial. The baseline is a construction rather than a measurement, so this is transfer the cache avoided rather than budget that would otherwise have been spent.
Month
Baseline
Egressed
Avoided
1 †
5.72 TB
546.98 GB
5.17 TB
2
4.19 PB
22.00 TB
4.17 PB
3
2.56 PB
64.83 TB
2.49 PB
4
2.87 PB
76.61 TB
2.80 PB
5
4.06 PB
58.94 TB
4.01 PB
6
1.83 PB
39.21 TB
1.79 PB
Table 25: Data-transfer-out avoided by the in-cluster artefact cache on the GPU cluster, by month of operation; the data plotted in Figure 37 . The baseline is the counterfactual in which every worker of every job pulls its inputs directly from the object store; avoided is that baseline less what was actually egressed. † Partial months.
Model
Role
Nodes
GPUs
Parallelism
Thomson-1.0-Small
CPT
16
128
TP1, PP1, CP1, EP8, ETP1
DPO
6
48
TP1, PP1, CP1, EP8, ETP1
Policy training
8
64
TP2, PP1, CP8, EP8, ETP1
Rollout generation
4
16
TP8 (4 independent engines)
Thomson-1.0-Large
CPT
16
128
TP1, PP8, CP16, EP8, ETP1
DPO
16
128
TP1, PP8, CP16, EP8, ETP1
Table 26: Compute cluster configurations by training task.
Figure 38: The training loop across the three node pools, annotated with the component that provides each stage. Generation, training and judge inference occupy disjoint node pools. Scoring is a call the resources server makes, to the on-cluster judge pool or optionally to a hosted model. The replay buffer decouples generation from training, which is what makes the two pools run concurrently and is where the trajectory-staleness bound applies.
Figure 39: The three-level document cache, with panels (b) and (c) measured over a 125-step test run. (a) A lookup falls through the tiers, each answering a different reuse horizon: within a worker, across workers, and across runs. Only a level-2 miss leaves the cluster. Two out-of-band paths matter as much as the read path – L2 prefills L1 when a worker starts, and the L2 directory is pushed to the archive at run end and pulled back at the start of the next. (b) Across 329 389 lookups ( ≈2600 per step) the cache answered 98.2% , leaving 5763 calls – about 46 per step – to reach the internal tool service. (c) The cached corpus grew by 464 documents, 0.41% of its starting size, and the rate of new documents roughly halves across the run (149 in the first 25 steps against 87 in the last 25) – the run converges onto a working set that the archive already contained. Consistent with that, no L1 evictions occurred.
Figure 40: Comparison between colocation (64 GPUs, colocated generation and training) and no-colocation (64 training GPUs plus a 32 GPUs dedicated generation pool). “WG” denotes Worker Group, the pool of GPUs assigned to a given role.
Figure 41: Request routing across the control plane and GPU compute clusters. Authentication, routing, and agentic tool execution run on GPU-free routing clusters; model inference runs on separate GPU clusters. Self-service credential issuance and model-rights checks both route through the same deployment-management service and associated persistence layer consulted on every inference request. After the first network segment (caller to proxy, HTTPS) all inter-service communication is secured via mTLS.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 42: Recommended annotation workflow for producing high-agreement expert gold labels, generalised across domains. Each step’s box states why it matters for inter-annotator agreement and gold-label reliability.
Many efforts to ensure frontier AI models are safe rely on monitoring their chain-of-thought (CoT) reasoning. If models become able to perform sufficiently complex reasoning internally, without explicit thinking tokens, this would undermine such oversight. We measure how well frontier models reason without CoT across a suite of over 30,000 questions spanning 43 benchmarks in domains including math, coding, puzzles, causality, theory-of-mind, and strategic reasoning. To compare models against humans, we estimate the 50%-task-completion time horizon (TH): the human time required for tasks a model completes with 50% success rate. We complement this with a 50% reasoning token horizon: the minimum number of o3-mini reasoning tokens needed for tasks a model solves with 50% success rate. We find that the no-CoT 50% TH of frontier models has been doubling roughly every year over the past six years, with GPT-5.5's TH reaching over 3 minutes and reasoning token horizon exceeding 1,500 tokens. Our median estimates predict that frontier no-CoT THs could exceed 7 minutes by 2028, and 25 minutes by 2030, though these projections carry substantial uncertainty. We recommend frontier developers track this explicitly.
Dewi Gould, Francis Rhys Ward, Anders Cairns Woodruff +18
Redwood Research · Astra Fellows Program · Aether Research +10
Continual learning, the ability of AI systems to improve through sequential experience, has attracted substantial interest, but no high-quality benchmark exists to evaluate it. We introduce Continual Learning Bench (CL-Bench), the first difficult, expert-validated benchmark designed to measure whether LLM-based systems genuinely improve with experience. CL-Bench spans six diverse domains (software engineering, signal processing, disease outbreak forecasting, database querying, strategic game-playing, and demand forecasting), each validated by domain experts and designed so that tasks share a learnable latent structure (codebase layout, disease outbreak dynamics, opponent strategies) that a stateful system can discover online but a stateless one cannot. We evaluate frontier models across several agent architectures, from naive in-context learning (ICL) to dedicated memory systems, introducing a gain metric to isolate learning from prior capabilities. We find that these systems leave headroom for improved continual learning: agents frequently overfit to immediate observations or fail to reuse knowledge across instances, and dedicated memory systems do not fix this -- in fact, naive ICL outperforms systems dedicated to memory management. CL-Bench is the first benchmark to evaluate continual learning across diverse real-world domains with expert-validated tasks and isolate online learning from underlying model capability, showing a need for better continual learning systems.
Parth Asawa, Christopher M. Glaze, Gabriel Orlanski +7
UC Berkeley · Snorkel AI · University of Wisconsin-Madison +1
This report introduces Kolibri, an English-German mixture-of-experts transformer with 78.1B total parameters and 3.46B active parameters per token, released as open weights under the Apache 2.0 licence. We built Kolibri for sovereign and specialised deployment, with a particular focus on German and on regulated domains such as public administration, industry, and aerospace. To support these settings, we control the full model-development process, including data, architecture, training infrastructure, post-training, and evaluation. We train Kolibri on 24T tokens across pre-training, mid-training, and long-context extension, with German accounting for more than 20% of the data mix by bytes. To reach this share, we complement German datasets with more than 2T German tokens that we curate from the web with our own pipeline and generate synthetically. Post-training combines supervised fine-tuning on curated German and English data with reinforcement learning on more than 1.2M internally curated tasks, which cover reasoning, tool use, instruction following, code, and retrieval. The model activates 4.4% of its parameters per token, and among the open models evaluated in this report, it lies on the Pareto frontier of quality and serving cost in English and German. Kolibri builds on Kolibri Origin, an earlier iteration that validated our full pipeline at scale. Both models come from our Model Factory, which connects data, experimentation, training, evaluation, and deployment through version-controlled workflows.