MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training

Yongwan Kim; Sungchul Park

cs.LGMar 26, 2026

MAGNET: Autonomous Expert Model Generation via Decentralized Autoresearch and BitNet Training

Authors: Yongwan Kim, Sungchul Park

Organizations: Holo Studio Co., Ltd.

Abstract

We present MAGNET (Model Autonomously Growing Network), a decentralized system for autonomous generation, training, and serving of domain-expert language models across commodity hardware. MAGNET integrates four components: (1) autoresearch, an autonomous ML research pipeline that automates dataset generation, hyperparameter exploration, evaluation, and error-driven iteration; (2) BitNet b1.58 ternary training, enabling CPU-native inference via bitnet.cpp without GPU hardware; (3) DiLoCo-based distributed merging for communication-efficient aggregation of domain specialists; and (4) on-chain contribution tracking on the HOOTi EVM chain. We validate autoresearch through three case studies: video safety classification (balanced accuracy 0.9287 to 0.9851), cryptocurrency directional prediction (41% to 54.9% hit rate), and BitNet hyperparameter optimization (10-phase sweep, -16.7% validation loss).

Figures & tables

Panel of Figure 1 — Figure 1: MAGNET four-pillar architecture. Arrows indicate data flow between subsystems.

Without…	Degradation	Alternative exists?
Autoresearch	Human bottleneck limits domains and update frequency.	Manual research
BitNet	GPU cost excludes most nodes. 4-bit quant is an option but less efficient on CPU.	GPTQ, AWQ
DiLoCo	Models remain isolated. FedAvg possible but less communication-efficient.	FedAvg, DeMo
On-Chain	Contributions hard to verify at scale. Reputation systems partially substitute.	Reputation, payments

Table 1: Component interdependencies: degradation when each subsystem is removed. Alternatives exist for each (4-bit quantization for BitNet, FedAvg for DiLoCo, reputation systems for on-chain), but each alternative weakens a specific property that MAGNET optimizes for.

Model	$t_{\text{infer}}$	$t_{\text{python}}$	Overhead ratio $\frac{t_{\text{python}}}{t_{\text{infer}}+t_{\text{python}}}$
175B (GPT-3)	$\sim$ 10 s	$\sim$ 5 $\mu$ s	$\sim$ 0.00005%
7B (Llama 2)	$\sim$ 500 ms	$\sim$ 5 $\mu$ s	$\sim$ 0.001%
618M (BitNet)	$\sim$ 10 ms	$\sim$ 5 $\mu$ s	$\sim$ 0.05%
100M (Edge)	$\sim$ 1 ms	$\sim$ 5 $\mu$ s	$\sim$ 0.5%

Table 2: Python FFI overhead as a fraction of total inference time per call. For small models, overhead becomes non-negligible; over

N

iterations total wasted time is

N\times t_{\text{python}}

Panel of Figure 2 — Figure 2: Zevor autoresearch progression. 9 versions, $\sim$ 5,000 configurations. Each version addresses specific failure modes identified in the previous version’s error analysis.

Ver.	bal_acc	FP	FN	Configs	Key Innovation
Transf.	0.9287	38	9	690	End-to-end transformer (failed)
v1	0.9647	24	1	$\sim$ 100	Tree models + pipeline_pred as feature
v2	0.9719	12	5	$\sim$ 200	ExtraTrees + class weights
v3	0.9733	13	4	$\sim$ 150	Per-context LE specialist
v4	0.9758	11	4	$\sim$ 200	Error analysis + model combination
v5	0.9783	9	4	298	Context-conditioned thresholds

Table 3: Zevor autoresearch progression: key innovations per version.

System	F1	Bal. Accuracy	Accuracy
ORPO LLM only	0.173	0.269	0.232
LGBM + Base LLM	0.371	0.371	0.374
LGBM + ORPO LLM	0.372	0.371	0.374
LGBM alone (no LLM)	0.380	0.379	0.384

Table 4: StockClaw: LLM-only vs ML judgment comparison.

Panel of Figure 3 — Figure 3: StockClaw hit rate progression across autoresearch versions. Data scale: 504 $\rightarrow$ 11,904 $\rightarrow$ 77,771 samples.

Ver.	Data Range	Samples	Features	Hit Rate	Key Innovation
v4	14 days	504	82+6	41.0%	Futures data integration (+8.3%)
v5	1 year	11,904	82+7	51.8%	Extended history, funding rate accum.
v6	6.5 years	77,771	82+10	54.9%	Full market cycle coverage

Table 5: StockClaw autoresearch progression.

Panel of Figure 4 — Figure 4: Genkidama autoresearch v5: best validation loss per phase. Each value is the best within that phase’s sweep, not the cumulative best-so-far across phases. Phase 7 (context length) shows the largest single improvement. Phase 10 validates top-3 configs at 4 $\times$ training steps.

Phase	Focus	Configs	Best val_loss	Key Finding
1	Architecture	8	7.6833	Wider $>$ Deeper (1024d $\times$ 12L)
2	Learning rate	8	7.6885	LR $5\times 10^{-4}$ – $10^{-3}$ optimal
3	Training method	6	7.7802	Two-stage WD (matches BitNet 2B4T [ 16 ] )
4	WSD schedule	6	7.7062	Warmup 1%, Decay 15%
5	$\beta_{2}$ , weight decay	6	7.7646	$\beta_{2}{=}0.98>0.95$
6	FFN multiplier	5	7.9323	2.67 $\times$ FFN optimal

Table 6: Genkidama autoresearch: phase results and key findings.

Parameter	Value
Architecture	BitNet b1.58 LLaMA (SwiGLU, RMSNorm, GQA, RoPE)
Parameters	618.3M (tied embeddings)
Vocabulary	128,256 (Llama 3 tokenizer)
Training corpus	3.8B tokens (trilingual)
Token:param ratio	6.1:1 (data-efficient per Spectra [ 10 ] )
LR schedule	WSD (Warmup-Stable-Decay)

Table 7: Genkidama 618M training specifications.

Domain	Versions	Configs	Improvement	Duration
Zevor (video safety)	9	$\sim$ 5,000	$+5.6\%$ bal_acc	2 days
StockClaw (crypto)	3	$\sim$ 100	$+13.9$ pp hit rate	3 days
Genkidama (BitNet LM)	10 phases	54	$-16.7\%$ val_loss	2.3 hours

Table 8: Cross-domain autoresearch summary.

Metric	Measurement	Verification
Data quality	Held-out evaluation loss $\Delta$	Designed for on-chain reproducibility
Training compute	Steps $\times$ model size	Checkpoint hash
Model quality	Benchmark scores	Evaluator consensus (implemented)

Table 9: On-chain contribution metrics (implemented, pending mainnet validation).

System	Auto.	Tern.	Merge	Chain	Notes
Bittensor	×	×	×	✓	Inference market
Gensyn	×	×	NoLoCo	In dev.	Compute protocol
Prime Intellect	×	×	✓	×	OpenDiLoCo, 10B
Nous (DeMo)	×	×	✓	×	Dist. optimizer
Together AI	×	×	×	×	Centralized GPU
MAGNET	✓	✓	✓ †	✓ ‡	This work

Table 10: Feature comparison with existing decentralized AI systems. We compare along MAGNET’s four pillars; we acknowledge this selection favors our system and note that competitors may excel on dimensions not shown (e.g., model scale, active participants, production maturity).

Automated Machine Learning

Automated machine learning (AutoML) builds and tunes data-to-model pipelines with minimal manual effort, balancing predictive quality, compute, and deployment constraints.

See all 96 papers

AutoTrainess: Teaching Language Models to Improve Language Models Autonomously
Jun 30, 2026Zhaojian Yu, Penghao Yin, Shuzheng Gao +3Tool-Augmented Language Model Agents LLM Tool Use
1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab
Dynamic Expert Pruning for Multi-Agent Systems
Oct 2, 2026Jabin Koo, Soheil Abbasloo, Sungjae Lee +1Mixture-of-Experts Pruning Mixture-of-Experts Language Models
Pohang University of Science and Technology (POSTECH) · Microsoft Research Asia
Auto-Research Arena: Benchmarking and Advancing Auto-Research in Language Modeling
Oct 26, 2026Xi Chen, Chongjian Ge, Yiran Xu +5Automated Machine Learning Efficient Language Model Training
University of California San Diego · Adobe Research

Jun 30, 2026J/K move · Enter open · S save

AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

Zhaojian Yu, Penghao Yin, Shuzheng Gao, Shilin He+2

1Tsinghua University · 2The Chinese University of Hong Kong · 3Simple Agent Lab

Tool-Augmented Language Model Agents LLM Tool Use LLM Agent Training LLM Post-Training Language Model Self-Improvement

Open paper AskPDF arXiv

Training language models (LMs) remains a highly human-intensive process, even as frontier language model agents become increasingly capable at software engineering and other long-horizon tasks. A central challenge is that autonomous post-training is not just a coding problem: it requires the agent to repeatedly plan iterations, construct benchmark-aligned data, run stable training jobs, evaluate checkpoints, and preserve experiment state across many hours of interaction. We present AutoTrainess, a LM agent that exposes these operations as a repository of agent-computer interfaces for planning, data preparation, training, evaluation, and logging. Rather than leaving the agent to operate in a raw CLI environment with an underspecified action space, AutoTrainess externalizes prior human experience as explicit workflows, rules, and execution constraints that guide the agent toward effective and reliable training behavior. On PostTrainBench, AutoTrainess consistently outperforms CLI-only baselines, achieving 26.94 average score with GPT-5.4 (Codex) versus 23.21 for CLI-only. It also generalizes across models and harnesses, improving DeepSeek-V4-Flash (OpenCode) from 12.13 to 19.58.

Abstract

Automated Machine Learning

Explore similar work

AutoTrainess: Teaching Language Models to Improve Language Models Autonomously

Dynamic Expert Pruning for Multi-Agent Systems

Auto-Research Arena: Benchmarking and Advancing Auto-Research in Language Modeling