GradReason: LLMs as Meta-Learners that Replace Training Epochs with Training-Data-Free Weight Updates
Authors: Ali Mohammadi Ruzbahani, Abbas Yazdinejad, Hadis Karimipour
Organizations: Smart Cyber-Physical Systems (SCPS) Lab, University of Calgary, Calgary, Alberta, Canada · Decentralized Cybersecurity and Artificial Intelligence(DCAILab), University of Regina, Regina, Saskatchewan, Canada
Standard training recipes make a fixed number of passes over the training data, even when an epoch mostly adds mini-batch noise. We study whether a large language model (LLM) that reads the weight and gradient history of a run can replace some of these epochs with updates that compute no gradient and read no training image. In GradReason, an LLM agent is called every ten epochs. In a restricted sandbox, it analyses recent weight snapshots, epoch-mean gradients, and validation metrics, then submits a Python program that computes new weights and a plan marking each of the next ten epochs as data or virtual. A virtual epoch is kept only if it passes a forward-only check on 5,000 held-out validation images. In 120 runs on CIFAR-10 and CIFAR-100 with three architectures, Claude Opus 5.5 used 32.8% fewer data epochs than the 80-epoch recipe at 0.31 percentage points lower test accuracy (95% CI -0.72 to +0.09; 0.94 points on ResNet-18), on the recipe's own frontier of data against accuracy. Unprompted, Opus averaged recent weight snapshots in every run. This choice produced the savings: a fixed rule with the same acceptance check but extrapolations from prior work saved 5.4% of the data epochs, and the same rule with averaging added saved 25.0%. The LLM's contribution is to find a working update within a single run; once found, a fixed rule implements it equally well. An Opus run cost 0.68 USD, but at this scale the agent's latency exceeded the GPU time it saved.
Figures & tables
Figure 1: Overview of GradReason: the epoch clock (top) and one window (bottom).
Parameters (M)
Data epoch (s)
Accuracy, B=80 (%)
Architecture
CIFAR-10 / 100
Recipe
CIFAR-10 / 100
CIFAR-10
CIFAR-100
ResNet-18
11.17 / 11.22
SGD, lr 0.1, wd 5×10−4 , warm-up 1
2.4 / 2.0
94.47 ± 0.05
76.25 ± 0.23
MobileNetV2
2.24 / 2.35
SGD, lr 0.1, wd 5×10−4 , warm-up 1
5.6 / 5.9
92.73 ± 0.25
72.20 ± 0.42
ViT-Tiny
5.36 / 5.38
AdamW, lr 10−3 , wd 0.05, warm-up 5, LS 0.1
5.8 / 5.8
83.15 ± 0.15
55.17 ± 0.81
Table 1: Architectures, recipes and test accuracy of the 80-epoch recipe (mean ± s.d., two seeds).
Method
D
Saved (%)
GPU saved (%)
Δ80 (pp) [95% CI]
W/L
p
ΔF (pp) [95% CI]
W/L
p
A-priori rule
75.7
5.4 ± 10.1
0.9 ± 11.4
+0.02 [ −0.17 , +0.22 ]
3/7
0.34
+0.13 [ −0.22 , +0.47 ]
5/7
0.97
Rule
60.0
25.0 ± 12.2
19.6 ± 14.2
−0.19 [ −0.43 , +0.04 ]
3/9
0.12
+0.29 [ −0.01 , +0.60 ]
8/4
0.09
Sonnet
63.8
20.3 ± 10.2
12.1 ± 11.2
−0.17 [ −0.33 , −0.01 ]
4/8
0.08
+0.15 [ −0.21 , +0.51 ]
7/5
0.68
Opus
53.8
32.8 ± 7.3
22.5 ± 8.8
−0.31 [ −0.72 , +0.09 ]
3/9
0.11
+0.18 [ −0.18 , +0.54 ]
7/5
0.52
Table 2: Pooled results, 12 runs per method, paired by setting and seed ( ± : s.d.; p : Wilcoxon).
Figure 2: Test accuracy against data epochs. Rows: CIFAR-10, CIFAR-100; columns: ResNet-18, MobileNetV2, ViT-Tiny.
Recipe
Rule
Sonnet
Opus
Dataset
Architecture
Acc.
Acc.
D
ΔF
Acc.
D
ΔF
Acc.
D
ΔF
CIFAR-10
ResNet-18
94.47
94.38
66.0
+0.06
94.39
61.5
+0.15
93.59
47.0
−0.31
MobileNetV2
92.73
92.56
66.0
−0.07
92.48
64.5
−0.18
92.48
51.0
−0.02
ViT-Tiny
83.15
82.89
62.0
+0.57
83.31
55.5
+1.20
82.75
62.0
+0.43
CIFAR-100
ResNet-18
76.25
75.63
58.0
+0.13
75.74
67.0
−0.11
75.23
51.0
−0.09
MobileNetV2
72.20
72.16
66.0
+0.03
71.97
72.0
−0.18
72.27
52.5
+0.25
Table 3: Accuracy (%), data epochs D and frontier gap ΔF (pp) per setting, two-seed means.
Size relative to a data epoch
Saved per run by
Method
Accepted V
Averages (%)
Median
<5% (%)
Median ΔAV (pp)
Substantial V
Near-identity V
Early end
Rule
150
49
0.135
49
+0.00
6.4
6.1
7.5
Sonnet
178
3
0.005
87
−0.02
2.0
12.8
1.4
Opus
301
94
0.727
6
+2.78
23.5
1.6
1.2
Table 4: Accepted virtual epochs and the source of the data epochs saved per run.
Figure 3: Epoch kinds in every run: (a) CIFAR-10, (b) CIFAR-100.
Figure 4: CIFAR-100, ResNet-18, seed 0: test accuracy (top) and gap to the 80-epoch recipe (bottom).
Figure 5: (a) GPU time saved; (b) API cost per run; (c) API requests per invocation.
School of Digital Arts, Jiangxi Arts & Ceramics Technology Institute, Jingdezhen 33001, China · School of Computing, Universiti Sains Malaysia, 11800 USM Penang, Malaysia