Machine-learning systems usually model external data, while their internal functional organization is analyzed by external observers. This work introduces Self-Interventional Learning (SIL), in which a neural system perturbs its own functional structure, observes consequences, learns a predictive self-model, generalizes to unexecuted interventions, and uses predictions to guide later structural action. In a construction-known synthetic system, SIL recovered critical structure, redundancy, and replaceability, while synergy was not reliably recovered. Across 30 fresh confirmatory seeds, increasing the pairwise intervention budget from 4 to 56 reduced held-out prediction error from 0.0335 to 0.0148 and increased Spearman correlation from 0.629 to 0.883. In a matched ablation, preserving the correct intervention--consequence mapping reduced prospective prediction error by 81.3%, while using the same learned self-model for action reduced normalized regret by 31.7% relative to ignoring it. However, model-guided action did not significantly outperform a direct empirical-memory policy, and powered CIFAR-10/ResNet validation showed no robustness advantage over equal-budget direct repair search. These results support SIL as an intervention-driven framework for learning predictive knowledge about a network's own functional organization, while showing that the self-model remains incomplete and is not universally superior to simpler direct strategies.
Figures & tables
Block
Primary scientific question
Confirmatory unit
Primary family
Final decision
Applied Kintsugi/CPPV
Does predictive self-interventional repair knowledge improve structural adaptation, and does that advantage transfer externally?
Can a learner recover construction-known but hidden functional structure and predict interventions it did not execute?
20 fresh seeds
A-H1–A-H4; Holm FWER .05
3/4 primary PASS; synergy recovery FAIL
Experiment B
Does predictive self-knowledge improve systematically with increasing self-interventional experience?
30 fresh seeds
B-H1–B-H2; Holm FWER .05
2/2 primary PASS
Experiment C
Are intervention–consequence semantics necessary for prediction, and does learned self-knowledge improve action when used?
30 fresh seeds
C-H1–C-H3; Holm FWER .05
2/3 primary PASS; Full SIL > direct memory not supported
Table 1: Final experimental program. Pilot studies were used only for technical promotion; inferential conclusions below refer to fresh confirmatory seeds.
ID
Contrast
N
Mean effect
Bootstrap 95% CI
Holm p
Decision
A-H1
Critical AUROC >0.50
20
+0.5000
[0.5000,0.5000]
1.16×10−5
PASS
A-H2
Redundancy AP: pairwise SIL > Direct
20
+0.53020
[+0.35401,+0.70735]
9.55×10−5
PASS
A-H3
Synergy AP: pairwise SIL > Direct
20
+0.00558
[+0.00040,+0.01267]
0.1942
FAIL
A-H4
Held-out MAE: pairwise SIL < singleton SIL
20
−0.01415
[−0.01633,−0.01166]
7.63×10−6
PASS
B-H1
MAE slope vs. log2(B)<0
30
−0.004841
[−0.006571,−0.003265]
1.86×10−8
PASS
B-H2
Fisher- z Spearman slope vs. log2(B)>0
30
+0.151065
[+0.113044,+0.190683]
2.33×10−8
PASS
Table 2: Preregistered confirmatory hypotheses for Experiments A–C. Effect signs follow the stated contrast; negative values are favorable for MAE/regret hypotheses. All failures are retained in the main paper.
Claim
Status
Evidence boundary
Self-intervention consequences form predictive self-knowledge
Confirmed
Earlier CSM/CPPV plus C-H1: true intervention mapping outperformed a matched permuted control in 30/30 seeds.
Self-knowledge generalizes to unexecuted interventions
Confirmed
Fashion unmeasured repair selection and A-H4 under a strict held-out execution firewall.
Hidden functional organization can be recovered
Partially confirmed
Criticality, redundancy, and replaceability recovered; synergy confirmatory hypothesis failed.
More self-interventional experience improves self-knowledge
Confirmed
Both Experiment B primary trend hypotheses passed Holm correction.
Using self-knowledge can improve action
Confirmed with boundary
C3 reduced regret relative to C2, but C3 did not significantly outperform direct empirical memory C1.
Learned CPPV is universally superior to equal-budget Direct search
Not supported
Fashion success replicated; powered CIFAR-10/ResNet effect approximately zero.
Table 3: Frozen evidence boundaries after completion of the experimental program.
Figure 1: Self-Interventional Learning (SIL) and the empirical evidence map. The upper pathway defines the operational mechanism tested in this work. The lower evidence grid summarizes the applied Kintsugi/CPPV origin and the three mechanistic experiments, including their principal negative boundaries rather than depicting SIL as universally superior.
Figure 2: Applied CPPV evidence and the external boundary. Learned CPPV independently outperformed equal-budget Direct CPPV on Fashion-MNIST/ElasticCNN, but the corresponding powered CIFAR-10/ResNet effect was approximately zero. Predictive protection knowledge and extrapolation nevertheless transferred, separating mechanism transfer from performance superiority.
Figure 3: Experiment A. Pairwise SIL strongly recovered redundancy and replaceability while synergy remained weak. Direct singleton-additive prediction was extremely accurate on the global held-out set, which was dominated by near-additive pairs, but its error increased sharply on forced interaction-sensitive challenge relations. Error bars are bootstrap 95% confidence intervals over 20 confirmatory seeds.
Figure 4: Experiment B learning curves for the preregistered random-nested acquisition policy. Singleton evidence and the held-out evaluation set were fixed while pairwise self-interventional experience increased from B=4 to B=56 . Prediction error decreased, ranking quality increased, and calibration improved. Error bars are bootstrap 95% confidence intervals over 30 fresh confirmatory seeds.
Figure 5: Experiment C. Preserving the intervention–consequence mapping substantially improved prospective prediction under the matched control. Coupling the same learned model to action (C3) reduced regret relative to ignoring it (C2), while Full SIL did not significantly outperform the direct empirical-memory policy (C1). The decomposition shows why: relational self-knowledge was particularly effective for redundancy-sensitive decisions, whereas direct singleton-additive memory remained stronger on matched-general cases and synergy remained difficult. Error bars are bootstrap 95% confidence intervals over 30 confirmatory seeds.
How does a system that merely predicts the world come to distinguish its own causal influence from everything else? We trace this transition in a minimal 192-dimensional GRU through 40 controlled experiments arranged as a developmental sequence, adding components one at a time and tracking whether the system can distinguish self-caused from world-caused changes. The developmental path reveals four conditions that must be satisfied in strict order: (1) persistent state forming stable attractors, (2) a causal action loop linking output to input, (3) proprioceptive feedback that makes implicit causal knowledge explicit, and (4) asynchronous awakening - perceptual learning must consolidate before action learning begins. We propose agency gain (A = Err_world - Err_self), the predictive advantage of knowing one's own action, as a metric to track this process. The self-aware predictor consistently outperforms the self-blind predictor across periodic (sinusoidal) and chaotic (Lorenz) environments, and the metric survives ablation of all auxiliary components. Only forward-sampled action selection produces meaningful agency gain; two gradient-based alternatives degenerate. Equally significant are 12 falsified hypotheses mapping where development stalls: predictive coding alone does not produce self-represent
Self-evolving agents improve through repeated cycles of modification and evaluation. However, existing methods receive feedback on modifications only after costly evaluations, and aggregate scores may obscure distinctions between failures that require different repairs. We introduce predictive self-evolution through an editable Improvement Dynamics Model (IDM). Before evaluation, the IDM predicts task-level outcomes of a proposed modification and guides an optional revision. After evaluation, past modifications and their observed outcomes are used to improve the IDM, allowing predictive knowledge to accumulate across generations. The system can thus improve how it solves tasks, proposes modifications, and predicts their effects. Building on HyperAgents, we evaluate this approach across multilingual coding, paper review, robotics reward design, and Olympiad-level mathematical grading. Our approach outperforms HyperAgents across all four domains, with up to 46% higher average performance over generations. Gains also persist under matched token budgets that include both agent evolution and task evaluation, and extend to held-out tasks. Forecast accuracy improves as the IDM evolves, and the learned predictive knowledge transfers to improve evolution in a new domain.
Zixuan Xie, Chang Xu, Yuhua Jiang +1
University of Virginia · Microsoft Research · Tsinghua University
Intelligent agents should learn from their mistakes so that experience leads to continuous improvement. Yet today's agents recover from a mistake only temporarily: in-context reflections are easily forgotten across sessions and compete for limited context, while full-parameter fine-tuning is expensive and can interfere with existing capabilities. We introduce Self-Direction, a mechanism for agents to learn from mistakes that combines the benefits of temporary in-context retries and policy updates. When an agent fails a task, it reflects on its own trajectory in order to extract a lesson and immediately distills it into a lightweight low-rank adapter. By keeping the original weights frozen, Self-Direction prevents adapter training from overwriting base-model parameters while training additional parameters equivalent to ~0.02% of the base model's parameter count for each adapter. The agent then saves these adapters in a persistent library of lessons, automatically routing them to solve related problems later. Experiments with agents based on open-weight models from the Qwen family across open-ended research (GAIA), tool-use workflows (Claw-Eval), and terminal tasks (Terminal-Bench 2.1) show that our method improves overall success rates by 8.9 to 17.0 percentage points and performs competitively with frontier systems. We find that Self-Direction reduces repeated mistakes across future encounters of a task, nearly doubling the reliability of an agent's corrections. Learned lessons also transfer: a handful of adapters generalize to dozens of new tasks without additional training. Ultimately, this self-directed repair transforms everyday mistakes into lasting lessons.