cs.AISep 14, 2026

Metacognitive Steering: Learning the Structure of Scientific Judgment

Authors: Vincent Karpf, Joseph Reth, Eike Gerhardt, Audrey Wang, Anna Butz, Jiehao Xing, Jialing Song, Larry Callahan

Organizations: Autopoiesis Sciences

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Long-horizon scientific discovery requires agents to alternate between exploration, disciplined execution, and critical reassessment as evidence changes. Current language models are trained primarily on the products of science and optimized using outcome-level signals, providing limited supervision for these process-level shifts in scientific judgment. We investigate whether such judgment can be recovered from scientist interaction traces and used to control the internal computation of a frozen frontier model. Using contrastive interventions collected during real scientific research, we identify a coordinated, low-dimensional control structure within Kimi 2.6, a trillion-parameter mixture-of-experts model. Residual analysis, attention-weight subspace alignment, and cross-layer singular value decomposition converge on a mid-depth control surface spanning key layers. We introduce Metacognitive Steering, an inference-time controller that reads the model's cognitive regime and dynamically composes layer-specific interventions for exploration, procedural convergence, or critical reassessment without modifying model parameters. Behavioral analyses show that this control produces more sustained exploration, explicit pruning, and evidence-responsive synthesis. We operationalize the method in Columbus-1, an autonomous research system that identified eight independently reproduced, attacker-reachable vulnerabilities in BlueZ and directed the design, simulation, and fabrication of a ten-foot rocket intended to land propulsively using non-throttleable solid motors. Together, these results show that process-level scientific judgment can provide supervision for interpretable, dynamic control over a model's reasoning strategy.

Figures & tables

Figure 1
Figure 3: Cross-layer singular value decomposition of concatenated post-erasure activation differences at layers 36, 38, 42, and 48. The left panel compares the stacked cross-layer spectrum with spectra computed at individual layers. The right panel compares the observed components with an independently shuffled cross-layer permutation null; the three leading components exceed its 97.5th percentile.
Figure 4: MCS compared with representative dynamic activation-steering configurations: PDS [ 13 ] , DMS [ 34 ] , CLAS [ 12 ] , A-LQR [ 26 ] , SVF [ 17 ] , Dynamic Activation Composition [ 25 ] , and SADI [ 30 ] . Circle area represents the total parameter count of the largest model configuration reported for each method. For MCS, 1T denotes total parameters; approximately 32B parameters are activated per token.
Control regimeFunctional behaviorCentroid geometry
Deliberative  
ExploreBroad search, speculative branching, and resistance to premature commitmentPrimary reasoning manifold
Methodical  
ConvergeControls, metrics, falsification criteria, and budget-aware executionApproximately 85∘85^{\circ} from deliberative; nearly orthogonal
Challenging  
ScrutinizeConstraint enforcement, error detection, and reversal after overextension130∘130^{\circ} – 142∘142^{\circ} from the other regimes; directionally opposed
Table 1: Activation-defined control regimes. Angles are measured between normalized regime-centroid directions in the state representation space.
Figure 5: Cognitive states, steering modes, and their conditional mapping. Layer-36 un-differenced activations form three state regimes, while layer-48 contrastive differences form finer intervention modes. The two spaces are clustered independently; color shows their subsequent correspondence.
Figure 6: Sparse state detection and dynamic steering composition. Layer-36 neurons identify the current regime, state-specific predictors assign probabilities to compatible modes, and those probabilities blend fixed residual directions at layers 36, 38, 42, and 48.
Figure 7: Baseline and Metacognitive Steering trajectories on a closed-loop inference-time research task. The comparison aligns cognitive state, steering mixture, and active research workstream across the investigation. The steered trajectory sustains exploration later into the campaign, revisits competing approaches, and explicitly prunes a developed proposal before synthesis.
Figure 8: Baseline and Metacognitive Steering trajectories for the design of a propulsively landing solid-motor rocket. The baseline commits early to a single recovery concept, whereas the steered trajectory maintains deliberative exploration across propulsion, avionics, vehicle design, descent control, and program constraints.
Figure 9: Columbus-1 system architecture. Goals provide soft and hard constraints to a learned bias policy, which exchanges strategic nudges and research context with the steered reasoning model. Both components interact with persistent semantic memory, while the reasoning model uses tools and human collaborators to search, analyze, simulate, design, and execute research tasks.
Figure 10: Closed-loop six-degree-of-freedom simulation of the propulsive landing system. The visualization shows descent trajectory, flight phases, vehicle telemetry, and sequential solid-motor ignition. Staggered ignition approximates throttle control by coordinating predicted time-to-contact with descent velocity and remaining motor impulse. View the flight simulation .
More in this topic

Inference-Time Guidance

Inference-time guidance steers pretrained generative models, language models, and policies toward desired outputs without requiring full retraining.

See all 104 papers

Explore similar work

CardsList
Oct 15, 2026

SciΠΠvot: Benchmarking Principal-Investigator Judgment at Pivotal Moments of Scientific AutoResearch Agents

Large language model (LLM) agents can keep scientific experiments running for months with little human supervision, but reliable autonomous research also requires scientific judgment. Evaluating scientific judgment is difficult because research decisions are open-ended and depend on project context. We construct SciΠΠvot, a 248-case benchmark of scientific judgment based on human corrections. The benchmark draws on records from six months of research in which senior researchers ran LLM agents on 26 projects across six scientific fields and repeatedly intervened to correct the agents' research judgments. In each case, an agent chooses between the original LLM plan and the human-corrected alternative, using only the project record available before the correction as context. We compare LLM and human performance and examine the effects of reasoning effort, access to the project record, and reconsideration. At medium reasoning effort with the project record, models systematically prefer the original LLM plan over the human-corrected alternative. Four of five frontier models perform below chance on the selected cases. The highest accuracy in the project condition across the tested models and reasoning effort settings is 63.5%, compared with 73.4% for fresh researchers. At medium effort, every model achieves lower accuracy with the project record than without it, and reconsideration does not consistently improve overall accuracy. Even with a human-corrected alternative available, frontier models struggle with the selected research decisions, while SciΠΠvot supports systematic evaluation of scientific judgment for LLM agents.

Youran Sun, Xingyu Ren, Chugang Yi +15

University of Maryland, College Park · The Chinese University of Hong Kong · Gradient Network +3

LLM Agent EvaluationAI Agent BenchmarksScientific Reasoning in Language ModelsAutonomous Research Agents
AskPDF
Apr 20, 2026

AI scientists produce results without reasoning scientifically

Large language model (LLM)-based systems are increasingly deployed to conduct scientific research autonomously, yet whether their reasoning adheres to the epistemic norms that make scientific inquiry self-correcting is poorly understood. Here, we evaluate LLM-based scientific agents across eight domains, spanning workflow execution to hypothesis-driven inquiry, through more than 25,000 agent runs and two complementary lenses: (i) a systematic performance analysis that decomposes the contributions of the base model and the agent scaffold, and (ii) a behavioral analysis of the epistemological structure of agent reasoning. We observe that the base model is the primary determinant of both performance and behavior, accounting for 41.4% of explained variance versus 1.5% for the scaffold. Across all configurations, evidence is ignored in 68% of traces, refutation-driven belief revision occurs in 26%, and convergent multi-test evidence is rare. The same reasoning pattern appears whether the agent executes a computational workflow or conducts hypothesis-driven inquiry. They persist even when agents receive near-complete successful reasoning trajectories as context, and the resulting unreliability compounds across repeated trials in epistemically demanding domains. Thus, current LLM-based agents execute scientific workflows but do not exhibit the epistemic patterns that characterize scientific reasoning. Outcome-based evaluation cannot detect these failures, and scaffold engineering alone cannot repair them. Until reasoning itself becomes a training target, the scientific knowledge produced by such agents cannot be justified by the process that generated it.

Martiño Ríos-García, Nawaf Alampara, Chandan Gupta +5

Laboratory of Organic and Macromolecular Chemistry (IOMC), Friedrich Schiller University Jena, Humboldstr. 10, 07743 Jena, Germany · Department of Civil Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India · School of Interdisciplinary Research, Indian Institute of Technology Delhi, Hauz Khas, New Delhi 110016, India +3

AI Agent ReliabilityScientific ReasoningLLM Agent EvaluationLLM ReliabilityAutonomous Scientific DiscoveryScientific Reasoning in Language Models+2
AskPDF
May 13, 2026

LLMs Know When They Know, but Do Not Act on It: A Metacognitive Harness for Test-time Scaling

Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succeed, and after solving it, they can judge whether their answer is likely to be correct. However, these signals are typically measured or elicited in isolation, rather than used to control inference. In this work, we ask whether LLMs possess latent metacognitive ability that can be turned into effective test-time control. Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning. For each problem, the model first reports a pre-solve feeling-of-knowing (FOK) signal; after each solve attempt, it reports a post-solve judgment-of-learning (JOL) signal. Rather than treating these signals as passive confidence estimates, the harness turns them into an explicit control interface for reasoning: it decides when to trust the current solution, when to retry with compact metacognitive feedback, and when to pass multiple attempts to a final aggregator. Across text, code, and multimodal reasoning benchmarks, our harness substantially improves a fixed Claude Sonnet-4.6 base model without parameter updates or benchmark-specific fine-tuning. On the evaluated public benchmark snapshots, it raises pooled accuracy from 48.3 to 56.9 and exceeds the strongest listed leaderboard entries on the three primary evaluation settings: HLE-Verified, LiveCodeBench v6, and R-Bench-V. These results suggest that strong LLMs may already possess useful metacognitive ability, but require an explicit control harness to act on it during reasoning.

Qi Cao, Yufan Wang, Peijia Qin +2

University of California, San Diego

LLM InferenceTest-Time ScalingMetacognition in Language ModelsLLM Reasoning
AskPDF