When the harness decides success: state that cannot be re-observed

Agent harnesses repeatedly invoke reasoning after tool feedback, even when an earlier call has already fixed the next action. We introduce HarnessMM (Harness Minus Minus), a method for improving harness token efficiency through guarded decision reuse and recursive self-improvement (RSI). RSI iteratively refines companion harness configurations from training feedback, with validation selecting the configuration for deployment. A controller at the model-tool boundary reuses resolved decisions to replace later reasoning callbacks while actor weights remain fixed. Executable Decision Graphs store exact actions, literal arguments, causal dependencies, and public guards. Guarded Decision Reuse activates these persistent decisions from observed receipts; Certified Callback Substitution executes them through the native interface and records removal of the registered slow callback. New semantic decisions, unsupported evidence, and selector abstention invoke the original actor again. The representation restricts what feedback may change, while complete-task measurements establish quality and cost. On HumanEval, HarnessMM preserves Codex success at 90.82% while reducing Tokens/task (k) from 66.72 to 47.02 (29.54% saving). For DeepSeek Harness, success rises from 71.43% to 85.71% while Tokens/task (k) fall from 96.28 to 11.62 (87.93% saving). A separate MBPP ablation measures 26.91% lower cost than single-use graphs and 19.52% lower cost than retained callbacks. These results establish guarded decision reuse as a practical way to reduce repeated reasoning in existing harnesses.

Context managers summarize or evict old messages so agents can operate beyond a finite window. This creates a reliability problem: removing history that still shapes the model's computation may change later actions, while retaining everything defeats compression. We study plans because agents create them early and are expected to use them later, yet behavioral success alone cannot reveal whether later calls still depend on the plan exchange. We make four contributions. First, we introduce replay pairing, which holds the action-observation trajectory fixed and compares pre-action hidden states with and without the plan exchange. We show that, on Llama-3.1-70B in ALFWorld, the mean contrast across 15 sampled layers decreases 4.1x from the first post-plan action to the second; Llama-3.1-8B shows a similar 5.6x decrease. Second, we show that a simple linear model using layer-32 hidden states predicts later contrast on entirely held-out tasks better than a baseline that only knows the action-call offset (R^2 = 0.847 vs. 0.575). Third, we identify confounds that limit this measurement: the paired histories differ in length, token positions, and discourse structure, and replayed reasoning can reintroduce plan-derived text. Reasoning traces therefore require symmetric controls, and native Qwen3 shows that the Llama timing pattern is not universal. Fourth, in a separate descriptive compression test, we find pooled ALFWorld run success of 56.7% with full history and 22.0% with naive eviction; preserving or re-surfacing only the plan does not recover the gap. Replay pairing measures sensitivity to retaining an exchange, not isolated plan content or later eviction. It gives researchers a concrete audit tool and suggests that context policies should preserve broader task state, not plans alone.

Recent studies in harness optimization have shown substantial performance gains in language model agents without updating model weights. A well-designed harness more effectively leverages the underlying model's capabilities by shaping context, tool use, and execution workflows. However, naïve success rates alone do not fully characterize whether optimized harness yields more reliable success on tasks already within reach (elicitation) or a broader range of solvable tasks (expansion). This distinction is crucial for recursive self-improvement and evolutionary search, as expansion may provide a broader range of successful experiences to push the capability boundaries. Despite the growing significance, this distinction remains underexplored in harness optimization. We conduct a systematic analysis to study the effect of gains from harness optimization. Building on pass@k analyses in the reinforcement learning (RL) literature that compare models before and after RL training, we compare optimized harnesses with their pre-optimization counterparts. Specifically, we assess changes in reachable task coverage, compare harness optimization with model scaling and increased inference-time compute, and examine how gains vary with task difficulty. Across static prediction and dynamic tasks, we find that pass@1 advantages of harness over stronger interventions can reverse at larger attempt budgets and aggregate gains conceal shortfalls on hard tasks, underscoring the need for a fine-grained diagnosis.