TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation

Akash Gokul; Srijan Bansal; Yingbo Zhou; Semih Yavuz

cs.AIOct 19, 2026

TEX: Test-Time Scaling Testing Agents via Execution-based Cross-Validation

Authors: Akash Gokul, Srijan Bansal, Yingbo Zhou, Semih Yavuz

Organizations: NVIDIA · Salesforce Research · Recursive

Abstract

Test-case generation is critical but remains challenging for language models. Existing test-time scaling methods either isolate candidates or aggregate traces, which do not scale to long-horizon tasks. We present TEX, a collaborative test-time scaling framework based on execution-driven cross-validation. TEX parallelizes agents to generate tests and patches, cross-executing each test against every patch to form a $K \times K$ feedback matrix. Agents use their matrix slices for refinement, enabling objective validation without verbose trace sharing. On SWT-Bench Verified, TEX achieves 86.4%, establishing a new state-of-the-art and improving +9.0% over single-agent baselines. Moreover, even random selection from TEX outputs (84.4%) surpasses the previous SOTA (84.0%), showing collaborative execution feedback improves solution quality, not just best-case samples. Our results highlight execution-grounded feedback as an effective scaling mechanism for long-horizon agentic tasks.

Figures & tables

Figure 1: TEX overview. At each round, $K$ agents are sampled in parallel and each agent outputs a test file and code patch. We perform cross-validation by running all generated code patches against all generated test files. Each agent receives execution feedback from running its code & test against all other tests & code, then continues sampling in the next round.

Baselines
Method	Pass@1	Best-of-4
Test-Only	71.3%	–
Test + Code (R0)	77.4%	88.7%
TEX-T
Round 1	84.1%	88.7%
Round 2	84.1%	88.9%

Table 1: SWT-Bench Verified results.

Baseline
Method	Pass@1	Best-of-4
Code Generation (R0)	57.8%	68.4%
TEX-C
Round 1	58.9%	67.7%
Round 2	59.4%	67.8%
Round 3	59.8%	67.9%

Table 2: SWE-Bench Verified results.

Figure 2: Ablation on SWT-Bench subset shows cross-candidate feedback (TEX) outperforms self-feedback under identical compute (

K=4

R=3

Test-Time Scaling

Test-time scaling studies how extra inference-time computation can improve model answers, decisions, and generated outputs.

See all 225 papers

Scaling Test-Time Compute for Agentic Coding
Apr 16, 2026Joongwon Kim, Wannan Yang, Kelvin Niu +13AI Coding Agents Test-Time Scaling
1Meta Superintelligence Labs · University of Washington · 3New York University +4
Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks
Apr 13, 2026Yoonsang Lee, Howard Yen, Xi Ye +1Test-Time Scaling Long-Horizon Agent Tasks
Princeton Language and Intelligence, Princeton University
ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling
May 21, 2026Woomin Song, Beomjun Kim, Daewon Choi +4Test-Time Scaling Multi-Agent Systems
1KAIST · 2Amazon AGI · 3Together AI

Apr 16, 2026J/K move · Enter open · S save

Scaling Test-Time Compute for Agentic Coding

Joongwon Kim, Wannan Yang, Kelvin Niu, Hongming Zhang+12

1Meta Superintelligence Labs · University of Washington · 3New York University+4

AI Coding Agents Test-Time Scaling Long-Horizon LLM Agents

Open paper AskPDF arXiv

Test-time scaling has become a powerful way to improve large language models. However, existing methods are best suited to short, bounded outputs that can be directly compared, ranked or refined. Long-horizon coding agents violate this premise: each attempt produces an extended trajectory of actions, observations, errors, and partial progress taken by the agent. In this setting, the main challenge is no longer generating more attempts, but representing prior experience in a form that can be effectively selected from and reused. We propose a test-time scaling framework for agentic coding based on compact representations of rollout trajectories. Our framework converts each rollout into a structured summary that preserves its salient hypotheses, progress, and failure modes while discarding low-signal trace details. This representation enables two complementary forms of inference-time scaling. For parallel scaling, we introduce Recursive Tournament Voting (RTV), which recursively narrows a population of rollout summaries through small-group comparisons. For sequential scaling, we adapt Parallel-Distill-Refine (PDR) to the agentic setting by conditioning new rollouts on summaries distilled from prior attempts. Our method consistently improves the performance of frontier coding agents across SWE-Bench Verified and Terminal-Bench v2.0. For example, by using our method Claude-4.5-Opus improves from 70.9% to 77.6% on SWE-Bench Verified (mini-SWE-agent) and 46.9% to 59.1% on Terminal-Bench v2.0 (Terminus 1). Our results suggest that test-time scaling for long-horizon agents is fundamentally a problem of representation, selection, and reuse.

Abstract

Test-Time Scaling

Explore similar work

Scaling Test-Time Compute for Agentic Coding

Agentic Aggregation for Parallel Scaling of Long-Horizon Agentic Tasks

ExComm: Exploration-Stage Communication for Error-Resilient Agentic Test-Time Scaling