cs.CLOct 14, 2026

Interlock: From Local Correctness to Task Completion in Long-Horizon Agents

Authors: Linji Hao, Yanji He, Zihan Wang, Hansi Yang, Jiayuan Zhang, Hongyan Hao, Qi Gu, Yan Feng, 

Organizations: Peking University · The Hong Kong University of Science and Technology (Guangzhou) · Tsinghua University · Meituan, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language model agents increasingly undertake long-horizon tasks in complex environments characterized by fragmented evidence, partially completed states, and dependencies across resources and artifacts. Existing agents may satisfy individual requirements while failing to maintain these relationships, producing locally correct but incomplete outcomes. These failures arise because task completion depends on relationships among environmental evidence, consequential decisions, and required effects. To address this completion gap, we propose Interlock, a framework for task synthesis that represents each task as a typed completion graph. Grounding edges connect public evidence to decisions, coupling hyperedges impose joint constraints across components, and realization edges connect decisions to all required downstream effects. Interlock compiles this graph into a resettable workspace, a public request, and an evaluation rubric. A source-coverage review checks whether the original requirements survive generation, while a public-input review checks whether the rubric is supported by the materials available to the solver. An agent-as-judge inspects execution artifacts and runs verification scripts, retaining trajectories judged to satisfy all mandatory requirements. To evaluate training utility, we fine-tune two Qwen3.6 backbones on their own accepted trajectories. Across the two backbones, training improves AutomationBench task pass rates by 4.00-4.50 percentage points, JobBench weighted scores by 3.20-6.64 points, and Harvey LAB criterion pass rates by 3.89-4.16 percentage points. These results demonstrate the training utility of the synthesized experience across external task environments.

Figures & tables

Figure 1: From local correctness to task completion. A correct flight booking can still leave schedule and budget updates unfinished. Interlock preserves grounding, coupling, and realization relations during task synthesis to connect evidence, decisions, and required downstream effects.
Figure 2: Interlock overview. Left: Grounding connects evidence to decision variables; Coupling imposes joint constraints across decisions and effects; Realization specifies the required consequences of each decision. These relations guide construction of the workspace W0W_{0} , public request qq , and private rubric VV , followed by source-coverage and public-support review. Right: Real execution and an agent-as-judge provide evidence for task admission and trajectory selection. Each backbone is fine-tuned on its own accepted rollouts.
Base ModelMethodAutomationBenchJobBenchHarvey LAB
MainEasy
 Original14.3314.1154.4673.96
Qwen3.6-27BClawGym-SFT9.0011.8248.0175.03
 Interlock-RFT18.3318.4661.1077.85
 Original9.0011.8151.8654.18
Qwen3.6-35B-A3BClawGym-SFT5.176.2540.6654.97
Table 1: Main results on unseen benchmark tasks. AutomationBench and Harvey LAB report task and criterion pass rates (%), respectively; JobBench Main and Easy report weighted scores (0–100). Each panel compares training conditions within one backbone. Higher is better, and the best score within each backbone is shown in bold.
Figure 3: Broader domain coverage improves transfer at a fixed trajectory count. The original checkpoint is labeled 0, and 128 denotes all domains in this model’s training set for the main experiment. AutomationBench reports task pass rate (%), JobBench Main and Easy report weighted scores (0–100), and Harvey LAB reports criterion pass rate (%). Lines connect the evaluated settings.
Trajectory filterJudge output ↓\downarrow (k tokens/rollout)Automation Bench ↑\uparrowJobBench Main ↑\uparrowJobBench Easy ↑\uparrowHarvey LAB ↑\uparrow
Original (no SFT)–14.3314.1154.4673.96
LLM-as-judge11.0313.1715.1456.5375.28
Agent-as-judge12.2818.3318.4661.1077.85
Table 2: Agentic verification analysis. Both filters use the same judge backbone, rubrics, and Qwen3.6-27B rollout pool. The resulting models share initialization and SFT settings. Judge output sums all verification calls and is averaged over all candidate rollouts, including accepted and rejected ones; tool returns are excluded ( k=103\mathrm{k}=10^{3} tokens).
Figure 4: Execution horizons of Interlock and ClawGym trajectories. Bars report the mean, median, and 95th percentile (P95) of recorded tokens, assistant turns, and tool calls. Each trajectory has equal weight, and K=1,024\mathrm{K}=1{,}024 tokens. Appendix C gives the measurement protocol.
Figure 5: Task-statement embeddings. Joint PCA–t-SNE projection of normalized BGE-M3 embeddings; “ours” denotes Interlock .
Figure 6: Case study. Left: recovery planning couples supply, whole-order service, and action costs. Right: training schedules couple due scope, shift windows, and dated qualifications.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Cumulative execution-horizon distributions. Horizontal axes show recorded trajectory tokens, assistant turns, and tool calls. The vertical axis is the fraction of trajectories at or below each horizontal value; a curve farther to the right indicates larger values at the same percentile. The curves use the same 32,370 Interlock and 24,497 ClawGym trajectories and token-counting rules as Figure 4 ; K=1,024\mathrm{K}=1{,}024 .
Figure 8: Trajectory-length and assistant-turn histograms. Each vertical value is the fraction of trajectories in the corresponding bin. The two corpora share bin boundaries within each panel; the plotted values are bin shares rather than probability densities. The panels retain the full observed range of the 32,370 Interlock and 24,497 ClawGym trajectories, with the same counting rules as Figure 4 ; K=1,024\mathrm{K}=1{,}024 tokens.
Figure 9: Eight task archetypes with multi-file workspaces and multi-criterion rubrics. All panels cover the 3,278 retained tasks: (a) archetype shares, (b) source workspace file counts, and (c) rubric criterion counts. Orange dashed lines mark medians.
Figure 10: Content inspection and recomputation dominate the declared checks. Bars show counts and shares of all 99,219 criteria, with each criterion counted once under its declared verification method.
Figure 11: Verification-method composition varies across task archetypes. Each row divides method counts by that archetype’s total criterion count and sums to 100%; criteria are unweighted by rubric scores.
Figure 12: More atomic outcome criteria are associated with longer executions. Points summarize the 3,278 retained tasks by averaging the available solver-specific medians over repeated runs; horizontal jitter aids display. Red curves show binned medians; shaded bands show the 25th–75th percentiles of task summaries. Annotations report Spearman correlations and 95% confidence intervals from bootstrap resampling clustered by source task. Token axes are logarithmic; K=1,024\mathrm{K}=1{,}024 .
Figure 13: A protocol must preserve both feasibility and interpretation. Five selected failures connect the risk rule, treatment timing, daily resources, payer boundary, and endpoint roles. AP5 and AP7 mean termination 4–6 and 6–8 days after planting, respectively; the risk threshold is two bushels per acre. Full saved Judge counts are 15/22 and 22/22 for attempts 1 and 2 of Qwen3.6-35B-A3B, respectively; TT denotes model turns.
Figure 14: Correct capacity calculations must constrain the final allocation. The diagram traces demand coverage, panel scope, energy-service windows, and hardware nameplates. Attempt 2 meets the highlighted object constraints but retains an energy deficit, missing existing peak loads, and a cost inconsistency. Both Qwen3.6-27B attempts remain incomplete, at 35/44 and 41/44 criteria; the FLT-207 counterpart is marked as future service.
Figure 15: Cash holds still require corresponding cost records. The three focal obligations concern E1004’s pending amount, E1011’s refund authority, and E1005’s gross-cost entry. The 1,296.80cashholdand1,296.80 cash hold and 1,760 gross cost refer to different required records. Saved Judge counts for the two Qwen3.6-27B attempts are 20/29 and 29/29; the independent audit retains the off-cycle tax error shared by both executions.
ParameterValue
Maximum sequence length200k200\mathrm{k} tokens
Learning rate2×10−52\times 10^{-5}
Batch size64
Training epochs5
Warmup ratio0.03
Learning-rate scheduleCosine
Appendix
Table 3: Shared SFT configuration for Qwen3.6-27B and Qwen3.6-35B-A3B.
More in this topic

Long-Horizon Agent Tasks

Long-Horizon Agent Tasks study how AI agents plan, remember, and act reliably across extended, multi-step interactions.

See all 193 papers

Explore similar work

CardsList
Oct 10, 2026

SkillHorizon: Constructing Long-Horizon Agent Tasks through Skill Composition

Long-horizon agents must complete workflows in which later activities depend on earlier outputs and changes to shared state. A key challenge for evaluation is to expand task requirements and their dependencies while keeping tasks executable and outcomes verifiable. We introduce SkillHorizon, a benchmark that constructs long-horizon tasks through skill composition in seven business environments. Its generators use domain-specific rules to turn reusable skill procedures into concrete business activities. These activities are connected through input-output dependencies, shared resources, and state updates, allowing workload and coordination demands to grow together. Successful reference executions validate task feasibility and measure interaction workload, while programmatic evaluators assess local completion and whole-task success. We evaluate five language models on 1,050 tasks with reference interaction workloads ranging from 64K to 1M tokens. Across these scales, mean local-goal completion falls from 90.4% to 46.8%, as completed work fails to keep pace with growing requirements. At larger workloads, agents more often submit unsuccessful final answers with substantial work unfinished and tool-call allowance remaining. Trace analysis also identifies cases where scripts execute successfully but their outputs fail downstream business checks. SkillHorizon provides a configurable testbed for studying sustained progress, dependency coordination, and complete task delivery in growing workflows. Our code is available at https://github.com/KnowComp/SkillHorizon.

Yuxuan Liu, Zhaochen Su, Hangyu Guo +11

The Hong Kong University of Science and Technology · Alibaba Group · Shanghai Jiao Tong University

Long-Horizon Agent TasksLong-Horizon Agent EvaluationAI Agent BenchmarksBusiness Process Automation
AskPDF
Aug 3, 2026

LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks

Large language model (LLM) agents increasingly undertake long-horizon tasks that require sustained reasoning, tool use, and revision across many interdependent steps. However, existing agent harnesses maintain task execution, task state, and completion assessment within a growing context, making the state difficult to track and allowing incorrect self-assessments to propagate into later decisions. We reformulate long-horizon execution as a task-state management problem and propose LongHorizon-Harness, which maintains the task state explicitly outside execution and updates it only with facts independently verified from the environment. Its Manage-Execute-Audit(MEA) loop uses a manager to maintain the task state and determine the next subtask, a fresh-context executor to perform it, and a read-only auditor to verify the resulting environment state before the next round. A lightweight AgentAdapter supports interchangeable model and harness backends without modifying their native agent loops. LongHorizon-Harness improves Qwen3.7-Plus from 51.8% to 80.7% on WeaveBench, from 69.7% to 77.2% on Terminal-Bench2.1, and from 2.8% to 8.3% on OSWorld2.0. It also raises Claude Opus4.7 from 20.0% to 34.3% on an OSWorld2.0 subset, demonstrating consistent gains across models, harnesses, and interaction domains.

Ziyu Ma, Hailang Huang, Shun Zou +5

DreamX Team, Alibaba Group

Long-Horizon Agent EvaluationLong-Horizon LLM AgentsLLM Agent Harnesses
AskPDF
Feb 28, 2026

LOGIGEN: Logic-Driven Generation of Verifiable Agentic Tasks

The evolution of Large Language Models (LLMs) from static instruction-followers to autonomous agents necessitates operating within complex, stateful environments to achieve precise state-transition objectives. However, this paradigm is bottlenecked by data scarcity, as existing tool-centric reverse-synthesis pipelines fail to capture the rigorous logic of real-world applications. We introduce LOGIGEN, a logic-driven framework that synthesizes verifiable training data based on three core pillars: Hard-Compiled Policy Grounding, Logic-Driven Forward Synthesis, and Deterministic State Verification. Specifically, a Triple-Agent Orchestration is employed: the Architect compiles natural-language policy into database constraints to enforce hard rules; the Set Designer initializes boundary-adjacent states to trigger critical policy conflicts; and the Explorer searches this environment to discover causal solution paths. This framework yields a dataset of 20,000 complex tasks across 8 domains, where validity is strictly guaranteed by checking exact state equivalence. Furthermore, we propose a verification-based training protocol where Supervised Fine-Tuning (SFT) on verifiable trajectories establishes compliance with hard-compiled policy, while Reinforcement Learning (RL) guided by dense state-rewards refines long-horizon goal achievement. On τ2τ^2-Bench, LOGIGEN-32B(RL) achieves a 79.5% success rate, substantially outperforming the base model (40.7%). These results demonstrate that logic-driven synthesis combined with verification-based training effectively constructs the causally valid trajectories needed for next-generation agents.

Yucheng Zeng, Weipeng Lu, Linyun Liu +9

Baidu Inc. · Tsinghua University

Synthetic Data Generation for LLM AgentsLLM Agent Trajectory SynthesisLLM Agent TrainingSynthetic Data Generation
AskPDF