cs.CLOct 24, 2026
SaveTrajectory2Gym: Decision-Grounded Compositional Interaction Synthesis for Tool-Use Agents
Organizations: Westlake University (westlake.edu) · Ant International (ant-intl.com)
Abstract
Successful tool-use trajectories are not necessarily useful supervision. Even successful interactions may underrepresent multi-step dependencies or contain actions that are unjustified by the information visible to the agent at decision time. We identify two coupled requirements for effective trajectory supervision: compositional coverage, so that important observation-action dependencies are represented, and decision grounding, so that each supervised action is supported by the learner-visible interaction history. We introduce Trajectory2Gym, which constructs executable interactions around under-supported compositional structures and derives supervision only from grounded decisions in their realized executions. Structural-support selection allocates the resulting experience, while parent-normalized supervised fine-tuning preserves the semantic interaction as the unit of training weight. This reframes trajectory generation as the deliberate construction of learnable interaction experience, rather than the collection of successful trajectories. Across five Qwen3 scales, Trajectory2Gym improves Tau1, Tau2, and BFCL at every tested scale, increasing the equal-weight three-benchmark mean by 16.9-29.8 percentage points. The gains are strongest in stateful and multi-turn interaction, where later decisions depend on earlier observations and feedback. Our results suggest that scaling tool-use agents depends not only on collecting more successful trajectories, but on constructing interactions whose dependencies are represented and whose decisions are learnable from the agent's visible history.
Figures & tables
Figure 1: Compositional interaction requires grounded decisions. Top: Trajectory2Gym constructs linked interactions (1) and checks each training action against that history (2). Bottom: a Tau2 Retail user requests a larger bottle and a dimmer lamp, then confirms only the lamp. Both rules judge the same two candidate actions. Filtering by execution keeps both because native tool replay accepts both requests. Trajectory2Gym uses catalog evidence and the latest confirmation to keep the lamp exchange and reject the additional bottle exchange. Green checks mark correct selection decisions; the red cross marks erroneous retention. This schematic illustrates supervision selection with paraphrased dialogue and a constructed second candidate.
Figure 2: Overview of Trajectory2Gym.
| Model | Tau1 | Tau2 | Mean | ||||||
|---|---|---|---|---|---|---|---|---|---|
| Retail | Airline | Avg. | Retail | Airline | Telecom | Avg. | |||
| Closed-source references | |||||||||
| GPT-5.2 (high) | 75.44 | 63.00 | 69.22 | 81.58 | 83.00 | 89.69 | 84.76 | 76.99 | |
| Claude-4.5-Opus (high) | 79.39 | 67.50 | 73.45 | 79.61 | 84.00 | 92.32 | 85.31 | 79.38 | |
| Re-evaluated agent-training checkpoints | |||||||||
| Simia Tau-SFT (Qwen3-8B) | 45.22 | 28.00 | 36.61 | 62.50 | 50.00 | 15.00 | 42.50 | 39.55 | |
Table 1: Tau1 and Tau2 results at all five scales (Pass@1, %; higher is better). Blue rows are Trajectory2Gym-trained; is the paired gain in Mean, the equal average of the two benchmark averages. Tau1 Airline is a seen-task, transductive setting (Appendix A ). Agent-training checkpoints are re-evaluated on the same complete suites: Simia [ Li et al., 2025b ] , MUA-RL [ Zhao et al., 2025 ] , LoopTool [ Zhang et al., 2026a ] , EnvFactory [ Xu et al., 2026 ] , and MidTool-RL [ Jiang et al., 2026 ] (all evaluated checkpoints in Appendix F.1 ); closed-source rows are reported references.
| Model | Non-live | Live | Multi-turn | Overall | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Simple | Par. | Mult. | Irrel. | Simple | Mult. | Irrel. | Base | |||
| Closed-source references | ||||||||||
| GPT-5.5 | 91.50 | 90.50 | 88.50 | 88.75 | 73.64 | 73.79 | 83.82 | 60.00 | 75.63 | |
| Gemini-3.0-Pro | 94.50 | 91.00 | 94.00 | 82.50 | 87.60 | 80.44 | 73.19 | 69.00 | 79.97 | |
| Agent-training references | ||||||||||
| ToolACE-2-8B | 88.00 | 92.50 | 92.50 | 95.41 | 70.93 | 79.01 | 84.80 | 49.00 | 73.12 | |
Table 2: BFCL results (%; higher is better). Overall equally averages the four-category Non-live mean, the three-category Live mean, and Multi-turn Base. Blue rows are Trajectory2Gym-trained; is the paired gain in Overall. Closed-source rows and ToolACE-2-8B follow Gecko Table 3 [ Zhang et al., 2026c ] without GATS test-time refinement; the other agent-training checkpoints are re-evaluated as in Table 1 . Par., Mult., and Irrel. abbreviate the Parallel, Multiple, and Irrelevance categories; a Parallel score of 0.00 is 0/100 correct, not missing data.
| Setting | Tau1 | Tau2 | BFCL | Mean | Gain |
|---|---|---|---|---|---|
| Compositional interaction synthesis | |||||
| Existing candidate pool | 51.00 | 75.83 | 80.39 | 69.08 | – |
| + Targeted expansion | 58.61 | 81.67 | 77.61 | 72.63 | +3.55 |
| Decision-grounded supervision | |||||
| Loose success-derived targets | 48.13 | 71.67 | 75.86 | 65.22 | – |
| + Decision grounding | 51.13 | 74.17 | 78.62 | 67.97 | +2.75 |
Table 3: Matched component ablations on Qwen3-8B (%). Each pair compares one Trajectory2Gym design choice with its stage-matched alternative. Mean equally averages Tau1, Tau2, and BFCL; Gain is the Trajectory2Gym choice minus its alternative, so positive values favor the proposed component. Matching conditions are given in Appendix F.3 .
Figure 3: Structural vs. random selection at a fixed 8B budget (14,819 parents, equal target tokens). (a) Canonical paths covered. (b) Score change of structural selection (points). Structural selection covers 942 more paths and improves both Tau benchmarks, with BFCL essentially unchanged.
| Depth | Trajectories | Target tokens | Tau1 | Tau2 | BFCL | Mean |
|---|---|---|---|---|---|---|
| 2 | 13,986 | 18.02M | 55.61 | 82.50 | 78.74 | 72.28 |
| 3 | 14,819 | 19.42M | 57.00 | 75.83 | 78.81 | 70.55 |
| 4 | 15,805 | 20.61M | 54.00 | 75.00 | 79.31 | 69.44 |
Table 4: Sensitivity to structural-support depth at Qwen3-8B (%). Depth controls the amount of repeated support allocated to certified interaction structures. These control-specific runs use a different training view from the headline 8B checkpoint of Tables 1 and 2 .
Figure 4: Same-task comparison of evidence use and action order. Simia overlooks visible evidence; MUA-RL violates a state-dependent ordering; Trajectory2Gym reuses the evidence and preserves the required dependency (native rewards 0, 0, and 1). Interactions are independently generated and abridged for display; a step-by-step account appears in Appendix G.1 .
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Tau1 | Tau2 | BFCL | Mean |
|---|---|---|---|---|
| MUA-RL | ||||
| 8B | 34.65 | 25.83 | 58.17 | 39.55 |
| 14B | 47.57 | 45.00 | 64.05 | 52.21 |
| 32B | 54.04 | 45.00 | 72.48 | 57.17 |
| Simia (Qwen3-8B) | ||||
| Tau-SFT | 36.61 | 42.50 | 43.08 | 40.73 |
Table 5: Complete public-checkpoint evaluation (%). Tau scores are domain macro averages; Mean equally averages Tau1, Tau2, and BFCL. Trajectory2Gym-8B is shown for reference.
Figure 5: Grounded supervision within a compositional interaction. A failed email lookup remains visible context without receiving imitation loss. Subsequent visible evidence supports identity recovery, cancellation, address modification, and catalog-grounded item updates. Dialogue and tool fields are abridged for display.
| Decision | Assistant decision | Preceding visible evidence | Supervision |
|---|---|---|---|
| 0 | Ask for identity | User requests cancellations and updates | Supervised |
| 1 | Lookup by email | User-supplied email; tool returns user not found | Context only |
| 2–4 | Recover identity | Error, then user-supplied name and ZIP 80281 | Supervised |
| 5–7 | Read two orders; confirm cancellations | User gives both order IDs; reads show pending status | Supervised |
| 8–9 | Cancel both orders | User explicitly confirms both cancellations | Supervised |
| 10–14 | Report results; confirm and change address | Third order is pending; user supplies and confirms the address | Supervised |
Table 6: Decision-level supervision in the multi-order Retail case. The failed lookup uses an email supplied by the user: its argument is visible, but its execution fails. The call and error remain context; user messages and tool responses receive no imitation loss.
Figure 6: Changing intent during a stateful interaction. Simia submits an exchange before a later return request, which the changed order state prevents. In the independently generated Trajectory2Gym dialogue, the user changes the request before submission and the return succeeds. The different histories make this a behavior illustration, not an identical-context comparison.
Figure 7: Delegating phone-side actions and using feedback. MUA-RL repeatedly attempts phone-side operations unavailable to the assistant. Trajectory2Gym delegates those operations to the user and adapts its continuation to the reported device state and connection speed, completing the task.
Figure 8: Recovery enables dependent operations. After a route error, Simia proposes a lookup but does not complete the booking. Trajectory2Gym retrieves the needed airport codes, books the flight, and reuses the returned booking identifier for invoice and support calls. Only the Trajectory2Gym interaction satisfies the native task checker.
Figure 9: An early return blocks later exchange attempts. LoopTool submits a return before confirmation, leaving later exchange and cancellation attempts invalid. Trajectory2Gym preserves the delivered state while obtaining confirmation, then submits the user-selected exchange.
Figure 10: A correct fare does not complete the workflow. Both agents report the correct fare, but Simia does not execute the requested booking. Trajectory2Gym books the flight and uses its returned identifier for the subsequent insurance operation, completing the dependent workflow.
Figure 11: Remaining failure against a successful external baseline. Simia completes payment, line resumption, and device repair. Trajectory2Gym transfers after sending a payment request, leaving the connectivity and overdue-bill checks unsatisfied. This case retains an external-agent success and a Trajectory2Gym failure.
More in this topic
Tool-Using Agents
Tool-using agents plan and act through external tools, APIs, software interfaces, and environments to complete tasks beyond language generation.
See all 289 papers