From Watching to Steering: Enabling Human-Coding Agent Collaboration through Execution Traces
Authors: Xiaoke Yan, Jiaying Lei, Danqing Shi, Zichen Wang, Nan Cao
Organizations: Intelligent Design & Innovation Lab, Tongji University, Shanghai, China · Department of Computer Science, LTH, Lund University, Sweden · School of Mathematical Sciences, Tongji University, Shanghai, China
Recent advances in large language models (LLMs) have enabled coding agents to autonomously perform complex software development tasks, yet their long-running execution remains difficult for users to inspect, understand, and steer. We present Vibetrace, an interactive trace visualization and intervention system for agentic programming built upon OpenCode. Vibetrace introduces a human-agent collaboration paradigm of Observe, Understand, Intervene, and Evolve. It (1) visualizes agent workflows as interactive execution traces for inspecting actions, intermediate states, and failures; (2) enables users to fork execution from prior steps and provide targeted corrections without restarting validated progress; and (3) distills traces and human feedback into reusable skills for continual agent evolution. On SWE-bench Verified, distilled skills reduce input tokens by 23.0% and cost by 15.5%, while improving pass rate from 65.0% to 68.0%. User studies further demonstrate improved accuracy and efficiency over conventional log-based interaction.
Figures & tables
Fig. 1: VibeTrace enables a continuous human–coding agent collaboration loop. The user first assigns a coding task, after which the agent executes it while VibeTrace records and visualizes the execution trace. By inspecting the trace and step details, the user can observe and understand the agent’s behavior, identify errors, and intervene by forking the execution and providing feedback. VibeTrace then distills successful human interventions into reusable skills and updates the skill pool, allowing the evolved agent to apply learned knowledge to subsequent tasks. This trace–feedback–skill loop transforms transient human interventions into persistent collaboration knowledge and supports continuous agent improvement. The source code is available at https://github.com/idvxlab/VibeTrace .
Fig. 2: System framework and running pipeline. VibeTrace transforms opaque agent execution into interactive traces that support workflow inspection, fine-grained intervention, and continuous skill evolution.
Fig. 3: Interactive user interface.
Fig. 4: Visualization of an execution shown in a trace card.
Fig. 5: Overview of the online trace-to-skill distillation process in VibeTrace. Execution traces and human feedback are analyzed to extract reusable skill patches, which are evaluated and integrated into the skill pool to continuously improve future agent behaviors.
Method
Pass (%) ↑
In-Tok. (Cached) ↓
Out-Tok. ↓
Cost ( 10−3)\downarrow$
Time (s) ↓
Calls ↓
J(Ω)↑
Base
65.00±3.16
1459.60±467.94 ( 1431.45 )
17.62±0.97
12.88±1.89
268.25±15.76
38.26±3.91
0.25±0.01
Trace-only
65.00±4.47
1303.26±284.74 ( 1275.60 )
16.43±2.40
12.04±1.66
257.05±25.05
36.35±3.83
0.26±0.02
Trace-fb (ours)
68.00±2.45
1123.33±137.32 ( 1098.68 )
15.56±1.34
10.88±0.94
229.32±16.00
34.47±2.64
0.28±0.04
TABLE I: Held-out SWE-bench Verified results over 20 tasks and five repeated runs. The split and stream order use seed 42. Values are mean ± population standard deviation. Tokens are in thousands and cost is in 10−3 $. Parentheses in the input-token column show mean cache-read tokens.
Fig. 6: Accuracy and response time comparison between VibeTrace and the baseline; ** denotes p<0.01 and *** denotes p<0.001 . (a) Overall accuracy: VibeTrace (95%) and the baseline (76%). (b) Average response time: VibeTrace (44.0 s) and the baseline (99.7 s). (c) Per-question accuracy across Q1–Q10. (d) Per-question response time across Q1–Q10.
Fig. 7: Distribution of interaction time across the four panels for each participant ( N=8 ).
Fig. 8: VibeTrace supports observing and understanding long-running agent execution through interactive visualization of agent execution traces. The figure illustrates a representative workflow: (a) the user asks the agent to investigate two UI techniques and synthesize the findings into a webpage; (b–c) the interaction trace branches into two subagent workflows, each exploring one technique; (d–f) the visualization reveals three failures in the second branch, including a rate-limit error, an oversized argument list, and an invalid tool invocation; (g) the terminal state summarizes the execution, identifies the root causes, and recommends corresponding fixes.
Fig. 9: VibeTrace lets the user intervene in the agent execution by forking the execution trace. Left: The user forks a new branch before development begins, prompting a request for design clarification. The greyed-out branch is the original trace that implements the application without consulting design styles, while the lower branch is the newly forked branch that includes user feedback. Middle: the user feedback interface of the forked branch for reminding the agent to settle the frontend design details before building. Right: On the forked branch, the agent uses the question tool to ask design details.
Fig. 10: VibeTrace distills a skill from human feedback and reuses it in a later task. Left: the agent automatically matches the distilled skill. Right: VibeTrace automatically distilled front-end-design-alignment-before-build.skill .