cs.HCSep 27, 2026
SaveParallelPilot: Supporting Coordination and Monitoring in Parallel AI Coding
Organizations: Microsoft Research AI Frontiers, Columbia University, USA · Microsoft Research AI Frontiers, USA
Abstract
As coding assistants become increasingly autonomous, developers run multiple sessions in parallel, shifting the challenge from code generation alone to coordinating and monitoring concurrent agent work. Through a formative study (N=14), we identified PILOT: five supervisory practices for Planning, Isolating, Logging, Observing, and Triaging parallel sessions. We present ParallelPilot, a design probe that instantiates PILOT through a planning interface, a run-logger, and an ambient dashboard alongside existing coding tools. In a counterbalanced within-subjects study (N=16), participants using ParallelPilot increased ticket throughput by 63% in short coding tasks and supervised an average of one more concurrent agent at peak, while their tracking effort and context switching dropped. ParallelPilot also clarified execution plans, task dependencies, and intervention cues, and 14 of 16 participants preferred it over their current setup. These gains were not accompanied by significant improvements in perceived control or perceived success in redirecting the agents. Our findings demonstrate the value of explicit supervision support and position PILOT as a scaffold for designing tools that help people supervise concurrent work within and beyond coding. We suggest that future coding assistants should pair high-level awareness with low-cost paths back to the implementation evidence developers need to judge and steer agent work.
Figures & tables
Figure 1. Overview of two challenges in parallel AI coding—coordination and monitoring (left)—the PILOT framework (center), and the ParallelPilot design probe (right). PILOT links five supervisory practices (Plan, Isolate, Log, Observe, Triage) to interface support structuring parallel work, preserving session context, and directing attention to agents that need input. ParallelPilot implements these practices through a planning interface, a run-logger, and an ambient dashboard, alongside existing coding tools. Conceptual overview diagram divided into three vertical sections: Challenges of Parallel AI Coding, PILOT Framework, and ParallelPilot as a Design Probe. Left section: Challenges are split into Coordination (task dependencies, overlapping edits) and Monitoring (scattered progress signals, missed input requests, costly context switching). Example task dependency graph shows tasks X, Y, Z, S, W with shared files and conflict warnings. Middle section: The PILOT framework, an acronym for Plan, Isolate, Log, Observe, Triage. PLAN structures tasks and context; ISOLATE identifies and separates dependent work; LOG preserves session context; OBSERVE monitors distributed progress; TRIAGE surfaces intervention cues. Dashed lines connect challenge problems to corresponding framework components. Right section: ParallelPilot design probe implementation. A Planning Interface performs dependency analysis and defines wave‑based agent task plans. Agents launch to execute tasks. Run‑Logger preserves context across raw JSON, mid‑length Markdown, and short summary formats. An Ambient Dashboard visualizes agent status (running, needs input, completed). A closed‑loop human feedback path: Inspect & Respond feeds back to the Planning Interface for iterative revision. Color coding: beige for planning/coordination, light blue for monitoring/intervention. Dashed connectors link problem, framework, and prototype components.
Figure 2. Current coding workflow distribution self-reported by participants ( ). Each bar shows the percentage of coding time spent with no AI tools, a single AI session, or multiple AI sessions in parallel. Participants are sorted by parallel AI usage (highest to lowest); the group average is shown at the top, separated by a dashed line. Three additional columns on the right indicate which tools each participant uses across CLI, IDE-integrated chat interfaces, and WEB/APP (standalone web/desktop AI applications), with aggregate usage counts displayed in the average row. Horizontal stacked bar chart of 14 participants' coding workflow distribution across three categories: no AI tools (orange), a single AI session (blue), and parallel AI sessions (green). Bars are ordered from highest to lowest parallel AI use (P7 at 100% down to P9 at 10%). The group average, shown at the top, is 6.1% no AI, 48.6% single AI, and 45.3% parallel AI. Three columns to the right show tool usage per participant across CLI, Web/App, and IDE interfaces.
Figure 3. P5 invoked the familiar developer trope depicted in xkcd #303 to describe how parallel AI coding changes the experience of waiting. Instead of disengaging while code compiles, developers remain actively engaged by monitoring, answering, and redirecting multiple concurrent AI sessions. Source: Randall Munroe, xkcd #303, https://xkcd.com/303/ , licensed under CC BY-NC 2.5. Two-panel illustration. Left: a participant shown from behind has a thought bubble containing a blank comic area and the text
NO LONGER THIS… BACK TO FULL-TIME ENGINEERING,'' illustrating how parallel AI coding replaces idle waiting with continuous interaction with AI coding agents. Right: the xkcd Compiling'' comic depicts two programmers sword-fighting while their code compiles, representing legitimate idle time.Figure 4. The PILOT framework for supervising parallel AI coding. Five complementary practices address coordination and monitoring challenges: Planning , Isolating , Logging , Observing , and Triaging . Logging bridges both challenge areas by preserving context across sessions. Examples beneath each pillar illustrate developer practices; the pillars are not a fixed sequence. A five-column diagram of the PILOT framework. Overlapping bands group Planning and Isolating under coordination challenges, Observing and Triaging under monitoring challenges, and Logging under both. Planning scopes tasks and prepares context through specs, task decomposition, and priorities. Isolating limits interference through separate files, worktrees, and tools. Logging preserves context by saving outcomes, tracking unfinished work, and rereading logs. Observing brings agent states into one view through tiled sessions, activity indicators, and a prominent primary task. Triaging helps developers notice completion, questions, errors, and unexpected delays. Each column includes an illustration and three example practices.
Figure 5. ParallelPilot instantiates PILOT through three integrated components: a planning interface for task preparation and dependency-aware scheduling (a–b), a run-logger for automatic capture and session memory (c–d), and an ambient dashboard for monitoring progress and triaging agent questions and completion (e–f). The interface thumbnails map these components to successive stages of the workflow; enlarged views of (b) and (e) appear in the system walkthrough. System workflow and six interface screenshots. On the left, tasks, repositories, and specifications feed into the Planning Interface, mapped to Plan and Isolate. Developers refine the plan and launch parallel CLI sessions. The Run-Logger, mapped to Log, captures raw records, session memory, and tracking summaries, which update the Ambient Dashboard, mapped to Observe and Triage. A return arrow shows developers inspecting cues and responding in the CLI sessions. Coordination and monitoring brackets overlap around the Run-Logger. On the right, six screenshots illustrate providing task context, refining the execution plan, launching sessions with automatic logging, reviewing Markdown memory, responding to agent questions, and verifying completion.
Figure 6. ParallelPilot supports dependency-aware plan structuring and iteration. The task analysis (left) distinguishes independent work, shared-file conflicts, and output dependencies. The execution timeline (center) organizes five tasks across three agent windows and two waves, with pagination starred for follow-up. In the assistant panel (right), Amelia compares a four-window alternative and retains three windows to keep supervision manageable. The planning interface has three columns. The left column distinguishes an independent empty-state task, a shared-file conflict between pagination and CSV export in tickets.py, and an output dependency from retry handling to regression tests. The center schedules five tasks across three agent windows and two waves: Window X runs pagination followed by CSV export; Window Y runs the empty-state task; and Window Z runs retry handling followed by regression tests. Pagination is marked with a gold Spotlight star. The right column shows Amelia discussing agent assignments with the assistant, comparing a four-window alternative, and retaining three windows.
Figure 7. ParallelPilot combines glanceable status, completion and attention cues, session memory, and terminal access to support monitoring and triage. After her bathroom break, Amelia catches up on parallel progress: 1a remains running (blue), 1c reports completion pending her review (green), and 2a needs her input on the CLI exit status for skipped records (yellow). The dashboard surfaces completion and attention cues alongside session memory and a progress summary, while direct terminal access lets her review results and answer the agent in its CLI session. The dashboard arranges agent windows in rows with Done, Running Now, and Next columns. Window X shows starred pagination task 1a running in blue, with CSV export queued. Window Y shows empty-state task 1c completed in green, alongside a prompt to review the result and confirm completion. Window Z shows retry task 2a highlighted in yellow, with a thought bubble asking whether skipped records should produce a non-zero CLI exit status. Regression tests remain queued. Cards include activity summaries, files touched, and terminal and session-memory controls. On the right, the assistant summarizes progress after Amelia's bathroom break. A floating approved-plan panel shows task assignments and statuses. Three overlapping terminal windows appear in the foreground; Window Z displays the same exit-status question with response options.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8. Session procedure for the within-subjects study ( , four groups of ). Each 80-minute session opens with a 5-minute consent and setup briefing, then two counterbalanced coding blocks that cross tool condition (ParallelPilot vs. baseline without ParallelPilot) with project (Sparkmatch vs. Kart), and closes with a 10-minute post-study survey and interview. A horizontal timeline with a 0 to 80 minute axis at the top and four rows below it, one per participant group. Each row shows a sequence of blocks sized to their duration in minutes: a short block for consent and setup, two adjacent blocks for the two coding conditions, and a final block for the post-study survey and interview. Each coding block is subdivided into a task briefing segment, an onboarding video segment shown only for the \ppcondition, a coding segment, and a survey segment. A plane icon marks which of the two coding blocks in each row is the \ppcondition. The four rows differ only in the order of the two conditions and which project, Sparkmatch or Kart, is paired with which condition, illustrating the study's counterbalancing scheme.
| Composite (item count) | Kart | Sparkmatch | [95% CI] | TOST |
| Supervision-specific load (3) | 3.54 | 3.33 | [ , ] | Inconclusive |
| Awareness, memory, and recall (3) | 4.61 | 4.42 | [ , ] | Equivalent |
| Planning, decomposition, and delegation (5) | 4.54 | 4.52 | [ , ] | Equivalent |
| Control, verification, and intervention (4) | 3.76 | 4.31 | [ , ] | Equivalent |
| Usability and experience (5) | 4.65 | 4.68 | [ , ] | Equivalent |
Table 1. Paired task comparisons ( ). . Confidence intervals shown are 95%; TOST equivalence decisions use 90% intervals and a margin.
| Composite (item count) | Session 1 | Session 2 | ||
| Supervision-specific load (3) | 3.60 | 3.27 | ||
| Awareness, memory, and recall (3) | 4.39 | 4.65 | ||
| Planning, decomposition, and delegation (5) | 4.40 | 4.66 | ||
| Control, verification, and intervention (4) | 3.90 | 4.17 | ||
| Usability and experience (5) | 4.69 | 4.64 |
Table 2. Paired session-order comparisons ( ). ; is the paired standardized mean difference. All comparisons had .
Figure 9. Task-phase time allocation for dual-completers with and without ParallelPilot. Panel A gives the mean share of task duration spent on Plan & Isolate versus Launch + Monitoring ; Panel B places the same means on a shared 0–16 min axis.
Figure 10. Participants’ use of ParallelPilot’s three components alongside existing coding tools. (a) Planning interface: reviewing the execution plan and dependencies on the left external monitor, with code on the right monitor and project tickets on the laptop. (b) Run-logger: following launch instructions on the left monitor to start sessions with automatic logging, alongside VS Code terminals on the right monitor and a terminal on the laptop. (c–e) Ambient dashboard: the left monitor keeps overall progress visible while the user (c) receives two completion notifications for four sessions running concurrently across the right monitor and laptop, prompting them to go back and inspect; (d) receives one stall notification from the left-monitor dashboard and inspects the flagged session on the right monitor where two sessions sit side by side, while an idle task from a previous wave remains untouched on the laptop; and (e) tiles five sessions on the right monitor, with the ticket open in VSCode on the laptop and a dependent task held back for a later execution wave. Participants and surrounding room details have been masked for privacy and visual clarity.
Figure 11. Exploratory associations between participant-level gains in coordination (top row) and monitoring (bottom row) and gains in ticket completion, throughput, and self-rated efficiency ( ). Each point represents one participant. Annotations report Spearman correlations and unadjusted -values; green panels and asterisks indicate . Black lines show descriptive OLS fits, with gray 95% confidence bands for the fitted means. Six scatterplots arranged in two rows and three columns. The top row plots coordination-composite gains against ticket-completion, throughput, and self-rated-performance gains; the bottom row plots monitoring-composite gains against the same outcomes. Blue points are labeled with participant IDs. Coordination correlations are .55, .62, and .34, respectively; monitoring correlations are -.14, .20, and .62. Green backgrounds highlight coordination with ticket completion and throughput, and monitoring with self-rated performance, based on unadjusted p-values below .05. Black fitted lines and gray confidence bands show the direction and uncertainty of descriptive linear trends. All axes represent condition differences, with positive values indicating improvement with ParallelPilot.