We Ran a Research Sprint With AI. The AI Worked. The Research Didn't
Organizations: Microsoft
Abstract
Generative AI tools for research are widely deployed, yet individual productivity gains do not guarantee better team research. We argue that how AI use is structured matters alongside access. Personal AI tools may amplify two failures under deadline pressure: a consumption bottleneck, in which generation outpaces team-visible evaluation, and AI-induced research silos, in which private human-AI loops displace shared decision-making. We motivate this position with a mechanism-generating pilot: a 100-minute research sprint with twelve researchers in three independently operating teams, within a seventeen-person session. Both patterns recurred across teams and evidence sources. Of 33 logged AI interactions, one was accepted outright and 27 lacked a recorded disposition; missing records alone do not establish absent evaluation. A parallel autonomous pipeline produced a complete draft without in-session human evaluation, illustrating production capacity rather than comparative quality. We propose Human-AI-Human collaboration: an interaction topology in which AI connects researchers' contexts and supports inspectable collective evaluation and decision-making. The pilot does not isolate AI-specific effects from time pressure or workflow practices. We close with seven modular, falsifiable design hypotheses, prioritizing shared context and team-visible evaluation checkpoints for controlled follow-up.
Figures & tables
| Mode | Unit of analysis | Status | Evaluation concern |
|---|---|---|---|
| Human-AI | One person, one model | Default | Private contexts may fragment team evaluation. |
| AI-AI | One pipeline, no human in loop | Trajectory [ 29 , 51 ] | Human verification deferred downstream. |
| Human-AI-Human | The team; AI mediates between researchers | Design agenda | Collective evaluation remains to be tested. |
| Prior work | Claim | What we add |
|---|---|---|
| Farach et al. [14] | A synchronous AI drafting protocol was associated with lower document quality and production. | A motivation to test shared context and collective evaluation, not a fixed sequence of human and AI turns. |
| Tankelevitch et al. [44] | A conceptual framework for individual metacognitive demands of generative AI. | A proposed team-level bottleneck when shared evaluation competes with production. |
| Messeri and Crockett [31] | AI may foster illusions of understanding in science. | Candidate team-level organizational mechanisms. |
| Vaccaro et al. [46] | Human+AI is not automatically better than the best of either. | Shared context and collective evaluation as specific team-level design variables. |
| Liang et al. [27] | Developers report difficulty controlling outputs and meeting code requirements. | Cross-person context and collective evaluation as additional design concerns. |
| Chiang et al. [8] | LLM devil’s advocates can promote appropriate group reliance on AI. | A broader topology encompassing this intervention. |
| ID | Requirement | Failure addressed | Falsifiable prediction |
|---|---|---|---|
| R1 | Shared context substrate across sessions and tools | Idle Claude Code session; cross-tool copy/paste | Tools sharing a substrate show higher cross-person reuse of AI outputs |
| R2 | Conflict and redundancy detection across workstreams | Parallel-session fragmentation in all teams | Tools that flag converging or contradictory sessions reduce late integration failures |
| R3 | Forced shared evaluation checkpoints | 27/33 dispositions unrecorded (Fig. 2 ) | Team-level forcing functions raise the share of AI outputs passing team-visible evaluation |
| R4 | Disposition logging team-visible by default | 82% blank rate on external observer log under load | Built-in logging produces a more complete record than external annotation |
| R5 | Role-aware routing of AI output to relevant expertise | Team 2 acronym-laden hypothesis summary | Routing to expertise-matched teammates with a translation pass reduces incomprehensibility reports |
| R6 | Decision-ownership tracking | First 10 min: research question silently delegated to Claude | Tools recording decision ownership produce more contested mid-session course corrections |
AI-Assisted Scientific Research
AI-assisted scientific research uses machine learning and AI agents to support discovery, analysis, experimentation, and scientific communication.
See all 261 papers