Recent advances in artificial intelligence (AI) have shown promise in automating key aspects of Agile project management, yet their impact on team cognition remains underexplored. In this work, we investigate cognitive offloading in Agile sprint planning by conducting a controlled, three-condition experiment comparing AI-only, human-only, and hybrid planning models on a live client deliverable at a mid-sized digital agency. Using quantitative metrics -- including estimation accuracy, rework rates, and scope change recovery time -- alongside qualitative indicators of planning robustness, we evaluate each model's effectiveness beyond raw efficiency. We find that while AI-only planning minimizes time and cost, it significantly degrades risk capture rates and increases rework due to unstated assumptions, whereas human-only planning excels at adaptability but incurs substantial overhead. Drawing on these findings, we propose a theoretical framework for hybrid AI-human sprint planning that assigns algorithmic tools to estimation and backlog formatting while mandating human deliberation for risk assessment and ambiguity resolution. Our results challenge the assumption that efficiency equates to effectiveness, offering actionable governance strategies for organizations seeking to augment rather than erode team cognition.
Figures & tables
Metric
AI-Only
Human-Only
Hybrid
Best
Planning Time
0.38 hrs
4.50 hrs
1.80 hrs
AI
Total Completion Time
78.5 hrs
91.0 hrs
82.0 hrs
AI
Cost per Story Point
$78.50
$91.00
$82.00
AI
Forecast Error
8.3%
7.1%
3.8%
Hybrid
Rework Rate
14.2%
9.1%
8.6%
Hybrid
Documented Risks
4
11
13
Hybrid
Table 1: Cross-Phase Summary Metrics
Risk Category
AI-Only
Human-Only
Hybrid
Technical Dependencies
20%
80%
100%
Client Behavior Risks
33%
100%
100%
Third-Party Service Risks
67%
67%
100%
Novel / Context-Specific
0%
67%
100%
Overall
36.4%
78.6%
86.7%
Table 2: Risk Capture Rate by Category Across Conditions
Sprint
AI-Only
Human-Only
Hybrid
Sprint 1
2.3%
1.4%
2.0%
Sprint 2
20.5%
12.8%
11.3%
Sprint 3
12.9%
7.5%
9.7%
Overall
14.2%
9.1%
8.6%
Table 3: Sprint-Level Rework Rate by Condition
Cost Component
AI-Only
Human-Only
Hybrid
Execution Cost
$3,689.50
$4,277.00
$3,854.00
Rework Cost
$521.70
$390.10
$333.70
Planning Ceremony Cost
$17.86
$211.50
$84.60
Total
$4,229.06
$4,878.60
$4,272.30
Table 4: Total Cost of Delivery by Condition
Low Contextual Ambiguity
High Contextual Ambiguity
High Computational Complexity
AI Delegation with Human Review. Velocity forecasting, throughput analysis, routine estimation. AI generates; human confirms applicability to current context.
Iterative Human-AI Collaboration. Scope change impact analysis, architectural refactoring estimation. Human frames context; AI models quantitative implications.
Low Computational Complexity
Full AI Automation. Administrative documentation, status reporting, retrospective formatting. No cognitive risk; full delegation appropriate.
Human Deliberation with AI Scaffolding. Risk identification, assumption articulation, contingency planning. Human leads; AI provides baseline structure to interrogate.
The sprint-based iterative approach in the Agile software development method allows continuous feedback and adaptation. One of the crucial Agile software development activities is the sprint planning session where developers estimate the effort required to complete tasks through a consensus-based estimation technique such as Planning Poker. In the Agile software development method, a common unit of measuring development effort is Story Point (SP) which is assigned to tasks to understand the complexity and development time needed to complete them. Despite the benefits of this process, it is an extremely time-consuming manual process. To mitigate this issue, in this study, we investigated if this manual process can be automated using Retrieval Augmented Generation (RAG) which comprises a "Retriever" and a "Generator". We applied two embedding models - bge-large-en-v1.5, and Sentence-Transformers' all-mpnet-base-v2 on 23 open-source software projects of varying sizes and examined four key aspects: 1) how retrieval hyper-parameters influence the performance, 2) whether estimation accuracy differs across different sizes of the projects, 3) whether embedding model choice affects accuracy, and 4) how the RAG-based approach compares to the existing baselines. Although the RAG-based approach outperformed the baseline models in several occasions, our results did not exhibit statistically significant differences in performance across the projects or across the embedding models. This highlights the need for further studies and refinement of the RAG, and model adaptation strategies for better accuracy in automatically estimating user stories.
Lamyea Maha, Tajmilur Rahman, Chanchal Roy
University of Saskatchewan, Saskatoon, SK, Canada · Gannon University, Erie, PA, USA
Agile estimation techniques, particularly T-shirt sizing, are widely used in software development for their simplicity and utility in scoping work. However, when we apply these methods to artificial intelligence initiatives -- especially those involving large language models (LLMs) and multi-agent systems -- the results can be systematically misleading. This paper shares an evidence-backed analysis of five foundational assumptions we often make during T-shirt sizing. While these assumptions usually hold true for traditional software, they tend to fail in AI contexts: (1) linear effort scaling, (2) repeatability from prior experience, (3) effort-duration fungibility, (4) task decomposability, and (5) deterministic completion criteria. Drawing on recent research into multi-agent system failures, scaling principles, and the inherent unreliability of multi-turn conversations, we show how AI development breaks these rules. We see this through non-linear performance jumps, complex interaction surfaces, and "tight coupling" where a small change in data cascades through the entire stack. To help teams navigate this, we propose Checkpoint Sizing: a more human-centric, iterative approach that uses explicit decision gates where scope and feasibility are reassessed based on what we learn during development, rather than what we assumed at the start. This paper is intended for engineering managers, technical leads, and product owners responsible for planning and delivering AI initiatives.
We propose a stylized model of human-AI collaboration that isolates a mechanism we call the novelty bottleneck: the fraction of a task requiring human judgment creates an irreducible serial component analogous to Amdahl's Law in parallel computing. The model assumes that tasks decompose into atomic decisions, a fraction ν of which are "novel" (not covered by the agent's prior), and that specification, verification, and error correction each scale with task size. From these assumptions, we derive several non-obvious consequences: (1) there is no smooth sublinear regime for human effort it transitions sharply from O(E) to O(1) with no intermediate scaling class; (2) better agents improve the coefficient on human effort but not the exponent; (3) for organizations of n humans with AI agents, optimal team size decreases with agent capability; (4) wall-clock time achieves O(E) through team parallelism but total human effort remains O(E); and (5) the resulting AI safety profile is asymmetric -- AI is bottlenecked on frontier research but unbottlenecked on exploiting existing knowledge. We show these predictions are consistent with empirical observations from AI coding benchmarks, scientific productivity data, and practitioner reports. Our contribution is not a proof that human effort must scale linearly, but a framework that identifies the novelty fraction as the key parameter governing AI-assisted productivity, and derives consequences that clarify -- rather than refute -- prevalent narratives about intelligence explosions and the "country of geniuses in a data center."