Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects

Syed Mohammad Kashif; Ruiyin Li; Peng Liang; Amjed Tahir; Qiong Feng; Zengyang Li; Mojtaba Shahin

cs.SEApr 7, 2026

Beyond Functional Correctness: Design Issues in AI IDE-Generated Large-Scale Projects

Authors: Syed Mohammad Kashif, Ruiyin Li, Peng Liang, Amjed Tahir, Qiong Feng, Zengyang Li, Mojtaba Shahin

Organizations: School of Computer Science, Wuhan University, China · School of Mathematical and Computational Sciences, Massey University, New Zealand · School of Computer Science, Nanjing University of Science and Technology, China · School of Computer Science, Central China Normal University, China ·

Abstract

New generation of AI coding tools, including AI-powered IDEs equipped with agentic capabilities, can generate code within the context of the project. These AI IDEs are increasingly perceived as capable of producing project-level code at scale. However, there is limited empirical evidence on the extent to which they can generate large-scale software systems and what design issues such systems may exhibit. To address this gap, we conducted a study to explore the capability of Cursor in generating large-scale projects and to evaluate the design quality of projects generated by Cursor. First, we propose a Feature-Driven Human-In-The-Loop (FD-HITL) framework that systematically guides project generation from curated project descriptions. We generated 10 projects using Cursor with the FD-HITL framework across three application domains and multiple technologies. We assessed the functional correctness of these projects through manual evaluation, obtaining an average functional correctness score of 91%. Next, we analyzed the generated projects using two static analysis tools, CodeScene and SonarQube, to detect design issues. We identified 1,305 design issues categorized into 9 categories by CodeScene and 3,193 issues in 11 categories by SonarQube. Our findings show that (1) when used with the FD-HITL framework, Cursor can generate functional large-scale projects averaging 16,965 LoC and 114 files; (2) the generated projects nevertheless contain design issues that may pose long-term maintainability and evolvability risks, requiring careful review by experienced developers; (3) the most prevalent issues include Code Duplication, high Code Complexity, Large Methods, Framework Best-Practice Violations, Exception-Handling Issues and Accessibility Issues; (4) these design issues violate design principles such as SRP, SoC, and DRY. The replication package is at https://github.com/Kashifraz/DIinAGP

Figures & tables

Figure 1. Overview of our research process

Figure 2. Overview of the Feature-Driven Human-In-The-Loop (FD-HITL) Framework

Inclusion Criteria
I1	The project should have at least 8k lines of code (LoC).
I2	The project should be generated in a technology stack with at least 3 technologies.
I3	The project is built in a complex architectural style with multiple architectural components.
E4	The project uses external dependencies.
Exclusion Criteria
E1	The project is small and not built in a complex architectural style with multiple architectural components.

Table 1. Inclusion and exclusion criteria for the AI-IDE generated projects

Project ID	Project Name	SonarQube Issues	CodeScene Issues
P1	CVBuilder	231	217
P2	VocabularyApp	350	90
P3	Ecommerce	268	113
P4	JobApplication	253	75
P5	ChartMaker	53	62
P6	FormPlugin	97	74

Table 2. Number of issues found by SonarQube and CodeScene across different projects

Project	LoC	Files	Functional Correctness	Dependencies	Technology Stack
P1_CVBuilder	26,798	137	86.36%	47	MERN stack
P2_VocabularyApp	11,066	83	84.61%	35	React Native, Spring Boot & MySQL
P3_Ecommerce	14,644	112	88.46%	37	React, Spring Boot & MySQL
P4_JobApplication	15,312	79	89.47%	28	MERN stack
P5_ChartMaker	9,391	49	92.85%	20	Vue.js, Django & MySQL
P6_FormPlugin	8,594	27	92.85%	4	PHP, MySQL & WordPress API

Table 3. Overview of large-scale projects generated by Cursor

Figure 3. An example of a mobile application ( P8_SocialApp ) generated using Cursor

Figure 4. An example of a Web application ( P7_BlogWebsite ) generated using Cursor

Figure 5. Overview of the categories of the design issues identified by CodeScene

Principle	CodeScene	SonarQube
Don’t Repeat Yourself (DRY) ( Fowler, 2008 )	Code Duplication	Dead, Duplicate or Redundant Code
Single Responsibility Principle (SRP) ( Martin, 2014 )	Complex Method, Large Method, Excess Number of Function Arguments	Design Principle Violation
Separation of Concerns (SoC) ( Tarr et al., 1999 )	Overall Code Complexity
Fail Fast ( Shore, 2004 )		Exception-Handling Issue
Keep It Simple, Stupid (KISS) ( Martin, 2009 )	Complex Conditional, Bumpy Road Ahead, Deep Nested Complexity	Code Complexity, Control Flow and Conditionals Issue

Table 4. Mapping of design principles to issues identified by CodeScene and SonarQube

Figure 6. An example of Code Duplication design issue

Figure 7. Distribution of cyclomatic complexity (CC) in Complex Method design issues and lines of code (LoC) in Large Method design issues in Cursor-generated projects

Figure 8. Examples of Complex Conditional design issue

Figure 9. An example of Bumpy Road Ahead design issue

Figure 10. An example of Excess Number of Function Arguments and Primitive Obsession design issues

Project	BLOCKER	CRITICAL	MAJOR	MINOR	INFO	Total
P1_CVBuilder	0	9	123	99	0	231
P2_VocabularyApp	0	41	277	32	0	350
P3_Ecommerce	8	65	145	48	2	268
P4_JobApplication	0	2	154	93	2	253
P5_ChartMaker	1	17	11	24	0	53
P6_FormPlugin	0	31	50	16	0	97

Table 5. Overview of project issues by severity level

Figure 11. Overview of the design issues identified by SonarQube

Figure 12. An example of Duplicated CSS Selectors design issue

Figure 13. An example of Generic Exceptions design issue

Figure 14. An example of Complex Ternary Operators design issue

Figure 15. An example of a Higher Number of Constructor Parameters design issue

Figure 18. An example of Merging Conditional design issue

Figure 19. Overview of overlap between issues identified by SonarQube and CodeScene

Software Engineering

Software engineering studies how to design, build, test, deploy, and maintain reliable software systems and the processes behind them.

See all 270 papers

Sep 11, 2026

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

AI coding assistants are becoming co-authors of production software, yet their evaluation centers on functional correctness, leaving open whether their code differs from human code in the quality dimensions dominating lifecycle cost. We compare human-written and AI-generated code at scale: 787,562 function pairs across Python, Java, and C, each human function mined from open-source repositories paired with implementations generated from its docstring by three AI assistants (OpenAI GPT models, DeepSeek-Coder, Qwen2.5-Coder). We characterize structural complexity and statistical naturalness, and map static-analysis findings onto Orthogonal Defect Classification for defects and the Common Weakness Enumeration for vulnerabilities, making authors and languages directly comparable. AI-generated code is structurally compressed and stylistically templated: roughly half the size and branching of human code, clustering apart at the style level. Defect profiles differ in kind: human code concentrates issues of mature codebases, AI code repetitive boilerplate; security is language-dependent, with LLMs producing more, and more severe, findings in Python and Java but fewer high-severity memory-safety findings than humans in C. Once size is controlled for, complexity metrics carry little signal, while naturalness separates authors. Finally, we release CQBench, a benchmark of 27,346 issue-prone tasks with baselines and an evaluation pipeline for quality assurance and security testing.

Cristina Improta, Pietro Liguori, Domenico Cotroneo

University of Naples Federico II, Naples, Italy · University of North Carolina at Charlotte, Charlotte, NC, USA

LLM Security Code Generation Evaluation Code Generation Benchmarks

Ask PDF

Apr 19, 2026

AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code

Practitioners have reported a directional pattern in AI-assisted code generation: AI-generated code tends to fail quietly, preserving the appearance of functionality while degrading or concealing guarantees. This paper introduces the Reward-Shaped Failure Hypothesis - the proposal that this pattern may reflect an artifact of optimization through human feedback rather than a random distribution of bugs. We define failure truthfulness as the property that a system's observable outputs accurately represent its internal success or failure state. We then present AIRA (AI-Induced Risk Audit), a deterministic 15-check inspection framework designed to detect failure-untruthful patterns in code. We report results from three studies: (1) an anonymized enterprise environment audit, (2) a balanced 600-file public corpus pilot, and (3) a strict matched-control replication comparing 955 AI-attributed files against 955 human-control files. In the final replication, AI-attributed files show 0.435 high-severity findings per file versus 0.242 in human controls (1.80x). The effect is consistent across JavaScript, Python, and TypeScript, with strongest concentration in exception-handling-related patterns. These findings are consistent with a directional skew toward fail-soft behavior in AI-assisted code. AIRA is designed for governance, compliance, and safety-critical systems where fail-closed behavior is required.

William M. Parris

BDB Labs | Jurisprudential AI Governance Initiative

LLM Auditing Static Code Analysis Secure Code Generation

Ask PDF

Apr 27, 2026

Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%

AI coding agents powered by large language models can read codebases and produce functional code, but they routinely violate team-specific product decisions that are invisible in the source code alone. We introduce a controlled benchmark measuring decision compliance, the rate at which an AI coding agent follows established product, design, and engineering decisions, across 8 realistic software engineering tasks containing 41 weighted decision points. We compare a baseline configuration (Claude Code with codebase access only) against an augmented configuration that adds Brief, a product-context retrieval system providing spec generation, mid-build consultation, and retrieval of recorded decisions, persona pain points, customer signals, and competitive intelligence. On identical prompts and the same repository, the augmented configuration achieves 95% decision compliance versus 46% for the baseline, a 49 percentage point improvement. Per-decision analysis reveals that the baseline achieves 100% compliance on decisions visible in the codebase and 0-33% on decisions requiring product context, suggesting that product-context retrieval is a key driver of the improvement. We release the benchmark repository, all 16 pull requests, and scoring harness for independent reproduction.

Drew Dillon, Kasyap Varanasi

Brief

Retrieval-Augmented Generation AI Coding Agents Code Generation Code Generation Benchmarks Coding Agent Benchmarks Retrieval-Augmented Code Generation

Ask PDF

Abstract

Software Engineering

Explore similar work

What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

AIRA: AI-Induced Risk Audit: A Structured Inspection Framework for AI-Generated Code

Context-Augmented Code Generation: How Product Context Improves AI Coding Agent Decision Compliance by 49%