cs.SEFeb 16, 2026

GenAI for Systems: Recurring Challenges and Design Principles from Software to Silicon

Authors: Arya Tschand, Chenyu Wang, Zishen Wan, Andrew Cheng, Ioana Cristescu, Kevin He, Howard Huang, Alexander Ingare, 

Organizations: Harvard University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generative AI is reshaping how computing systems are designed, optimized, and built, yet research remains fragmented across software, architecture, and chip design communities. This paper takes a cross-stack perspective, examining how generative models are being applied from code generation and distributed runtimes through hardware design space exploration to RTL synthesis, physical layout, and verification. Rather than reviewing each layer in isolation, we analyze how the same structural difficulties and effective responses recur across the stack. Our central finding is one of convergence. Despite the diversity of domains and tools, the field keeps encountering five recurring challenges (the feedback loop crisis, the tacit knowledge problem, trust and validation, co-design across boundaries, and the shift from determinism to dynamism) and keeps arriving at five design principles that independently emerge as effective responses (embracing hybrid approaches, designing for continuous feedback, separating concerns by role, matching methods to problem structure, and building on decades of systems knowledge). We organize these into a challenge--principle map that serves as a diagnostic and design aid, showing which principles have proven effective for which challenges across layers. Through concrete cross-stack examples, we show how systems navigate this map as they mature, and argue that the field needs shared engineering methodology, including common vocabularies, cross-layer benchmarks, and systematic design practices, so that progress compounds across communities rather than being rediscovered in each one. Our analysis covers more than 275 papers spanning eleven application areas across three layers of the computing stack, and distills open research questions that become visible only from a cross-layer vantage point.

Figures & tables

Figure 1: Growth of AI for Systems publications on arXiv, based on an analysis of over 7,800 papers (2017–2025). (a) Publication count by domain (log scale), showing a 23 ×\times overall increase. (b) Per-domain growth normalized to 2017 levels: although Software dominates in volume, Hardware (43 ×\times ) and Chip Design (60 ×\times ) are rapidly catching up with significantly higher growth rates compared to Software (21 ×\times ), reflecting the field’s broadening from software-centric applications toward the full computing stack.
Figure 2: Scope and organization of the survey. We examine generative AI across three layers of the computing stack (software, hardware architecture, and chip design) and synthesize cross-stack challenges, design principles, and open research questions that emerge from a unified analysis.
SubsectionChallengesKey principlesDescription
3.1 Software engineeringC1, C2, C3P1, P2, P3Generative and agentic systems that produce code edits, patches, and refactorings in large repositories, evaluated through build systems, test suites, and CI pipelines where implicit project conventions and non-local correctness constraints dominate outcomes.
3.2 Performance engineering & code optimizationC1, C2, C4P2, P4, P5Learning-driven techniques for optimizing code, compiler flags, and program transformations using profiling feedback, where performance gains depend on hardware-specific behavior and tight coupling between compilers, runtimes, and microarchitectures.
3.3 GPU kernelsC1, C3, C4P2, P4Automated synthesis and tuning of GPU kernels (e.g., CUDA, Triton), where correctness and performance are validated via expensive compile-run loops and are highly sensitive to memory access patterns, numerical behavior, and runtime interfaces.
3.4 Distributed systemsC3, C4, C5P1, P2, P3Generative approaches to producing system configurations and policies for scheduling, networking, and collective communication, where changes affect live deployments and must remain safe under topology constraints and dynamically varying workloads.
4.1 Performance prediction & DSEC1, C2, C4P1, P2, P4Surrogate modeling and generative search methods for architectural design-space exploration, where expensive simulation or measurement limits feedback and proxy metrics encode tacit assumptions about performance and efficiency.
4.2 Hardware accelerators & AI mappingsC2, C4, C5P2, P4, P5AI-assisted accelerator design and mapping techniques that determine dataflows, scheduling, and memory placement for ML workloads, tightly integrating model structure with compiler stacks and adapting to rapidly evolving architectures and models.
Table 1: Cross-stack challenges and design principles for applying Generative AI across the systems stack (Sections 3 - 5 ). The table highlights recurring challenges (C1-C5) and the most relevant principles (P1-P5) that guide effective and trustworthy deployment.
Figure 3: Challenge–principle response map with diagnostic trajectory. Rows correspond to recurring cross-stack challenges (C1–C5) and columns correspond to design principles (P1–P5). Color intensity reflects the relative dominance of each principle as a response to a given challenge, derived from the annotations in Table 1 . The numbered trajectory illustrates a common maturation pattern: systems often begin with the feedback loop crisis (C1), shift toward trust and validation (C3) as iteration accelerates, and later expose cross-boundary co-design challenges (C4). The map supports diagnosis and iterative navigation rather than prescribing a fixed workflow.
Figure 4: Benchmark progression from 2021 to 2025, illustrating the shift from function-level code generation to repository-scale and agent-based evaluation.
Figure 5: Large language models approach expert-level performance on isolated optimization tasks but lag on real-world repository-level optimizations. The figure compares prompt-based and fine-tuned models across benchmark categories.
Figure 6: Taxonomy of AI-driven GPU kernel optimization, illustrating the evolution from empirical autotuning to learned cost models and generative synthesis.
Figure 7: Taxonomy of generative AI applications in distributed system integration, spanning network control, scheduling, and collective communication.
Figure 8: Relationship between computing system complexity and prediction model complexity across representative performance prediction studies. The figure illustrates the progression from early regression models to deep graph networks, hybrid analytical–ML systems, and regression language models as hardware systems grow in scale and heterogeneity [ 69 , 27 , 162 , 119 , 210 , 7 , 257 ] .
Figure 9: Trends in AI accelerator specialization across recent generations, illustrating how domain-specific features enable compute scaling despite power and memory constraints.
Figure 10: The evolution of performance priorities in memory and data systems from 1965 to 2025, showing how metric families such as cache, bandwidth, tail latency, and ML KPIs wax and wane over time.
Figure 11: Taxonomy of learning-based optimizations for LLM serving, spanning orchestration, system control, and generation engine mechanisms.
Figure 12: Monthly public GitHub repository creations for RTL compared to non-RTL languages, illustrating the severity of data scarcity.
Figure 13: Evolution of placement algorithm performance over time, illustrating the transition from classical optimization to GenAI-driven approaches. Performance is measured by Half-Perimeter Wirelength normalized to the analytical baseline RePlAce [ 273 , 139 , 37 , 129 , 78 , 154 , 115 , 114 , 33 , 250 , 120 , 223 ] .
Figure 14: Survey of ML in hardware verification papers by ML method across formal methods and functional verification. Colors encode aggregate ML method.
Required CapabilityMethodsBenchmarksTools & Infra.Deployment EvidenceCross-Domain Transfer
1. Fast, structured feedback loop Feedback Loop Crisis [C1] →\rightarrow Continuous Feedback [P2]⚫◗◗◗❍
2. Independent validation infrastructure Trust & Validation [C3] →\rightarrow Hybrid Approaches [P1]◗◗◗◗❍
3. Adaptive policies with safety guarantees Determinism to Dynamism [C5] →\rightarrow Hybrid Approaches [P1]◗❍❍◗❍
4. Generator-verifier separation Trust & Validation [C3] →\rightarrow Separate Concerns by Role [P3]◗❍❍❍❍
5. Captured and updatable domain knowledge Tacit Knowledge [C2] →\rightarrow Build on Systems Knowledge [P5]◗❍❍❍❍
6. Cross-layer optimization interfaces Co-Design Across Boundaries [C4] →\rightarrow Match Problem Structure [P4]◗❍❍❍❍
Table 2: The AI-for-Systems Readiness Matrix. Rows represent six core capabilities derived from the challenge–principle pairings in this survey, ordered by aggregate maturity. Columns represent evidence dimensions, ordered by pipeline progression. Ratings: ⚫ established, ◗ emerging, ❍ early. See text for definitions and discussion.
More in this topic

Electronic Design Automation

Electronic Design Automation uses algorithms and software to design, verify, optimize, and manufacture electronic circuits and systems.

See all 213 papers

Explore similar work

CardsList

  1. Generative AI Technologies, Techniques & Tensions: A Primer

    Apr 19, 2026John T. BehrensGenerative AI in EducationHuman-Centered AI

    University of Notre Dame

  2. Thinking Outside the [Chat]Box: Bridging Computer Science and Industrial Design for Cognitive-Inclusive Generative AI

    Jun 12, 2026Virginia Francisco, Daniel Guasch, Raquel HervásHuman-AI InteractionHuman-Centered AI

    Facultad de Inform´atica Universidad Complutense de Madrid Madrid, Spain · Universitat Polit`ecnica de Catalunya Villanova i la Geltru, Spain · Instituto de Tecnolog´ıa Del Conocimiento Universidad Complutense de Madrid Madrid, Spain

  3. Grounded Inference: Principles for Deterministically Encapsulated Generative Models

    Jun 18, 2026Marty O'NeillGenerative ModelingGenerative AI Adoption

    Odenton, MD, USA