cs.SEFeb 16, 2026

GenAI for Systems: Recurring Challenges and Design Principles from Software to Silicon

Authors: Arya Tschand, Chenyu Wang, Zishen Wan, Andrew Cheng, Ioana Cristescu, Kevin He, Howard Huang, Alexander Ingare, 

Organizations: Harvard University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generative AI is reshaping how computing systems are designed, optimized, and built, yet research remains fragmented across software, architecture, and chip design communities. This paper takes a cross-stack perspective, examining how generative models are being applied from code generation and distributed runtimes through hardware design space exploration to RTL synthesis, physical layout, and verification. Rather than reviewing each layer in isolation, we analyze how the same structural difficulties and effective responses recur across the stack. Our central finding is one of convergence. Despite the diversity of domains and tools, the field keeps encountering five recurring challenges (the feedback loop crisis, the tacit knowledge problem, trust and validation, co-design across boundaries, and the shift from determinism to dynamism) and keeps arriving at five design principles that independently emerge as effective responses (embracing hybrid approaches, designing for continuous feedback, separating concerns by role, matching methods to problem structure, and building on decades of systems knowledge). We organize these into a challenge--principle map that serves as a diagnostic and design aid, showing which principles have proven effective for which challenges across layers. Through concrete cross-stack examples, we show how systems navigate this map as they mature, and argue that the field needs shared engineering methodology, including common vocabularies, cross-layer benchmarks, and systematic design practices, so that progress compounds across communities rather than being rediscovered in each one. Our analysis covers more than 275 papers spanning eleven application areas across three layers of the computing stack, and distills open research questions that become visible only from a cross-layer vantage point.

Figures & tables

Figure 1: Growth of AI for Systems publications on arXiv, based on an analysis of over 7,800 papers (2017–2025). (a) Publication count by domain (log scale), showing a 23 ×\times overall increase. (b) Per-domain growth normalized to 2017 levels: although Software dominates in volume, Hardware (43 ×\times ) and Chip Design (60 ×\times ) are rapidly catching up with significantly higher growth rates compared to Software (21 ×\times ), reflecting the field’s broadening from software-centric applications toward the full computing stack.
Figure 2: Scope and organization of the survey. We examine generative AI across three layers of the computing stack (software, hardware architecture, and chip design) and synthesize cross-stack challenges, design principles, and open research questions that emerge from a unified analysis.
SubsectionChallengesKey principlesDescription
3.1 Software engineeringC1, C2, C3P1, P2, P3Generative and agentic systems that produce code edits, patches, and refactorings in large repositories, evaluated through build systems, test suites, and CI pipelines where implicit project conventions and non-local correctness constraints dominate outcomes.
3.2 Performance engineering & code optimizationC1, C2, C4P2, P4, P5Learning-driven techniques for optimizing code, compiler flags, and program transformations using profiling feedback, where performance gains depend on hardware-specific behavior and tight coupling between compilers, runtimes, and microarchitectures.
3.3 GPU kernelsC1, C3, C4P2, P4Automated synthesis and tuning of GPU kernels (e.g., CUDA, Triton), where correctness and performance are validated via expensive compile-run loops and are highly sensitive to memory access patterns, numerical behavior, and runtime interfaces.
3.4 Distributed systemsC3, C4, C5P1, P2, P3Generative approaches to producing system configurations and policies for scheduling, networking, and collective communication, where changes affect live deployments and must remain safe under topology constraints and dynamically varying workloads.
4.1 Performance prediction & DSEC1, C2, C4P1, P2, P4Surrogate modeling and generative search methods for architectural design-space exploration, where expensive simulation or measurement limits feedback and proxy metrics encode tacit assumptions about performance and efficiency.
4.2 Hardware accelerators & AI mappingsC2, C4, C5P2, P4, P5AI-assisted accelerator design and mapping techniques that determine dataflows, scheduling, and memory placement for ML workloads, tightly integrating model structure with compiler stacks and adapting to rapidly evolving architectures and models.
Table 1: Cross-stack challenges and design principles for applying Generative AI across the systems stack (Sections 3 - 5 ). The table highlights recurring challenges (C1-C5) and the most relevant principles (P1-P5) that guide effective and trustworthy deployment.
Figure 3: Challenge–principle response map with diagnostic trajectory. Rows correspond to recurring cross-stack challenges (C1–C5) and columns correspond to design principles (P1–P5). Color intensity reflects the relative dominance of each principle as a response to a given challenge, derived from the annotations in Table 1 . The numbered trajectory illustrates a common maturation pattern: systems often begin with the feedback loop crisis (C1), shift toward trust and validation (C3) as iteration accelerates, and later expose cross-boundary co-design challenges (C4). The map supports diagnosis and iterative navigation rather than prescribing a fixed workflow.
Figure 4: Benchmark progression from 2021 to 2025, illustrating the shift from function-level code generation to repository-scale and agent-based evaluation.
Figure 5: Large language models approach expert-level performance on isolated optimization tasks but lag on real-world repository-level optimizations. The figure compares prompt-based and fine-tuned models across benchmark categories.
Figure 6: Taxonomy of AI-driven GPU kernel optimization, illustrating the evolution from empirical autotuning to learned cost models and generative synthesis.
Figure 7: Taxonomy of generative AI applications in distributed system integration, spanning network control, scheduling, and collective communication.
Figure 8: Relationship between computing system complexity and prediction model complexity across representative performance prediction studies. The figure illustrates the progression from early regression models to deep graph networks, hybrid analytical–ML systems, and regression language models as hardware systems grow in scale and heterogeneity [ 69 , 27 , 162 , 119 , 210 , 7 , 257 ] .
Figure 9: Trends in AI accelerator specialization across recent generations, illustrating how domain-specific features enable compute scaling despite power and memory constraints.
Figure 10: The evolution of performance priorities in memory and data systems from 1965 to 2025, showing how metric families such as cache, bandwidth, tail latency, and ML KPIs wax and wane over time.
Figure 11: Taxonomy of learning-based optimizations for LLM serving, spanning orchestration, system control, and generation engine mechanisms.
Figure 12: Monthly public GitHub repository creations for RTL compared to non-RTL languages, illustrating the severity of data scarcity.
Figure 13: Evolution of placement algorithm performance over time, illustrating the transition from classical optimization to GenAI-driven approaches. Performance is measured by Half-Perimeter Wirelength normalized to the analytical baseline RePlAce [ 273 , 139 , 37 , 129 , 78 , 154 , 115 , 114 , 33 , 250 , 120 , 223 ] .
Figure 14: Survey of ML in hardware verification papers by ML method across formal methods and functional verification. Colors encode aggregate ML method.
Required CapabilityMethodsBenchmarksTools & Infra.Deployment EvidenceCross-Domain Transfer
1. Fast, structured feedback loop Feedback Loop Crisis [C1] →\rightarrow Continuous Feedback [P2]⚫◗◗◗❍
2. Independent validation infrastructure Trust & Validation [C3] →\rightarrow Hybrid Approaches [P1]◗◗◗◗❍
3. Adaptive policies with safety guarantees Determinism to Dynamism [C5] →\rightarrow Hybrid Approaches [P1]◗❍❍◗❍
4. Generator-verifier separation Trust & Validation [C3] →\rightarrow Separate Concerns by Role [P3]◗❍❍❍❍
5. Captured and updatable domain knowledge Tacit Knowledge [C2] →\rightarrow Build on Systems Knowledge [P5]◗❍❍❍❍
6. Cross-layer optimization interfaces Co-Design Across Boundaries [C4] →\rightarrow Match Problem Structure [P4]◗❍❍❍❍
Table 2: The AI-for-Systems Readiness Matrix. Rows represent six core capabilities derived from the challenge–principle pairings in this survey, ordered by aggregate maturity. Columns represent evidence dimensions, ordered by pipeline progression. Ratings: ⚫ established, ◗ emerging, ❍ early. See text for definitions and discussion.
More in this topic

Electronic Design Automation

Electronic Design Automation uses algorithms and software to design, verify, optimize, and manufacture electronic circuits and systems.

See all 213 papers

Explore similar work

CardsList
Apr 19, 2026

Generative AI Technologies, Techniques & Tensions: A Primer

Generative AI systems have entered everyday academic, professional, and personal life with remarkable speed, yet most users encounter them as mysterious artifacts rather than intelligible systems. This chapter discusses large language models within a broader historical shift in computing paradigms and argues that many of the confusions surrounding their use arise from a mismatch between how these systems are built, how they behave, and how people expect computers to behave writ large. Rather than treating generative AI as a monolithic technology, the chapter decomposes it into interacting components, spanning data, models, product features, and user inputs, each introducing distinct affordances and tensions. Particular attention is given to the statistical and data-based foundations of these systems and to the fact that their surface behavior is explicitly human-like, a combination that places them squarely within the intellectual traditions of educational and behavioral research. From this perspective, educational researchers are unusually well positioned to study, evaluate, and productively use generative AI systems, drawing on established methods for modeling latent processes, managing uncertainty, and interpreting complex human-system interactions. The goal is to equip readers with a conceptual map that supports more informed experimentation, critical interpretation, and responsible use as these systems continue to evolve.

John T. Behrens

University of Notre Dame

Generative AI in EducationHuman-Centered AI
AskPDF
Jun 12, 2026

Thinking Outside the [Chat]Box: Bridging Computer Science and Industrial Design for Cognitive-Inclusive Generative AI

Current Generative AI (GenAI) interfaces remain largely constrained to chatbox interaction, which can impose high cognitive demands on users and create substantial barriers for people with intellectual disabilities (ID), including prompt formulation difficulties, response overload, and limited mechanisms to assess information reliability. To explore alternative interaction models for cognitive accessibility, we conducted a cross-disciplinary co-design challenge in which two student cohorts (Computer Science and Industrial Design) developed interface concepts from the same set of functional requirements (e.g., prompt scaffolding, structured output, GUI-based refinement, transparency, and personalization). Comparing the resulting proposals reveals both convergence on foundational requirements (notably initial calibration, proactive prompting, and direct manipulation of response fragments) and complementary contributions that outline a multi-layered support system. Computer Science teams primarily produced structural scaffolding, emphasizing predictability, navigability, and trust through mechanisms such as reliability indicators, explicit sources, and context management for long conversations. Industrial Design teams emphasized experiential scaffolding, focusing on pacing, attention guidance, multimodality, and proactive agency, including step-by-step response flows, focus modes, and assistant-like integrations. We synthesize these findings into a dual-layer scaffolding framework that expands the design space for cognitively accessible GenAI interaction beyond chat-centric models and motivates future work on expert refinement, technical feasibility, and empirical validation with users with ID.

Virginia Francisco, Daniel Guasch, Raquel Hervás

Facultad de Inform´atica Universidad Complutense de Madrid Madrid, Spain · Universitat Polit`ecnica de Catalunya Villanova i la Geltru, Spain · Instituto de Tecnolog´ıa Del Conocimiento Universidad Complutense de Madrid Madrid, Spain

Human-AI InteractionHuman-Centered AI
AskPDF
Jun 18, 2026

Grounded Inference: Principles for Deterministically Encapsulated Generative Models

The incorporation of generative models into traditional computational systems presents both enormous opportunity and tremendous peril. Although many early adopters have realized these perils at great expense, the field still requires foundational frameworks to de-risk incorporation of AI into traditional systems. This manuscript establishes this foundation through the definition of four specific primitives of AI blended architecture, designed to enable deterministic encapsulation of probabilistic models. It further establishes two overarching anti-patterns broadly represented across industry to serve as warnings for engineers in this field. This framework was designed to enable successful integration of AI into traditional systems while providing a foundation upon which generative model providers could build the next generation of generative model interfaces.

Marty O'Neill

Odenton, MD, USA

Generative ModelingGenerative AI Adoption
AskPDF