cs.AINov 14, 2026

Large Language Models for AI Infrastructure

Authors: Bolin Shen, Shaoyi Huang, Yushun Dong

Organizations: Florida State University, Computer Science, Tallahassee, FL, USA · Stevens Institute of Technology, Computer Science, Hoboken, NJ, USA

Abstract

Large language models (LLMs) are increasingly used to design, optimize, and operate the infrastructure behind AI training and serving, yet the literature remains fragmented across hardware design, machine learning systems, electronic design automation, and high-performance computing. We present a comprehensive survey of LLM-assisted AI infrastructure across its lifecycle, from accelerator design and kernel engineering to compilation, deployment, operations, data management, and energy optimization. We organize the literature by the primary resource the LLM manages: artifact-centric methods for computational artifacts, operational-centric methods for deployment context and system state, and resource-centric methods for data and energy. These categories span the lifecycle and cover AI-specific concerns such as distributed-training configuration, inference serving, GPU-cluster scheduling, and data loading. Building on this taxonomy, we conduct a cross-study quantitative synthesis of reported results. Because studies adopt heterogeneous benchmarks, baselines, hardware, and metrics, we compare methods only where evaluation conditions permit and align results along common dimensions: functional correctness, execution performance, throughput, resource and energy consumption, operational outcomes, and optimization cost, keeping each result's native metric, baseline, and setting rather than collapsing them into a single score. This comparison reveals where LLM-driven methods yield consistent gains, how reported gains in workload orchestration depend on the evaluation environment, from simulators to real clusters and production, and where evidence is still lacking. Together, the taxonomy, lifecycle perspective, and quantitative evaluation provide an evidence-grounded reference for LLM-driven engineering of AI infrastructure.

Figures & tables

Explore similar work

CardsList
  1. ΦΦ-Bench: Can Large Language Models Engineer the Infrastructure That Powers Them?

    Sep 9, 2026Leilei Ding, Shumin Wang, Yuting Huang +10LLM EvaluationLarge Language Model-Guided Optimization

  2. Multi-Level Modeling of Large Language Model Inference Latency and Energy via Hybrid Analytical--Machine-Learning Predictors

    Aug 7, 2026Saeid Shokoufa, Mohammad Erfan Sadeghi, Mehdi Kamal +1LLM InferenceEnergy-Aware LLM Inference

  3. LLM-Guided Runtime Parameter Optimization for Energy-Efficient Model Inference

    Apr 29, 2026Katelyn Crumpacker, Dimitrios NikolopoulosLLM Inference EfficiencyEnergy-Efficient ML