MLOps

Follow this topic Ask about this topic

Latest papers

11 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?
Oct 27, 2026Yangshen Deng, Jiaqi Ruan, Yeqi Huang +3AI Coding Agents Coding Agent Benchmarks
University of Edinburgh · Tencent Hunyuan
AgentDP: Towards Automated Model Deployment for Heterogeneous Embedded AI Applications
Oct 22, 2026Leming Shen, Qiang Yang, Xinyu Huang +2MLOps On-Device Inference
The Hong Kong Polytechnic University · University College London · U2University of Cambridge
Trustworthy Data- and ML-Ops for Intelligent Transportation Systems and Logistics
Oct 1, 2026Antonio Emanuele Cinà, Giovanni Scodeller, Cecilia Caterina Pasquale +5Intelligent Transportation Systems Trust in AI
Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps
Aug 30, 2026Sagar Srinivas Sakhinana, Venkataramana RunkanaMLOps Software Engineering Agents
Tata Research Development and Design Centre
SMOCS: A Streaming Framework for Simplified Deployment, Monitoring, and Optimization of ML Systems in Production
Jul 2, 2026Armen Kasparian, Kishansingh Rajput, Malachi Schram +1MLOps
Trustworthy Self-Composable Big-Data-as-a-Service: An LLM-Orchestrated Multi-Agent Framework for Automated Data Engineering, AutoML, MLOps Deployment, and Drift-Aware Lifecycle Optimization
Jun 16, 2026Aueaphum Aueawatthanaphisut, Badri Raj LamichhaneMulti-Agent LLM Systems Automated Machine Learning
School of Information, Computer, and Communication Technology Sirindhorn International Institute of Technology, Thammasat University Pathum Thani, Thailand
Architecturally Significant MLOps Guidelines for ML Model Integration and Deployment: a Gray Literature Review
Jun 3, 2026Faezeh Amou Najafabad, Markus Haug, Keerthiga Rajenthiram +2MLOps
Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · Technical University of Munich, Munich, Germany
Robust and Reliable AI for Predictive Quality in Semiconductor Materials Manufacturing with MLOps and Uncertainty Quantification
May 8, 2026Min Gao, Julia Maria Perathoner, Anton Ludwig Bonin +2Concept Drift Semiconductor Manufacturing
Merck Group, Versum Materials US LLC, Wilmington, USA · Merck Group, Merck Electronics KGaA, Darmstadt, Germany · Technical University of Darmstadt, Darmstadt, Germany +1
FlightSense: An End-to-End MLOps Platform for Real-Time Flight Delay Prediction via Rotation-Chain Propagation Features and Agentic Conversational AI
May 8, 2026Aditi J. Shelke, Renuka J. Shelke, Yash M. Kamerkar +1Spatiotemporal Forecasting MLOps
Stevens Institute of Technology Hoboken, NJ, USA · Axtria Inc. Berkeley Heights, NJ, USA · Meta Menlo Park, CA, USA
Foundations and Architectures of Artificial Intelligence for Motor Insurance
Mar 19, 2026Teerapong PanboonyuenMLOps Financial Services
AI4EOSC: a Federated Cloud Platform for Artificial Intelligence in Scientific Research
Dec 18, 2025Ignacio Heredia, Álvaro López García, Fernando Aguilar Gómez +28AI-Assisted Scientific Research Scientific Workflow Automation
Instituto de Física de Cantabria (IFCA), CSIC-UC, Avda. los Castros s/n, Santander, 39006, Cantabria, Spain · Instituto de Instrumentación para Imagen Molecular (I3M), Centro Mixto CSIC - Universitat Politècnica de València (UPV), Camino de Vera s/n, Valencia, 46022, Valencia, Spain · Karlsruher Institut für Technologie, Kaiserstraße 12, Karlsruhe, 76131, Germany +7

Oct 27, 2026J/K move · Enter open · S save

InfraDev: Can Coding Agents Develop AI Infrastructure, and How Can We Tell?

Yangshen Deng, Jiaqi Ruan, Yeqi Huang, Nick Lau+2

University of Edinburgh · Tencent Hunyuan

AI Coding Agents Coding Agent Benchmarks Software Engineering Agents MLOps

Open paper AskPDF arXiv

Coding agents are beginning to develop the production-grade AI infrastructure that trains and serves large models, yet existing benchmarks cover only single kernels or single-GPU tasks, and grade with tests and prompts that misjudge both correct and incorrect patches. We introduce InfraDev, a benchmark of 36 tasks built from real-world pull requests to Megatron-LM, vLLM, and verl, covering infrastructure development for training, inference, and reinforcement learning. Each task asks an agent to reimplement a real change that spans many files and must run correctly across many GPUs. To grade such changes, we jointly synthesize hidden probes, which enforce the requirements the original tests leave unchecked, and a prompt that states every interface the probes call without revealing the solution. We also design a shared execution environment that reuses dependencies and artifacts across tasks, cutting storage by 52-57% and build time by 13-14% on Megatron-LM and vLLM. Across six frontier models, the best resolves only 38.9% of the tasks; the probes reject 32% of the submissions that pass every official test, and with the original pull request descriptions as prompts, no model resolves a single task. Failure analysis shows that structural and state errors dominate and that agents' own tests rarely check memory, time, or traffic. Turning these findings into an agent skill raises the resolved rate of a frontier model on the Megatron-LM tasks by 10.5%.