Beyond Local Code Optimization: Multi-Agent Reasoning for Software System Optimization
Organizations: Purdue University · Amazon Robotics · Loyola University Chicago
Abstract
Large language models and AI agents have recently shown promise in automating software performance optimization, but existing approaches predominantly rely on local, syntax-driven code transformations. This limits their ability to reason about program behavior and capture whole system performance interactions. As modern software increasingly comprises interacting components - such as microservices, databases, and shared infrastructure - effective code optimization requires reasoning about program structure and system architecture beyond individual functions or files. This paper explores the feasibility of whole system optimization for microservices. We introduce a multi-agent framework that integrates control-flow and data-flow representations with architectural and cross-component dependency signals to support system-level performance reasoning. The proposed system is decomposed into coordinated agent roles - summarization, analysis, optimization, and verification - that collaboratively identify cross-cutting bottlenecks and construct multi-step optimization strategies spanning the software stack. We present a proof-of-concept on a microservice-based system that illustrates the effectiveness of our proposed framework, achieving a 36.58% improvement in throughput and a 27.81% reduction in average response time.
Figures & tables
| Captured Information | Description |
| Component Summary Agent | |
| Component inventory | Identifies Functions, classes, packages, and services from code structure. |
| Hierarchical composition | Derives Parent-child relationships (service package class method). |
| Exported interfaces | HTTP endpoints, and API entry points. |
| Static dependencies | Extracts Call-based, type-based, and resource-based component dependencies. |
| Evidence mapping | Associate with precise source file locations |
| Static Evidence | Interpretation | Suggestion |
| O1: HTTP Client | ||
| Jersey Client created per RESTClient constructor. Expensive initialization on service call paths. | Duplicate connection pools across microservices increase resource overhead, degrading throughput and increasing tail latency on service-to-service calls. | Singleton pattern using SHARED_CLIENT _HTTP/S for thread-safe reuse with connection pooling benefits. |
| O2: Volatile Flag | ||
| synchronized on setMaintenance -ModeInternal() (CodeQL analysis). | Unnecessary lock contention on request-critical paths introduces thread blocking, causing latency spikes under concurrent access. | Replace synchronized method with volatile boolean flag for lock-free access while ensuring memory visibility. |
| O3: ObjMapper | ||
| new ObjectMapper() per request in session methods of AbstractUI -Servlet class. | Per-request allocation in the base servlet propagates across all UI request paths, increasing GC pressure and CPU serialization overhead. | Shared static final instance that remains thread-safe after configuration to eliminate allocations. |
Software Engineering
Software engineering studies how to design, build, test, deploy, and maintain reliable software systems and the processes behind them.
See all 270 papers