Model serving is one of the largest cost drivers in production recommender systems. Maximizing its throughput requires navigating a deeply layered hierarchy: GPU kernels, the ML framework computation graph, the model server, and on-demand feature processing -- each demanding specialized domain expertise. Such cross-layer expertise is inherently difficult to acquire, and does not scale with a workload that continuously grows and evolves, leaving significant cost efficiency gains unrealized. While recent AI agents have demonstrated human expert level efficiency in standalone GPU kernel optimization, automated tuning and optimization for the rest of the serving stack remain largely unexplored. We present FlashVector, an agentic system that optimizes performance across all layers of the model serving stack. The key contribution is an extensible framework to generalize the single kernel optimization agent paradigm to heterogeneous technical stacks, and to deliver performance improvements holistically. After deployment in Unity's Vector advertising platform, FlashVector achieved up to 2x throughput increase and up to 1.98x latency speedup on model server, and up to 1.6x throughput increase on feature store. These optimizations were discovered not only at the GPU kernel and computation graph levels, but also across the other components of the model serving stack, such as the model server (NVIDIA Triton's C++ codebase) and the on-demand feature transformation service (Python codebase), demonstrating the extensibility of the framework to more complex system architectures.
Figures & tables
Figure 1. A typical model serving stack.
Figure 2. The three-stage workflow of FlashVector.
Figure 3. The layer agent abstraction. Every layer implements the same four stages — Profile, Diagnose, Optimize, Verify — plus the post-run Refine that writes findings back to the knowledge base; the loop itself is identical across layers. Cell entries name the concrete instance each production layer implements today.
Model
Latency Speedup
Throughput Increase
Retrieval model
1.67 ×
1.10 ×
Ranking model 1
1.36 ×
1.80 ×
Ranking model 2
1.98 ×
2.00 ×
Ranking model 3
1.30 ×
1.33 ×
Ranking model 4
1.30 ×
1.35 ×
Gamer model
1.34 ×
1.15 ×
Table 1. Model Serving Stack optimizations delivered by FlashVector on production models.
Figure 4. Input Serialization bottleneck found between Triton Server and its Python backend
Figure 5. Fused Attention Kernel
Figure 6. Optimized Embedding Layer
Python Functions Replaced
Kernal speedup
pd.to_datetime, once per request
350 ×
one numpy assignment per feature
25.1 ×
bytes → unicode per input
12.4 ×
6 universal functions dispatches × 180 sites
∼ 8 ×
per-row dict for one membership query
4.4–5.8 ×
Table 2. Native Cython kernels and the operations they replace, with speedup at the call site
Stage
RPS / pod
vs. prev.
vs. base
Baseline
940
—
—
+ native kernels, assembly
∼ 1.2k
+28%
1.3×
+ zero-copy request I/O
∼ 1.5k
+25%
1.6×
Table 3. Throughput per preprocessor pod as each round of optimizations was tested in production environment. Each row is measured on top of the one above it.
inst. / batch
chosen trial
Model
SLO
base
tuned
RPS
p99
Model A
250
7 / 64
6 / 6
1321
192
Model B
250
10 / 64
7 / 3
736
271
Model C
300
8 / 2
4 / 2
238
251
Model D
400
12 / 3
3 / 1
139
285
Table 4. Configurations selected by the agent from the sweep results. SLO and p99 in ms; RPS is the throughput the chosen trial sustained at the SLO.