Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Services

Tao Zhang; Bin Liao; Tao Zhou; Yanping Liu

cs.DCSep 20, 2026

Co-occurrence Patterns of LoRA Adapters in Production Diffusion Model Inference Services

Authors: Tao Zhang, Bin Liao, Tao Zhou, Yanping Liu

Organizations: School of Information Engineering, Guizhou University of Traditional Chinese Medicine, Guiyang, 550025, China · School of Big Data and Statistics, Guizhou University of Finance and Economics, Guiyang, 550025, China · School of Mathematics and Statistics, Guizhou University of Finance and Economics, Guiyang, 550025, China · School of Mathematics and Statistics, Beijing Technology and Business University, Beijing, 100048, China

Abstract

Low-rank adaptation (LoRA) has become a key technology for serving large-scale personalized large language models and diffusion models in the cloud. However, the co-occurrence patterns, resource contention relationships, and evolutionary regularities of adapters under production inference workloads have not been systematically or quantitatively studied. Based on GenTD26, Alibaba's production diffusion model inference dataset, this paper adopts a graph-theoretic framework to construct an adapter co-occurrence network and conducts a characterization from both static structure and dynamic evolution. Our main findings are as follows. (1) The co-occurrence network is extremely sparse, and adapter usage frequency follows a significant heavy-tailed distribution. (2) Introducing the first adapter incurs a 66.1% execution-latency overhead, with diminishing marginal costs afterwards. (3) Co-occurrence relationships are driven by base models: in 90.6% of multi-adapter requests, all adapters share the same dominant base model; 66.2% of significant co-occurrence edges connect same-model adapter pairs; and in 85.8% of multi-adapter requests, all adapter pairs form significant co-occurrence edges. (4) The adapter ecosystem exhibits a core-periphery bipolar structure, with a weekly Jaccard similarity of 0.696 at the model level and a churn rate of 54.5% for the top-10 hottest models within a 12-hour window. Based on these findings, we propose a preloading strategy built on top-k co-occurrence statistics; offline experiments show that it covers 81.0% of test-set co-occurrence pairs at k=3, and sensitivity analyses across frequency thresholds and time windows verify the robustness of the conclusions. These results provide a data-driven basis for cache preloading, adaptive scheduling, and GPU memory management in LoRA inference services.

Figures & tables

Attribute	Value
Provider	Alibaba commercial image generation
Duration	November 15 – December 8, 2024 (24 days)
Number of requests	26,823 ( lora_request_trace )
Number of processing records	68,195 ( data_trace_processed )
GPU sampling points	157,417 (duty cycle)
Containers	143

Table 1: Summary of the GenTD26 dataset

Figure 1: LoRA adapter frequency distribution. (a) Top-30 adapters by occurrence, annotated with the Gini coefficient (0.749) and the top-5 share (19.7%), with colors distinguishing frequency tiers (dark blue: ranks 1–5; red: 6–10; light blue: 11–30); (b) log–log frequency–rank plot with the rank–frequency regression fit line ( $\alpha=1.30$ , a descriptive measure; strict power-law fitting is given in the main text).

Number of LoRAs	Number of requests	Mean (s)	Standard deviation (s)	Degradation
0	52,993	23.6	13.4	(baseline)
1	14,112	39.2	25.1	+66.1%
2	754	40.4	21.2	+71.2%
3	277	44.7	16.0	+89.4%
4	43	40.8	12.6	+72.9%
5	10	38.8	11.1	+64.4%

Table 2: Execution time vs. number of LoRA adapters

Figure 2: LoRA count distribution and latency impact. (a) Distribution of the LoRA count per request (77.7% none, 20.7% single, 1.6% multiple); (b) mean execution time with percentage degradation over the no-LoRA baseline (23.6 s).

Figure 3: LoRA loading latency. (a) Histogram of 27,415 loading operations (mean = 4.40 s, median = 4.38 s, $P_{90}=6.14$ s); (b) CDF of loading latency versus execution-time baselines (no-LoRA 23.6 s, single-LoRA 39.2 s); a typical loading operation accounts for only 18.6% of the no-LoRA execution time.

Figure 4: Evolution of the co-occurrence network with the frequency threshold $\tau_{\min}$ . (a)–(e) Network topology at $\tau_{\min}=1,5,10,20,50$ : node size is proportional to adapter frequency, color represents degree centrality (with a unified mapping across panels), gray dots are isolated adapters that only satisfy the frequency threshold, and the red box marks the default threshold 10; (f) network-scale metrics (number of nodes, number of edges, number of connected nodes, number of non-trivial connected components) versus $\tau_{\min}$ on a log scale. The co-occurrence edge threshold is fixed at $w_{\min}=3$ .

Figure 5: Connected core of the co-occurrence network (degree $>0$ ; 51 nodes, 68 edges, edge weight $\geq 5$ ). Node size is proportional to adapter frequency, and color denotes the dominant base model of each adapter (gray = other models). The two largest connected components contain 15 and 8 adapters, respectively.

Figure 6: Bipartite graph of base models and LoRA adapters (top 8 models with their top 10 adapters each; 70 edges). Squares = base models (size proportional to the number of processing-trace records; together accounting for 80.2% of records); circles = adapters (size proportional to frequency; color = dominant base model; black border = cross-model adapter).

$\tau_{\min}$	$\|V\|$	$\|E\|$	Density (%)	Communities	Non-trivial comps.	Same-model over-expr.
5	463	83	0.08	412	13	2.7 $\times$
10	274	68	0.18	233	10	2.6 $\times$
20	154	49	0.42	125	8	2.8 $\times$
50	63	23	1.18	49	5	3.5 $\times$

Table 3: Sensitivity of network characteristics to the frequency threshold

\tau_{\min}

(

w_{\min}=3

)

$\tau_{\min}$	$\|V\|$	$\|E\|$	Density (%)	Communities	Non-trivial comps.	Largest comp.
5	463	83	0.08	412	13	16
10	274	68	0.18	233	10	15
15	195	52	0.27	163	9	13
20	154	49	0.42	125	8	13
30	108	31	0.54	90	5	12
50	63	23	1.18	49	5	11

Table 4: Complete sensitivity analysis across the frequency threshold

\tau_{\min}

(

w_{\min}=3

)

Figure 7: Temporal evolution. (a) Daily LoRA request share (request trace, 24 days; mean 14.7%; no LoRA requests on November 16–17); (b) daily active base model count (mean 34.2; adjacent-day model Jaccard mean 0.497, weekly model Jaccard mean 0.696); (c) distribution of adapter cross-base-model spread ( $\leq 2$ models: 786; 3–17: 83; $\geq 18$ : 5).

Metric	SoCC’25	This paper
Unique adapters	705	874
Proportion of requests carrying LoRA adapters	21.2%	22.3%
Multi-adapter requests	—	1.6%
Rank–frequency exponent $\alpha$	—	1.30
Adapter frequency Gini coefficient	—	0.749
Latency degradation (1 adapter)	69%	66.1%

Table 5: Comparison of LoRA characterization findings

Diffusion Model Serving

Diffusion model serving optimizes the scheduling, execution, and resource use of diffusion models to meet latency and throughput goals.

See all 19 papers

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective
May 7, 2026Suoxin Zhang, Run He, Di Fang +3LLM Fine-Tuning Adapter Tuning
South China University of Technology, China · Zhejiang University, China
PLoRA: Efficient Concurrent LoRA Training for Large Language Models
Aug 4, 2025Minghao Yan, Zhuang Wang, Zhen Jia +2LLM Fine-Tuning Efficient Language Model Training
University of Wisconsin-Madison · Amazon Web Services
When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning
Jun 26, 2026Tanguy Dieudonné, Giulia Lanzillotta, Enis Simsar +2Continual Learning Low-Rank Adaptation
Department of Computer Science, ETH Zurich, Zurich, Switzerland.

May 7, 2026J/K move · Enter open · S save

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective

Suoxin Zhang, Run He, Di Fang, Xiang Tan+2

South China University of Technology, China · Zhejiang University, China

LLM Fine-Tuning Adapter Tuning Low-Rank Adaptation Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

Low-rank adaptation (LoRA) is a widely used parameter-efficient fine-tuning method that places trainable low-rank adapters into frozen pre-trained models. Recent studies show that using fewer LoRA adapters may still maintain or even improve performance, but existing methods still distribute adapters broadly, leaving where to place a limited number of adapters to maximize performance largely open. To investigate this, we introduce PAGE (Projected Adapter Gradient Energy), a gradient-based sensitivity probe that estimates the initial trainable gradient energy available to each candidate LoRA adapter. Surprisingly, we find that PAGE is highly concentrated on a single shallow FFN down-projection across two model families and four downstream tasks. We term this module the dominant adaptation module and show that its layer index is architecture-dependent but task-stable. Motivated by this finding, we propose DomLoRA, a placement method that places a single adapter at the dominant adaptation module. With only 0.7% of vanilla LoRA's trainable parameters, DomLoRA outperforms it on average across downstream tasks, including instruction following, mathematical reasoning, coding, and multi-turn conversation. This method also matches or improves other LoRA variants and reduces training time by up to 2.74

\times

compared with broad placement, supporting the dominant adaptation module perspective as a practical placement guideline.

Abstract

Diffusion Model Serving

Explore similar work

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective

PLoRA: Efficient Concurrent LoRA Training for Large Language Models

When One Adapter Speaks for Many: Discovering Low-Rank Redundancy in Continual Fine-Tuning