LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows

Lingyun Yang; Suyi Li; Tianyu Feng; Xiaoxiao Jiang; Zhipeng Di; Weiyi Lu; Kan Liu; Yinghao Yu

cs.DCApr 9, 2026

LegoDiffusion: Micro-Serving Text-to-Image Diffusion Workflows

Authors: Lingyun Yang, Suyi Li, Tianyu Feng, Xiaoxiao Jiang, Zhipeng Di, Weiyi Lu, Kan Liu, Yinghao Yu,

Organizations: Hong Kong University of Science and Technology · Alibaba Group

Abstract

Text-to-image generation executes a diffusion workflow comprising multiple models centered on a base diffusion model. Existing serving systems treat each workflow as an opaque monolith, provisioning, placing, and scaling all constituent models together, which obscures internal dataflow, prevents model sharing, and enforces coarse-grained resource management. In this paper, we make a case for micro-serving diffusion workflows with LegoDiffusion, a system that decomposes a workflow into loosely coupled model-execution nodes that can be independently managed and scheduled. By explicitly managing individual model inference, LegoDiffusion unlocks cluster-scale optimizations, including per-model scaling, model sharing, and adaptive model parallelism. Collectively, LegoDiffusion outperforms existing diffusion workflow serving systems, sustaining up to 3x higher request rates and tolerating up to 8x higher burst traffic.

Figures & tables

Figure 1. Top : a basic diffusion workflow using Flux-Dev ( Labs, 2024 ) . Middle : add ControlNets ( Zhang et al., 2023b ) alongside the base diffusion model, which process an additional input reference image and pass the intermediaries to diffusion model to control the composition in image generation. Bottom : Further add LoRA ( Hu et al., 2022 ) to change image styles by patching LoRA weights onto diffusion model weights.

Figure 2. Latent parallelism and ControlNets parallelism.

Figure 3. Left : Loading time of workflow scaling and base diffusion model (DM) scaling. Right : Latency-throughput tradeoff of models in a SD3 workflow. Both use H800 GPUs.

Figure 4. Left: Model sharing reduces request latency. Right: Adaptive parallelization reduces request latency.

Figure 5. An overview of LegoDiffusion .

Class	API	Description
Model	init()	Create a model instance
	call()	Express a model invocation in the frontend
	setup_io()	Define model inputs and outputs
	load()	Load the model in the backend
	execute()	Execute model inference in the backend
	add_patch()	Attach a patchable adapter to a model

Table 1. Primitives in LegoDiffusion ’s Python library for defining models and composing diffusion workflows.

Figure 6. A simplified case of integrating Flux with LegoDiffusion .

Figure 7. A simplified diffusion workflow using Flux ( Labs, 2024 ) .

Figure 8. Illustrating data fetch. For simplicity, we primarily illustrate tensor fetch process in data store. C.N. : ControlNet.

Setting	Diffusion Model	Workflow
Single-model Deployments (3 workflows each)
S1	SD3 ( Esser et al., 2024 )	(Basic, +C.N. 1, +C.N. 2)
S2	SD3.5-Large ( stabilityai, 2025 )	(Basic, +C.N. 1, +C.N. 2)
S3	Flux-Schnell ( Labs, 2024 )	(Basic, +C.N. 1, +C.N. 2)
S4	Flux-Dev ( Labs, 2024 )	(Basic, +C.N. 1, +C.N. 2)
Mixed-model Deployments (6 workflows each)

Table 2. Evaluation Settings . We use workflows of representative diffusion models. S1–S4 represent single-model deployments, each including three workflow variants: a Basic workflow (text encoders, diffusion model, and decoder), plus two using adapters ( Basic + C.N. 1 and Basic + C.N. 2 ). S5–S6 represent mixed-model deployments. C.N. : ControlNet.

Figure 9. End-to-end performance across six settings (S1–S6 in Table 2 ), evaluated under varying request traffic rates (a–f, j), SLO requirements (g), traffic burstiness (h), and testbed sizes (i).

Figure 10. Left: Normalized latency of LegoDiffusion across different numbers of available GPUs with intra-/inter-node parallelism (§ 5 ). Flux-S: Flux-Schnell. Right: Effectiveness of admission control (A.C.) in settings S1-4. RS : Rate Scale.

Figure 11. Left: Data fetching latency of varying sizes of tensor blocks. Right: The distribution of tensor block sizes found in typical SD3 and Flux workflows.

Technique	LOC / (Support adaptive adjustment?)
Technique	Katz ( Li et al., 2025b )	xDiT ( Fang et al., 2024 )	LegoDiffusion
Latent parallel	92 (No)	68 (No)	74 (Yes)
ControlNet parallel	127 (No)	N.A.	79 (Yes)
Async LoRA loading	182 (Yes)	N.A.	61 (Yes)

Table 3. Effective LOC and adaptive-runtime support.

Diffusion Model Serving

Diffusion model serving optimizes the scheduling, execution, and resource use of diffusion models to meet latency and throughput goals.

See all 18 papers

ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform
Jul 29, 2026Xiaoxiao Jiang, Suyi Li, Sheng Yao +5Efficient Neural Network Inference T2I Generation
Hong Kong University of Science and Technology · Alibaba Group
PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference
Mar 24, 2026Qirui Wang, Qi Guo, Yiding Sun +4Diffusion Model Serving T2I Diffusion Models
School of Software Engineering, Xi’an Jiaotong University, China · Nankai University, China
BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving
Jul 9, 2026Yuanjie Zhu, Liangwei Yang, Ke Xu +4LLM Inference Scheduling Diffusion Model Serving
University of Illinois Chicago, USA

Jul 29, 2026J/K move · Enter open · S save

ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform

Xiaoxiao Jiang, Suyi Li, Sheng Yao, Tianyu Feng+4

Hong Kong University of Science and Technology · Alibaba Group

Efficient Neural Network Inference T2I Generation

Open paper AskPDF arXiv

Text-to-image (T2I) workflows are increasingly deployed on serverless platforms because users often compose customized workflows and invoke them intermittently. Existing platforms typically deploy each workflow as an opaque GPU function, provisioning, placing, and scaling all constituent models in the workflow together. This monolithic design obscures workflow structure, inflates scaling overhead, forces users to manage low-level GPU coordination, and limits fine-grained fairness in multi-tenant clusters. In this paper, we present ServerlessT2I, a serverless-native system that decomposes a T2I workflow into loosely coupled model functions that can be independently managed and scheduled. By explicitly managing individual model execution, ServerlessT2I enables per-model scaling, declarative workflow composition, transparent GPU-resident communication, and fairness-aware scheduling. To make this decomposition efficient, ServerlessT2I harvests slack GPU memory left idle by compute-bound T2I inference to build a data plane that reduces model loading and data communication overheads. \sys{} further introduces a fair scheduler for multi-tenant serving. Using production traces, ServerlessT2I sustains up to 2

\times

higher request rates than existing T2I workflow serving systems with the same GPU budget; for a fixed request rate, it saves up to 3

\times

GPU resources while satisfying service level objectives (SLOs).

Abstract

Diffusion Model Serving

Explore similar work

ServerlessT2I: Efficient Text-to-Image Workflow Serving on a Serverless Platform

PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference

BlockServe: Block-Grained Continuous Batching for High-Throughput Diffusion LLM Serving