cs.CV · 2609.16572 Copy arXiv ID · Sep 15, 2026 Save Efficient Text-to-Image Generation: An Adaptive Step Schedule Controller for Diffusion Models Authors: Kuluhan Binici , Cihan Acar , Shivam Aggarwal , Siying Liu , Tulika Mitra
Organizations: 1SAP · 2National University of Singapore · Institute for Infocomm Research (I2R), A*STAR, Singapore
Abstract Text-to-image diffusion models often use a fixed number of denoising steps, balancing time costs and image quality. However, the optimal number of steps depends on the complexity of the input text prompt. We propose an adaptive diffusion controller that dynamically adjusts the number of steps to generate high-quality images efficiently, without additional model training. By leveraging a mixture of step schedules with varying step sizes and evaluating the error term discrepancy at each timestep, our method transitions between schedules to optimize performance. Experiments on COCO and DiffusionDB show that our approach reduces inference time while maintaining visual fidelity, offering a more efficient alternative for text-to-image diffusion models.
Explore similar work Aug 10, 2026 · Yushun Tang, Weiming Chen, Siyi Liu +3 Text-To-Image Diffusion Models Text-To-Image
Sep 29, 2025 · Wenkun He, Yuchao Gu, Junyu Chen +11 Text-To-Image Diffusion Models Image Generation
Aug 10, 2026 · cs.CV J/K move · Enter open · S save
Yushun Tang, Weiming Chen, Siyi Liu, Yi Zhang +2
Text-to-image diffusion models have achieved remarkable success in generating high-quality images from a given text prompt. Subject-driven generation aims to synthesize customized images to mimic the appearance of subjects in given reference images within different visual contexts specified by the text prompts. The central challenge here is that, when the reference image changes, the diffusion model cannot efficiently adapt to different visual contexts while consistently maintaining the subject identity. Existing methods either train the model with a large domain-specific dataset or fine-tune the model using the reference image for hundreds of iterations before actual image generation. In this work, we explore a new approach, called \textit{In-Loop Model Adaptation} (IMA), which adapts the core diffusion model at each generation step during the actual process of image generation, without being trained on the reference image before the generation process. To this end, we establish a DDIM inversion chain that maps the reference image to a sequence of latent, as well as a text-to-image generation chain which generates the image from the text prompt only. We then introduce a masked latent consistency loss and a noise regularization loss to characterize the latent-noise difference between the diffusion model and these two chains at each generation step. This coupled latent-noise loss is used to guide the in-loop model adaptation to preserve the subject identity specified by the reference image while maintaining accurate alignment with the text prompt, resulting in high-fidelity text-to-image generation. Our extensive experiments demonstrate that our proposed IMA method significantly improves the performance of subject-driven text-to-image generation.