RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models

Ritika Allada; Pinar Yanardag

cs.CVAug 14, 2026

RankT2I: A Submodular Framework for Discovering Interpretable and Diverse Semantics in Text-to-Image Models

Authors: Ritika Allada, Pinar Yanardag

Organizations: Virginia Tech, Blacksburg, VA, USA

Abstract

Recent advances in text-to-image (T2I) models have revolutionized the field of image generation and editing. However, identifying semantics that a T2I model can successfully edit in an image continues to be a challenging task. Most existing approaches require users to manually specify semantics to modify a particular image, a time-consuming process that often involves extensive trial and error. In this paper, we present RankT2I, a novel, training-free, and model-agnostic framework that automates the discovery of editable semantics in diffusion and FLUX-based models. Given a visual domain, we first utilize a multimodal vision-language model to gather a broad set of candidate semantics. We then frame semantic discovery as a set selection problem and use a submodular objective to identify semantics that are relevant, editable, and diverse. Our method helps users efficiently identify a wide range of semantics for text-to-image editing models across several domains while outperforming existing methods.

Figures & tables

Figure 1: Motivating Examples. T2I editing methods like InstructPix2Pix [ 4 ] and FLUX.1 Kontext [ 32 ] can apply some semantics, but fail on others. This underscores the need for an automated approach to uncover the full range of editable semantics in generative models.

Figure 2: An Overview of RankT2I. First, we prompt a multimodal vision-language model (MMVLM) [ 62 ] to obtain a list of potentially editable semantics for a domain. Then, we feed these semantics and a set of unedited images into a T2I model [ 3 , 4 , 32 ] , to get a set of edited images. Lastly, we input the attributes list, unedited images, and edited images into our submodular framework, “RankT2I,” which suggests relevant, editable, and diverse semantics.

Figure 3: Semantic Discovery for Real-Image Editing Using a Closed-Source Model. RankT2I finds both diverse and interpretable semantics in real images in many domains. GPT-Image-1.5 (see Footnote 1 ) was the base model that was used to generate these edits. The original images are from Unsplash (see Footnote 2 ).

Figure 4: Top Discovered Semantics in FLUX-Based Models. RankT2I successfully identifies several semantics in FLUX-based T2I models compared to SliderSpace across various domains, including ancient ruins, planets, and cityscapes.

Figure 5: Top Discovered Semantics in Diffusion-Based Models. Compared to existing diffusion-based semantic discovery methods, like NoiseCLR and SliderSpace, RankT2I successfully identifies a wide range of diverse and interpretable semantics.

Figure 6: Comparisons with Other Attribute Retrieval Methods. RankT2I successfully identifies editable, diverse, and interpretable semantics compared to other attribute retrieval methods (e.g., first-K images, random selection, k-means, etc.).

Figure 7: Generalizability of Semantics. The top semantics discovered by RankT2I can be applied to edit different images (Base Model: FLUX.1 Kontext).

Model	Method	$\downarrow$ I2I	$\uparrow$ TCE	$\uparrow$ TIE	$\uparrow$ CLIP-T	$\uparrow$ TIFA	$\uparrow$ VQA
SD	NoiseCLR	0.8439	13.8819	34.2546	0.2378	0.3402	0.5484
SD	SliderSpace	0.8682	16.5061	32.9884	0.2355	0.3872	0.5236
SD	Ours	0.7800	26.4701	41.1987	0.2595	0.6078	0.8177
FLUX	SliderSpace	0.9210	12.5769	35.2775	0.2542	0.3462	0.6737
FLUX	Ours	0.8405	22.8102	40.7330	0.2840	0.7159	0.9206

Table 1: Quantitative Comparisons with Semantic Discovery Methods. RankT2I identifies more diverse and interpretable attributes in both diffusion- and FLUX-based T2I models compared to existing methods.

Model	Method	$\downarrow$ I2I	$\uparrow$ TCE	$\uparrow$ TIE	$\uparrow$ CLIP-T	$\uparrow$ TIFA	$\uparrow$ VQA
SD	First-K	0.9039	12.5140	26.4074	0.2577	0.2290	0.7012
SD	Random	0.8611	14.5577	32.4777	0.2595	0.3356	0.6972
SD	K-Means	0.8433	19.1433	34.5400	0.2635	0.4156	0.7592
SD	Cos. Sim.	0.9002	13.2874	27.5639	0.2764	0.4846	0.7459
SD	Ours	0.8436	19.6326	35.5105	0.2688	0.5021	0.8048
FLUX	First-K	0.8962	12.6479	30.7635	0.2631	0.4224	0.7028

Table 2: Quantitative Comparisons with Attribute Retrieval Methods. RankT2I discovers more diverse and interpretable semantics than the majority of these attribute retrieval methods.

Figure 8: Hyperparameter Ablation Studies. These charts show how changing the hyperparameters ( $\alpha$ , $\beta$ , and $\lambda$ ) affects the number of categories suggested by RankT2I. Note: (D) and (F) in the line graphs indicate images that were generated using diffusion- and FLUX-based T2I models.

Model	Method	Keyword	Training	Generation	Ranking	Total
SD	NoiseCLR	0	1114m 3s	3m 11s	0	1117m 14s
SD	SliderSpace	0	389m 33s	2m 15s	0	391m 48s
SD	Ours	24s	0	42m 21s	29s	43m 14s
FLUX	SliderSpace	0	287m 36s	3m 1s	0	290m 37s
FLUX	Ours	25s	0	112m 40s	1m 13s	114m 18s

Table 3: Semantic Discovery Time Comparisons. RankT2I discovers 100 semantics for text-to-image editing significantly faster than existing methods.

Figure 9: Editability Score vs. Identity Preservation. As shown above, semantics with editability scores below approximately 0.03 may alter the identity of the subject.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 10: Generalizability of Woman Semantics. This image shows examples of the top 10 woman semantics discovered by our method using FLUX.1 Kontext and how they can be applied to edit different images.

Model	$\uparrow$ CLIP-T	$\uparrow$ TIFA	$\uparrow$ VQA
SD	0.282	0.724	0.897
FLUX	0.276	0.799	0.912

Appendix

Table 4: Generalization of Semantics across Base Models. Higher image–text alignment and visual question answering scores indicate that RankT2I’s semantic edits consistently match the intended prompts across different images, demonstrating the generalizability of the discovered semantics.

Prompt Template

Generate a Python string array containing distinct prompts for editing an image of <SUBJECT>. The prompts should span a wide range of attribute categories to produce diverse variations of the scene. Example Attribute Structure: Broad Category 1: attribute 1, attribute 2, attribute 3, …

Broad Category 2: attribute 1, attribute 2, attribute 3, …

Broad Category 3: attribute 1, attribute 2, attribute 3, …

Broad Category 4: attribute 1, attribute 2, attribute 3, …

etc.

Appendix

Table 5: Prompt Template. The template is used to generate diverse editing prompts spanning multiple semantic attribute categories.

Prompt Example (Ancient Ruins)

Generate a Python string array containing distinct prompts for editing an image of ancient ruins. The prompts should span a wide range of attribute categories (e.g., time of day, weather, ruins condition, ruins design, style, etc.). Example Attribute Categories: Time of day: sunrise, sunset, night, etc.

Weather: bright and sunny, overcast, rainy, etc.

Ruins design: faded murals, domed structures, etc.

Style: sepia-toned, monochrome, etc.

etc.

Appendix

Table 6: Example Prompt Used to Retrieve a List of Potential Attributes. This table shows an example of the prompt used to retrieve a list of potential semantics for the ancient ruins domain.

Potential List of Semantics (Ancient Ruins)

"Make the ruins at sunrise",\ "Make the ruins at sunset",\ "Make the ruins in the early morning",\ "Make the ruins at noon",\ "Make the ruins in the afternoon",\ "Make the ruins in the evening",\ "Make the ruins at dusk",\ "Make the ruins at night",\ "Make the ruins under a full moon",\ "Make the ruins bright and sunny",\ "Make the ruins overcast",\ "Make the ruins rainy",\ "Make the ruins misty",\ "Make the ruins dust-filled",\ "Make the ruins with broken columns",\ "Make the ruins with fallen pillars",\ "Make the ruins with cracked stone walls",\ "Make the ruins with collapsed arches",\ "Make the ruins with scattered stone rubble",\ "Make the ruins partially buried in sand",\ "Make the ruins submerged in shallow water",\ "Make the ruins heavily eroded",\ "Make the ruins structurally intact",\ "Make the ruins crumbling",\ "Make the ruins with ancient runes",\ "Make the ruins with carved stone patterns",\ "Make the ruins with faded murals",\ "Make the ruins with engraved symbols",\ "Make the ruins with monumental statues",\ \ldots\ ]

Appendix

Table 7: Example List of Potential Semantics. Examples of prompts used to edit images of ancient ruins , spanning diverse attribute categories including time of day, weather, structural condition, and decorative elements, are shown above.

Diffusion-Based Image Editing

Diffusion-based image editing changes image content, style, or attributes while aiming to preserve the parts users want to keep.

See all 225 papers

Semantic Browsing: Controllable Diversity for Image Generation
Jun 22, 2026Sara Dorfman, Maya Vishnevsky, Omer Dahary +2T2I Generation Controllable Image Generation
Tel Aviv University
DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models
May 7, 2026Juntong Wang, Jiarui Wang, Huiyu Duan +3Learning to Rank Benchmark Design
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, Shanghai, China
Text-to-Image Models Need Less from Text Encoders Than You Think
Jun 2, 2026Nurit Spingarn, Noa Cohen, Tamar Rott Shaham +1Text Embeddings Conditional Image Generation
Technion – Israel Institute of Technology · MIT CSAIL

Jun 22, 2026J/K move · Enter open · S save

Semantic Browsing: Controllable Diversity for Image Generation

Sara Dorfman, Maya Vishnevsky, Omer Dahary, Or Patashnik+1

Tel Aviv University

T2I Generation Controllable Image Generation

Open paper AskPDF arXiv

Modern text-to-image models excel in visual fidelity and prompt adherence. However, this strict adherence comes at the cost of diversity: generated samples tend to collapse into a single visual interpretation. Existing methods to improve diversity produce outputs driven by incidental variations rather than meaningful design choices. This motivates a new variant of the diversity task where structure is enforced on the generated samples. We introduce a method for controlled diversity that enables Semantic Browsing, where users can navigate structured image galleries and experience creative exploration through a systematic traversal of meaningful, interpretable axes of variation. Achieving this level of semantic control requires a deep understanding of the scene. We exploit the fact that recent text-to-image models are trained on elaborated captions, effectively decoupling semantic decision-making from pixel generation. This enables a paradigm shift: instead of relying on stochastic variation within the text-to-image model, we induce diversity directly at the text level. By leveraging rich textual representations, we allow a Vision Language Model (VLM) to operate on the full scene context. To overcome the generic outputs typical of standard VLMs, we employ an agentic workflow that explicitly enforces structured variation attuned to the original prompt. We demonstrate that our method produces diverse and navigable design spaces where every variation corresponds to a specific, user-understandable semantic decision.

Abstract

Diffusion-Based Image Editing

Explore similar work

Semantic Browsing: Controllable Diversity for Image Generation

DynT2I-Eval: A Dynamic Evaluation Framework for Text-to-Image Models

Text-to-Image Models Need Less from Text Encoders Than You Think