DreamReader: An Interpretability Toolkit for Text-to-Image Models
Authors: Nirmalendu Prakash, Narmeen Oozeer, Michael Lan, Luka Samkharadze, Phillip Howard, Roy Ka-Wei Lee, Dhruv Nathawani, Shivam Raval, Amirali Abdullah
Organizations: Singapore University of Technology and Design · Martian · Thoughtworks · NVIDIA · Harvard University
Despite the rapid adoption of text-to-image (T2I) diffusion models, causal and representation-level analysis remains fragmented and largely limited to isolated probing techniques. To address this gap, we introduce DreamReader: a unified framework that formalizes diffusion interpretability as composable representation operators spanning activation extraction, causal patching, structured ablations, and activation steering across modules and timesteps. DreamReader provides a model-agnostic abstraction layer enabling systematic analysis and intervention across diffusion architectures. Beyond consolidating existing methods, DreamReader introduces three novel intervention primitives for diffusion models: (1) representation fine-tuning (LoReFT) for subspace-constrained internal adaptation; (2) classifier-guided gradient steering using MLP probes trained on activations; and (3) component-level cross-model mapping for systematic study of transferability of representations across modalities. These mechanisms allows us to do lightweight white-box interventions on T2I models by drawing inspiration from interpretability techniques on LLMs. We demonstrate DreamReader through controlled experiments that (i) perform activation stitching between two models, and (ii) apply LoReFT to steer multiple activation units, reliably injecting a target concept into the generated images. Experiments are specified declaratively and executed in controlled batched pipelines to enable reproducible large-scale analysis. Across multiple case studies, we show that techniques adapted from language model interpretability yield promising and controllable interventions in diffusion models. DreamReader is released as an open source toolkit for advancing research on T2I interpretability.
Figures & tables
Figure 1: DreamReader provides a unified abstraction of the diffusion stack, exposing four core interpretability operators—Localization, Steering, Stitching, and Sparse Autoencoders (SAEs)—that can be composed within a shared interface. Shown above are the 4 steps in the analysis (shown here is an example for ‘Steering’).
Workflow
Unique configuration / interventions
Workflow-specific metrics & outputs
Localization
Intervention type (scaling/zeroing/corruption), strength, and timestep schedule.
Effect ranking over sites/timesteps; influence reports (e.g. which hook sites are more causal).
Dead-feature rate, sparsity stats; feature cards with top activations.
Table 1: DreamReader workflows. Shared inputs/metrics/outputs are listed in the common block; each row reports only workflow-specific configuration knobs and deliverables. Technical terms are defined in Table 4 .
Figure 2: LoREFT steering results on sample prompts for SDXL-Turbo. We steer the model to add spectacles by training a LoREFT module on cross-attention activations from different U-Net regions (down, mid, and up blocks). Steering effectiveness varies by prompt: for Simba , down-block steering succeeds, whereas for Jack Sparrow , up-block steering performs best. CLIP score measures prompt–image alignment; FID and LPIPS quantify the deviation from the baseline (unsteered) output. Results are logged and visualized with W&B.
Figure 3: Steering. We use CAA to extract a steering direction from a fine-tuned SD 1.5 model using a minimal contrastive setup with two prompts (e.g., “a photo of a Black man” vs. “a photo of a man” ). We map this direction to the base model with our learned mapper and apply it during generation. The figure compares baseline samples (left) with the corresponding steered samples (right) for two example prompts.
Capability
Dream Reader (Ours)
TransformerLens
nnsight
Primary focus
Diffusion / T2I
Autoregressive LLMs
LLMs (+ diffusion)
Abstraction level
End-to-end workflows
Mechanistic primitives
Tracing/editing primitives
Tracing & editing
(steer, ablate, stitch)
Patch-only
Patch + causal
SAE support
Built-in workflow
External add-ons
External add-ons
Experiment infrastructure
YAML pipelines + standardized outputs
Python scripts
Python scripts
Visualization & reporting
Integrated tables
External tooling
External tooling
Table 2: Comparison of Dream Reader with existing interpretability libraries. “Tracing & editing” encompasses activation extraction, intervention (e.g., patching, steering, ablations), and model stitching. “Experiment infrastructure” refers to configuration-driven experiment setup and standardized output formats. TransformerLens focuses on low-level mechanistic analysis for language models, while nnsight provides a general tracing and intervention graph framework. Dream Reader differs by providing diffusion-first, workflow-oriented pipelines with integrated experiment management and reporting.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
System / Paper
Interpretability Capabilities
Visualization & Workflow
Head/Block Patching
Concept Directions
SAE Training
SAE Intervene
Model Stitching
Trained Steering (LoREFT/K-Steer)
Attention Maps
Intermed. Viz ( x^0 /steps)
Workflow & Reports
Patch Explorer ( Grabe et al., 2025 )
✓
–
–
–
–
–
∘
∘
–
One-Step is Enough (SAE) ( Surkov et al., 2024 )
–
–
✓
✓
–
–
–
–
∘
Cross/Self-Attn for Editing ( Liu et al., 2024 )
–
–
–
–
–
–
✓
–
–
Distilling Diversity & Control ( Gandikota and Bau, 2025 )
–
–
–
–
∘
–
∘
✓
∘
SliderSpace ( Gandikota et al., 2025 )
–
✓
–
∘
–
–
–
✓
∘
Appendix
Table 3: T2I interpretability repositories summary. Left block: core interpretability capabilities; right block: visualization and workflow support. Symbols: ✓supported; ∘ partial/limited; –not reported/applicable.
Term
Definition
Scales
Intervention strength parameters (e.g., how strongly activations are modified).
Schedules
Temporal specifications describing when an intervention is applied (e.g., which denoising steps/timesteps).
Steering vector
A learned direction in activation space that, when added to activations, steers model behavior.
Mapper
A neural network (affine/linear or MLP) transforming activations, used in steering and stitching workflows.
Feature ablation
Removing or zeroing specific features to test their causal role.
CLIP alignment
Semantic similarity between text prompt and generated image using CLIP embeddings.
Appendix
Table 4: Glossary of key terms used across DreamReader workflows.
Diffusion models have consistently driven progress in text-to-image generation. However, it is challenging to attribute recent progress to specific modeling and data choices: state-of-the-art open-weight models provide limited ablations, and do not disclose their training data and full training details. The research community needs fully open (weights, data, and code) models as a foundation for further research; yet existing fully open models still fall significantly short of leading models in performance. In this project, we conduct a systematic investigation of the modeling and data design choices in text-to-image diffusion training and inference with 300+ controlled experiments totaling 700K+ TPU v6e hours. Our experiments highlight several empirical findings (e.g., equal weighting is a strong default for mixing curated datasets) and simple design decisions (e.g., larger text encoder adapters improve performance with minimal added parameters) for training strong models. Guided by these insights, we train i1, a 3B-parameter text-to-image diffusion model using only publicly available datasets. i1 is competitive with leading models on five representative benchmarks (GenEval, DPG, PRISM, CVTG-2K, and LongText), and outperforms the best existing fully open model by 29.5 absolute percentage points on average. We provide the i1 checkpoints, training and inference code, and the data processing pipeline. Together, our findings and the i1 recipe establish a practical foundation for future open research in text-to-image diffusion models. Our code is available at https://github.com/zlab-princeton/i1.
Numerous studies on text-to-image (T2I) generative models have utilized cross-attention maps to boost application performance and interpret model behavior. However, the distinct characteristics of attention maps from different attention heads remain relatively underexplored. In this study, we show that selectively aggregating cross-attention maps from heads most relevant to a target concept can improve visual interpretability. Compared to the diffusion-based segmentation method DAAM, our approach achieves higher mean IoU scores. We also find that the most relevant heads capture concept-specific features more accurately than the least relevant ones, and that selective aggregation helps diagnose prompt misinterpretations. These findings suggest that attention head selection offers a promising direction for improving the interpretability and controllability of T2I generation.
Jungwon Park, Jungmin Ko, Dongnam Byun +1
Department of Intelligence and Information, Seoul National University, Seoul, Republic of Korea · Interdisciplinary Program in Artificial Intelligence, Seoul National University, Seoul, Republic of Korea
Multimodal Diffusion Transformers (MM-DiTs) jointly process visual and textual representations throughout generation. These models repeatedly update the text tokens through multimodal attention, forming dynamic contextual tokens whose function is not well understood. In this work, we introduce a framework for reading this contextual space through natural-language interrogation. We train a lightweight bottleneck network that maps intermediate contextual tokens into the input space of a frozen Large Language Model (LLM), allowing the LLM to answer questions about the emerging image directly from these hidden representations. Our reader reveals that contextual tokens encode a rich, global representation of the emerging scene: generation-specific semantics, including attributes left underspecified by the prompt, are accessible surprisingly early in denoising, while increasingly fine-grained details become readable over time. Remarkably, this information remains decodable even when the MM-DiT receives an empty prompt, showing that contextual tokens accumulate substantial image-specific information from the evolving visual representation itself. We further find that generations with more readable contextual representations tend to receive higher human-preference scores. Building on these observations, we introduce Contextual Alignment, a training technique that explicitly reinforces the visual-semantic information encoded in the contextual tokens, improving generation quality and distributional coverage. Together, our results establish contextual tokens as both an interpretable view into the internal dynamics of MM-DiTs and an effective target for improving generative models.