cs.HCApr 15, 2026

Creo: From One-Shot Image Generation to Progressive, Co-Creative Ideation

Authors: Zoe De Simone, Angie Boggust, Fredo Durand, Ashia Wilson, Arvind Satyanarayan

Organizations: MIT CSAIL, Cambridge, MA, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Text-to-image (T2I) systems enable rapid generation of high-fidelity imagery but are misaligned with how visual ideas develop. T2I systems generate outputs that make implicit visual decisions on behalf of the user, often introduce fine-grained details that can anchor users prematurely and limit their ability to keep options open early on, and cause unintended changes during editing that are difficult to correct and reduce users' sense of control. To address these concerns, we present Creo, a multi-stage T2I system that scaffolds image generation by progressing from rough sketches to high-resolution outputs, exposing intermediary abstractions where users can make incremental changes. Sketch-like abstractions invite user editing and allow users to keep design options open when ideas are still forming due to their provisional nature. Each stage in Creo can be modified with manual changes and AI-assisted operations, enabling fine-grained, step-wise control through a locking mechanism that preserves prior decisions so subsequent edits affect only specified regions or attributes. Users remain in the loop, making and verifying decisions across stages, while the system applies diffs instead of regenerating full images, reducing drift as fidelity increases. A comparative study with a one-shot baseline shows that participants felt stronger ownership over Creo outputs, as they were able to trace their decisions in building up the image. Furthermore, embedding-based analysis indicates that Creo outputs are less homogeneous than one-shot results. These findings suggest that multi-stage generation, combined with intermediate control and decision locking, is a key design principle for improving controllability, user agency, creativity, and output diversity in generative systems.

Figures & tables

Figure 1. Creo is a multi-stage image generation workflow. Unlike current text-to-image systems, Creo starts from a rough sketch, allowing users to progressively make visual decisions, like viewpoint, composition, color, lighting, and style.
Figure 2. Creo decomposes image generation into multiple stages. From a prompt, it generates (1) multiple viewpoints , after which the illustrator (2) refines composition , (3) color , (4) lighting , and (5) style in any order. Stages can be completed in any order.
Figure 3. Creo supports non-linear creative workflows by combining edits in earlier stages (e.g., adding a hat to the composition ) with existing upstream decisions (e.g., dog color ).
MetricCreoGPT
Exploration & Anchoring  
Direction changes (# changes per session)1.6±101.6\pm 100.4±50.4\pm 5
Exploration breadth (# iterations per branch)3.3±183.3\pm 182.5±172.5\pm 17
Concept drift (% unintended drift actions)14.4±10214.4\pm 10214.2±8914.2\pm 89
Intentional pivots (% pivot actions)3.9±343.9\pm 342.9±432.9\pm 43
Control & Predictability  
Table 1. Creo ’s staged generation outperforms GPT’s one-shot prompting across exploration and control metrics ( μ±σ\mu\pm\sigma ).
Figure 4. Creo reduces design anchoring and homogenization. Visual embeddings of the images that users produced using Creo are less tightly clustered than those from GPT.
Figure 5. Creo supports people’s non-linear creative workflows. Participants P4-P9 frequently explored stages out of order and revisited stages to generate their images.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6. Two entry points into Creo ’s progressive ideation workflow. Users can begin with an existing image. This image is reverse engineered into the lighting , color and composition stages. Users can edit each of the representations, as shown in the bottom half of the image, moving across stages and propagating decisions across stages. This allows for iterative refinement of visual decisions on existing images.
Figure 7. Creo decomposes image generation into multiple stages. From a prompt, it generates (1) multiple viewpoints , after which the illustrator (2) refines composition , (3) color , (4) lighting , and (5) style in any order. Stages can be completed in any order.
SequenceTool OrderPrompt Order
ACreo →\rightarrow BaselineLiving Room →\rightarrow Kitchen
BCreo →\rightarrow BaselineKitchen →\rightarrow Living Room
CBaseline →\rightarrow CreoLiving Room →\rightarrow Kitchen
DBaseline →\rightarrow CreoKitchen →\rightarrow Living Room
Appendix
Table 2. Counterbalancing design used in the study.
Figure 8. Diagram depicting how editing instructions and constrains are dynamically passed to each layer.
Figure 9. Diagram depicting how editing instructions and constrains passed across layers; revision loops etc.
More in this topic

T2I Generation

Text-to-image generation creates images from natural-language prompts using generative models, with research focused on fidelity, control, efficiency, and safety.

See all 299 papers

Explore similar work

CardsList
Mar 9, 2026

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning. However, existing CoT-based T2I methods largely rely on abstract natural-language planning, which lacks the precision required for complex spatial layouts, structured visual elements, and dense textual content. In this work, we propose CoCo (Code-as-CoT), a code-driven reasoning framework that represents the reasoning process as executable code, enabling explicit and verifiable intermediate planning for image generation. Given a text prompt, CoCo first generates executable code that specifies the structural layout of the scene. The code is then executed in a sandboxed environment to render a deterministic draft image. Subsequently, the model refines this draft through fine-grained image editing to produce the final high-fidelity result. To support this training paradigm, we construct CoCo-10K, a curated dataset containing structured draft-final image pairs designed to teach both structured draft construction and corrective visual refinement. Empirical evaluations on StructT2IBench, OneIG-Bench, and LongText-Bench show that CoCo achieves improvements of 68.83%, 54.8%, and 41.23%, respectively, over direct generation, while also outperforming other CoT-enhanced generation methods. These results demonstrate that executable code is an effective and reliable reasoning paradigm for precise, controllable, and structured text-to-image generation.

Haodong Li, Chunmei Qing, Huanyu Zhang +11

The Chinese University of Hong Kong · South China University of Technology · Institute of Automation, Chinese Academy of Sciences +1

T2I GenerationImage GenerationControllable Image GenerationCompositional T2I Generation
AskPDF
May 28, 2026

GenClaw: Code-Driven Agentic Image Generation

Image generation models have evolved from text-conditioned pixel synthesis toward multimodal agents endowed with visual comprehension and tool invocation capabilities. Yet, existing agents remain at the mercy of underlying black-box image models. Their workflow is trapped in a repetitive cycle of prompt rewriting for generation refinement, leaving them with no mechanism to directly manipulate the canvas. In essence, the potential of LLMs to serve as a genuine "brush" for precise visual construction remains largely untapped. In this paper, we propose GenClaw, a code-driven agentic image generation paradigm that empowers the agent to create like a human artist: first conceptualizing, then sketching, and finally coloring. Specifically, the agent first constructs the conceptual knowledge and context through search and reasoning. It then utilizes code (e.g., SVG, HTML, ThreeJS) to render executable visual sketches. Finally, it employs an image generation model to supplement textures, materials, and photorealism. In this workflow, code serves as a controllable intermediate canvas bridging linguistic reasoning and pixel synthesis, seamlessly integrating programmatic logic with the visual expressiveness of generative models. By transforming image generation from a black-box paradigm into a staged process akin to authentic human creation, GenClaw offers a step toward for highly controllable and interpretable visual generation systems.

Junyan Ye, Jun He, Zilong Huang +4

1Tencent Hunyuan · 2Sun Yat-sen University · 3Tsinghua University +1

Image GenerationAgentic Image GenerationTool-Using Vision-Language AgentsControllable Image Generation
AskPDF
May 18, 2026

Generation Navigator: A State-Aware Agentic Framework for Image Generation

Despite rapid advances in text-to-image generation, faithfully realizing user intent remains challenging, often requiring manual multi-turn trial and error. To automate this process, existing systems rely on either simple prompt rewriting or closed-loop agents driven by hand-crafted rules, rather than learning to adapt actions to the evolving generation process. In this paper, we reformulate image generation as a state-conditioned action-making problem and propose Generation Navigator, a multi-turn T2I agent that learns to dynamically steer the generation trajectory and output the next action. However, training this agent via reinforcement learning introduces a critical credit assignment challenge: naively rewarding a trajectory based solely on a single state assigns equal credit to all actions in the rollout, ignores the quality dynamics across turns, and fails to distinguish actions that improve the trajectory from those that degrade it or waste turns without progress. We resolve this with PRE-GRPO (Peak-Retention-Efficiency Group Relative Policy Optimization), a trajectory-level reinforcement learning objective that explicitly rewards discovering a high-quality image (Peak), avoiding subsequent quality degradation across turns (Retention), and minimizing unnecessary turns (Efficiency). Experiments show substantial improvements across benchmarks, reaching a WISE score of 0.90 and 79.06% reasoning accuracy on T2I-ReasonBench.

Jinming Liu, Ruoyu Feng, Yuqi Wang +2

Shanghai Jiao Tong University · Independent · Eastern Institute of Technology, Ningbo, China

T2I GenerationAgentic Image GenerationRL for Image GenerationControllable Image Generation
AskPDF