SketchDirector: Complex Scene Sketching via Agentic Layout Reasoning and Detail Refinement
Organizations: GVC Lab, Great Bay University
Abstract
Existing sketch generation methods often rely on image guidance and are largely limited to individual objects or simplified scenes, leaving text-guided complex scene sketching underexplored. We present SketchDirector, an agentic framework that mimics the human sketching workflow for automatic multi-concept scene sketching. Given a scene-level text prompt, SketchDirector first employs a large language model~(LLM) to decompose the scene into multiple draft assets, where each is represented by a set of Bézier curves. Then, an agentic reasoning loop iteratively plans and executes their spatial arrangement, guided by self-evolving memory. Finally, a detail refinement module leverages diffusion priors to enhance sketch quality while preserving geometric structure. To support future research and applications in multi-concept scene sketch generation, we further introduce the MCSketch8K dataset, comprising editable vector assets, compositional layouts, and object-level manipulation annotations. Extensive experiments show that SketchDirector outperforms state-of-the-art baselines in semantic and compositional quality while effectively preserving overall contour organization during refinement.
Figures & tables
| Method | Sem. | Compo. | Compl. | Aes. | Size | Time |
|---|---|---|---|---|---|---|
| SketchAgent | 0.742 | 0.598 | 0.589 | 0.714 | 0.693 | 20s |
| OpenAI GPT-5.5 | 0.911 | 0.730 | 0.811 | 0.813 | 0.820 | 20s |
| VectorFusion | 0.466 | 0.252 | 0.338 | 0.417 | 0.412 | 17min |
| SVGDreamer | 0.690 | 0.546 | 0.620 | 0.650 | 0.695 | 24min |
| DiffSketcher | 0.567 | 0.374 | 0.381 | 0.567 | 0.537 | 13min |
| SketchDirector | 0.916 | 0.772 | 0.821 | 0.836 | 0.821 | 6min |
| Method | Sem. | Comp. | Compl. | Aes. | Size | Iter. | Ver. |
|---|---|---|---|---|---|---|---|
| Gemini-3.1-pro | 0.858 | 0.723 | 0.723 | 0.819 | 0.707 | 5.332 | 0.535 |
| Claude-Opus-4-7 | 0.879 | 0.681 | 0.791 | 0.739 | 0.721 | 5.843 | 0.477 |
| Qwen3.7-Plus | 0.629 | 0.477 | 0.526 | 0.552 | 0.507 | 10.743 | 0.436 |
| OpenAI GPT-5.5 | 0.926 | 0.795 | 0.811 | 0.871 | 0.785 | 1.656 | 0.803 |
| Method | Sem. | Comp. | Compl. | Aes. | Size | Iter. | Ver. |
|---|---|---|---|---|---|---|---|
| w/o Plan | 0.676 | 0.503 | 0.571 | 0.65 | 0.468 | 12.375 | 0.357 |
| w/o Geo. Ver. | 0.698 | 0.550 | 0.570 | 0.660 | 0.589 | 1.857 | 0.633 |
| w/o Sem. Ver. | 0.603 | 0.512 | 0.645 | 0.558 | 0.547 | 1.438 | 0.468 |
| w/o Short Mem. | 0.737 | 0.658 | 0.629 | 0.654 | 0.681 | 2.875 | 0.635 |
| w/o Long Mem. | 0.781 | 0.679 | 0.697 | 0.762 | 0.729 | 1.600 | 0.650 |
| SketchDirector | 0.839 | 0.765 | 0.732 | 0.768 | 0.742 | 1.375 | 0.873 |
Image Generation
Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.
See all 504 papers