How Far Can Coding Agents Go in Visual Generation?
Organizations: University of California, Los Angeles · Tsinghua University
Abstract
Modern image generation largely relies on specialized generative models that leave the process of image construction implicit in learned parameters. We explore an alternative: general-purpose coding agents that construct images through executable programs. Our study of zero-shot drawing on AgentDrawBench, a 200-prompt benchmark, reveals a distinctive capability profile. Agents excel at preserving structure and satisfying precise visual constraints, achieving competitive results in domains where compositional accuracy matters more than photorealistic appearance, while remaining limited in perceptual richness. Their visual expressiveness varies with the drawing interface, and a qualitative study on reconstruction shows how procedural skills extracted from human demonstrations support different artistic media. We then examine code as an external visual language of thought: a structured, executable representation that agents can construct and inspect. We test it in visual question answering task and find reconstructed visual code retains substantial information for question answering and, in some domains, supports higher accuracy than the original image. These abilities raise a further question: do agents draw on a shared internal representation of scene content beyond a particular drawing program? We explore an internal visual language-of-thought hypothesis, in which a shared, compositionally structured representation supports visual operations across drawing languages. Cross-harness drawings and images assembled from independent single-pixel queries exhibit related layouts and tones. Agents also maintain self-generated room arrangements across viewpoints after visible drawings are removed. These results demonstrate that the coding agent would maintain a consistent and coherent world inside its imagine even before the harness gets involved and the generation starts.
Figures & tables
| Harness | Canvas 2D | Shapes | SVG | Blender | Native | |||
|---|---|---|---|---|---|---|---|---|
| Agent | Astra | Kimi K3 | DeepSeek | Astra | Astra | Astra | GPT-image-2 | GPT-image-1 |
| Scenes | 76.5 | 52.8 | 58.8 | 68.1 | 71.9 | 62.0 | 85.4 | 76.8 |
| People / organic | 81.1 | 54.7 | 69.4 | 71.4 | 79.1 | 73.2 | 93.6 | 80.1 |
| Objects / 3D | 82.0 | 60.5 | 70.3 | 63.3 | 77.8 | 71.5 | 87.0 | 67.1 |
| Stylized illustration | 88.5 | 49.3 | 73.6 | 82.3 | 85.8 | 71.9 | 92.4 | 74.7 |
| Design / symbolic | 94.6 | 84.7 | 89.5 | 75.2 | 93.9 | 92.3 | 94.4 | 58.1 |
| Harness | Canvas 2D | Shapes | SVG | Blender | ||
|---|---|---|---|---|---|---|
| Agent | Astra | Kimi K3 | DeepSeek | Astra | Astra | Astra |
| artifacts | 45.2 | 32.3 | 29.0 | 28.7 | 48.9 | 39.7 |
| object form | 46.1 | 24.3 | 25.3 | 30.2 | 39.0 | 32.6 |
| anatomy | 15.4 | 9.9 | 8.2 | 10.4 | 18.9 | 13.6 |
| typography | 77.6 | 57.6 | 61.2 | 38.1 | 74.8 | 58.6 |
| perspective | 28.0 | 14.8 | 15.1 | 11.6 | 23.2 | 25.8 |
| Harness | Canvas 2D | Shapes | SVG | Blender | ||
|---|---|---|---|---|---|---|
| Agent | Astra | Kimi K3 | DeepSeek | Astra | Astra | Astra |
| Scenes | 13.0 | 0.0 | 1.9 | 0.0 | 1.9 | 0.0 |
| People / organic | 6.0 | 0.0 | 4.0 | 0.0 | 0.0 | 0.0 |
| Objects / 3D | 24.4 | 14.0 | 13.1 | 9.5 | 24.4 | 21.2 |
| Stylized illustration | 35.2 | 7.4 | 11.1 | 18.5 | 24.1 | 11.8 |
| Design / symbolic | 92.0 | 64.0 | 77.3 | 62.0 | 87.3 | 76.5 |
| Context after the first view | Same pieces | Relations | Plan wall | Order |
|---|---|---|---|---|
| Own drawings kept | 0.97 | 1.00 | 0.99 | 0.92 |
| Own drawings removed | 0.99 | 0.96 | 0.99 | 0.79 |
| Drawings and list removed | 0.88 | 0.79 | 0.99 | 0.80 |
| Fresh call per view | 0.71 | 0.61 | 0.24 | 0.63 |
| Chance | – | – | 0.31 | 0.50 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Harness | Renderer | Representation / interaction |
|---|---|---|
| Pixel | PIL | Individual pixel assignments |
| Shapes | PIL | Ordered geometric marks |
| Canvas 2D | Node + Canvas 2D | JavaScript on a persistent canvas |
| SVG | resvg | Complete vector-document replacement |
| GUI | JS Paint / Chromium | Mouse and keyboard actions with screenshots |
| Blender Cycles | Cycles | Procedural 3D scenes with render feedback |
Agentic Image Generation
Agentic image generation uses reasoning, tools, search, and iterative feedback to create and refine images with greater control.
See all 43 papers