Advances in Agentic Computer Graphics
Organizations: University of Texas at Austin · Stanford University · Simon Fraser University · University of California, Berkeley ·
Abstract
Computer graphics is built on code. Large language model (LLM) agents can now generate graphics code and iteratively refine it through various feedback. This capability opens the door to a potentially new paradigm of computer graphics, with graphics algorithms increasingly built around agentic systems to automate progressively complex tasks. Over the past years, research on agentic graphics has expanded rapidly across a diverse range of tasks. This rapid growth makes it increasingly difficult for manual literature reviews to keep pace with new developments. At the same time, similar ideas are often described using different terminology across these tasks, further complicating efforts to develop a systematic understanding of the field. In this report, we address these challenges through two contributions. First, we introduce a unified taxonomy for characterizing agentic systems across diverse computer graphics tasks. Second, we develop an agentic literature system that can assist human researchers in analyzing existing and newly published papers at scale. Building on these contributions, we analyze emerging research trends and discuss open challenges in coordinating agents and tools across graphics tasks, designing interfaces and representations for human-agent collaboration, sustaining long-horizon planning and refinement, and reliably evaluating interactive graphics systems.
Figures & tables
| Output module | SubDGuide [ 52 ] | PLAYGROUND [ 68 ] |
| Scope | Core, mesh-to-SubD reconstruction | Core, 2D layout generation |
| Input | Modeling, 3D Representation, Graphics Artifact | 2D Graphics Authoring, Language, Image, Graphics Artifact |
| Base model | VLM, Off-the-Shelf | VLM, Supervised Fine-Tuning (retriever, planner, generator), Off-the-Shelf (verifier) |
| Harness | Hybrid, Sequential (feedback loop), Local Patch, Rollback, Candidate Selection, Self-verification, Programmatic, Binary, Score, Diagnosis, Visual, Structural Scene State, Short-Term Memory, Local Memory | Hybrid, Sequential (feedback loop), Regeneration, Candidate Selection, Replanning, Critic, Ranking, Binary, Diagnosis, Visual, Structural Scene State, Short-Term Memory, Global Memory, Local Memory |
| Graphics interface | Structured Specification, Tool/Function Call, Agent-Adapted, Graphics Operations, Perception Models, Validators, External reference resources not reported, Geometry, Pixels | Structured Specification, Agent-Adapted, Authoring, Observation, Examples & Trajectories, Documentation & Rules, Scene Structure and Layouts, Pixels |
| Design insight | Separating semantic decisions from geometric execution allows proposed repairs to be checked before replacing the current result. | Conditioning each planning step on the current render adapts grouping and placement to the evolving canvas. |
| Method | Base Model | Harness | Graphics Interface & Tooling | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | Training | Update | Verifier | Assessment | Observation | Action | Software | Tools | Resources | Representation | |
| Decomate [ 100 ] | – | ||||||||||
| LogoMotion [ 82 ] | – | ||||||||||
| PSDesigner [ 114 ] | – | ||||||||||
| SketchAgent [ 129 ] | – | – | |||||||||
| PaperBanana [ 185 ] | – | ||||||||||
| Method | App. | Base Model | Harness | Graphics Interface & Tooling | ||||||||
| Type | Training | Update | Verifier | Assessment | Observation | Action | Software | Tools | Resources | Representation | ||
| aDSL [ 133 ] | Object | |||||||||||
| LL3M [ 84 ] | Object | |||||||||||
| L3GO [ 152 ] | Object | |||||||||||
| Idea23D [ 18 ] | Object | |||||||||||
| ShapeCraft [ 171 ] | Object | |||||||||||
| Method | Base Model | Harness | Graphics Interface & Tooling | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Type | Training | Update | Verifier | Assessment | Observation | Action | Software | Tools | Resources | Representation | |
| Articraft [ 183 ] | |||||||||||
| Artisan [ 9 ] | |||||||||||
| Articulate-Anything [ 67 ] | |||||||||||
| Cutscene Agent [ 38 ] | |||||||||||
| ArtiMo [ 189 ] | |||||||||||
Agentic Workflows
Agentic workflows coordinate AI agents, tools, and humans across multi-step tasks, emphasizing reliable execution, efficiency, and oversight.
See all 230 papers