Embodied agents for creative tasks like photography must bridge the semantic gap between high-level language commands and geometric control. We introduce PhotoAgent, an agent that achieves this by integrating Large Multimodal Models (LMMs) reasoning with a novel control paradigm. PhotoAgent first translates subjective aesthetic goals into solvable geometric constraints via LMM-driven, chain-of-thought (CoT) reasoning, allowing an analytical solver to compute a high-quality initial viewpoint. This initial pose is then iteratively refined through visual reflection within a photorealistic internal world model built with 3D Gaussian Splatting (3DGS). This ``mental simulation'' replaces costly and slow physical trial-and-error, enabling rapid convergence to aesthetically superior results. Evaluations confirm that PhotoAgent excels in spatial reasoning and achieves superior final image quality.
Figures & tables
Fig. 2: Overall cognitive architecture of PhotoAgent.
Fig. 3: Intention Parsing workflow demonstration. “Take a photo of the toys with visual tension, like the reference.”
Fig. 4: Closed-loop reflective reasoning. Starting from an internal 3DGS “imagined world”, the agent renders candidate views, critiques them via the LMM, and issues optimized motion commands.
Fig. 5: Performances of our method and baselines on three tasks of different levels.
Method
Easy
Medium
Hard
Direct-6-DoF
3.00 ( 2/3 )
N/A ( 0/3 )
N/A ( 0/3 )
Direct-6-DoF w/ Reflection
3.00 ( 1/3 )
N/A ( 0/3 )
N/A ( 0/3 )
PhotoAgent (ours)
2.33 ( 3/3 )
2.00 ( 3/3 )
2.00 ( 2/3 )
TABLE I: Performance Comparison on Gradient Tasks
ID
Scene Name
User Instruction
(a)
Girl_Portrait ‡
Take a close-up from the front.
(b)
Girl_Reading ‡
Capture her reading intently.
(c)
Man_Whiteboard
Capture a thoughtful-looking expression.
(d)
Girl_Library ‡
Take a beautiful photo.
(e)
Teddy_Lab
Give the teddy bear a close-up shot.
(f)
Dolls_Confrontation
Take a photo of the toys with visual tension, like the reference.
TABLE II: Experimental scenarios and user instructions. § denotes simulated scenes; ‡ indicates same subject under different instructions.
Fig. 6: Qualitative examples from real (a) and simulated (b) environments. Each row shows: third-person view, baseline, and PhotoAgent’s output.
Category (N)
Metric
Baseline
PhotoAgent (ours)
Δ
Portraits (4)
MOS
2.86
3.82
+0.96
GoB
25.2%
68.5%
+43.2 pp
IAWR
—
89.5%
—
Still Life (4)
MOS
2.88
3.94
+1.07
GoB
28.2%
71.2%
+43.0 pp
IAWR
—
96.2%
—
TABLE III: Performance comparison between PhotoAgent (ours) and the baseline across categories.
Fig. 7: Stage-2 instruction adherence. Per-scene win rates under paired comparison. Scene IDs (a–h) are defined in Table II .
Virtual photography asks an agent to enter a prepared 3D scene with no preselected camera pose or reference image, infer a suitable shot from scene information and a language intent, choose executable camera parameters, and render the final photograph. Recent progress in vision-language models makes this kind of spatial agent increasingly plausible, but the task stresses two capabilities that remain hard to evaluate together: complex 3D spatial understanding and abstract aesthetic judgment. We introduce PhotoFlow, a Director-Reviewer-Reflector agent for closed-loop camera search. The Director builds a soft photographic blueprint and proposes diverse candidate cameras; the Reviewer combines rule checks, visual critique, and pairwise incumbent selection; and the Reflector converts failures into region memory, dead-zone suppression, and high-explore relocation. We also introduce VPhotoBench, a benchmark of 47 open-license Blender scenes and 141 language-conditioned photography missions spanning subject placement, relational composition, and atmosphere/style. On held-out experiments, PhotoFlow achieves the strongest external quality-alignment composite and success rate among one-shot prediction, single-chain reflection, anchor-bank selection, and random search under a six-round rendering budget. To our knowledge, this is the first work to make language-conditioned virtual photography in arbitrary Blender scenes an executable agent task, and our results show that an LLM-centered spatial agent can already produce strong photographs in a setting designed to challenge both 3D reasoning and aesthetic choice.
Jiarui Guo, Haojia Wei, Yiming Zhang +5
Shanghai Jiao Tong University · Northeastern University · University of California, Los Angeles +3
With the recent fast development of generative models, instruction-based image editing has shown great potential in generating high-quality images. However, the quality of editing highly depends on carefully designed instructions, placing the burden of task decomposition and sequencing entirely on the user. To achieve autonomous image editing, we present PhotoAgent, a system that advances image editing through explicit aesthetic planning. Specifically, PhotoAgent formulates autonomous image editing as a long-horizon decision-making problem. It reasons over user aesthetic intent, plans multi-step editing actions via tree search, and iteratively refines results through closed-loop execution with memory and visual feedback, without requiring step-by-step user prompts. To support reliable evaluation in real-world scenarios, we introduce UGC-Edit, an aesthetic evaluation benchmark consisting of 7,000 photos and a learned aesthetic reward model. We also construct a test set containing 1,017 photos to systematically assess autonomous photo editing performance. Extensive experiments demonstrate that PhotoAgent consistently improves both instruction adherence and visual quality compared with baseline methods. The project page is https://mdyao.github.io/PhotoAgent/.
Mingde Yao, Zhiyuan You, King-Man Tam +2
MMLab, CUHK · CPII under InnoHK · Institute of Science Tokyo +2
Text-to-image (T2I) models can produce visually compelling images, yet they remain limited on open-world tasks that require complex semantic understanding, multi-step reasoning, and the integration of external world knowledge. Existing efforts introduce agent capabilities into image generation, but they either prescribe a fixed workflow or place only a subset of the open-world image generation process under agent control. Consequently, reasoning, tool invocation, and image generation are not coordinated by a single policy. We propose ToolArtist, a fully agentic image generation model obtained by post-training a Unified Multimodal Model (UMM). ToolArtist dynamically orchestrates reasoning, external tool use, and native image generation within one unified policy. During Supervised Fine-Tuning (SFT), we equip a teacher agent with search tools alongside an image-generation tool. We then convert the collected trajectories into a UMM compatible format, where the image-generation tool is concealed while the resulting generated images are retained. During Reinforcement Learning (RL), we develop an agentic RL infrastructure for UMMs and introduce Reason-Act-Draw GRPO (RAD-GRPO), which uses complementary intent and quality rewards to jointly optimize the model. Experiments show that placing the entire open-world image-generation process under an agent policy consistently outperforms approaches with fixed pipelines or only partially agent-controlled components. We release the training data and the complete post-training infrastructure.