Unified multimodal large language models (MLLMs) and multi-agent systems have advanced visual generation. However, three limitations remain. (1) Existing methods often distill task-specific experience with limited generalizability. (2) Reflection is often deferred until task completion. (3) Knowledge is often acquired only in response to downstream task demands. To address these limitations, we introduce OmniHarness, a framework for generalizable visual generation via symbolic policy learning. OmniHarness abstracts verified executions into symbolic policies for visual generation task families, capturing shared procedures and applicability conditions while removing instance-specific inputs. The harness instantiates, adapts, and composes these policies for new tasks. Intermediate verification guides refinement and failure recovery during execution. Through self-directed inquiry, OmniHarness autonomously generates and executes practice tasks near its capability limits before downstream objectives are specified. Execution feedback continually refines the policies while model parameters remain fixed. Experiments across six benchmarks, three MLLM backbones, and three visual agent frameworks demonstrate strong performance and continual capability expansion. On ComfyBench's Creative tasks, OmniHarness achieves a 95.0% resolve rate, exceeding the strongest baseline by 27.5 percentage points. Frozen policy snapshots improve existing visual agent systems through plug-and-play reuse.
Figures & tables
Figure 1: Comparison of visual generation paradigms. (a) End-to-end unified MLLMs enable fast multimodal generation but lack deliberate reasoning and self-correction. (b) Existing multi-agent systems improve collaborative reasoning but lack unified coordination and persistent knowledge accumulation. (c) OmniHarness integrates self-directed inquiry and feedback-guided execution to learn reusable symbolic policies for generalizable visual generation.
Figure 2: Architecture of OmniHarness. Self-directed inquiry and feedback-guided execution drive symbolic policy learning for generalizable visual generation. The policy library evolves during OmniHarness execution, while frozen snapshots support plug-and-play reuse across visual agents.
Figure 3: Qualitative comparison on challenging tasks from the Complex and Creative subsets.
Agent
Vanilla
Complex
Creative
Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
GPT-4o + Zero-shot
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
GPT-4o + Few-shot [ 3 ]
32.0 ↑ 32.0
27.0 ↑ 27.0
16.7 ↑ 16.7
8.3 ↑ 8.3
7.5 ↑ 7.5
0.0 = 0.0
22.5 ↑ 22.5
16.0 ↑ 16.0
GPT-4o + CoT [ 53 ]
44.0 ↑ 44.0
29.0 ↑ 29.0
11.7 ↑ 11.7
8.3 ↑ 8.3
12.5 ↑ 12.5
0.0 = 0.0
28.0 ↑ 28.0
17.0 ↑ 17.0
GPT-4o + CoT-SC [ 51 ]
45.0 ↑ 45.0
34.0 ↑ 34.0
11.7 ↑ 11.7
5.0 ↑ 5.0
15.0 ↑ 15.0
0.0 = 0.0
29.0 ↑ 29.0
18.5 ↑ 18.5
Claude-3.5-Sonnet + RAG [ 29 ]
27.0 ↑ 27.0
13.0 ↑ 13.0
23.0 ↑ 23.0
6.7 ↑ 6.7
7.5 ↑ 7.5
0.0 = 0.0
22.0 ↑ 22.0
8.5 ↑ 8.5
Table 1: Quantitative comparison on ComfyBench [ 65 ] .
Figure 4: Qualitative comparison on representative text-to-image tasks.
Method
Single Obj. ↑
Two Obj. ↑
Counting ↑
Colors ↑
Position ↑
Attr. Bind. ↑
Overall ↑
Frozen Text-Encoder Mapping Methods
SDv1.5 [ 45 ]
0.97
0.38
0.35
0.76
0.04
0.06
0.43
SDv2.1 [ 45 ]
0.98 ↑ 0.01
0.51 ↑ 0.13
0.44 ↑ 0.09
0.85 ↑ 0.09
0.07 ↑ 0.03
0.17 ↑ 0.11
0.50 ↑ 0.07
SD-XL [ 43 ]
0.98 ↑ 0.01
0.74 ↑ 0.36
0.39 ↑ 0.04
0.85 ↑ 0.09
0.15 ↑ 0.11
0.23 ↑ 0.17
0.55 ↑ 0.12
DALL-E 2 [ 44 ]
0.94 ↓ 0.03
0.66 ↑ 0.28
0.49 ↑ 0.14
0.77 ↑ 0.01
0.10 ↑ 0.06
0.19 ↑ 0.13
0.52 ↑ 0.09
SD3-Medium [ 12 ]
0.99 ↑ 0.02
0.94 ↑ 0.56
0.72 ↑ 0.37
0.89 ↑ 0.13
0.33 ↑ 0.29
0.60 ↑ 0.54
0.74 ↑ 0.31
Table 2: Quantitative comparison on GenEval [ 17 ] .
Method
Object ↑
Attribute ↑
Count ↑
Position ↑
Verb ↑
Overall ↑
Stable Diffusion Model Series
SD 2.1 [ 45 ]
55.1
30.4
22.3
11.7
17.8
27.46
SDXL [ 43 ]
74.1 ↑ 19.0
42.4 ↑ 12.0
28.7 ↑ 6.4
16.0 ↑ 4.3
28.9 ↑ 11.1
38.02 ↑ 10.56
SD3 [ 12 ]
87.0 ↑ 31.9
65.5 ↑ 35.1
49.9 ↑ 27.6
41.0 ↑ 29.3
46.7 ↑ 28.9
58.02 ↑ 30.56
SD3.5-Large [ 12 ]
91.6 ↑ 36.5
70.3 ↑ 39.9
52.2 ↑ 29.9
39.5 ↑ 27.8
55.6 ↑ 37.8
61.84 ↑ 34.38
State-of-the-Art Text-to-Image Models
Table 3: Quantitative comparison on GenEval2 [ 25 ] .
Table 7: Ablation of task selection in self-directed inquiry on the Creative subset.
Agent
Complex T2I
Complex I2I
Complex Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
w/o Planning
80.0 ↓ 20.0
40.0 ↓ 50.0
87.0 ↓ 13.0
56.5 ↓ 34.8
86.7 ↓ 13.3
55.0 ↓ 28.3
w/o Intermediate Verification
70.0 ↓ 30.0
60.0 ↓ 30.0
73.9 ↓ 26.1
69.6 ↓ 21.7
75.0 ↓ 25.0
68.3 ↓ 15.0
w/o Localized Recovery
90.0 ↓ 10.0
80.0 ↓ 10.0
95.7 ↓ 4.3
82.6 ↓ 8.7
93.3 ↓ 6.7
76.7 ↓ 6.6
OmniHarness
100.0
90.0
100.0
91.3
100.0
83.3
Table 8: Ablation of feedback-guided execution on the Complex subset.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Modality
Capability Category
Description
Text-to-Image (T2I)
Photorealistic Generation
Generates realistic people, animals, objects, and indoor and outdoor scenes.
Futuristic & Fantasy Generation
Generates science-fiction, futuristic, and fantasy scenes, objects, and concepts.
Illustration & Comic Generation
Generates illustrations, cartoons, comics, and multi-panel visual narratives.
Poster & Graphic Design
Generates posters, covers, advertisements, and other graphic compositions with specified layouts.
In-Image Text Generation
Renders specified text with correct content, legibility, appearance, and placement.
Position-Constrained Generation
Places objects or visual elements in specified image regions or locations.
Appendix
Table 9: Generative capability space G for self-directed inquiry.
Field
Description
Modality
Specifies whether the workflow supports T2I or I2I generation.
Capability Categories
Records the capabilities in G supported by the workflow.
Workflow Template
Stores a reusable ComfyUI graph with node types, shared parameters, connections, and execution settings. Instance-specific inputs are removed and supplied when the policy is instantiated for a new task.
Workflow Name
Provides a unique identifier for indexing, retrieval, and invocation.
Workflow Description
Summarizes the shared procedure and the task family it supports.
Preconditions
Specifies required input modalities, image properties, task conditions, available models, and execution resources.
Appendix
Table 10: Schema of the workflow library Lt .
Entry Type
Field
Description
Execution Failure Record
Task τt
Stores the description qτt , modality mτt , required capabilities Gτt , and source image xτt when applicable.
Failure Context
Records the input conditions and generation constraints under which the failure occurred.
Failed Workflow πt
Stores the failed task’s ComfyUI workflow, including nodes, parameters, connections, and execution settings.
Failure Evidence ot
Preserves execution status, intermediate and final outputs, source images when applicable, execution feedback, and verifier results.
Corrective Strategies
Root Cause
Identifies the workflow or generation factor responsible for the failure.
Workflow Antipattern
Describes workflow structures, node configurations, parameter settings, or generation strategies to avoid.
Appendix
Table 11: Schema of the failure library Ft .
Field
Description
Modality Coverage
Counts stored T2I and I2I workflows to identify imbalances in modality coverage.
Capability Coverage
Counts workflows associated with each capability in G to identify gaps in workflow coverage.
Reliability Distribution
Counts Provisional , Validated , and Suspended workflows in Lt .
Source Image Pool X
Contains all original ComfyBench images for I2I practice. Video inputs are excluded.
Source Image Metadata
Records image file names, resolutions, and formats to support image selection and task generation.
Appendix
Table 12: Fields of the current scene context ct .
Figure 6: Visualization of the workflow library after 50 iterations of self-directed inquiry. (a) Word cloud summarizing the capabilities represented by stored policies. (b) Distribution of workflow entries across capability categories.
Figure 7: Resolve rates with different numbers of self-directed inquiry iterations on ComfyBench [ 65 ] .
Figure 8: Symbolic policy learning during downstream execution on ComfyBench [ 65 ] . (a) Growth in total and validated workflow entries in the workflow library, starting after 50 inquiry iterations and continuing through the Vanilla, Complex, and Creative subsets. (b) Changes in the proportions of Validated , Provisional , and Suspended workflows before and after ComfyBench execution.
Figure 9: Evolution and reuse of symbolic policies during downstream execution on ComfyBench [ 65 ] . (a) Expansion of the image-only workflow library acquired after 50 inquiry iterations to include T2I, I2I, T2V, I2V, and V2V workflows after ComfyBench execution. (b) Distribution of direct reuse, compositional reuse, and no-prior-match cases among resolved Vanilla, Complex, and Creative tasks, together with the overall distribution.
Figure 10: Capability distribution shift in the image workflow library during downstream execution on ComfyBench [ 65 ] . The bars compare workflow proportions across capability categories after 50 inquiry iterations and after ComfyBench execution. Percentages are normalized over T2I and I2I workflows only. The comparison shows how the capabilities represented by stored symbolic policies change during downstream execution.
Figure 11: Reasoning efficiency of OmniHarness on the 200 ComfyBench tasks [ 65 ] , reported over consecutive groups of 20 tasks. The bars show the average token consumption per task, while the line shows the average reasoning time per task. Tasks 1–100, 101–160, and 161–200 correspond to the Vanilla, Complex, and Creative subsets, respectively. Reasoning time includes agent-side planning, workflow synthesis, verification, and recovery, but excludes visual generation model execution.
Agent
Vanilla
Complex
Creative
Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
GPT-4o + ComfyAgent [ 65 ]
67.0
46.0
48.3
21.7
40.0
15.0
56.0
32.5
GPT-4o + ComfyMind [ 18 ]
100.0
92.0
100.0
85.0
100.0
57.5
100.0
83.0
Gemini-2.5-Flash + SymbOmni [ 37 ]
100.0
95.0
100.0
83.3
100.0
67.5
100.0
86.0
Gemini-2.5-Flash + OmniHarness
100.0
96.0
100.0
81.7
100.0
87.5
100.0
90.0
GPT-4o + OmniHarness
100.0
95.0
100.0
76.7
100.0
95.0
100.0
89.5
Appendix
Table 13: Backbone ablation of OmniHarness on ComfyBench [ 65 ] . The first three rows report representative existing agents, followed by OmniHarness instantiated with different reasoning backbones and planner configurations. Pass and Resolve are reported as percentages, with higher values indicating better performance.
Agent
Vanilla
Complex
Creative
Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
OmniHarness with XComfyBench
100.0
97.0
100.0
83.3
100.0
95.0
100.0
92.5
OmniHarness with XGen
100.0
97.0
100.0
83.3
100.0
90.0
100.0
91.5
Appendix
Table 14: Source image pool isolation study on ComfyBench [ 65 ] . XComfyBench denotes the original ComfyBench source image pool, while XGen replaces it with independently generated images from the same generative capability space.
Figure 12: Qualitative comparison on representative I2I tasks from ComfyBench [ 65 ] . The examples illustrate target localization, instruction following, and preservation of unrelated scene content.
Figure 15: GPT-score comparison on Reason-Edit [ 23 ] across seven reasoning-intensive image-editing categories. Higher scores indicate better performance.
Figure 16: Qualitative comparison on representative WISE [ 41 ] tasks spanning cultural, temporal, spatial, biological, physical, and chemical knowledge.
Method
Factual ↑
Conceptual ↑
Procedural ↑
Overall ↑
Closed-Source Models
GPT-Image-1
79.80
81.37
78.32
80.09
Gemini 2.0 Flash Experimental
65.26
59.65
62.90
62.41
Doubao
63.30
62.23
54.17
60.70
Open-Source Models
BAGEL-Think [ 10 ]
55.77
59.44
39.26
53.36
Appendix
Table 15: Quantitative comparison on KRIS-Bench [ 58 ] . Scores evaluate factual, conceptual, and procedural knowledge, together with overall performance. Higher values indicate better performance.
Figure 17: Qualitative results of OmniHarness on representative KRIS-Bench tasks [ 58 ] , covering factual, conceptual, and procedural knowledge through knowledge-guided generation, editing, visual reasoning, and multi-image composition.
Agent
Vanilla
Complex
Creative
Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
ComfyAgent [ 65 ]
67.0
46.0
48.3
21.7
40.0
15.0
56.0
32.5
ComfyMind [ 18 ]
100.0
92.0
100.0
85.0
100.0
57.5
100.0
83.0
SymbOmni [ 37 ]
100.0
95.0
100.0
83.3
100.0
67.5
100.0
86.0
ComfyAgent [ 65 ] + Kinquiry
87.0
69.0
73.3
46.7
67.5
42.5
79.0
57.0
ComfyMind [ 18 ] + Kinquiry
100.0
94.0
100.0
86.7
100.0
75.0
100.0
88.0
Appendix
Table 16: Quantitative plug-and-play evaluation on ComfyBench [ 65 ] . The first three rows report the original host agents, while the last three augment them with the same frozen policy snapshot Kinquiry learned through self-directed inquiry, without model fine-tuning.
Agent
T2V
I2V
V2V
Total
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
Pass ↑
Res. ↑
GPT-4o + ComfyAgent [ 65 ]
68.4
47.4
48.0
20.0
40.0
13.3
56.4
32.1
GPT-4o + ComfyMind [ 18 ]
100.0
71.1
100.0
76.0
100.0
53.3
100.0
69.2
Gemini-2.5-Flash + SymbOmni [ 37 ]
100.0
81.6
100.0
88.0
100.0
66.7
100.0
80.8
GPT-4o + OmniHarness
100.0
84.2
100.0
92.0
100.0
80.0
100.0
85.9
Appendix
Table 17: Quantitative cross-modal generalization results on the video-generation subset of ComfyBench [ 65 ] . T2V, I2V, and V2V denote text-to-video, image-to-video, and video-to-video tasks, respectively. GPT-4o + OmniHarness uses the frozen policy snapshot Kinquiry , learned through self-directed inquiry in an image-only capability space without exposure to video tasks during inquiry.
Figure 18: Qualitative cross-modal generalization results of OmniHarness on representative video-generation tasks from ComfyBench [ 65 ] . Sampled frames illustrate generated content and source-content preservation in T2V, I2V, and V2V tasks using the frozen policy snapshot Kinquiry learned through image-only inquiry.
Figure 19: Reuse of symbolic policies learned through image-only inquiry during cross-modal generalization on ComfyBench [ 65 ] . The bars report invocation counts of workflow entries from the frozen image-only workflow library for T2V, I2V, and V2V tasks.
Figure 20: Web interface used in the 40-task user study. Participants assess an anonymized generation result against the task instruction and, when required, the input image. Ratings cover task description alignment, input-structure fidelity, photorealism, visual quality, and overall satisfaction on a five-point Likert scale.
Figure 21: User-study comparison across five evaluation criteria. Mean ratings over 40 ComfyBench [ 65 ] tasks are reported on a five-point Likert scale, with higher scores indicating better performance.
Multimodal large language models (MLLMs) have made rapid progress on visual reasoning, yet many tasks require evidence that is difficult to obtain through internal reasoning alone: our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains. Such tasks demand grounded perception, external knowledge, and numerical measurements obtained through heterogeneous tools, whose outputs must be made interpretable to the model and retained across reasoning steps. We introduce OpenVLHarness, a multimodal reasoning harness organized around a capability layer and persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations. Persistent multimodal memory retains the corresponding images, structured data, and web references as addressable artifacts throughout each query. A coding interface executes generated programs over stored data, enabling computations that combine results from multiple tools. Together, these mechanisms support an iterative process of acquiring, inspecting, and composing multimodal evidence. Evaluated on 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7-12.6 points over matched no-tool baselines, including +11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, and +9.9 with GPT-6 Astra. With GPT-6 Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by 7.7-8.5 points while incurring at most 37% of their model-token cost.
Visual generation is increasingly ubiquitous in diverse domains, from text-to-image/video synthesis to multimodal interactive creation. Yet prevailing monolithic models remain fundamentally constrained by their inability to learn cumulatively and evolve autonomously, which is a limitation we term the "perpetual novice" problem. They lack mechanisms for structuring experience into reusable knowledge and therefore rely on brittle, "from-scratch" reasoning for each task, resulting in poor compositional generalization and inefficient knowledge retention. Motivated by these limitations, we propose SymbOmni, an agentic omni-model designed for cumulative evolution through Symbolic Concept Learning. At its core is the Symbolic Concept Box, an optimizable memory module that abstracts low-level operations into reusable Symbolic Workflow Instructions. SymbOmni operates through an induction-transduction cycle: experiences are abstracted into symbolic concepts (induction), which are then adaptively composed to solve novel tasks (transduction). The training is done by verbalized backpropagation with language-based feedback to enable continuous self-improvement without gradient-based model fine-tuning. Comprehensive experiments validate that (I) SymbOmni significantly outperforms existing agent-based systems for iterative creation and also surpasses closed-source models (e.g., Nano Banana, GPT-Image-1) in both image quality and task success rates; (II) SymbOmni effectively reduces token consumption by over 40% while maintaining competitive generation quality; and (III) SymbOmni enables effective continual learning by achieving cumulative gains across multiple online-learning benchmarks and setting a new state of the art.
Jinxiu Liu, Jianru Li, Tanqing Kuang +4
South China University of Technology · Westlake University · Johns Hopkins University +2
The central bottleneck in visual generation and editing has migrated from achieving pixel-level fidelity to satisfying densely constrained user specifications through increasingly agentic workflows. Prevailing agentic generation pipelines remain shallow generate-verify-rewrite loops treating VLM judgments as coarse visual feedback, either discarding repair experience or admitting ungated updates that poison skill library. With memorize and govern periods left open, they degrade into ungoverned iterators of repeated trial-and-error rather than continual self-evolving harness. Towards this issue, we present VEvo-Harness, the first causally-guarded continual self-evolving harness system for agentic visual generation. VEvo-Harness introduces four coupled innovations: (1) a visual traceback that densifies sparse judgments into root-cause attributions with a hierarchical verification tree; (2) a visual continual governance that admits only causally valid skill patches through counterfactual dual-arm image generation replay; (3) a lifecycle-managed harness state that hierarchically distills root-cause-anchored episodes across working, experience, and pattern tiers into a persistent skill library of routable and causally verified capabilities; (4) a heterogeneous sandbox policy that orchestrates a diverse action space of repair primitives, dynamically routing cost-aware interventions based on root causes to select the most effective strategy. VEvo-Harness surpasses state-of-the-art agentic methods and rivals proprietary generators (e.g., GPT-Image, Nano-Pro) while maintaining the continual harness evolution and generalization.
Yuan Wang, Yuhang Yu, Mengting Chen +4
Taobao & Tmall group of Alibaba · Tsinghua University