OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning

Xu Xu; Jinxiu Liu; Zhangbo Qiao; Jiaxing Lu; Xiangyu Zhang; Yubin Gu; Fangwei Ning; Yan Shi

cs.LGSep 13, 2026

OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning

Authors: Xu Xu, Jinxiu Liu, Zhangbo Qiao, Jiaxing Lu, Xiangyu Zhang, Yubin Gu, Fangwei Ning, Yan Shi

Organizations: Beihang University · The Chinese University of Hong Kong · National University of Singapore

Abstract

Unified multimodal large language models (MLLMs) and multi-agent systems have advanced visual generation. However, three limitations remain. (1) Existing methods often distill task-specific experience with limited generalizability. (2) Reflection is often deferred until task completion. (3) Knowledge is often acquired only in response to downstream task demands. To address these limitations, we introduce OmniHarness, a framework for generalizable visual generation via symbolic policy learning. OmniHarness abstracts verified executions into symbolic policies for visual generation task families, capturing shared procedures and applicability conditions while removing instance-specific inputs. The harness instantiates, adapts, and composes these policies for new tasks. Intermediate verification guides refinement and failure recovery during execution. Through self-directed inquiry, OmniHarness autonomously generates and executes practice tasks near its capability limits before downstream objectives are specified. Execution feedback continually refines the policies while model parameters remain fixed. Experiments across six benchmarks, three MLLM backbones, and three visual agent frameworks demonstrate strong performance and continual capability expansion. On ComfyBench's Creative tasks, OmniHarness achieves a 95.0% resolve rate, exceeding the strongest baseline by 27.5 percentage points. Frozen policy snapshots improve existing visual agent systems through plug-and-play reuse.

Figures & tables

Figure 1: Comparison of visual generation paradigms. (a) End-to-end unified MLLMs enable fast multimodal generation but lack deliberate reasoning and self-correction. (b) Existing multi-agent systems improve collaborative reasoning but lack unified coordination and persistent knowledge accumulation. (c) OmniHarness integrates self-directed inquiry and feedback-guided execution to learn reusable symbolic policies for generalizable visual generation.

Figure 2: Architecture of OmniHarness. Self-directed inquiry and feedback-guided execution drive symbolic policy learning for generalizable visual generation. The policy library evolves during OmniHarness execution, while frozen snapshots support plug-and-play reuse across visual agents.

Figure 3: Qualitative comparison on challenging tasks from the Complex and Creative subsets.

Agent	Vanilla		Complex		Creative		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
GPT-4o + Zero-shot	0.0	0.0	0.0	0.0	0.0	0.0	0.0	0.0
GPT-4o + Few-shot [ 3 ]	32.0 $\uparrow$ 32.0	27.0 $\uparrow$ 27.0	16.7 $\uparrow$ 16.7	8.3 $\uparrow$ 8.3	7.5 $\uparrow$ 7.5	0.0 $=$ 0.0	22.5 $\uparrow$ 22.5	16.0 $\uparrow$ 16.0
GPT-4o + CoT [ 53 ]	44.0 $\uparrow$ 44.0	29.0 $\uparrow$ 29.0	11.7 $\uparrow$ 11.7	8.3 $\uparrow$ 8.3	12.5 $\uparrow$ 12.5	0.0 $=$ 0.0	28.0 $\uparrow$ 28.0	17.0 $\uparrow$ 17.0
GPT-4o + CoT-SC [ 51 ]	45.0 $\uparrow$ 45.0	34.0 $\uparrow$ 34.0	11.7 $\uparrow$ 11.7	5.0 $\uparrow$ 5.0	15.0 $\uparrow$ 15.0	0.0 $=$ 0.0	29.0 $\uparrow$ 29.0	18.5 $\uparrow$ 18.5
Claude-3.5-Sonnet + RAG [ 29 ]	27.0 $\uparrow$ 27.0	13.0 $\uparrow$ 13.0	23.0 $\uparrow$ 23.0	6.7 $\uparrow$ 6.7	7.5 $\uparrow$ 7.5	0.0 $=$ 0.0	22.0 $\uparrow$ 22.0	8.5 $\uparrow$ 8.5

Table 1: Quantitative comparison on ComfyBench [ 65 ] .

Figure 4: Qualitative comparison on representative text-to-image tasks.

Frozen Text-Encoder Mapping Methods
Method	Single Obj. $\uparrow$	Two Obj. $\uparrow$	Counting $\uparrow$	Colors $\uparrow$	Position $\uparrow$	Attr. Bind. $\uparrow$	Overall $\uparrow$
SDv1.5 [ 45 ]	0.97	0.38	0.35	0.76	0.04	0.06	0.43
SDv2.1 [ 45 ]	0.98 $\uparrow$ 0.01	0.51 $\uparrow$ 0.13	0.44 $\uparrow$ 0.09	0.85 $\uparrow$ 0.09	0.07 $\uparrow$ 0.03	0.17 $\uparrow$ 0.11	0.50 $\uparrow$ 0.07
SD-XL [ 43 ]	0.98 $\uparrow$ 0.01	0.74 $\uparrow$ 0.36	0.39 $\uparrow$ 0.04	0.85 $\uparrow$ 0.09	0.15 $\uparrow$ 0.11	0.23 $\uparrow$ 0.17	0.55 $\uparrow$ 0.12
DALL-E 2 [ 44 ]	0.94 $\downarrow$ 0.03	0.66 $\uparrow$ 0.28	0.49 $\uparrow$ 0.14	0.77 $\uparrow$ 0.01	0.10 $\uparrow$ 0.06	0.19 $\uparrow$ 0.13	0.52 $\uparrow$ 0.09
SD3-Medium [ 12 ]	0.99 $\uparrow$ 0.02	0.94 $\uparrow$ 0.56	0.72 $\uparrow$ 0.37	0.89 $\uparrow$ 0.13	0.33 $\uparrow$ 0.29	0.60 $\uparrow$ 0.54	0.74 $\uparrow$ 0.31

Table 2: Quantitative comparison on GenEval [ 17 ] .

Stable Diffusion Model Series
Method	Object $\uparrow$	Attribute $\uparrow$	Count $\uparrow$	Position $\uparrow$	Verb $\uparrow$	Overall $\uparrow$
SD 2.1 [ 45 ]	55.1	30.4	22.3	11.7	17.8	27.46
SDXL [ 43 ]	74.1 $\uparrow$ 19.0	42.4 $\uparrow$ 12.0	28.7 $\uparrow$ 6.4	16.0 $\uparrow$ 4.3	28.9 $\uparrow$ 11.1	38.02 $\uparrow$ 10.56
SD3 [ 12 ]	87.0 $\uparrow$ 31.9	65.5 $\uparrow$ 35.1	49.9 $\uparrow$ 27.6	41.0 $\uparrow$ 29.3	46.7 $\uparrow$ 28.9	58.02 $\uparrow$ 30.56
SD3.5-Large [ 12 ]	91.6 $\uparrow$ 36.5	70.3 $\uparrow$ 39.9	52.2 $\uparrow$ 29.9	39.5 $\uparrow$ 27.8	55.6 $\uparrow$ 37.8	61.84 $\uparrow$ 34.38
State-of-the-Art Text-to-Image Models

Table 3: Quantitative comparison on GenEval2 [ 25 ] .

Dedicated T2I Models
Method	Cultural $\uparrow$	Time $\uparrow$	Space $\uparrow$	Biology $\uparrow$	Physics $\uparrow$	Chemistry $\uparrow$	Overall $\uparrow$
SDv1.5 [ 45 ]	0.34	0.35	0.32	0.28	0.29	0.21	0.32
SDv2.1 [ 45 ]	0.30 $\downarrow$ 0.04	0.38 $\uparrow$ 0.03	0.35 $\uparrow$ 0.03	0.33 $\uparrow$ 0.05	0.34 $\uparrow$ 0.05	0.21 $=$ 0.00	0.32 $=$ 0.00
SD-XL [ 43 ]	0.43 $\uparrow$ 0.09	0.48 $\uparrow$ 0.13	0.47 $\uparrow$ 0.15	0.44 $\uparrow$ 0.16	0.45 $\uparrow$ 0.16	0.27 $\uparrow$ 0.06	0.43 $\uparrow$ 0.11
SD3-Medium [ 12 ]	0.42 $\uparrow$ 0.08	0.44 $\uparrow$ 0.09	0.48 $\uparrow$ 0.16	0.39 $\uparrow$ 0.11	0.47 $\uparrow$ 0.18	0.29 $\uparrow$ 0.08	0.42 $\uparrow$ 0.10
SD3.5-Medium [ 12 ]	0.43 $\uparrow$ 0.09	0.50 $\uparrow$ 0.15	0.52 $\uparrow$ 0.20	0.41 $\uparrow$ 0.13	0.53 $\uparrow$ 0.24	0.33 $\uparrow$ 0.12	0.45 $\uparrow$ 0.13

Table 4: Quantitative comparison on WISE [ 41 ] .

Figure 5: Qualitative results on (a) Understanding Scenarios, (b) Reasoning Scenarios.

Method	Understanding Scenarios				Reasoning Scenarios
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$
InstructPix2Pix [ 2 ]	21.58	0.72	0.09	22.76	24.23	0.71	0.08	19.41
MagicBrush [ 70 ]	18.12 $\downarrow$ 3.46	0.68 $\downarrow$ 0.04	0.14 $\uparrow$ 0.05	22.62 $\downarrow$ 0.14	22.10 $\downarrow$ 2.13	0.69 $\downarrow$ 0.01	0.11 $\uparrow$ 0.03	19.76 $\uparrow$ 0.34
InstructDiffusion [ 16 ]	23.26 $\uparrow$ 1.68	0.74 $\uparrow$ 0.02	0.07 $\downarrow$ 0.02	23.08 $\uparrow$ 0.32	21.45 $\downarrow$ 2.78	0.67 $\downarrow$ 0.04	0.12 $\uparrow$ 0.03	19.52 $\uparrow$ 0.11
SmartEdit-7B [ 23 ]	22.05 $\uparrow$ 0.47	0.73 $\uparrow$ 0.01	0.09 $\approx$ 0.00	23.61 $\uparrow$ 0.85	25.26 $\uparrow$ 1.02	0.74 $\uparrow$ 0.04	0.06 $\downarrow$ 0.03	20.95 $\uparrow$ 1.54
SmartEdit-13B [ 23 ]	23.60 $\uparrow$ 2.02	0.75 $\uparrow$ 0.03	0.07 $\downarrow$ 0.02	23.54 $\uparrow$ 0.77	25.76 $\uparrow$ 1.52	0.75 $\uparrow$ 0.04	0.05 $\downarrow$ 0.03	20.78 $\uparrow$ 1.36

Table 5: Quantitative comparison on Reason-Edit [ 23 ] .

Agent	Vanilla		Complex		Creative		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
w/o Self-Directed Inquiry	100.0 $=$ 0.0	96.0 $\downarrow$ 1.0	100.0 $=$ 0.0	81.7 $\downarrow$ 1.6	100.0 $=$ 0.0	72.5 $\downarrow$ 22.5	100.0 $=$ 0.0	87.0 $\downarrow$ 5.5
w/o Online Policy Updates	100.0 $=$ 0.0	96.0 $\downarrow$ 1.0	100.0 $=$ 0.0	76.7 $\downarrow$ 6.6	100.0 $=$ 0.0	87.5 $\downarrow$ 7.5	100.0 $=$ 0.0	88.5 $\downarrow$ 4.0
OmniHarness	100.0	97.0	100.0	83.3	100.0	95.0	100.0	92.5

Table 6: Quantitative ablation results on ComfyBench [ 65 ] .

Agent	Creative T2I		Creative I2I		Creative Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
w/o Capability Novelty	100.0 $=$ 0.0	66.7 $\downarrow$ 16.6	100.0 $=$ 0.0	86.7 $\downarrow$ 6.6	100.0 $=$ 0.0	87.5 $\downarrow$ 7.5
w/o Competence Frontier	100.0 $=$ 0.0	83.3 $=$ 0.0	100.0 $=$ 0.0	80.0 $\downarrow$ 13.3	100.0 $=$ 0.0	85.0 $\downarrow$ 10.0
OmniHarness	100.0	83.3	100.0	93.3	100.0	95.0

Table 7: Ablation of task selection in self-directed inquiry on the Creative subset.

Agent	Complex T2I		Complex I2I		Complex Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
w/o Planning	80.0 $\downarrow$ 20.0	40.0 $\downarrow$ 50.0	87.0 $\downarrow$ 13.0	56.5 $\downarrow$ 34.8	86.7 $\downarrow$ 13.3	55.0 $\downarrow$ 28.3
w/o Intermediate Verification	70.0 $\downarrow$ 30.0	60.0 $\downarrow$ 30.0	73.9 $\downarrow$ 26.1	69.6 $\downarrow$ 21.7	75.0 $\downarrow$ 25.0	68.3 $\downarrow$ 15.0
w/o Localized Recovery	90.0 $\downarrow$ 10.0	80.0 $\downarrow$ 10.0	95.7 $\downarrow$ 4.3	82.6 $\downarrow$ 8.7	93.3 $\downarrow$ 6.7	76.7 $\downarrow$ 6.6
OmniHarness	100.0	90.0	100.0	91.3	100.0	83.3

Table 8: Ablation of feedback-guided execution on the Complex subset.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Modality	Capability Category	Description
Text-to-Image (T2I)	Photorealistic Generation	Generates realistic people, animals, objects, and indoor and outdoor scenes.
	Futuristic & Fantasy Generation	Generates science-fiction, futuristic, and fantasy scenes, objects, and concepts.
	Illustration & Comic Generation	Generates illustrations, cartoons, comics, and multi-panel visual narratives.
	Poster & Graphic Design	Generates posters, covers, advertisements, and other graphic compositions with specified layouts.
	In-Image Text Generation	Renders specified text with correct content, legibility, appearance, and placement.
	Position-Constrained Generation	Places objects or visual elements in specified image regions or locations.

Appendix

Table 9: Generative capability space

\mathcal{G}

for self-directed inquiry.

Field	Description
Modality	Specifies whether the workflow supports T2I or I2I generation.
Capability Categories	Records the capabilities in $\mathcal{G}$ supported by the workflow.
Workflow Template	Stores a reusable ComfyUI graph with node types, shared parameters, connections, and execution settings. Instance-specific inputs are removed and supplied when the policy is instantiated for a new task.
Workflow Name	Provides a unique identifier for indexing, retrieval, and invocation.
Workflow Description	Summarizes the shared procedure and the task family it supports.
Preconditions	Specifies required input modalities, image properties, task conditions, available models, and execution resources.

Appendix

Table 10: Schema of the workflow library

\mathcal{L}_{t}

Entry Type	Field	Description
Execution Failure Record	Task $\tau_{t}$	Stores the description $q_{\tau_{t}}$ , modality $m_{\tau_{t}}$ , required capabilities $\mathcal{G}_{\tau_{t}}$ , and source image $x_{\tau_{t}}$ when applicable.
	Failure Context	Records the input conditions and generation constraints under which the failure occurred.
	Failed Workflow $\pi_{t}$	Stores the failed task’s ComfyUI workflow, including nodes, parameters, connections, and execution settings.
	Failure Evidence $o_{t}$	Preserves execution status, intermediate and final outputs, source images when applicable, execution feedback, and verifier results.
Corrective Strategies	Root Cause	Identifies the workflow or generation factor responsible for the failure.
	Workflow Antipattern	Describes workflow structures, node configurations, parameter settings, or generation strategies to avoid.

Appendix

Table 11: Schema of the failure library

\mathcal{F}_{t}

Field	Description
Modality Coverage	Counts stored T2I and I2I workflows to identify imbalances in modality coverage.
Capability Coverage	Counts workflows associated with each capability in $\mathcal{G}$ to identify gaps in workflow coverage.
Reliability Distribution	Counts Provisional , Validated , and Suspended workflows in $\mathcal{L}_{t}$ .
Source Image Pool $\mathcal{X}$	Contains all original ComfyBench images for I2I practice. Video inputs are excluded.
Source Image Metadata	Records image file names, resolutions, and formats to support image selection and task generation.

Appendix

Table 12: Fields of the current scene context

c_{t}

Figure 6: Visualization of the workflow library after 50 iterations of self-directed inquiry. (a) Word cloud summarizing the capabilities represented by stored policies. (b) Distribution of workflow entries across capability categories.

Figure 7: Resolve rates with different numbers of self-directed inquiry iterations on ComfyBench [ 65 ] .

Figure 8: Symbolic policy learning during downstream execution on ComfyBench [ 65 ] . (a) Growth in total and validated workflow entries in the workflow library, starting after 50 inquiry iterations and continuing through the Vanilla, Complex, and Creative subsets. (b) Changes in the proportions of Validated , Provisional , and Suspended workflows before and after ComfyBench execution.

Figure 9: Evolution and reuse of symbolic policies during downstream execution on ComfyBench [ 65 ] . (a) Expansion of the image-only workflow library acquired after 50 inquiry iterations to include T2I, I2I, T2V, I2V, and V2V workflows after ComfyBench execution. (b) Distribution of direct reuse, compositional reuse, and no-prior-match cases among resolved Vanilla, Complex, and Creative tasks, together with the overall distribution.

Figure 10: Capability distribution shift in the image workflow library during downstream execution on ComfyBench [ 65 ] . The bars compare workflow proportions across capability categories after 50 inquiry iterations and after ComfyBench execution. Percentages are normalized over T2I and I2I workflows only. The comparison shows how the capabilities represented by stored symbolic policies change during downstream execution.

Figure 11: Reasoning efficiency of OmniHarness on the 200 ComfyBench tasks [ 65 ] , reported over consecutive groups of 20 tasks. The bars show the average token consumption per task, while the line shows the average reasoning time per task. Tasks 1–100, 101–160, and 161–200 correspond to the Vanilla, Complex, and Creative subsets, respectively. Reasoning time includes agent-side planning, workflow synthesis, verification, and recovery, but excludes visual generation model execution.

Agent	Vanilla		Complex		Creative		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
GPT-4o + ComfyAgent [ 65 ]	67.0	46.0	48.3	21.7	40.0	15.0	56.0	32.5
GPT-4o + ComfyMind [ 18 ]	100.0	92.0	100.0	85.0	100.0	57.5	100.0	83.0
Gemini-2.5-Flash + SymbOmni [ 37 ]	100.0	95.0	100.0	83.3	100.0	67.5	100.0	86.0
Gemini-2.5-Flash + OmniHarness	100.0	96.0	100.0	81.7	100.0	87.5	100.0	90.0
GPT-4o + OmniHarness	100.0	95.0	100.0	76.7	100.0	95.0	100.0	89.5

Appendix

Table 13: Backbone ablation of OmniHarness on ComfyBench [ 65 ] . The first three rows report representative existing agents, followed by OmniHarness instantiated with different reasoning backbones and planner configurations. Pass and Resolve are reported as percentages, with higher values indicating better performance.

Agent	Vanilla		Complex		Creative		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
OmniHarness with $\mathcal{X}_{\mathrm{ComfyBench}}$	100.0	97.0	100.0	83.3	100.0	95.0	100.0	92.5
OmniHarness with $\mathcal{X}_{\mathrm{Gen}}$	100.0	97.0	100.0	83.3	100.0	90.0	100.0	91.5

Appendix

Table 14: Source image pool isolation study on ComfyBench [ 65 ] .

\mathcal{X}_{\mathrm{ComfyBench}}

denotes the original ComfyBench source image pool, while

\mathcal{X}_{\mathrm{Gen}}

replaces it with independently generated images from the same generative capability space.

Figure 12: Qualitative comparison on representative I2I tasks from ComfyBench [ 65 ] . The examples illustrate target localization, instruction following, and preservation of unrelated scene content.

Figure 13: Qualitative comparison on representative GenEval [ 17 ] tasks covering single-object generation, two-object co-occurrence, counting, color, spatial relations, and attribute binding.

Figure 14: Qualitative comparison on representative GenEval2 [ 25 ] tasks covering object generation, attribute rendering, counting, spatial relations, and transitive verb relations.

Figure 15: GPT-score comparison on Reason-Edit [ 23 ] across seven reasoning-intensive image-editing categories. Higher scores indicate better performance.

Figure 16: Qualitative comparison on representative WISE [ 41 ] tasks spanning cultural, temporal, spatial, biological, physical, and chemical knowledge.

Closed-Source Models
Method	Factual $\uparrow$	Conceptual $\uparrow$	Procedural $\uparrow$	Overall $\uparrow$
GPT-Image-1	79.80	81.37	78.32	80.09
Gemini 2.0 Flash Experimental	65.26	59.65	62.90	62.41
Doubao	63.30	62.23	54.17	60.70
Open-Source Models
BAGEL-Think [ 10 ]	55.77	59.44	39.26	53.36

Appendix

Table 15: Quantitative comparison on KRIS-Bench [ 58 ] . Scores evaluate factual, conceptual, and procedural knowledge, together with overall performance. Higher values indicate better performance.

Figure 17: Qualitative results of OmniHarness on representative KRIS-Bench tasks [ 58 ] , covering factual, conceptual, and procedural knowledge through knowledge-guided generation, editing, visual reasoning, and multi-image composition.

Agent	Vanilla		Complex		Creative		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
ComfyAgent [ 65 ]	67.0	46.0	48.3	21.7	40.0	15.0	56.0	32.5
ComfyMind [ 18 ]	100.0	92.0	100.0	85.0	100.0	57.5	100.0	83.0
SymbOmni [ 37 ]	100.0	95.0	100.0	83.3	100.0	67.5	100.0	86.0
ComfyAgent [ 65 ] + $\mathcal{K}_{\mathrm{inquiry}}$	87.0	69.0	73.3	46.7	67.5	42.5	79.0	57.0
ComfyMind [ 18 ] + $\mathcal{K}_{\mathrm{inquiry}}$	100.0	94.0	100.0	86.7	100.0	75.0	100.0	88.0

Appendix

Table 16: Quantitative plug-and-play evaluation on ComfyBench [ 65 ] . The first three rows report the original host agents, while the last three augment them with the same frozen policy snapshot

\mathcal{K}_{\mathrm{inquiry}}

learned through self-directed inquiry, without model fine-tuning.

Agent	T2V		I2V		V2V		Total
Agent	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$	Pass $\uparrow$	Res. $\uparrow$
GPT-4o + ComfyAgent [ 65 ]	68.4	47.4	48.0	20.0	40.0	13.3	56.4	32.1
GPT-4o + ComfyMind [ 18 ]	100.0	71.1	100.0	76.0	100.0	53.3	100.0	69.2
Gemini-2.5-Flash + SymbOmni [ 37 ]	100.0	81.6	100.0	88.0	100.0	66.7	100.0	80.8
GPT-4o + OmniHarness	100.0	84.2	100.0	92.0	100.0	80.0	100.0	85.9

Appendix

Table 17: Quantitative cross-modal generalization results on the video-generation subset of ComfyBench [ 65 ] . T2V, I2V, and V2V denote text-to-video, image-to-video, and video-to-video tasks, respectively. GPT-4o + OmniHarness uses the frozen policy snapshot

\mathcal{K}_{\mathrm{inquiry}}

, learned through self-directed inquiry in an image-only capability space without exposure to video tasks during inquiry.

Figure 18: Qualitative cross-modal generalization results of OmniHarness on representative video-generation tasks from ComfyBench [ 65 ] . Sampled frames illustrate generated content and source-content preservation in T2V, I2V, and V2V tasks using the frozen policy snapshot $\mathcal{K}_{\mathrm{inquiry}}$ learned through image-only inquiry.

Figure 19: Reuse of symbolic policies learned through image-only inquiry during cross-modal generalization on ComfyBench [ 65 ] . The bars report invocation counts of workflow entries from the frozen image-only workflow library for T2V, I2V, and V2V tasks.

Figure 20: Web interface used in the 40-task user study. Participants assess an anonymized generation result against the task instruction and, when required, the input image. Ratings cover task description alignment, input-structure fidelity, photorealism, visual quality, and overall satisfaction on a five-point Likert scale.

Figure 21: User-study comparison across five evaluation criteria. Mean ratings over 40 ComfyBench [ 65 ] tasks are reported on a five-point Likert scale, with higher scores indicating better performance.

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 389 papers

OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
Oct 19, 2026Yu Zhou, Cheng-Fu Yang, Rui Sun +8Tool Use in VLMs Multimodal Reasoning
University of California, Los Angeles · Apple
SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning
Jul 13, 2026Jinxiu Liu, Jianru Li, Tanqing Kuang +4Memory-Augmented Neural Networks Agentic Image Generation
South China University of Technology · Westlake University · Johns Hopkins University +2
VEvo-Harness: Causally-Guarded Self-Evolving Harness for Agentic Image Generation
Oct 29, 2026Yuan Wang, Yuhang Yu, Mengting Chen +4Agentic Image Generation Self-Improving Agents
Taobao & Tmall group of Alibaba · Tsinghua University

Oct 19, 2026J/K move · Enter open · S save

OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory

Yu Zhou, Cheng-Fu Yang, Rui Sun, Di Wu+7

University of California, Los Angeles · Apple

Tool Use in VLMs Multimodal Reasoning VLM Reasoning MLLM Reasoning Memory-Augmented VLMs

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have made rapid progress on visual reasoning, yet many tasks require evidence that is difficult to obtain through internal reasoning alone: our controlled study on 22 benchmarks shows that simply increasing the inference budget of the base model yields only modest gains. Such tasks demand grounded perception, external knowledge, and numerical measurements obtained through heterogeneous tools, whose outputs must be made interpretable to the model and retained across reasoning steps. We introduce OpenVLHarness, a multimodal reasoning harness organized around a capability layer and persistent multimodal memory. The capability layer combines task-level tool interfaces with evidence rendering, providing consistent access to heterogeneous backends and converting their outputs into model-readable observations. Persistent multimodal memory retains the corresponding images, structured data, and web references as addressable artifacts throughout each query. A coding interface executes generated programs over stored data, enabling computations that combine results from multiple tools. Together, these mechanisms support an iterative process of acquiring, inspecting, and composing multimodal evidence. Evaluated on 22 benchmarks with eight open and proprietary backbones, OpenVLHarness improves average scores by 9.7-12.6 points over matched no-tool baselines, including +11.2 with Qwen3-VL-8B, +9.7 with Kimi K3, and +9.9 with GPT-6 Astra. With GPT-6 Luna at medium reasoning effort, OpenVLHarness outperforms the base model, Codex, and Visual Sketchpad at maximum effort by 7.7-8.5 points while incurring at most 37% of their model-token cost.

Abstract

LLM Agent Harnesses

Explore similar work

OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory

SymbOmni: Evolving Agentic Omni Models via Symbolic Concept Learning

VEvo-Harness: Causally-Guarded Self-Evolving Harness for Agentic Image Generation