VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning

Yingbo Ma; Kuangshi Ai; Gordon Dai; Chenglong Yu; Chaoli Wang; Hua Shen

cs.AIOct 9, 2026

VETR: Execution-Grounded Self-Improving Agent for Symbolic Graphics Reasoning

Authors: Yingbo Ma, Kuangshi Ai, Gordon Dai, Chenglong Yu, Chaoli Wang, Hua Shen

Organizations: New York University · University of Notre Dame

Abstract

Visual agents for symbolic graphics reasoning depend on executable world representations, yet continued improvement remains bottlenecked by ambiguous supervision from final renders. Recent agentic systems refine scenes through repeated interaction, but they lack mechanisms to recursively extract actionable evidence from execution traces, overlooking which proposed symbolic edits survived, disappeared, or interfered during program execution. To address this, we propose VETR (Verified Execution Trajectories for Refinement), a framework that enables agentic recursive self-improvement over executable world representations for symbolic graphics reasoning. Within each task, VETR retains aligned program/render records with bounded runtime observations and introduces an execution refiner that compiles observed writes and state changes into constrained alternative programs. The refiner validates their endpoints and preservation conditions against captured runtime state; the ordinary visual verifier then assesses goal relevance. Repair construction makes no additional language-model calls, but can require additional executions. Evaluations on BlenderGym (280 tasks) and BlenderBench (27 released tasks) with GPT-5.6-luna and Qwen3.8-27B show that VETR consistently outperforms existing refinement strategies, including BlenderAlchemy and VIGA. Controlled ablations evaluate the contributions of execution refinement and role-asymmetric evidence organization under fixed generator-interaction caps.

Figures & tables

Figure 1: Conceptual overview of VETR. VETR refines scenes through execution-grounded evidence, bridging a scene program, its execution trace, and the realized 3D scene. Given a start and goal image, the system performs iterative updates guided by observed writes and realized bindings. This evidence-supported refinement targets viewpoint alignment, appearance (lighting/material) editing, and geometry and placement adjustments. Runtime checks test supported effects, while visual feedback guides subsequent proposals toward the goal.

Figure 2: Execution-evidence refinement before visual assessment. Each complete program executes from $B$ . The private refiner constructs alternatives, compiles supported changes and compatible bindings, and checks their realization relative to the candidate’s reference record. For the numerical channel, $W_{h}$ contains selected cells, $\mathcal{C}_{h}\setminus W_{h}$ contains captured cells to preserve, and properties outside $\mathcal{C}_{h}$ are not covered by that contract. Goal-image comparison occurs only in the ordinary visual verifier. Delivery, shortcuts, and the six-execution cap are specified in Algorithm 1 . Repair construction makes no additional language-model calls.

Model	Setting	All		Shape		Placement		Geometry		Lighting		Material		Impr. (%) $\uparrow$
Model	Setting	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	Impr. (%) $\uparrow$
GPT-5.6-luna	One-Shot	1.389	2.583	0.460	1.629	0.955	2.966	4.350	5.754	1.906	1.555	2.109	5.237	–
	BlenderAlchemy	5.675	9.032	1.176	3.567	0.895	4.993	10.506	14.553	9.969	11.421	11.429	17.588	(236.65)
	VIGA	1.354	4.017	0.265	0.864	0.551	1.608	4.242	10.977	0.916	1.161	1.276	7.155	16.92
	VETR	1.034	2.397	0.246	0.775	0.527	1.270	2.914	4.660	0.495	0.681	1.326	5.661	41.22
Qwen3.8-27B	One-Shot	2.160	3.520	0.522	1.849	0.923	2.820	3.956	6.201	1.748	1.697	5.978	7.412	–

Table 1: Visual editing performance on BlenderGym across editing families.

PL

and

N\text{-}CLIP

are scaled by 100 (

\downarrow

). Parentheses indicate negative improvement.

Model	Setting	Level 1		Level 2		Level 3		Impr.
		$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	$N\text{-}CLIP\downarrow$	$\uparrow$
GPT-5.6-luna	One-Shot	10.9403	14.8074	2.7780	4.4044	14.3415	26.3542	—
	VIGA	11.2304	14.7821	5.3481	5.0353	21.7647	22.8812	$(24.65\%)$
	VETR	6.3523	10.1947	0.6519	2.7021	5.3922	6.3201	$\mathbf{+54.45\%}$
Qwen3.8-27B	One-Shot	12.9096	19.3977	3.9448	5.1091	18.5005	36.1053	—
	VIGA	9.4124	14.2022	4.5780	4.8665	13.6108	15.1571	$+21.17\%$

Table 3: Visual editing on native BlenderBench by level. Improvement is the mean relative reduction from the same-backbone One-Shot row over the six level–metric cells, as defined in Eq. ( 5 ).

Figure 3: Qualitative comparison of visual editing. Columns display the goal, start, VIGA final output, and VETR final output across three editing categories: light (color), geometry (chair position & orientation), and shape (facial expression). Red arrows and boxes highlight local regions of comparison. Reported values denote PL $\times 100$ (lower is better). VETR achieves near-zero PL in the displayed light and geometry tasks and lower error in the displayed shape task. These output comparisons illustrate visual fidelity, not an independent verification of the refiner’s runtime mechanism.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Variant	$n$	Interactions	Model calls	Blender renders	Agent time (s)
VETR $N=1$	277	8.74	18.58	7.16	204
VETR $N=4$	280	8.76	66.17	28.56	733

Appendix

Table 5: Recorded cost in the GPT Blender 3.6 cohort. Means use tasks with saved cost records (

n

); completion is reported separately. Quality for both widths is reported in Table 9 .

Variant	Metric	$n$	Default	Ablated	Red. (%) [CI]	W/L/T	$p$
w/o full history (3 msgs)	$N\text{-}CLIP$	280	2.603	3.865	$+32.7$ [ $22.5$ , $41.8$ ] †	177/99/4	$3.10\!\times\!10^{-6}$
	$PL$	280	0.996	1.784	$+44.2$ [ $25.7$ , $56.9$ ] †	183/97/0	$3.08\!\times\!10^{-7}$
w/o role asymmetry	$N\text{-}CLIP$	280	2.603	2.935	$+11.3$ [ $3.7$ , $23.5$ ] †	164/113/3	$.0026$
	$PL$	280	0.996	1.233	$+19.2$ [ $2.9$ , $31.7$ ] †	169/108/3	$.00030$
w/o visual verifier	$N\text{-}CLIP$	280	2.603	2.774	$+6.2$ [ $-5.6$ , $17.0$ ]	159/118/3	$.016$
	$PL$	280	0.996	1.278	$+22.0$ [ $-4.2$ , $40.8$ ]	161/117/2	$.0098$

Appendix

Table 6: Context ablations in one cohort (GPT-5.6-luna,

N{=}1

, Blender 3.6.0, 280 tasks). Means are scaled by 100. Red.: relative reduction of the default mean error against the ablated mean (positive favors the default); the main text instead expresses each ablated error relative to the default. CI: 95% task bootstrap (5,000 resamples, seed 0). W/L/T: default wins/losses/ties.

p

: exact two-sided sign test excluding ties.

\dagger

: CI excludes zero and

p<.05

Tasks	$n$	Metric $\downarrow$	Rebuild	Persistent	Red. (%) [CI]	W/L/T	$p$
Overall	276	$N\text{-}CLIP$	2.371	3.585	$+33.9$ [ $19.8$ , $45.8$ ] †	154/120/2	$.0460$
Overall	276	$PL$	1.047	1.417	$+26.1$ [ $-2.4$ , $43.6$ ]	147/129/0	$.3062$
Geometry	52	$N\text{-}CLIP$	4.821	10.341	$+53.4$ [ $34.7$ , $67.2$ ] †	35/16/1	$.0110$
Geometry	52	$PL$	3.111	5.039	$+38.3$ [ $-3.4$ , $61.1$ ]	38/14/0	$.0012$

Appendix

Table 7: Scene origin matters under complete-program execution. Both arms retain generator history; only the scene-persistence setting is changed. Means use jointly valid tasks and are scaled by

100

. Red.: reduction of rebuild relative to persistent; CI: 95% task bootstrap (5,000 resamples, seed 0). W/L/T: rebuild wins/losses/ties.

p

: exact two-sided sign test excluding ties;

\dagger

: CI excludes zero and

p<.05

. Both arms complete 278/280 tasks. Family results are in Appendix 5 .

Figure 5: A complete program can compound edits left in a persistent scene. Selected execution attempts from the scene-origin ablation; both arms retain the full generator history. (a) On geometry22 , the persistent trajectory elongates the object until it leaves the visible frame, while rebuild retains a target-like object. Rebuild’s round 9 has no render; its scored answer is round 8 ( $N\text{-}CLIP\times 100=1.44$ ), versus persistent’s round 9 ( $23.52$ ). (b) On geometry20 , the first programs are byte-identical and produce equal scores; later persistent renders become progressively flatter, while rebuild finishes closer to the goal ( $1.29$ vs. $7.19$ ). The shared surface.add_geomod helper creates a new modifier on each call. Appendix P separately executes the identical first program repeatedly and measures modifier counts, mesh volume, and object bounds.

Family	$n$	Metric $\downarrow$	Rebuild	Persistent	Red. (%) [CI]	W/L/T	$p$
Blend-shape	84	$N\text{-}CLIP$	0.934	0.883	$(5.8)$ [ $-49.2$ , $24.5$ ]	43/40/1	$.8264$
Blend-shape	84	$PL$	0.301	0.313	$+4.0$ [ $-26.3$ , $26.9$ ]	42/42/0	$1.0000$
Placement	50	$N\text{-}CLIP$	1.487	1.490	$+0.2$ [ $-25.7$ , $21.3$ ]	26/24/0	$.8877$
Placement	50	$PL$	0.603	0.580	$(3.9)$ [ $-19.6$ , $12.0$ ]	22/28/0	$.4799$
Geometry	52	$N\text{-}CLIP$	4.821	10.341	$+53.4$ [ $34.7$ , $67.2$ ] †	35/16/1	$.0110$
Geometry	52	$PL$	3.111	5.039	$+38.3$ [ $-3.4$ , $61.1$ ]	38/14/0	$.0012$

Appendix

Table 8: Scene-origin ablation by task family. Same definitions and joint criterion as Table 7 . Both means use the paired task set; means are

\times 100

. Intervals bootstrap the relative reduction of mean error.

Variant	Valid	$N\text{-}CLIP\downarrow$	$PL\downarrow$
VETR $N=1$	278	2.616	1.149
VETR $N=4$	280	2.397	1.034
Paired red. ( $n=278$ )		$+7.7$ [ $-4.6$ , $18.4$ ]	$+9.4$ [ $-14.7$ , $26.6$ ]

Appendix

Table 9: Candidate breadth in the GPT Blender 3.6 cohort. Means use each variant's valid tasks; the paired row uses their intersection. Costs are in Table 5 . The saved candidate logs contain 2,453 interactions for

N=4

, with 497 nonzero winners; selected indices 0/1/2/3 occur 1,955/222/194/81 times.

System (paired subset)	$n$	$N\text{-}CLIP\downarrow$	$PL\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	Edge F1 $\uparrow$	LPIPS $\downarrow$	CLIP $\uparrow$
One-Shot (N1 pairs)	212	2.607	1.402	24.67	0.883	0.771	0.129	0.974
VETR $N=1$	212	1.555	0.613	29.15	0.923	0.833	0.076	0.984
One-Shot (N4 pairs)	214	2.583	1.389	25.17	0.884	0.773	0.127	0.974
VETR $N=4$	214	1.220	0.573	30.85	0.924	0.840	0.069	0.988
One-Shot (VIGA pairs)	214	2.583	1.389	25.17	0.884	0.773	0.127	0.974
VIGA	214	2.198	0.877	28.18	0.911	0.812	0.087	0.978

Appendix

Table 10: Matched-task GPT image quality. Each system is paired with One-Shot on their valid-task intersection; the One-Shot reference is recomputed for that exact set. The benchmark errors are scaled by 100. Paired effects and tests are in Tables 11 and 14 .

		$N\text{-}CLIP$			$PL$
Comparison	$n$	Red. (%) [95% CI]	Wins	$p$	Red. (%) [95% CI]	Wins	$p$
GPT-5.6-luna
N1 vs. VIGA	277	$+36.3$ [ $24.3$ , $46.4$ ] †	161	$.0054$	$+16.7$ [ $-9.1$ , $36.6$ ]	149	$.2294$
N4 vs. VIGA	279	$+40.6$ [ $30.2$ , $49.3$ ]	149	$.1845$	$+24.6$ [ $5.6$ , $40.0$ ]	152	$.1506$
N1 vs. One-Shot	212	$+40.3$ [ $29.4$ , $50.5$ ] †	159	$8.41\!\times\!10^{-14}$	$+56.3$ [ $44.6$ , $66.7$ ] †	144	$7.70\!\times\!10^{-8}$
N4 vs. One-Shot	214	$+52.7$ [ $44.9$ , $60.3$ ] †	168	$1.19\!\times\!10^{-18}$	$+58.8$ [ $45.9$ , $68.8$ ] †	156	$2.19\!\times\!10^{-12}$

Appendix

Table 11: Detailed paired comparisons by backbone. N1/N4: VETR

N{=}1/4

; Alchemy: BlenderAlchemy (

2\times 2

for Qwen).

n

: tasks valid for both systems; Red.: relative reduction of the first system's mean error against the second (positive favors the first); CI: 95% task-bootstrap interval (5,000 resamples, seed 0); Wins: strictly lower error;

p

: exact two-sided sign test, excluding ties.

\dagger

: CI excludes zero and

p<.05

(unadjusted). Qwen includes all ten method pairs; BlenderAlchemy uses depth 2, breadth 2, one try, and disabled thinking (Appendix K ).

Metric	w/o refinement	VETR	Red. (%) [CI]	W/L	$p$
$N\text{-}CLIP$	9.453	6.406	$+32.2$ [ $14.4$ , $55.5$ ]	18/9	$.122$
$PL$	7.099	4.132	$+41.8$ [ $6.7$ , $49.0$ ] †	24/3	$4.9\!\times\!10^{-5}$

Appendix

Table 12: Paired effect of execution refinement on BlenderBench (GPT-5.6-luna, 27 tasks). Means are scaled by 100. Red.: relative reduction of the default mean error against the refinement-disabled run (positive favors refinement); CI: 95% task bootstrap (5,000 resamples, seed 0); W/L: per-task wins/losses of the default (no ties);

p

: exact two-sided sign test;

\dagger

: CI excludes zero and

p<.05

Metric	VIGA	VETR	Red. (%) [CI]	W/L	$p$
$N\text{-}CLIP$	14.233	6.406	$+55.0$ [ $29.3$ , $69.8$ ]	19/8	$.052$
$PL$	12.781	4.132	$+67.7$ [ $37.7$ , $74.1$ ]	15/12	$.701$

Appendix

Table 13: Paired comparison with VIGA on BlenderBench (GPT-5.6-luna, 27 tasks). Means are scaled by 100. Red.: relative reduction of the VETR mean error against VIGA; CI: 95% task bootstrap (5,000 resamples, seed 0); W/L: per-task wins/losses of VETR (no ties);

p

: exact two-sided sign test;

\dagger

: CI excludes zero and

p<.05

GPT-5.6-luna					Qwen3.8-27B
Metric	$n$	$\Delta$ [CI]	W/L	$p$	Metric	$n$	$\Delta$ [CI]	W/L	$p$
N1 vs. VIGA					N1 vs. VIGA
PSNR	277	$+1.38$ [ $0.13$ , $2.64$ ]	149/128	$.2294$	PSNR	261	$+1.51$ [ $0.35$ , $2.75$ ] †	152/109	$.0092$
SSIM	277	$+2.7$ [ $1.1$ , $4.4$ ]	149/125	$.1646$	SSIM	261	$+4.1$ [ $2.0$ , $6.3$ ]	146/114	$.0543$
Edge F1	277	$+5.1$ [ $1.9$ , $8.6$ ]	135/110	$.1250$	Edge F1	261	$+4.5$ [ $2.0$ , $6.9$ ]	141/120	$.2157$
LPIPS	277	$+14.1$ [ $5.7$ , $22.0$ ]	150/127	$.1861$	LPIPS	261	$+24.0$ [ $16.3$ , $31.3$ ] †	155/106	$.0029$

Appendix

Table 14: Paired comparisons on the extended metrics. N1/N4 denote VETR at

N=1/4

n

: paired sample size;

\Delta

: PSNR difference in dB and relative improvement (%) for the other metrics, in the first system's favorable direction; CI: 95% task bootstrap (5,000 resamples, seed 0); W/L: per-task wins/losses (ties excluded);

p

: exact two-sided sign test (unadjusted);

\dagger

: recorded CI excludes zero and

p<.05

	One-Shot	VETR $N{=}1$ , round 1
Valid	34	98 / 100
Median retention of start lines	0.10	0.96
Programs retaining $\geq 50\%$ (valid)	8 (7)	99
Programs retaining $<50\%$ (valid)	92 (27)	1
Programs identical to the start program	0	–
Median changed lines	188	–

Appendix

Table 15: One-Shot program form. GPT-5.6-luna on the 100 geometry and material tasks under Blender 3.6, and the first-round program of the same backbone inside VETR

N{=}1

on the same tasks. Valid: tasks with a render for One-Shot; rendered programs / tasks run for round 1. First-error counts name the exception that ended the failed program.

Evidence / source	Tasks	Blender	Configuration and comparison
GPT systems (Table 2 )	280 each	3.6	GPT-5.6-luna; One-Shot, BlenderAlchemy, VIGA, VETR $N=1/4$ . Each system uses its own instructions and runner; BlenderAlchemy uses four candidates per interaction.
Context ablations (Table 6 )	280 each	3.6.0	GPT, $N=1$ ; one default run and three variants (three-message generator history, shared generator/verifier memory, no visual verifier); quality scored after sampling finished.
Scene origin (Table 7 )	280 each	3.6	GPT, $N=1$ ; fixed base vs. saved scene; both retain 24 messages and use identical complete-program instructions.
Guidance (Table 18 )	280 / 85	3.6	GPT, $N=1$ , fixed origin, 24 messages. VIGA role-prompt substitution on 280 tasks; recipe removal on 85 blend-shape tasks. Full is the scene-origin arm above.
Qwen systems (Appendix K )	280 each	3.6	Qwen3.8-27B; all five systems. VIGA/VETR use thinking; BlenderAlchemy uses depth 2, breadth 2, one try, with thinking disabled. Timeout caps differ by system.

Appendix

Table 16: Experiment map. Each row identifies a separate source cohort. Iterative VIGA/VETR runs allow ten generator interactions;

N

is the number of candidate actions per interaction. Within each ablation, the backbone, Blender version, tools, prompts, and budget are fixed except for the named switch. Completion and paired task intersections are reported with the corresponding results.

Metric	First program	Final answer	Reduction (%) [CI]	W/L/T	$p$
$N\text{-}CLIP$	3.922	2.255	$+42.5$ [ $33.7$ , $50.6$ ] †	190/50/16	$2.11\!\times\!10^{-20}$
$PL$	1.710	0.829	$+51.5$ [ $38.4$ , $61.7$ ] †	188/52/16	$2.91\!\times\!10^{-19}$

Appendix

Table 17: Refinement of the same initial proposals. All 256 jointly available tasks; means are scaled by 100. Reduction compares final versus first executed program. Bootstrap, sign tests, and

\dagger

use the same definitions as the main experiments. W/L/T favors the final answer.

Control	$n$	Metric	Full reduction (%) [CI]	W/L/T	$p$
VIGA role prompts	276	$N\text{-}CLIP$	$+9.4$ [ $-5.4$ , $21.9$ ]	148/128/0	$.2527$
	276	$PL$	$(6.7)$ [ $-36.2$ , $14.8$ ]	145/130/1	$.3986$
Without body-skin recipe	83	$N\text{-}CLIP$	$(0.7)$ [ $-24.2$ , $19.4$ ]	42/39/2	$.8243$
	83	$PL$	$+3.6$ [ $-20.4$ , $24.7$ ]	43/40/0	$.8264$

Appendix

Table 18: Guidance controls under the same refinement configuration. Positive reductions favor Full. Complete paired intervals and exact sign tests are shown, with ties excluded from the latter. The no-recipe control is restricted to blend-shape tasks.

Figure 6: The same program has different repeated-execution semantics. Measured modifier count and evaluated mesh volume on geometry20 . Persistent appending creates one additional modifier per execution and halves the current volume each time. Rebuild and explicit modifier replacement coincide at one modifier and constant volume. All three use the same archived program and Blender 3.6.0; the volume is normalized by each policy’s identical first execution.

LLM Self-Refinement

Large language model self-refinement studies how models can improve outputs through repeated critique, verification, revision, and feedback.

See all 114 papers

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning
Oct 9, 2026Jiaqi Liu, Ruinan Jin, Xiaoxiao LiVisual Program Synthesis Visually Grounded Reasoning
Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute
TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning
Jul 31, 2026Binnan Liu, Yechi Ma, Tian Xie +1Visual Reasoning Visually Grounded Reasoning
1Zhejiang University · 2Zhejiang Lab
EGL-SCA: Structural Credit Assignment for Co-Evolving Instructions and Tools in Graph Reasoning Agents
May 11, 2026Zike Yuan, Yukun Cao, Han Zhang +7Program Synthesis LLM Agent Training
Harbin Institute of Technology, Shenzhen, China · School of Computer Science and Technology, Xidian University · Peng Cheng Laboratory, Shenzhen, China

Oct 9, 2026J/K move · Enter open · S save

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning

Jiaqi Liu, Ruinan Jin, Xiaoxiao Li

Department of Electrical and Computer Engineering, University of British Columbia · Vector Institute

Visual Program Synthesis Visually Grounded Reasoning

Open paper AskPDF arXiv

Learning reusable skills can improve visual reasoning agents, yet applying a skill still requires the agent to implement or invoke a procedure and produce the final answer. Errors and hallucinations at these steps can prevent acquired experience from translating into reliable single-run performance. We propose VisualCoding, which combines execution-guided acquisition of reusable visual programs with direct adoption of their answers on applicable inputs. Execution-Guided Program Acquisition composes visual primitives and additional code into family-level solvers, then iteratively tests and revises them on labeled training instances. Scope-Aware Direct Execution runs a retained solver and returns its valid in-scope answer. We evaluate VisualCoding on four models across four benchmarks spanning 69 task families. VisualCoding exceeds the strongest evaluated baseline in mean accuracy by 4.45 percentage points with Gemini-3.1-Pro and by 14.78 points when reusing the frozen Pro-acquired artifacts with Flash-Lite. Ablations support execution-guided acquisition and show that direct execution outperforms textual or callable reuse of the same programs. Code is available at https://github.com/jiaq-liu/VisualCoding.

Abstract

LLM Self-Refinement

Explore similar work

VisualCoding: Learning Executable Skills for Reliable Visual Reasoning

TraceViT: Grounded Trace Supervision for Visual Abstract Reasoning

EGL-SCA: Structural Credit Assignment for Co-Evolving Instructions and Tools in Graph Reasoning Agents