RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization

Jiawei Lu; Chen Zhang; Jinwei Lu; Yuanwei Song; Yuanfeng Song; Raymond Chi-Wing Wong

cs.AIOct 26, 2026

RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization

Authors: Jiawei Lu, Chen Zhang, Jinwei Lu, Yuanwei Song, Yuanfeng Song, Raymond Chi-Wing Wong

Organizations: The Hong Kong Polytechnic University, Hong Kong, China · The Hong Kong University of Science and Technology, Hong Kong, China

Abstract

Users often request charts without knowing which database or table holds the relevant data. Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts. A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.

Figures & tables

Figure 1. Three challenges in real-world text-to-visualization. The assistant agent needs to discover relevant databases, resolve ambiguous requests before charting, and refine the chart across dialogue turns. Three side-by-side scenarios: discovering databases for a salary chart, clarifying gross versus net sales, and refining a regional sales chart over three dialogue turns.

Dataset	Scale	Domain	Target	Open- domain	Cross- domain	Multi-turn dialogue	Typed ambiguity	Cross- database a
Text2Vis ( Rahman et al., 2025 )	1985 NLQs	103	Python	✗	✗	✓	✓	✗
nvBench 2.0 ( Luo et al., 2025 )	7,878 NLQs	153	Vega-Lite	✗	✗	✗	✓	✗
TACO ( Deng et al., 2026 )	14,500 NLQs	31	SQL	✓	✗	✗	✓	✓
Dial-NVBench ( Song et al., 2024 )	7,244 dialogues	105	VQL	✗	✗	✓	✗	✗
CoSQL ( Yu et al., 2019a )	3,007 dialogues	138	SQL	✗	✓	✓	✓	✗
MultiVis-Bench ( Lu et al., 2026c )	1,202 NLQs	105	Python	✗	✗	✗	✗	✗

Table 1. Comparison of ODAC-Vis with existing benchmarks.

Figure 2. Mean dataset quality scores for ODAC-Vis , Dial-NVBench, and NVBench 2.0 on 100 randomly sampled tasks per benchmark. Panel (a) reports expert blind review; panel (b) reports LLM evaluation with the same three dimensions.

Scale and split
Tasks	640	Tables	1,056
Train / val / test	448 / 96 / 96	Visible databases	206
Domains	139	Test domains	15
Complexity tiers
Single-table	192 30.0%
Single-db join	320 50.0%

Table 2. Scale statistics of ODAC-Vis .

Figure 3. An illustrative task of ODAC-Vis . The first request is unambiguous, so the model retrieves the relevant data and writes a bar chart. A later iteration leaves the measure unspecified; the model asks a clarification question and emits the line chart after the user selects weight.

Figure 4. Architecture of RSIV-Harness . The inner task-solving loop answers one assistant turn under the deployed harness, and the outer self-improvement loop revises that harness from traces accumulated across tasks.

Method	Model	Executability	Route.	Stance	Exec Acc.	Clarify	Chart
Direct Prompt	DeepSeek	$81.32$	$57.59$	$62.25$	31.91	$4.32$	$52.23$
Direct Prompt	GPT	$63.04$	$44.36$	$61.00$	$24.51$	$3.02$	$47.46$
Static Harnesses	DeepSeek	$64.98$	$48.25$	$51.25$	$24.51$	$3.72$	$41.46$
Static Harnesses	GPT	$51.75$	$30.35$	$54.00$	$26.07$	$2.02$	$42.01$
ReAct	DeepSeek	$79.38$	$57.59$	$64.25$	$27.24$	$3.93$	$49.57$
ReAct	GPT	$65.76$	$44.75$	$61.75$	$21.40$	$2.27$	$50.37$

Table 3. Overall results on the ODAC-Vis test split across different methods.

Panel of Figure 5 — Figure 5. Case study on a ODAC-Vis task. The user asks for average driver age by driver, and chooses color by make. Direct Prompt and ReAct change the plotted ages and drivers. $H_{0}$ concatenates names from the two sources, and MultiVis-Agent changes the bar heights. RSIV-Harness successfully matches user requirements.

Method	Exec.	Route.	Stance	Exec Acc.	Clarify	Chart
RSIV-Harness	93.00	62.26	67.00	31.52	5.15	75.47
- w/o. rule	$83.27$	$59.14$	$64.50$	$28.02$	$2.41$	$65.94$
H0	$51.36$	$48.25$	$51.25$	$24.51$	$3.72$	$41.46$
+ prompt only	$87.94$	62.26	$64.00$	$29.96$	3.76	$71.13$
+ tool only	$84.82$	62.26	$65.25$	$27.63$	$2.84$	$66.67$
+ memory only	$85.99$	$59.92$	$64.00$	$26.85$	$2.56$	$67.69$

Table 4. Ablation of RSIV-Harness on the ODAC-Vis test split. The first variant removes the stability rules. The remaining variants add one evolved component to

H_{0}

LLM Agent Harnesses

LLM agent harnesses are the software and runtime systems that organize an agent’s tools, context, state, execution, and safeguards.

See all 387 papers

ChartEvo: Self-Evolving Agent Harnesses for Long-Tail Chart Reasoning
Oct 20, 2026Qi Wang, Peijie Wang, Jun Cao +4LLM Agent Self-Improvement LLM Agent Harnesses
State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · New Laboratory of Pattern Recognition, CASIA +1
ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement
Sep 14, 2026Siwei Wu, Jincheng Ren, Yizhi Li +12Coding Agents Agent Harness Optimization
University of Manchester · IQuest Research · M-A-P +3
Harness^2: Recursive Agent Harnessing for an Open World
Oct 9, 2026Ruiyao Xu, Yanfei Chen, Zhongying CuiZhu +6Agent Harness Optimization LLM Agent Harnesses
Northwestern University · Google Cloud AI Research

Oct 20, 2026J/K move · Enter open · S save

ChartEvo: Self-Evolving Agent Harnesses for Long-Tail Chart Reasoning

Qi Wang, Peijie Wang, Jun Cao, Haoyang Tong+3

State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA · School of Artificial Intelligence, University of Chinese Academy of Sciences · New Laboratory of Pattern Recognition, CASIA+1

LLM Agent Self-Improvement LLM Agent Harnesses LLM Agent Skill Learning

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have substantially advanced chart understanding and reasoning, yet their performance remains highly uneven across chart types. While modern MLLMs perform well on common charts, their accuracy drops sharply on long-tail charts with diverse visual structures and reasoning requirements. Tool-augmented agents can improve complex chart reasoning by introducing specialized visual capabilities, but manually engineering strategies and tools for the heterogeneous and evolving long tail is difficult to scale. We introduce ChartEvo, a recursive self-improvement framework that enables chart agents to adapt to underperforming chart types through experience-driven harness-level self-improvement. ChartEvo jointly evolves reusable chart-specific Skills and executable Tools, while learning reusable local tool-call chains to compose evolved capabilities and control the inference overhead of tool-augmented reasoning. On long-tail charts from ChartBench and ChartX, ChartEvo improves accuracy over its base MLLM, Qwen3.7-Plus, by 45.6% and 32.5%, respectively, demonstrating the effectiveness of recursive self-improvement for long-tail chart adaptation.

Abstract

LLM Agent Harnesses

Explore similar work

ChartEvo: Self-Evolving Agent Harnesses for Long-Tail Chart Reasoning

ModularRSI: Modular and Generalizable Recursive Harness Self-Improvement

Harness^2: Recursive Agent Harnessing for an Open World