Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

Zekai Li; Jiayi Yu; Jinlu Zhang; Luyang Niu; Zekun Qi; Jun Zheng; Xuchuan Chen; Dairu Liu

cs.ROOct 19, 2026

Where is Physical RSI capability from? An Observational Scaling Law of LLM-as-policy

Authors: Zekai Li, Jiayi Yu, Jinlu Zhang, Luyang Niu, Zekun Qi, Jun Zheng, Xuchuan Chen, Dairu Liu,

Organizations: Galbot Team

Abstract

In this paper, we systematically investigate the mechanisms, scaling behavior, and generalization of physical Recursive self-improvement. We introduce PhysRSI-eval to quantify physical RSI through a five-axis metric: First-Attempt Performance, Efficiency, Success, Budget and Improvement from failure. Different VLMs are evaluated on 18 manipulation tasks under a fixed interaction budget. The analysis of experimental results motivates the Physical Barrel Hypothesis: different capabilities may constrain different parts of improvement. The five-axis metrics also distinguish systems with close public benchmark averages and show that high scores need not come with early or efficient gains. We study observational scaling from two perspectives: predicting the relative performance of VLM agents from their public capabilities, and forecasting later progress from early physical feedback. Together, these analyses ask how quickly and how far an agent can improve. Moreover, we conduct workspace interventions and then analyze what improvement leaves behind. Removing history reduces late-round success, frozen workspaces remain useful on unseen layouts, and three cross-task cases achieve success earlier. The results suggest that scene-specific adaptation and reusable experience can coexist.

Figures & tables

Figure 1: PhysRSI-eval at a glance. (Q1) Capability sources and the Physical Barrel Hypothesis. (Q2) Five-axis quantification and observational scaling of performance, gain timing and model rankings. $T_{50}$ is the half-gain round, the first round at which the best mean score reaches half of its K1–K17 gain. (Q3) Overfitting and generalization across tasks and scenes.

Figure 2: Two stages, one interface. Top: the task input and robot interface shared by both stages (observations, physical actions, memory and files, history). Left, Evaluate (first-attempt round K0): one in-context episode of observe–reason–act through the shared tools, with no workspace; history and notes stay inside the episode. Right, Optimize (self-improvement rounds K1–K $H$ ): each round runs from a persistent workspace of skills, tools, policy and memory, is graded natively, revised with robot actions closed, and committed as the start of the next round, until the first native success or the round limit.

Figure 3: Difficulty-balanced task sampling. Three tasks per difficulty stratum, from D1 (highest official RoboDojo success rate) to D6 (lowest), covering RoboDojo’s five official task categories. Strata are assigned by ranking the tasks on the official RoboDojo success rate [ 3 ] and are fixed before the main evaluation; they do not describe the subsequent refinement outcomes.

Figure 4: Capability associations and the barrel hypothesis. The matrix reports exploratory rank associations across six systems on the 18 tasks. Each barrel represents one RSI axis, with a stave per capability group. Stave heights illustrate a proposed bottleneck interpretation of the associations.

System	Tasks	A1	A2	A3	A4	A5
GPT-6 Astra	18	0.3676	0.8594	0.8509	0.8429	0.7500
GPT-6.1 Sol	18	0.2843	0.8224	0.7028	0.8081	0.5714
GPT-6 Sol	18	0.0306	0.4658	0.4722	0.3718	0.3889
GPT-5.6 Sol	18	0.0120	0.2247	0.1037	0.3013	0.0556
Claude Opus 5.5	18	0.1769	0.6091	0.7444	0.5794	0.6875
Claude Sonnet 5.5	18	0.1241	0.6946	0.7006	0.6984	0.5625

Table 1: Five-axis profiles. K0 scores are averaged over three runs per task. A1–A5 denote First-Attempt Performance, Efficiency, Success, Budget and Improvement. ∗ Gemini 4 is evaluated on ten tasks with one K0 run per task.

Figure 5: Five-axis profiles. Left: seven overlaid profiles. Right: one panel per system. Axes clockwise from the top are First-Attempt Performance (A1), Budget (A4), Success (A3), Efficiency (A2) and Improvement (A5), on a shared 0–1 scale. The six reference systems use all 18 tasks, as in Table 1 ; ∗ Gemini 4 uses its ten evaluated tasks ( Section 3.3 ).

Figure 6: Public capability and physical RSI separate differently. Left: cumulative success during refinement on 18 tasks; the shaded region marks the gap between GPT-6.1 Sol and GPT-6 Sol. Right: mean public score over 17 selected benchmarks versus

100\times(A_{1}+A_{2}+A_{3}+A_{4}+A_{5})/5

. The physical summary is an arithmetic mean, not radar area.

Figure 7: Predicting the timing of gains from public capability profiles. Left: four capability profiles of each system, computed from 17 public benchmarks as normalized ranks among the six systems. Right: best mean score $\bar{q}_{k}$ on the 18 tasks. Circles mark $T_{50}$ , the first round at which $\bar{q}_{k}$ reaches half of its K1–K17 gain. Labels give predicted / observed ranks: $T_{50}$ beside the circles and best score reached by K17 on the right. Each pair of systems is predicted from the other four.

Figure 8: Forecasting GPT-6.1 Sol from Astra’s improvement shape. Left: Astra’s mean score and the Weibull template fitted to it. Right: GPT-6.1 Sol with K1–K5 available. The template is calibrated on these rounds and forecasts the hidden rounds through K17; shading shows the 90% paired-task bootstrap band. All 18 tasks are used.

Figure 9: Cross-task workspace transfer. Top: original self-improvement trajectories for tasks first solved at K6 or later. Bottom: trajectories initialized from a workspace pooled from all other tasks, excluding the target task. First success advances from K9 to K1 for Sonnet 5.5 on Cover Blocks, from K14 to K7 for Opus 5.5 on Organize Table, and from K15 to K8 for Opus 5.5 on Deposit Coin. These examples illustrate faster adaptation with transferred workspaces. Red crosses indicate unsuccessful attempts, dark checks mark first success, and pale checks carry success forward without additional evaluation.

Figure 10: Tool Libraries and Reuse across Models. (a) Total tool counts and the proportions of perception and geometry tools versus action tools. (b) Fraction of tool lineages reused in subsequent rounds, among those with later rounds available. Reuse includes direct use, modification and derivation of existing tools. (c) Reuse breadth, defined as one minus the share of subsequent calls accounted for by the top 10% of reused lineages. Higher values indicate reuse distributed across more lineages. Astra is omitted because it creates very few tools, used only for verification. GPT-6.1 Sol’s estimates reflect a small library of seven tools.

#	Task	Steps	Scoring (tiers, %)
1	arrange_largest_number	1050	graded [5,15,30,100] or [5,15,25,40,100]
2	classify_objects_by_language	1100	graded [10,40,100]
3	cover_blocks	800	graded [0,0,5,15,30,100]
4	deposit_coin	300	graded [20,100]
5	fill_pen_holder	1100	graded [10,25,40,90]
6	insert_key	300	graded [15,100]

Table 2: The 18 PhysRSI-eval tasks. Step limits are per round (25 Hz). Graded tasks score the highest tier reached (tiers in percent of the native score); binary tasks score 0 or 1. The success check is independent of the tiers.

Item	GPT-6 Astra	GPT-6.1 Sol	GPT-6 Sol	GPT-5.6 Sol	Opus 5.5	Sonnet 5.5
Client	Codex CLI 0.156.1	Codex CLI a	Codex CLI 0.156.1	Codex CLI 0.156.1	Claude Code a	Claude Code a
Reasoning effort	xhigh	xhigh a	xhigh	xhigh	high a	xhigh
Reasoning budget per round	4200 s	4200 s a	4200 s	4200 s	4200 s a	4200 s
Revision window	1800 s	1800 s a	1800 s	1800 s	600 s a	600 s
Selected K1–K17 entries	91	143	255	306	175	175
Counted extra attempts	13	9	35	12	21	40

Table 3: System configurations and evaluation coverage. Each system comprises a model and its runtime configuration, so cross-system comparisons are descriptive. a Settings from session logs; other settings come from configuration files.

System	$q_{0}$	$q_{17}$	$q_{\max}$	$\tau_{90}$	$C_{1}$	$C_{17}$	rec.
GPT-6 Astra	0.368	0.842	0.869	9	0.333	0.833	9
GPT-6.1 Sol	0.284	0.689	0.728	7	0.222	0.667	8
GPT-6 Sol	0.031	0.472	0.472	17	0.000	0.389	7
GPT-5.6 Sol	0.012	0.104	0.104	17	0.000	0.056	1
Opus 5.5	0.177	0.744	0.744	15	0.111	0.722	11
Sonnet 5.5	0.124	0.700	0.708	13	0.111	0.611	9

Table 4: Curve summaries for A1–A5 (18 tasks).

q_{0}

and

q_{17}

are the task-mean scores at K0 and K17,

q_{\max}

the peak,

\tau_{90}

the first round reaching 90% of the peak gain,

C_{1}

and

C_{17}

the fractions of tasks solved by K1 and K17, and “rec.” the number of tasks failed at K1 and solved later.

Task	Astra	6.1 Sol	6 Sol	5.6 Sol	Opus	Sonnet
Arrange Largest Number	1	2	17	–	3	2
Classify Objects by Language	1	1	–	–	4	2
Cover Blocks	1	3	4	17	10	9
Deposit Coin	4	–	–	–	15	–
Fill Pen Holder	3	8	–	–	–	–
Insert Key	–	–	–	–	–	–

Table 5: First successful Harness round on each task. A dash denotes no success within K17.

Group	Benchmark item	Source
Spatial	Object detection (mAP@50)	Roboflow
	OCR text localization	Roboflow
	Visual grounding	ParseBench
Harness	ctr-optimization	Terminal-Bench 4.0
	shadow-relay	Terminal-Bench 4.0
	Instruction following	LiveBench

Table 6: Public benchmarks behind the capability profiles. Scores were collected on 15 October 2026; each group averages normalized benchmark ranks with equal weight.

Name	Condition	Workspace	Grader	Images	Actions
Full	Full loop	✓	✓	✓	✓
No-img	No earlier-round images	✓	✓	✗	✓
No-grader	No grader feedback	✓	✗	✓	✓
WS+FB	Workspace and grader only	✓	✓	✗	✗
No-hist	No history	✗	✗	✗	✗
Fixed	Full loop, fixed layout	✓	✓	✓	✓

Table 7: Information from earlier rounds in each ablation condition. Workspace: committed notes, skills and code. Grader: result, score and failure reason after each round, and earlier-round outcome files; step-level success flags in action replies and logs are not part of this channel. Images and actions: observation images and action logs from earlier rounds. Current-round observations, including camera images, are available in every condition.

$K$	Full	Fixed	No-img	No-grader	WS+FB	No-hist
1	1	1	1	0	0	1
2	2	3	1	2	1	2
3	6	6	3	2	2	1
4	6	8	4	2	7	0
5	5	7	3	2	3	0
6	4	7	5	1	7	3

Table 8: Per-round online success in the Astra ablation. Successes out of 12 chains per condition and round (K1–K6 run in full).

Condition	Semantics	Sources	Succ./ $n$	Rate
E3-full	Full-loop workspaces; test agent may read prior-round raw records	12	23/60	38.3%
E3-notes	Full-loop workspaces; workspace files only	12	25/60	41.7%
HO-Fixed	Fixed workspaces on new layouts	12	27/60	45.0%
HO-NoImg	No-img workspaces; incomplete	9 (6)	13/35	37.1%
HO-NoGrader	No-grader workspaces; incomplete	6 (1)	5/21	23.8%

Table 9: Frozen held-out test of Astra K6 workspaces. 12 sources

\times

5 held-out layouts for the complete conditions; HO-NoImg and HO-NoGrader are incomplete and not comparable with the complete conditions.

Self-Improving Agents

Self-improving agents use experience and feedback to update their models, memories, skills, or workflows for better future task performance.

See all 167 papers

Toward Recursive Self-Improvement in Physical AI
Oct 21, 2026Meng Luo, Zechen Bai, Shanqing Xu +9Recursive Self-Improvement Robot Learning
National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University +2
Who Improves the Robot? Toward Physical Recursive Self-Improvement
Oct 13, 2026Hai Zhou, Fan Yang, Zeyi Chen +27Recursive Self-Improvement Continual Robot Learning
Shanghai Jiao Tong University · Google · Microsoft +13
AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents
Oct 14, 2026Peidong Wang, Ying Chang, Zhiming Ma +8Self-Improving Agents Robotics
Northeastern University · AgiBot · University of Chinese Academy of Sciences

Oct 21, 2026J/K move · Enter open · S save

Toward Recursive Self-Improvement in Physical AI

Meng Luo, Zechen Bai, Shanqing Xu, Yanlin Li+8

National University of Singapore · Huazhong University of Science and Technology · Shanghai Jiao Tong University+2

Recursive Self-Improvement Robot Learning

Open paper AskPDF arXiv

Recursive self-improvement (RSI) describes a system that improves not only its behavior but also the process by which it improves. Recent digital agents test this idea by rewriting their own code and checking each version with executable tests. Bringing these mechanisms into the physical world requires learning from partial observations through costly trials, while accounting for changing operating conditions and potentially irreversible consequences. We position RSI within robot learning and trace pathways from improving robot capabilities to improving the processes that generate them. We examine how physical experience can strengthen these improvement mechanisms and when the resulting gains can support further self-improvement of the mechanisms themselves. Gains carry forward only when feedback addresses remaining learning needs, the receiving component can use the change, its value survives changes in tasks or hardware, and continued practice remains safe and affordable. Existing studies, concentrated in manipulation, support behavioral improvement, benefits to later learning, and procedure improvement. Some systems also change the improver's knowledge or tools, but the physical studies reviewed here have yet to isolate gains attributable to recursion. We synthesize these findings into evaluation principles and research directions for machines that, trial by trial, safely improve the way they improve.

Abstract

Self-Improving Agents

Explore similar work

Toward Recursive Self-Improvement in Physical AI

Who Improves the Robot? Toward Physical Recursive Self-Improvement

AdaRSI: On-Demand Recursive Self-Improvement for Embodied Agents