Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

Kairui Hu; Siyuan Hu; Fangzhou Hong; Zhaoxi Chen; Ziwei Liu

cs.ROOct 8, 2026

Embodied Turing Machines: Stateful Code for Robot Recursive Self-Improvement

Authors: Kairui Hu, Siyuan Hu, Fangzhou Hong, Zhaoxi Chen, Ziwei Liu

Organizations: Nanyang Technological University · Ropedia

Abstract

Most robot policies keep a model in the control loop: a VLA maps observations to actions, and an Agent Harness, such as Agent-as-Policy or Harness VLA queries a VLM for decision making at run time. We propose a different view: the embodied world is an Embodied Turing Machine, whose tape is the robot and environment state and rules are the policy. If this state can be represented accurately, the decision making can be written entirely in code. We therefore propose Code-Only-as-Policy (COAP): code measures and tracks the robot, environment, and task state from camera images and proprioception, and makes every decision from it. The same code applies across episodes, and different tasks share one library without a VLM or VLA in the loop. Compared with VLAs and Agent Harnesses, we analyze three advantages of COAP: (i) Explicit State: the state can be stored in code; (ii) Execution: code makes decision making controllable, recovers from failures flexibly, and runs fast and cheaply online; (iii) Extensibility: new tasks reuse, inherit, or extend the shared library, so capabilities can accumulate over tasks. These advantages make COAP a suitable medium for recursive self-improvement (RSI): coding agents develop the library in a closed loop, and each change is explicit and controllable. On RoboDojo's 42 bimanual tasks, the resulting library reaches a success rate of 70.24% without a model at test time. The upper bound of COAP lies in how accurately the state is represented for decision making and how robust the code logic is. We thus propose COAP as a new paradigm for embodied tasks; since it applies across episodes, it can also serve as an efficient data engine for VLAs and Agent Harnesses.

Figures & tables

Figure 1: Code-Only-as-Policy (COAP). We view the embodied world as an Embodied Turing Machine whose tape is the state of the robot and its environment (top middle). If this state is represented accurately and the code is robust enough, no agent is needed in the control loop (top right). COAP is a shared library: different tasks use the same library, and within a task the code runs across episodes. The policy is therefore code only and reaches a 70.24% success rate on RoboDojo.

Figure 2: Comparison of code-based robot policies. In prior work, a model writes code-as-policy online, turn by turn, as the robot runs (left), or one program takes its state from a learned model or the simulator oracle (middle). Our COAP reuses code across episodes and measures the state by code (right).

	Agent-as-Policy	Harness VLA	VLA / WAM	COAP (ours)
Explicit State
Inspectable	$\triangle$ reasoning text	$\triangle$ memory text	✗ black box	✓ state and rules
Persistent	$\triangle$ model context	$\triangle$ text memory	✗ recent frames	✓ code variables
Accurate	$\triangle$ pixels, some tools	$\triangle$ pixels, some tools	✗ image pixels	✓ measured values
Execution
Controllable	$\triangle$ steered by prompt	$\triangle$ steered by prompt	✗ weights decide	✓ code decides

Table 1: Four ways to build a robot policy with foundation models , compared on the properties of the three advantages in Section 4.3 . ✓ yes,

\triangle

in part, ✗ no. In the rest of the paper, we refer to Agent-as-Policy and Harness VLA together as an Agent Harness.

Figure 3: Code-Only-as-Policy. Development (left): coding agents improve the code offline in a loop. They run the current code $\theta_{k}$ on development layouts, read the state, and run alternative diffs on branches in parallel. Shared library (middle): task programs call the library from the shared code. Execution (right): at runtime the robot runs one fixed task program across episode, and an agent and a VLA are not required.

Method	Memory	Precision	Open	Generalization	Long-Horizon	Overall
	(6)	(8)	(8)	(12)	(8)	SR	Score
Agent Harness
PhysicalRSI [ 15 ] (agent + VLA)	46.6	32.5	24.9	15.6	37.3	31.38	36.27
GPT-6 Astra [ 56 ]	38.7	4.0	31.0	30.5	8.2	22.48	28.97
World Action Model (WAM)
Awomo-0.5 [ 3 ]	41.2	33.5	22.9	23.8	26.8	29.64	35.34

Table 2: Results on RoboDojo. SR is the success rate (%), the share of episodes in which the whole task succeeds; every column except Score reports SR. Score is the benchmark’s progress score.

Figure 4: Three advantages of COAP , each with three properties.

Figure 5: Explicit State. COAP keeps five categories of state as variables (left), remembers states and values (middle), and succeeds more often with more accurate measurements (right).

Figure 6: Execution. Backtracking keeps failures from compounding over long tasks (left, middle), and code runs on a CPU with a small fraction of a model’s compute (right; COAP’s compute as a share of each model’s, from reported model sizes).

Figure 7: Extensibility. New tasks run mostly library code (left), are developed from scratch within hours without data or training (middle; baseline lines are schematic), and add to what the policy can do (right, schematic).

Figure 8: Code structure. (a) The layers of the library. (b) Functions are abstracted at a sensible level: grasping is written by grasp method, not by object. (c) A simplified task program; each call is colored by its layer.

Figure 9: Why Code-Only-as-Policy is a suitable medium for RSI. Each row is an attribute important for RSI, which comes from an advantage in Section 4.3 (check: yes; half circle: partly; cross: no).

Figure 10: Parallel branching over the variants of one decision. (a) Development as a BFS: eight grasps of a mug run at once, and the best code logic is chosen (decisions 2 and 3 are schematic). (b) Of 36 branches, each measured against its parent, few help. (c) With more branches, an accepted change is more likely and larger. (d) With more branches, an accepted change comes sooner in wall-clock time, for more GPU time.

Figure 11: Four classes of error. Each column gives the share of failures in one class, two frames of one failed episode from the official video, and the fix.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Outcome	Share	Failed step (reattempted)	Share	Saved
Reattempted, succeeded	12%	grasp missed or slipped	49%	17%
Reattempted, failed	22%	dropped while carrying	32%	48%
Noticed and reported	42%	stuck on release	18%	23%
no further method	37%	press did not latch	8%	100%
step budget	4%	pour incomplete	1%	100%
Hit the step limit	1%

Appendix

Table 3: What COAP did when something went wrong. Left: outcomes of these episodes. Right: failed steps of the reattempted episodes.

Figure 12: Compute moves offline. COAP uses models only offline; a VLA and an Agent Harness run them online.

Step limit	Tasks	COAP	Agent Harness	VLA	GPT-as-policy
200–500	12	78.1	47.3	21.9	26.5
550–800	15	78.4	30.6	38.0	26.9
900–1900	15	52.0	13.2	22.1	14.8

Appendix

Table 4: Success (%) by the step limit of a task. GPT-as-policy: an Agent Harness without a VLA.

Figure 13: One example of each grasp method.

Method	Approach	Contact	Examples	Tasks
Top	from above	across the body	blocks, digits, a lying pen, eggs	28
Side	level, toward the axis	around the body or neck	a standing bottle, a cup	6
Handle	toward the handle	around the bar	a basket’s arch, a broom, a mallet	3
Rim	down onto the wall	one jaw inside, one outside	a bowl, a mug, a plate	3
Edge	along the thin side	on the two faces	a coin, a bread slice, a key	4
Cloth pinch	down onto the hem	fabric layers	a long-sleeved top	1

Appendix

Table 5: Grasp methods. Tasks: number of task programs using it.

	make_toast	hang_mugs	sweep_blocks	fill_pen_holder
First evaluation	3.2 h	3.2 h	3.9 h	0.2 h
First success	4.7 h	5.2 h	6.4 h	4.0 h
First merge	5.8 h	6.8 h	7.9 h	2.7 h
Success at merges	50 $\to$ 77	7 $\to$ 50 $\to$ 57	0 $\to$ 60 $\to$ 50	0 $\to$ 20 $\to$ 27

Appendix

Table 6: Timeline of the new tasks. Hours from the start, and validation success (%) at each merge.

Figure 14: Extension without breaking. (a) The code each new task runs, by layer. (b) One library function reaches many tasks. (c) sweep_blocks reuses slide, written for push_T.

Layer	Holds	Lines	Used by
Task programs	order, parameters, special cases	23.5k	2.5%
Object families	what an object is, how to hold it	28.5k	18%
Manipulation	pick, carry, place, push; grasping	5.4k	78%
Perception	look, measure, confirm	5.3k	100%
Motion	inverse kinematics, planning, collision	3.3k	100%

Appendix

Table 7: Layers of the library. Used by: share of the 40 task programs that use the layer’s most used module.

Class	Typical cases
Perception error	unrecognized randomized look; wrong final check; split or merged object
Contact error	slip or drop in the grip; push stops short; tip at release
Incomplete logic	no grasp that also reaches the goal; untried order; unchecked collision
Poor code design	fixed size filter; fixed grip-width band; fixed match threshold
Unknown	no lead in the trace

Appendix

Table 8: Typical cases of each class of error.

Task	SR	Score	Task	SR	Score
Memory			Generalization
cover_blocks	100.0	100.0	push_T	97.3	97.3
press_by_number	100.0	100.0	stack_blocks	94.7	95.2
swap_T	100.0	100.0	stack_bowls	93.3	94.0
swap_blocks	100.0	100.0	fold_clothes	90.0	91.6
imitate_sorting_sequence	74.7	78.5	pour_liquid_into_cup	89.3	89.3

Appendix

Table 9: Per-task results of COAP , by dimension. † Task without a program.

Figure 15: Case studies: memory, recovery, and one program in two scenes.

Figure 16: Execution trajectories (1/4). Successful official episodes.

Figure 17: Execution trajectories (2/4). Successful official episodes.

Figure 18: Execution trajectories (3/4). Successful official episodes.

Figure 19: Execution trajectories (4/4). Successful official episodes.

Explore similar work

Jun 15, 2026cs.RO

RHO: Your Coding Agent is Secretly a Roboticist

Code-as-Policies (CaP) has shown that large language models (LLMs) can write code to solve robotics tasks by composing perception, planning, and control primitives. Recent CaP systems, however, rely on multi-turn code-generation loops at test time, which is often infeasible for real-time robot control. We introduce Robotics Harness Optimization (RHO), a novel paradigm in which tool-enabled coding agents, at training time, propose and search for interpretable, neurosymbolic multi-file policy repositories (Repositories-as-Policies) that compose these primitives rather than a single prompt, function, or file. RHO searches with reflective feedback from environment reward and execution rather than teleoperation demonstrations. It generalizes to perturbed pick-and-place settings like LIBERO-PRO, where OpenVLA scores 0.0% and

π_{0.5}

averages 12.83%. Using the same low-level primitives, RHO reaches a 45.0% success rate, 2.5x higher than the strongest multi-turn agentic system, and 3.5x higher than

π_{0.5}

. On Robosuite, RHO sets a new state-of-the-art of 70.0%, exceeding the prior multi-turn record of 68.29% using single-turn execution with no corrective LLM code edits at deployment. When an LLM is used in the control loop, as on RAI's O3DE benchmark, RHO optimizes the deployed agent's multi-file harness of prompts, tools, and control code, improving held-out success from 23.5% to 44.3% with 20% less wall-clock time and 27% fewer tool calls.

Karim Elmaaroufi, Justin Svegliato, Sarunas Kalade +3

University of California, Berkeley · Embodied Science · AMD

Robot Policy Learning AI Coding Agents Large Language Model-Based Robot Planning Program Synthesis Agent Harness Optimization AI Control+2

Ask PDF

Sep 24, 2026cs.RO

RACaP: Agentic Reasoning, Acting, and Coding as Policies for Evolvable Robot Learning

General-purpose robot agents must learn from experience, transfer to new tasks, and act efficiently. Code as Policies (CaP) methods generate and repair programs at runtime, incurring latency and entangling reusable mechanisms with task-specific decisions. We introduce RACaP, an agentic framework that moves coding to evolution and uses a Reasoning-and-Acting (ReAct) loop to call frozen, typed Policy APIs at deployment. A two-phase strategy combines capability curriculum learning with autonomous self-evolution to improve the APIs, the ReAct harness, and experience memory. The APIs encode reusable physical mechanisms while exposing arguments for runtime adaptation. ReAct combines task-specific working memory, long-term experience memory, and visual feedback to select actions, verify outcomes, and recover from failures without modifying source code. RACaP achieves 54.4% success on LIBERO-90, 45.0% on zero-shot LIBERO-PRO, and 46.0% on LIBERO-Long, compared with at most 4.0% for CaP baselines on long-horizon tasks. On LIBERO-PRO, it achieves 2.5 times the success rate of CaP baselines and a 1.9-fold speedup in median policy time. For efficient on-robot deployment, rejection-sampled fine-tuning distills GPT-5.6 ReAct decisions into Qwen3-VL-8B-Instruct, yielding a 13.2-fold per-decision inference speedup and reducing repeated physical calls from 16 to 4. These results show that separating reusable code from runtime decisions supports continued evolution, effective transfer, and efficient long-horizon control.

Zexi Li, Yehang Zhang, Haojian Huang +10

The Chinese University of Hong Kong · Knowin AI · The Hong Kong University of Science and Technology (Guangzhou)

Robot Policy Learning Robot Policy Generalization Self-Evolving Agents

Ask PDF

Jun 2, 2026cs.RO

Revisiting Embodied Chain-of-Thought for Generalizable Robot Manipulation

Embodied chain-of-thought (CoT) aims to bridge linguistic reasoning and robotic control, but its effective form and integration strategy remain underexplored. In this paper, we revisit embodied CoT for vision-language-action (VLA) models at large scale. We construct the largest embodied CoT corpus to date, comprising 978,743 trajectories, 226.3M samples, and 2592.5 hours of robot data. Through extensive experiments, we find that effective embodied CoT should ground high-level semantic understanding into concrete action guidance, such as end-effector movement descriptions and image-space trajectories, while high-level reasoning alone brings only marginal gains. We further show that explicit CoT does not scale reliably when used as an autoregressive action prefix, as it suffers from compounding inference errors and unstable reasoning-action coupling. To address these limitations, we propose ERVLA, a VLA model that uses embodied CoT as representation-shaping supervision rather than mandatory test-time reasoning. ERVLA is trained with a reasoning-dropout strategy, enabling the model to absorb rich reasoning traces during training while predicting actions directly without CoT decoding during inference. This design improves scalability with increasing pre-training data and avoids autoregressive instability. ERVLA achieves state-of-the-art performance on LIBERO-Plus with an 86.9% success rate and reaches 53.2% success rate on VLABench, demonstrating strong out-of-distribution generalization. In real-robot experiments, ERVLA further outperforms competitive state-of-the-art baselines, especially on tasks requiring semantic disambiguation and long-horizon execution.

Nan Sun, Yuan Zhang, Yongkun Yang +10

Tsinghua University · Xiaomi Robotics · Peking University +6

Long-Horizon Robotic Manipulation CoT Reasoning Generalization in Robotic Manipulation Robotic Manipulation Vision-Language-Action Models

Ask PDF