Can AI agents design robots that work in the real world? We introduce RoboBuilder, a benchmark in which coding agents co-design a robot's hardware and program from standard parts, motors and a custom 3D-printed part, and every design that passes in simulation is built and tested on a physical course. Simulation alone is not enough: under public part data, none of 15 initial passing designs crossed the real course, so an agent learns the simulator from measurements it requests and videos of real runs. Across four frontier coding agents, ten obstacles at three difficulty levels, 16,427 simulations of 12,774 distinct robots and 135 physical robots built in an estimated 240 hours, agents solve nearly all easy obstacles and invent mechanisms our search baselines do not reach, but falter at harder levels, when one robot must cross every obstacle, and in the real world, where 66% of simulated passes hold across all levels. Closing the loop helps: shown a video of its real failure, an agent redesigns the robot, sometimes correcting its simulator, and removes 72% of level-1 and level-2 failures. RoboBuilder measures progress toward agents that engineer real machines. Project Page: https://ghadinehme.com/robobuilder.github.io/
Figures & tables
Figure 2: Agent-designed robots, built by hand on their real courses. Twelve of the 135 robots we built: whegs, printed hoops and cam wheels, cord trolleys, lever arms, legs and a spoked wheel that vaults the wall. More designs in Appendix A .
Figure 3: RoboBuilder asks for a machine that can be built, and closes the gap to the real world. Left: agents combine inventory parts, motors and one custom CadQuery part type, printed in up to four copies, under design rules, and write an open-loop program; tasks and their simulation budgets; the ten obstacles, each at three levels. Right: one turn of the hardware loop (level-2 bumps, Opus 5.5): spoked wheels cross in simulation but spin against a ridge for real; after the agent lowers the friction of plastic on the bump rods the simulator fails the same way, and the redesign on worm-driven whegs crosses for real. Bottom right: share of simulated passes that cross for real (levels 1 and 2) with the simulator from public part data (initial designs), after simulator learning, and after one round of the loop’s redesigns.
Work
Functional task
Real parts
Built for real
New mechanisms
Agent loop
Real feedback
RoboGrammar [ 94 ]
✓
✗
✗
∘
✗
✗
RoboMorph [ 63 ]
✓
✗
✗
∘
✓
✗
BesiegeField [ 93 ]
✓
✗
✗
✓
✓
✗
Text2Robot [ 68 ]
✓
∘
✓
∘
✗
✗
BrickGPT [ 60 ]
✗
✓
✓
✗
✗
✗
Blox-Net [ 22 ]
✗
✓
✓
✗
✓
✗
Table 1: RoboBuilder is the first agentic design benchmark with real feedback into design and simulator. Functional task: judged on doing a task. New mechanisms: designs beyond tuning parameters. Agent loop: an agent iterates on feedback. Real feedback: real outcomes revise the design or simulator. ∘ : partly.
Table 2: Agents pass most tasks, optimize and invent; search over the full inventory passes nothing. Cross : tasks passed per level; + Print : with Cross+Print , all levels. Sims: median simulations to a pass. All-in-one : obstacles crossed by one robot. Light , Fast : mean mass and time over shared level-1 tasks, and scores (of 30). Score: combined (of 100). Novelty: distance of the designs from the nearest search designs in function space, in units of the search designs’ own spacing (search ≈ 1). Diversity: effective number of distinct designs among 100 (Vendi). ↑ / ↓ : higher/lower is better; colour: red (worst), yellow, green (best) per column. Full-inventory search: random search and a GA, 30 simulations per task, not run on All-in-one , Light or Fast ( Tab. 17 ); single LLM call: best model ( Tab. 20 ).
Figure 4: Designs cluster by obstacle, not by agent. t-SNE map of the DINOv2 [ 57 ] features of all simulated designs, coloured by obstacle; outlined: passed; diamonds: All-in-one robots; thumbnails: designs near cluster centres; corners: each agent’s All-in-one robot. Per agent in Figure 17 .
Figure 5: On hard obstacles, agents keep one machine until the printed part arrives. Four level-3 runs, first simulation to first plain-run crossing (the official pass, with the nudged starts, can come later). Frame colour: red, fails; green, passes; light green, passes with the printed part, which is available from simulation 31. The labels are our own reading of the renders.
Figure 6: Real failures stall or fall where the simulated twin crosses. Failure modes of level-1 and level-2 designs that pass in simulation but fail for real, with each mode’s share, a real frame and the simulation at the same time.
Figure 7: The loop turns real failures into crossings. One case per agent: (a) the original design failing for real, with its failure mode (shares in Figure 6 ); (b) the redesign crossing in simulation and (c) on the real course.
Appendix figures & tables48 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Robots that cross the real course range from trolleys and girders to whegs, cam climbers and walkers. Benchmark designs and loop redesigns (printed parts in orange) with agent, obstacle, level and mechanism. Bottom: each agent’s All-in-one robot.
Figure 9: The designs as simulated and as built. Render and real crossing frame for eight robots that crossed the real course.
Figure 10: On the hardest obstacles agents invent printed wheels, stilts, legs, vaulters and booms, though many never pass. Top two rows passed in simulation, bottom two did not (best progress). Cards: agent, obstacle, level and task; printed parts in orange.
Figure 11: Agents refine their first concept; under 6% of steps change concept. (a–d) Every simulation of every task (rows: level and obstacle; tick: start of Cross+Print ), coloured by the change from the previous design ( Sec. B.1 ); circles: first pass. (e) Share of steps per class. (f) Concept changes before and after the first pass and in failed runs.
Figure 12: Agents mostly make small edits, stall on failed tasks, and big moves pay off more often. (a) Step mix per agent. (b) Longest run without improving (dashed: median). (c) Share of steps that improve, local edits vs. new concepts.
Figure 13: All 40 simulations of the run in Figure 15 . Bar: progress; faded: program-only change; navy frame: new machine; green: first pass; photo: the built robot crossing.
Figure 14: Agents refine one machine, although new ideas pay more, and exploring more does not predict success. (a) Steps between consecutive simulated designs. (b) Rise of the best progress for a step into a new idea basin, or back to an old one, relative to a step within the same basin, under three design distances (95% intervals). (c) Per agent, ideas tried before the first pass against runs passed; marker size and (%): new ideas that beat the run’s best so far.
Figure 15: One run from first simulation to real crossing (level-3 bumps; simulations shown up to the first plain-run crossing, while the official pass, which also needs the nudged starts, comes later, Tab. 18 ). Most of the final design is present from the first simulation. Each circle is one simulation, placed at the progress the robot made and coloured by the share of the final design already present. The line shows the best progress so far. All 40 runs are in Figure 13 .
Table 3: The best agent starts closer to a working design rather than exploring more. Per agent: share of steps between consecutive simulations that keep the same machine or change the concept; ideas (basins) tried before the first pass; runs that pass in their first idea; new ideas that beat the run’s best so far; tasks passed ( Cross or Cross+Print , as in Tab. 2 ). A new idea pays 3.1 to 3.8 times more than a refining step under every design distance ( Sec. B.3 ).
Distance
silhouette
basins/run
never leave
steps switching
new-basin gain
return gain
concept steps
Look + parts (main)
0.64
2.0
34%
6.4%
3.8 × (2.1–5.8)
0.6 ×
70%
Parts (Jaccard)
0.66
2.0
42%
7.1%
3.4 × (1.9–5.2)
0.8 ×
86%
Function
0.71
2.5
25%
11.6%
3.1 × (1.8–4.9)
1.0 ×
76%
Appendix
Table 4: New basins pay under every design distance. Idea basins over the 120 counted runs with three distances. Gain ratio: mean rise of the best progress for a step into a new basin (or back to an old one) divided by that of a same-basin step, with 95% bootstrap intervals over runs. Concept steps: share of the concept changes of Figure 11 that switch basin.
Agent
Passed
First basin (%)
Basins to pass
Switches that pay (%)
Opus 5.5
80%
57 / 63 / 47
1.5 / 1.3 / 1.6
43 / 35 / 33
Sonnet 5.5
64%
25 / 32 / 29
2.2 / 1.9 / 2.3
15 / 12 / 23
GPT-6 Astra
80%
40 / 43 / 43
1.9 / 2.0 / 2.1
13 / 20 / 14
GPT-6.1 Sol
87%
53 / 50 / 50
1.5 / 1.7 / 1.9
15 / 23 / 14
All
78%
44 / 47 / 42
1.7 / 1.7 / 1.9
20 / 21 / 20
Appendix
Table 5: More exploration does not predict success. Per agent, under the three design distances of Tab. 4 (look + parts / parts / function), over the 118 runs with at least two simulations. Passed: runs with at least one passing simulation in phase A or X (before the final design review and re-scoring, so above the official pass rates of Tab. 2 ). First basin: share of runs that pass in the basin of their first design. Basins to pass: basins visited up to the first pass, in passed runs. Switches that pay: share of steps into a new basin that raise the run’s best progress.
Figure 16: New ideas pay, but agents stop exploring early. Idea basins over the 120 counted runs. (a) Gain of a step into a new basin, or back to an old one, relative to a same-basin step, under the three design distances of Tab. 4 , with 95% bootstrap intervals; dotted line: no difference. (b) Share of steps that enter a new basin, by simulation; the printed part is allowed from simulation 31. (c) Basins visited per run, per agent. (b) and (c) use the look-and-parts distance.
Agent
steps
same
< 10%
10–50%
concept
plateau P/F
switches P/F
Opus 5.5
660
33%
38%
24%
4.4%
2 / 39.5
0.46 / 0.42
Sonnet 5.5
1059
53%
17%
24%
5.9%
3.5 / 42.5
0.39 / 0.65
GPT-6 Astra
808
28%
26%
36%
11.0%
2 / 28
1.14 / 1.06
GPT-6.1 Sol
817
31%
29%
30%
9.7%
3 / 25
1.04 / 0.89
All
3344
38%
26%
28%
7.8%
2 / 33
0.83 / 0.74
Appendix
Table 6: Search behaviour per agent. Step mix over all steps; columns bin steps by the share of parts changed (concept: over half, without the look criterion of Sec. B.1 , hence above the 4% of Sec. 4 ); plateau: median longest run without improving the best progress; switches: new concepts per ten steps (passed / failed tasks).
Agent
wheels
tracks
other
chain
gears
printed
mass (g)
parts
motors
Opus 5.5
20
2
8
30%
100%
23%
486
135
1.9
Sonnet 5.5
22
1
7
33%
93%
27%
449
73.5
1.7
GPT-6 Astra
21
5
4
40%
97%
30%
523
142
1.9
GPT-6.1 Sol
24
0
6
57%
100%
30%
475
140
1.8
Appendix
Table 7: Final designs per agent (accepted, or best attempt; 30 tasks each). Locomotion counts; share of designs with chains, gears and a printed part; medians of mass and parts; mean motors; part types used.
Agent
B mass
C time
tokens in/out (M)
A hours
B+C sims
B+C hours
Opus 5.5
0.52
0.22
16.7/12.4
82
3000
463
Sonnet 5.5
0.68
0.32
11.6/8.5
34
2689
267
GPT-6 Astra
0.70
0.47
7.2/3.0
58
3000
326
GPT-6.1 Sol
0.60
0.58
12.7/6.6
79
3000
310
Appendix
Table 8: Lightest (B) and fastest (C) designs and cost. Median ratios to the agent’s passing design of the same task; pass-round tokens exclude cache reads (Codex input includes cached tokens, output includes reasoning).
Figure 17: The design map of Figure 4 , per agent. (a) All simulated designs, coloured by obstacle; (b) the same map split by agent.
Figure 18: The same map, one obstacle at a time: agents reach different machines for the same task. Each panel shows only the designs simulated on one obstacle, all levels, at their positions in Figure 4 (grey: did not pass; coloured by agent: passed), zoomed to their extent. Below each panel, three passing designs far apart in the map (the design nearest each of three k -means centres of the passing designs, different agents preferred).
Figure 19: The same designs projected with UMAP. Same features and colours as Figure 4 ; the same organisation by obstacle.
Method
Designs
Function NN
Shah
Variety
Concepts
Vendi (function)
Vendi (look)
Non-wheeled
Random search
729
1.11
0.35
10.0
12
2.2
2.0
0%
CMA-ES
805
0.98
0.34
7.1
9
1.9
1.8
0%
Genetic algorithm
1,136
0.94
0.38
16.6
21
2.1
2.3
0%
Opus 5.5
789
19.6
0.53
50.7
135
13.0
4.0
11%
Sonnet 5.5
454
15.4
0.49
51.1
89
8.9
3.7
12%
GPT-6 Astra
511
16.9
0.52
58.6
129
11.5
4.4
29%
Appendix
Table 9: By how they work, agents’ designs are far from the baselines and more varied. Pool: the parametric baselines; against the full-inventory search (6,489 designs, 25% on treads), agents’ function NN is 9.3 (Opus 5.5), 8.1 (Sonnet 5.5), 9.3 (GPT-6 Astra) and 8.8 (GPT-6.1 Sol), and the search’s Vendi (function) is 3.4 ( Tab. 2 ). Novelty and diversity of each method’s designs (agents: unique passing geometries; baselines: unique simulated geometries). Function NN: mean distance to the 5 nearest baseline designs in the function space, in units of the baselines’ own nearest-neighbour distance. Shah: a-posteriori novelty over 8 function attributes (0 to 1). Variety: distinct attribute combinations among 100 random designs (mean of 50 draws); concepts: distinct combinations overall. Vendi: effective number of distinct designs among 100 random designs, by function and by DINOv2 appearance. Non-wheeled: share of designs that do not roll only on standard wheels.
Table 10: Passing designs use far more pieces than they need. Medians over the 86 designs that pass Cross or Cross+Print . Conn.: connections. Reorient.: re-orientations during assembly. Nmin/N : share of parts that are necessary. DFA eff.: design-for-assembly efficiency, 3s⋅Nmin/T , where T is the estimated assembly time ( Sec. C.3 ). Higher is leaner.
Robots
Pieces
Hours
Print (h)
Simulator learning
26
∼ 1,130
∼ 27
–
Benchmark designs (sim passes)
87
9,390
156
∼ 46
All-in-one robots
4
231 ∗
23
∼ 23
Hardware loop redesigns
18
2,170
35
∼ 37
Total
135
∼ 13,000
∼ 240
∼ 106
Appendix
Table 11: Every real result is a robot built by hand. Robots built and tested, all levels. Hours: 43 s of assembly per piece plus 30 min per robot and course for build fixes (repairing parts broken in an earlier failed try, back to the design as specified), course resets and filming, both measured on a 125-piece robot. All-in-one robots are tested on ten courses each. Hours exclude 3D printing, which is machine time (last column).
Figure 20: Simulated and real progress of four level-1 designs. Top: real passes follow their simulation. Bottom: real failures end short of the goal.
Table 12: The hardware loop fixes most real failures for every agent. Levels 1 and 2: share of simulated passes that cross the real course before and after one round of the loop, and real failures.
Agent
failures
redesign
vote
real
Opus 5.5
6
5
5
3/5
Sonnet 5.5 ∗
4
4
4
4/4
GPT-6 Astra
4
4
4
4/4
GPT-6.1 Sol
5
5
5
3/5
All ∗
19
18
18
14/18
Appendix
Table 13: Closing the loop on every real failure. Per agent: level-1–2 real failures sent to the loop, redesigns that pass the verdict and the hand-in vote, and redesigns that cross for real. ∗ Includes Sonnet 5.5’s U-turn level-1 design, filmed as its Light design and left out of the real pass rates; its redesign crosses for real.
Failed partway
Failed early
All
New concept
7/7
1/2
8/9
Adjustment
1/2
5/7
6/9
Appendix
Table 14: Agents replace the mechanism when the robot fails partway and adjust it when it barely starts. The 18 loop redesigns (with Sonnet 5.5’s U-turn) by response and by how far the original got in its real tries; cells give redesigns that cross for real / redesigns.
Figure 21: A simulator pass is only the middle of the loop. One design (Opus 5.5, bumps L2): simulated pass (2), build and film (3–4), real failure compared with the simulation (5), updated simulator and a redesign that crosses for real (6).
Run
Agent
official
robust
10/10
Baseline
Opus 5.5
5
6
4
Baseline
Sonnet 5.5
3
3
2
Baseline
GPT-6 Astra
6
7
4
Baseline
GPT-6.1 Sol
5
5
2
Req.-first
Sonnet 5.5
2
2
–
Req.-first
GPT-6 Astra
0
7
7
Appendix
Table 15: The requirements-first harness does not reliably beat single agents on the All-in-one task. Obstacles crossed (of 10): official verdict, robust ( ≥ 6 of 10 perturbed repeats) and all ten repeats. + fix, + repair: follow-up repair sessions; pools, v2, optimizer, fork: Sec. E.2 .
Figure 22: Design changes per simulation with the requirements-first harness overlaid. As Figure 11 , with the runs of the requirements-first harness added for comparison.
Figure 23: Specialists solve nine obstacles alone, but one robot keeps seven. GPT-6 Astra, level-1 All-in-one task. (a) Specialist pools and a synthesizer. (b) Synthesized pairs: both pass (solid) or not (dashed). (c) Obstacles crossed by a single agent, the synthesized robot and the specialists alone.
Method
crossed /10
real /10
Single agent
6
5
Specialist pools + synthesizer
7
–
specialists alone (best per obstacle)
9
–
Appendix
Table 16: Integration, not single obstacles, limits the All-in-one robot. All-in-one task (level 1, GPT-6 Astra, 300 simulations): obstacles crossed by the submitted robot (official verdict).
Cross (%)
+ Print
Sims
Light
Fast
Score
Novelty
Diversity
Method
L1
L2
L3
(%)
(med.)
score
score
(/100)
(function)
(Vendi)
Full inventory, random
0
0
0
0
–
–
–
0.0
1.1
3.4
Full inventory, GA
0
0
0
0
–
–
–
0.0
0.9
3.2
Parametric, random
10
0
0
3
9
0.0
0.0
0.8
1.1
2.2
Parametric, CMA-ES
0
0
0
0
–
0.0
0.0
0.0
1.0
1.9
Parametric, template
0
0
0
0
–
0.0
0.0
0.0
–
–
Appendix
Table 17: No search baseline passes more than one task. All non-agentic baselines, in the columns of Tab. 2 . Full inventory: random search and a GA over build actions on the whole inventory (30 simulations per task; novelty and diversity against this pool, as in Tab. 2 ). Parametric: each searches a hand-built family of cars (19 parameters: motor, wheel, rails, wheelbase, gearing, drive, deck and speed program) with the same simulator, verdict and budget as the agents.
Table 18: Every task, every agent. Pass: simulation at which the task passed ( P : in Cross+Print ), or ✗ if it failed in simulation; Light : lightest mass (g); Fast : fastest time (s); Real: crosses ( ✓ ) or fails ( ✗ ) for real, → loop redesign ( ✗ also when no redesign passed the vote); –: none ( Light , Fast : no design; Real: not built, as the task failed in simulation). ‡ See Sec. F.2 .
Method
Cross/Print
All
Light
Fast
mean
geo.
rank
95% CI
Random search
0.03
0.00
0.00
0.00
0.8
1.4
7.1
–
CMA-ES
0.00
0.00
0.00
0.00
0.0
1.0
7.6
–
Template search
0.00
0.00
0.00
0.00
0.0
1.0
7.6
–
Genetic algorithm
0.03
0.00
0.02
0.01
1.7
1.7
5.9
–
Single LLM call
0.07
0.00
0.00
0.00
1.7
1.6
6.8
–
Opus 5.5
0.80
0.50
0.75
0.90
73.9
72.2
1.4
60.9–87.1
Appendix
Table 19: The ranking does not depend on how the score is aggregated. Mean: headline score; geo.: geometric mean; rank: mean rank over the four components (1 = best); 95% CI: bootstrap over obstacles ( Sec. F.2 ).
Figure 24: Agents pass nearly every level-1 obstacle; harder levels bring legs, whegs, printed parts and fewer passes. One design per agent, level and obstacle. Frame: pass (dark green), pass with a printed part (light green), no pass (grey, best progress). Badge: crosses (dot) or fails (cross) for real.
Method
L1
L2
L3
All-obst.
Random search (parametric)
1
0
0
–
CMA-ES (parametric)
0
0
0
–
Template search
0
0
0
–
Genetic algorithm
1
0
0
–
Single LLM call, Opus 5.5
2
0
0
0
Single LLM call, Sonnet 5.5
1
0
0
0
Appendix
Table 20: Agents pass nearly every level-1 task but under half at level 3; baselines pass almost none. Tasks passed (of 10 per level; in parentheses, with the printed part). All-obst.: obstacles crossed by one robot (of 10).
Lightest ( Light )
Fastest ( Fast )
Agent
L1
L2
L3
total
L1
L2
L3
total
Opus 5.5
9.2
8.3
5.1
22.6
9.5
9.6
8.0
27.1
Sonnet 5.5
8.4
5.3
3.5
17.2
6.4
3.1
1.5
11.0
GPT-6 Astra
7.4
6.5
4.2
18.1
4.1
4.0
2.0
10.1
GPT-6.1 Sol
7.8
4.7
6.0
18.5
4.2
2.0
3.4
9.6
Appendix
Table 21: Opus 5.5 leads both optimization objectives; speed separates the agents more than mass. Per task: best mass (time) among agents divided by the agent’s, 0 if none; summed per level (max 10).
Agent
sim /10
real /10
Opus 5.5
5
6
Sonnet 5.5
3
2
GPT-6 Astra
6
5
GPT-6.1 Sol
5
6
Appendix
Table 22: One robot for all ten obstacles crosses at most six. Obstacles passed by the handed-in level-1 robot in simulation (official verdict: plain run and both nudged starts) and for real. Opus 5.5 and GPT-6.1 Sol cross one obstacle more for real than in simulation: Opus 5.5’s robot passes bumps in a rescore of the same design and in 7 of 10 perturbed repeats, and GPT-6.1 Sol’s robot passes gap-drop in the plain run but not with a 50 ms late start.
L1
L2
L3
Agent
cross
tested
cross
tested
cross
tested
Opus 5.5
8
10
4
8
3
6
Sonnet 5.5
7
9
3
4
1
3
GPT-6 Astra
8
10
7
9
2
4
GPT-6.1 Sol
7
10
5
7
2
6
All
30
39
19
28
8
19
Appendix
Table 23: About two in three simulated successes cross for real, fewer at harder levels. Per try, 127 of 188 level-1 and level-2 tries cross (68%).
Figure 25: Agents build wheeled machines and use legs, arms and rollers only where needed. (a) Final designs per locomotion type and obstacle, with each type’s pass rate. (b) Mean parts per design by category, one dot per agent. (c) Lightest ( Light ) and fastest ( Fast ) designs relative to the passing design; line: median.
Figure 26: Simulation and real results per task. One cell per agent, obstacle and level: pass in Cross , pass in Cross+Print or no pass (shade: best progress); symbols: built and filmed, crosses or fails for real.
Table 24: Light and Fast leaderboard. Tasks with a design and tasks won (lightest mass, fastest time) per agent; ties go to the first listed.
Figure 27: Lightest and fastest designs on bridge level 1. One column per agent; mass of the Light design, time to the goal of the Fast design.
before
after
Sim passes crossing for real
0/15
57/86 (66%)
First-round agent designs predicted
0/15
20/22
Labelled runs predicted
–
35/38
of which validation
–
9/10
Appendix
Table 25: Before the first round no simulated pass held for real; after it, two in three do. Rows: simulated passes that cross for real; labelled runs predicted (all, validation); first-round agent designs (before: the 15 that passed; after: all 22 filmed). The before and after rows use different designs.
Measurement
Result
Stud pull-off: motor on 4 / 2 studs, hanging 2 × 4 bricks
515 g (1.26 N per stud) / about 214 g (1.05 N per stud)
Stud peel: long brick on 2 studs, hanging 1 × 16 beams
35–47 g at 105 mm (about 36–48 N mm)
Reference car tilt test on the course board
slides at 23 ∘ and 15 ∘ (two orientations)
Plain brick tilt test on the course board
slides at 23 ∘ ; μ=0.42
Friction-pin torque: beams hung on a pinned joint
holds about 4.5 N mm (24.5 assumed before)
Lowest speed level that starts a car
level 1 on the flat; level 3 (not 2) on 32 ∘
Appendix
Table 26: Bench measurements the agent asked for after the first builds failed , made with the parts themselves: tilting the course boards, and hanging bricks of known mass from a joint until it gives.
Physics added
Real observation it explains
Stud joints release under overload and on impact
the battery box comes off on a steep ramp
Joints peel progressively
the battery stack peels off its stud joints on the steps
Loosely built bevel-on-spur meshes slip
the motor whines and the wheels never turn
Battery and receiver supply sag under load
fitted to the measured loaded speeds
Chains skip teeth above a load per tooth
set from a public filmed lifting test; slack rule
Free axles rub under radial load
cars teeter onto free middle wheels over the gap
Appendix
Table 27: What the simulator after the first round adds, and the real observation behind each change.
Figure 28: Ten obstacles at three levels, and one robot for all ten. Each obstacle is drawn from the same viewpoint and at the same scale at all three levels; the number under each course is the dimension that changes with the level: ramp angle, gap width, plank width, cord incline, step height, rod height, seesaw tilt, drop to the landing deck, wall height and plate tilt. Levels also tighten the time limit. Right: each agent’s single robot for all ten level-1 obstacles.
Figure 29: One box of parts. The owner’s stock of standard parts, one tile per part type with the number of pieces; agents may use only what is in the box, so every design can be built. Motors, receivers and the battery box are in the last rows.
We propose BrickBench, a benchmark for agentic text-conditioned LEGO-set design. Given a prompt, an agent is tasked with producing an assembly that not only satisfies semantic and design criteria, but that can also be physically built. To do so, it must select parts from a discrete library and reason jointly about local and global constraints. We score validity, alignment, and design across three settings that vary in scale and part availability. We provide BrickAgent, an environment for coding agents to construct, inspect, and validate their designs. We find that leading agents largely satisfy verifiable physical and semantic requirements, but fall short of human designs. We release our benchmark and environment at http://www.brickben.ch
Peter Kulits, Yiqing Xu, R. Kenny Jones +2
Stanford University · Max Planck Institute for Intelligent Systems · Inria
Realistic, fast robotics simulations could enable large-scale training of embodied AI models that transfer to the real world. However, despite decades of progress, available simulations cover only a narrow range of real-world tasks and physical phenomena. The core bottleneck is that creating better simulations requires significant expert effort and therefore does not scale. To address this, we introduce Gym-Anything for Robotics, a framework in which coding agents create realistic robotics simulations. Our core insight is that simulation creation is a coding agent task, so improvements in coding agents translate directly into richer and more faithful simulations. We select simulations based on U.S. GDP data to cover economically valuable physical work, define what makes a good simulation and a good task, and develop an adversarial creation-audit loop that checks each simulation and task against these definitions. Using this pipeline, we build RoboWorld, a set of 25 simulations with 25 test and 175 training RL environments of varying difficulty. Even the best frontier models fail on every RoboWorld test task, achieving an average reward of only 0.06. Building on RoboWorld, we introduce several new ways for frontier models to improve, including in-context learning across repeated attempts, a privileged teacher that reads the simulator state and teaches a student in context, self-assisted learning from the model's own privileged experience, and code policies that generate demonstrations across varied scenes to train vision-language-action models (VLAs) such as π0.5. Each of these substantially improves performance. We release our pipeline, RL environments, and benchmark to facilitate future research. Code and data are available at https://gym-anything.github.io/robotics
We introduce Debate2Create (D2C), a multi-agent LLM framework that formulates robot co-design as structured, iterative debate grounded in physics-based evaluation. A design agent and control agent engage in a thesis-antithesis-synthesis loop, while criterion-specific LLM judges provide multi-objective feedback to steer exploration. Across five MuJoCo locomotion benchmarks, D2C achieves the highest default-normalized score among the evaluated LLM-based and black-box baselines, with gains up to 3.2x on Ant and nearly 9x on Swimmer. Iterative debate yields 18-35% gains over compute-matched zero-shot generation, and D2C-generated rewards transfer to default morphologies in 4/5 tasks. These results suggest that structured, simulator-grounded multi-agent interaction is a useful mechanism for joint morphology-reward optimization under a fixed-topology, per-candidate-RL protocol. Project page: debate2create.github.io.