Simulated environments enable scalable training data generation for embodied agents, but more data does not necessarily provide what a particular agent needs to learn. Existing environment-generation pipelines largely optimize generic properties such as realism, diversity, or task coverage, while different pretrained policies can fail for different environmental reasons. We therefore introduce WEDGE, a framework for weakness-guided navigation learning in which the failure modes of a pretrained agent guide the generation of training environments at each adaptation round. Given a pretrained policy, WEDGE uses an LLM to propose executable weakness features from contrasting successful and failed rollouts, automatically validates them on the diagnostic set, and selects the strongest validated weakness based on its success-rate gap. The validated weaknesses are then mapped to the environmental factors that control them. WEDGE varies these factors to construct environments in the difficulty regimes where the policy struggles, producing targeted training experience for adaptation. Across three navigation agents with different adaptation mechanisms, we find that policies exhibit distinct weakness profiles and benefit from different training environments. Weakness-conditioned adaptation consistently outperforms matched random and hard-example alternatives on ProcTHOR, while the resulting gains for Uni-NaVid and Gemini also extend to HM3D after ProcTHOR adaptation. Iterative re-diagnosis further shows that dominant weaknesses can persist or shift as the policy improves, providing updated guidance for subsequent environment construction. Together, these results show that policy failures are not only diagnostic signals, but actionable guidance for constructing training environments that target a policy's current weaknesses.
Figures & tables
Figure 1: Overview of Wedge . Unlike existing environment generators, which are largely policy-agnostic, Wedge uses agent failures to guide generation. (a) We mine policy-specific weaknesses from successful and failed rollouts. (b) The mined weaknesses determine which environmental factors to control and which learnable regimes to target. Training on the resulting environments improves the agent and can yield cross-domain transfer beyond the generation simulator.
Figure 2: LLM-guided discovery and validation of executable policy weaknesses. (a) We construct a diagnostic set by contrasting successful and failed episodes represented with a shared vocabulary of measurable primitives. (b) An LLM uses these structured examples to propose candidate weakness features and their executable definitions. (c) Each candidate is evaluated on the full diagnostic set, and (d) retained only if it satisfies the validation criteria for coverage, non-redundancy, predictiveness, and incremental value. Candidates that fail any criterion trigger a new round of episode sampling and feature proposal. This proposal–validation process repeats until the stopping criterion is met, with validated weaknesses retained for subsequent environment generation.
Figure 3: Weakness-conditioned environment construction and rollout verification. (a) Each policy is associated with its top mined weakness. (b) Candidate episodes are constructed by modifying the factors relevant to that weakness, such as target placement for target_visibility or start state and layout for blind_path_ratio . Candidates are retained only when their feature values fall within the weakness range, with new layouts generated when existing ones are insufficient. (c) The policy is rolled out five times in each retained candidate, and episodes with two or three successful rollouts are selected for training.
Agent
Diagnosed weakness
Adaptation setting
ProcTHOR
HM3D
SR ↑
SPL ↑
SR ↑
SPL ↑
DD-PPO
Target visibility ( Δ SR = 28.5)
Pretrained
82.1
58.2
–
–
Random env.
82.3
58.8
–
–
Hard-mined
82.4
58.1
–
–
Wedge R1 (Ours)
85.4
59.8
–
–
Wedge R2 (Ours)
85.6
59.8
–
–
Table 1: Experimental results on policy-specific adaptation and generalization. For each agent, we report the validated weaknesses identified in ProcTHOR, adaptation performance on ProcTHOR, and performance when evaluated back on HM3D. Bold denotes the best performance. † Uni-NaVid’s original HM3D training data are unavailable, so the pretrained model is shown as a reference.
Figure 4: Analysis of weakness-conditioned adaptation. (a) We re-diagnose each agent after every adaptation round and track the SR gap associated with its dominant weakness. DD-PPO and Uni-NaVid retain the same dominant weakness, while Gemini develops a new weakness after the first round. (b) Adaptation gains are larger in weakness-associated regimes. For Uni-NaVid and Gemini, we compare gains across blind_path_ratio regimes. For DD-PPO, gains are reported by target category, with larger improvements on small, hard-to-see targets.
DD-PPO
# Env.
SR ↑
Pretrained
–
82.1
All weakness-matched
4,215
84.4
Calibrated (2–3/5)
668
85.4
Table 2: Effect of rollout calibration. Proc. and HM3D report SR; Hardest: share of hardest episodes.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Adaptation data
SR ↑
SPL ↑
Random self-rollout
46.6
34.1
Weakness-mined existing
49.2
34.9
Weakness-generated only
51.3
35.4
Mined + Generated ( Wedge )
54.5
37.3
Appendix
Table 3: Ablation of mined and generated adaptation on ProcTHOR
Agent
Discovered weakness
Executable definition
SR drop
DD-PPO
target_visibility
Fraction of standing positions in the target room from which the target is visible
↓28.5
doorway_visibility
Whether the target is visible from a doorway of the target room, evaluated within low-visibility rooms
↓22.5
Uni-NaVid
blind_path_ratio
Fraction of the shortest path traversed before the target first becomes visible
↓37.3
room_hops
Number of room-to-room transitions along the shortest path to the target
↓34.8
Gemini
blind_path_ratio
Fraction of the shortest path traversed before the target first becomes visible
↓37.6
Appendix
Table 4: Policy-specific weaknesses. Validated weakness features and the corresponding success-rate drop between the diagnosed weak regime and the contrasting regime. Bold denotes the top-ranked weakness used for environment generation.
Group
Primitive
Meaning
Scene house structure
houseArea
floor area of the house (m 2 )
roomsTotal
number of rooms
numDoorsHouse
number of doors in the house
startRoomType
room type the episode starts in
roomPair
start room type → target room type
numSameTypeRooms
rooms of the same type as the target room
Appendix
Table 5: Measurement primitives. Every evaluated episode is summarised by a fixed vocabulary of simulator-measured quantities. Scene , Target and Path primitives depend only on the layout and the ground-truth shortest path, so they are controllable at generation.
Generative simulation can automate robot environment construction, but physically valid environments are not necessarily useful for training. The challenge is to identify environments that address a policy's failures and supervision that helps it overcome them. We introduce CoDA, a framework in which a simulation data engine and an action policy co-evolve through a shared heterogeneous memory of environment construction, policy failures, and learning progress. The memory compiles validated diagnoses from a vision-language critic into generation programs that recreate difficulties and training programs that help the policy overcome them. We further propose Text2potential to convert these diagnoses into staged potentials for action-chunk value learning, providing dense supervision for policy updates while preserving the task's optimal policy under standard shaping assumptions. Measured learning progress guides further generation and program refinement. Across four manipulation task families, CoDA improves out-of-distribution success from 5.3% after supervised fine-tuning to 27.6%, exceeding the strongest closed-loop baseline by 9.5 percentage points with the same generation budget. On LIBERO-Pro, 15 rounds of generation and training raise success from 25% to 84% under position perturbations and from 1% to 76% under instruction perturbations.
Tianjiao Yu, Zhilin Ou, Yifan Shen +5
University of Illinois Urbana-Champaign · The Chinese University of Hong Kong, Shenzhen · Amazon
Robust navigation policies for autonomous agents must generalize across continuously varying environmental conditions such as turn rates, obstacles, friction, pits, and slopes. Curriculum generation provides a principled mechanism for improving generalization by progressively adapting training environments, but designing such curricula in a sample-efficient and automated manner remains challenging. This paper proposes a reparameterized curriculum generation framework for structured continuous environment parameters using unidirectional gradient-based optimization. To improve robustness in multimodal observation spaces consisting of image-based and scalar inputs, a distribution-shift regularization objective is incorporated to encourage the learning of finer-grained latent representations. The proposed method is evaluated across two continuous-control OpenAI Gym environments: a 2D obstacle-based Car Racing variant and Bipedal Walker variant, where coupled environment parameters jointly influence policy performance. Across five random seeds, our method consistently outperforms vanilla policy training, random parameter sampling, manual curricula, frontier-based methods, Self-Paced Reinforcement Learning (SPRL), Absolute Learning Progress with Gaussian Mixture Models (ALP-GMM), and reverse curriculum learning baselines. Ablation studies further demonstrate the effectiveness of the reparameterized curriculum mechanism across both environments, while highlighting environment-dependent benefits of the auxiliary regularization objective.
LLM agents learn by interacting with environments, yet these environments are hand-built and static: blind to an agent's weaknesses, and quickly left behind as it improves. While recent environment generation methods attempt to address this, they require domain-specific pipelines, rely on expensive or unreliable verifiers, and still produce static environments. To alleviate the engineering burden of rebuilding environments from scratch, we propose Environment Harness (EnvHarness), a programmable layer of plug-in components that wraps a static environment to reshape its behavior without modifying the underlying logic. Operating through standard interfaces, EnvHarness applies across diverse domains while ensuring every reshaped environment retains its original verifier. To automate this process, we introduce EnvRigger, which treats the target policy as a black box, observing its execution trajectories to synthesize EnvHarness components targeting diagnosed flaws, and validating them via fresh rollouts. Across five benchmarks in four domains, EnvHarness outperforms both original environments and domain-specific environment generation pipelines, achieving up to a 9.0-point improvement on held-out instances with 9.8% fewer execution steps. Furthermore, EnvHarness provides a superior optimization signal for reinforcement learning, enabling continuous, targeted co-evolution of the policy and its environment.
Chengsong Huang, Zifeng Wang, Rujun Han +14
Washington University in St. Louis · Google Cloud AI Research · University of North Carolina at Chapel Hill +1