ALIGN: Aligning Condition Estimation and Expression in Advantage-Conditioned Behavior Cloning

Zechen Wang; Zhenqi Qiu; Houlong Xiong; Suohang Zhang; Yiyu Ren; Wanting Xu; Hongfei Niu; Chengyang He

cs.ROOct 19, 2026

ALIGN: Aligning Condition Estimation and Expression in Advantage-Conditioned Behavior Cloning

Authors: Zechen Wang, Zhenqi Qiu, Houlong Xiong, Suohang Zhang, Yiyu Ren, Wanting Xu, Hongfei Niu, Chengyang He,

Organizations: PrimeBot · ShanghaiTech University · Shanghai Jiao Tong University · Zhejiang University ·

Abstract

Human-collected robot manipulation data often mix successful behavior with retries, pauses, recoveries, and failures. Advantage-Conditioned Behavior Cloning (AC-BC) can exploit such mixed-quality data, but only if the conditioning signal is both accurately constructed from offline trajectories and effectively incorporated into the policy's action-vector-field prediction. Existing pipelines can weaken this signal by thresholding estimator-derived advantage scores into noisy conditions, and by routing conditions through prompts that weakly affect the action vector field. We introduce ALIGN, a condition-aligned AC-BC framework for vision-language-action policies. ALIGN directly predicts categorical pairwise conditions in the policy-facing label space and injects them into a DiT action expert through AdaRMS modulation fused with flow time. We evaluate ALIGN on deliberately mixed real-robot datasets for ARX X5 bowl stacking and Franka key hanging, each combining standard demonstrations with retry-heavy trajectories. ALIGN improves condition fidelity, strengthens conditional vector-field separability, and achieves 90.0% and 56.7% closed-loop success, compared with 56.7% and 33.3% for the strongest AC-BC baseline. These results indicate that aligning conditions from estimation to action prediction is an effective way to extract robust manipulation policies from data in which success, recovery and error coexist.

Figures & tables

Figure 1: Overview of ALIGN. Left: condition estimation. Given a historical observation, the current observation, robot states, and the task instruction, the direct categorical estimator predicts the AC-BC condition $\hat{c}_{t}$ from an ordered observation pair. Right: AC-BC policy training with estimated conditions. The current observation stream is encoded by the VLM, while AdaRMS combines $\hat{c}_{t}$ with the flow-time embedding to generate per-layer modulation parameters for the DiT action expert, so the predicted action vector field is explicitly conditioned on the estimated training condition.

Figure 2: Validation-set episodes for ARX X5 bowl stacking and Franka key hanging.

		ARX X5 bowl stacking			Franka key hanging
Estimator	Output space	Precision	Recall	F1	Precision	Recall	F1
$\pi^{*}_{0.6}$ -style value	continuous $\rightarrow\{+,-\}$	98.1 %	71.1%	82.5%	91.4%	85.8%	88.5%
$\chi_{0}$ -style stage advantage	continuous $\rightarrow\{+,-\}$	97.8%	70.9%	82.3%	92.7 %	75.6%	83.3%
Direct tri-class	categorical $\rightarrow\{+,-\}$	97.6%	79.0%	87.3%	90.6%	86.5%	88.5%
Direct binary-class (ALIGN)	$\{+,-\}$	97.7%	79.2 %	87.5 %	90.7%	89.1 %	89.9 %

Table 1: Condition-estimator ACF on held-out ARX X5 and Franka validation sets.

Figure 3: Estimator-induced AC-BC conditions on a held-out trajectory.

Figure 4: Condition-dependent vector-field separation across injection mechanisms. Left: t-SNE of vector-field effects at flow time $t=0.2$ . Right: CVFS over flow time, where stronger post-backbone injection yields larger separation than prompt conditioning.

Figure 5: Condition-injection ablation with a fixed direct estimator. Bars report flow-time-averaged

\overline{\mathrm{CVFS}}

and 30-rollout closed-loop success for ARX X5 bowl stacking and Franka key hanging. Hatched bars denote

\overline{\mathrm{CVFS}}

on the left axis, while solid bars denote success rate on the right axis.

Policy variant	Condition source	Injection	ARX bowl stacking	Franka key hanging
Condition construction comparison
SFT, 100 demonstrations	none	none	46.7%	6.7%
SFT, 100 demonstrations + 100 retry trajectories	none	none	50.0%	3.3%
$\pi^{*}_{0.6}$ -style threshold	continuous progress	prompt/default	50.0%	26.7%
$\chi_{0}$ -style threshold	continuous stage advantage	prompt/default	56.7%	33.3%
2-class direct estimator	binary categorical	prompt	70.0%	40.0%

Table 2: Closed-loop policy success over 30 rollouts on ARX bowl stacking and Franka key hanging.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 6 — Figure 6: Real-robot hardware platforms used in our experiments. Left: the ARX X5 platform used for bowl stacking. Right: the Franka platform used for key hanging.

Platform	Robot embodiment	End effectors	Cameras	Control
ARX X5	Dual 6-DoF arms	Dual 1-DoF parallel grippers	Front RGB + two wrist RGB, 30 Hz	100 Hz position control
Franka Research 3	Dual 7-DoF arms	Dual parallel grippers	Front RGB + two wrist RGB, 30 Hz	200 Hz Cartesian impedance

Appendix

Table 3: Real-robot hardware and observation interfaces used in our experiments.

Annotation rule	$\pi^{*}_{0.6}$ F1	$\chi_{0}$ F1	Binary F1	Tri F1	Best F1
Baseline manual labels	82.5	82.3	87.5	87.3	Binary
Ignore boundary by 1 frame	83.2	83.0	88.4	88.3	Binary
Ignore boundary by 5 frames	83.3	83.2	88.6	88.4	Binary
Ignore boundary by 10 frames	83.5	83.3	88.7	88.5	Binary
Expand bad window by 1 frame	82.5	82.2	87.5	87.3	Binary
Expand bad window by 5 frames	82.5	82.2	87.5	87.3	Binary

Appendix

Table 4: Annotation-boundary robustness of ACF-F1 on the held-out ARX X5 validation set. The raw row gives the baseline manual labels reported in Table 1 ; other rows perturb the manually annotated retry/failure boundaries to account for boundary bias.

Annotation rule	$\pi^{*}_{0.6}$ F1	$\chi_{0}$ F1	Binary F1	Tri F1	Best F1
Baseline manual labels	88.5	83.3	89.9	88.5	Binary
Ignore boundary by 2 frames	89.2	83.9	89.5	89.2	Binary
Ignore boundary by 5 frames	90.1	84.8	90.4	90.1	Binary
Ignore boundary by 10 frames	91.7	86.3	92.0	91.7	Binary
Expand bad window by 2 frames	88.4	83.3	88.7	88.4	Binary
Expand bad window by 5 frames	88.3	83.4	88.5	88.2	Binary

Appendix

Table 5: Annotation-boundary robustness of ACF-F1 on the held-out Franka validation set. The main text uses the baseline manual labels; other rows perturb the manually annotated retry/failure boundaries to account for boundary bias.

Hyperparameter	Value
Policy backbone	$\pi_{0.5}$ -style VLA policy with a flow-matching action expert
Action horizon	50
Training precision	bfloat16
Optimizer	AdamW
AdamW betas / epsilon	$(0.9,0.95)$ / $10^{-8}$
Weight decay	$10^{-10}$

Appendix

Table 6: Policy fine-tuning settings.

Setting	Value
Pair sampling	randomly sample two frames from the same trajectory without sorting their temporal order
Direct-estimator objective	cross-entropy on binary $g^{(2)}_{\tau}(j-i)$ or three-class $g^{(3)}_{\tau}(i,j)$ labels
Value/progress objective	MSE regression for $V_{\psi}(o_{t})$ on scalar value or progress target
Local comparison horizon	$N=50$
Retry validation window	$K_{r}=20$ frames
Failure validation window	$K_{f}=30$ frames

Appendix

Table 7: ALIGN and

\pi^{*}_{0.6}

-style estimator settings.

Setting	Value used in our reproduction
Pair sampling for SA	randomly sample two frames from the same trajectory without sorting their temporal order
SA supervision target	pairwise stage-advantage target $A^{\chi}_{i,j}$
Stage annotation	semantic sub-goals; 4 stages for ARX X5 bowl stacking and 2 stages for Franka key hanging
Positive-label rule	rank local stage-advantage scores; label the top 30% as positive and the rest as negative
Local comparison horizon	$N=50$ , same as ALIGN and $\pi^{*}_{0.6}$ -style baselines
Optimizer	AdamW, $(\beta_{1},\beta_{2})=(0.9,0.95)$ , $\epsilon=10^{-8}$

Appendix

Table 8: Matched

\chi_{0}

-style estimator settings.

Task	Prompt-only success	AdaRMS success
Unscrew bottle cap	0.0%	73.3%
Stack bowls	26.7%	80.0%
Hang key	0.0%	50.0%

Appendix

Table 9: Multitask categorical-condition injection on the Franka platform. Each row reports closed-loop success on the evaluated physical task. Prompt-only uses only the task-specific language prompt, while AdaRMS uses the same prompt together with the matched task category.

Method	Success (%)	First sub-goal (%)	Mean time (s)
Demo	28.0	36.0	50.2
Demo + retry	34.0	46.0	51.7
$\pi^{*}_{0.6}$ thresh. + AdaRMS	64.0	70.0	42.5
Filtered-SFT	76.0	86.0	50.5
ALIGN	86.0	90.0	43.2

Appendix

Table 10: Additional AGI G2 results, with 50 rollouts per method. The first sub-goal is engaging the handle. The final column is the mean time-to-success over successful rollouts only.

Data pool	Composition	Total
Estimator training	400 standard	400
Policy training	100 standard + 100 retry-heavy	200
Held-out ACF	100 standard + 100 retry + 50 failure	250

Appendix

Table 11: Trajectory counts per single-task setting. The estimator-training, policy-training, and held-out ACF pools are disjoint.

Policy	ARX X5	Franka
Condition construction comparison
SFT, standard	14/30; 46.7 [30.2, 63.9]	2/30; 6.7 [1.8, 21.3]
SFT, standard + retry	15/30; 50.0 [33.2, 66.8]	1/30; 3.3 [0.6, 16.7]
$\pi^{*}_{0.6}$ , prompt/default	15/30; 50.0 [33.2, 66.8]	8/30; 26.7 [14.2, 44.4]
$\chi_{0}$ , prompt/default	17/30; 56.7 [39.2, 72.6]	10/30; 33.3 [19.2, 51.2]
Direct binary, prompt	21/30; 70.0 [52.1, 83.3]	12/30; 40.0 [24.6, 57.7]

Appendix

Table 12: Success counts and 95% Wilson score intervals for every policy entry in Table 2 , with

n=30

rollouts per policy and task. Each cell reports

k/n

; success percentage [95% interval], with interval endpoints also in percent. SFT, standard uses 100 standard trajectories; SFT, standard + retry uses 100 standard and 100 retry-heavy trajectories. These are marginal intervals for each policy, not confidence intervals for paired method differences.

Injection	Condition space	ARX X5 (%)	Franka (%)
Prompt	Binary	70.0	40.0
Prompt	Three-class	66.7	20.0
Prompt	Four-class	66.7	16.7
AdaRMS	Binary	90.0	56.7
AdaRMS	Three-class	86.7	33.3

Appendix

Table 13: Closed-loop success for different condition granularities. Four-class results are reported for prompt injection only.

Robot Imitation Learning

Robot imitation learning trains robots to perform tasks by reproducing and adapting behaviors shown in demonstrations.

See all 659 papers

Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment
Jul 15, 2026Dwip Dalal, Shivansh Patel, Chahit Jain +8VLA Model Training Vision-Language-Action Models
University of Illinois Urbana-Champaign · Texas A&M University · University of California, Irvine
Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation
Oct 14, 2026Xiaoman Li, Zhiqiang Yang, Jiayu Hu +6Robot Imitation Learning Behavior Cloning
Zhejiang University, State Key Lab of CAD & CG · The Institute of Artificial Intelligence, China Telecom (TeleAI) · South China University of Technology +4
Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments
Jun 14, 2026Mingyu Liu, Zeju Li, Jiuhe Shu +4Robot Skill Learning Robotic Manipulation
Zhejiang University · Hong Kong University of Science and Technology (GZ) · Shanghai Innovation Institute

Jul 15, 2026J/K move · Enter open · S save

Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

Dwip Dalal, Shivansh Patel, Chahit Jain, Jeonghwan Kim+7

University of Illinois Urbana-Champaign · Texas A&M University · University of California, Irvine

VLA Model Training Vision-Language-Action Models Vision-Language Alignment Robustness of VLA Models VLA Model Adaptation

Open paper AskPDF arXiv

Finetuning a pretrained vision-language model (VLM) on robot demonstrations via behavior cloning (BC) has become the standard recipe for vision-language-action (VLA) policies. However, BC finetuning progressively overwrites the pretrained representations that support visual and semantic generalization. Co-training on web image-text data, a common remedy, applies language and action losses to separate observations, leaving VLAs with language-action misalignment that standard manipulation benchmarks do not expose. We propose Anchor-Align, which augments BC with two objectives: Vision-Language Anchoring distills layer-wise representations from a frozen VLM copy to prevent this drift, while Language-Action Alignment converts each action target into a discrete motion-direction label and jointly trains language and action prediction on the same robot observation. We conduct real-world evaluations across eight manipulation settings on single-arm xArm7 and bimanual YAM robots, using two VLA architectures with regression and flow-matching action heads. Across these settings, Anchor-Align consistently improves over BC on novel targets, layouts, and motion-sensitive bimanual tasks requiring coordinated control. At scale in simulation, we demonstrate consistent improvements on OOD perturbations, perceptual robustness, and long-horizon control across LIBERO-PRO, LIBERO-Plus, and CALVIN, respectively, suggesting that preserving pretrained representations and effective action learning are not fundamentally at odds. Project page: anchoralignvla.github.io

Abstract

Robot Imitation Learning

Explore similar work

Generalizable VLA Finetuning via Representation Anchoring and Language-Action Alignment

Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation

Perfect Demo Makes Poor Teacher: Learning Robust Alignment from Critical Motion Segments