Making two action heads agree: coordination mechanisms and a runtime collapse certificate for flow-matching policies

Jinhui Sun; Wei Zhou; Bowen Yang; Xinliang Xiao; Li Yang

cs.ROAug 16, 2026

Making two action heads agree: coordination mechanisms and a runtime collapse certificate for flow-matching policies

Authors: Jinhui Sun, Wei Zhou, Bowen Yang, Xinliang Xiao, Li Yang

Organizations: School of Automation, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China

Abstract

A dual-representation flow-matching policy decodes each predicted motion into joint and end-effector spaces, and the residual between the two kinematically equivalent decodings provides a physically interpretable runtime signal. On multimodal tasks, however, independently sampled branches may choose different valid modes, causing false alarms. We study how to coordinate the two branches and at what cost. Across two robot environments and a non-robotic testbed, the tested mechanisms fall into four classes. An auxiliary latent shared by both branches but absent from the flow-matching construction is erased at the population optimum, a provable dead end confirmed within a prespecified 2% equivalence band. Sharing source noise can coordinate or anti-coordinate: its effect changes sign with the representation map and tracks the alignment of decoder mode basins. Consistency regularization gives intermediate coordination but reduces the valid-pair rate, while training-supported discrete partitions achieve near-ceiling coordination robustly. We further derive a chance-corrected coordination bound based only on each branch's Gini-Simpson diversity, yielding an attainable region and a label-free certificate that separates coordination from collapse when zero mismatch is ambiguous. On LIBERO-Plus, benign multimodality adds 1.57 percentage points of false alarms to the residual, which remains the strongest evaluated failure signal; the preregistered token intervention does not meet its false-alarm criterion or produce a seed-robust detection change. Code, models, and per-run configurations are available at https://github.com/kimo423/dual-head-coordination.

Figures & tables

Figure 1: Attainable region for the two robot environments ( $K=2$ ). The horizontal axis is modal-diversity capacity $C$ and the vertical axis is the chance-corrected effect $\Delta=\mathrm{BL}-\mathrm{obs}$ , both formed per condition and aggregated with both-valid weights. The diagonal is the feasibility boundary. Each marker is one training run; fill indicates valid-pair rate. Seventeen runs have exact both-valid marginals. The remaining 50 use stored head-valid marginals and are therefore approximate; six lie at most $0.6\%$ above the line, whereas every re-estimable condition satisfies the bound under exact accounting. The dashed path is the shared-source-noise consistency sweep.

Figure 2: Why two correct branches can disagree. The same condition is decoded into two kinematically equivalent action representations. Independent samples can select different valid routes, so the reconciled forward-kinematics residual becomes large even though both endpoints are correct. The drawing is schematic; measured decompositions appear in Section 7.1 .

This paper	Agreement-statistics object
$\mathrm{obs}$	observed disagreement $1-p_{o}$
$\mathrm{BL}=1-\sum_{k}q_{k}p_{k}$	chance disagreement, own-marginal convention ( Cohen, 1960 )
$\Delta=\mathrm{BL}-\mathrm{obs}$	numerator of Cohen’s kappa; at $K{=}2$ , $2(P_{LL}P_{RR}-P_{LR}P_{RL})$
$\Delta/\mathrm{BL}$	Cohen’s kappa
$C_{K}=\sqrt{G(q)G(p)}$	denominator of Gorodkin’s $R_{K}$ ( Gorodkin, 2004 )
$\rho=\Delta/C_{K}$	$R_{K}$ ; at $K{=}2$ , the phi coefficient/MCC ( Matthews, 1975 )

Table 1: Correspondence with chance-corrected agreement statistics (arbitrary

K

		head-valid marginals			exact both-valid marginals			excursions
Family	$n$	$\Delta$	$C$	$\rho$	$\Delta$	$C$	$\rho$	HV $\to$ BV
C $z_{s}$ dim-0 anchor	3	$+0.0020$	0.4858	$+0.4\%$	$+0.0020$	0.4857	$+0.4\%$	0/0 $\to$ 0/0
D continuous $z_{s}$ prior	3	$-0.0006$	0.4708	$-0.1\%$	$-0.0009$	0.4708	$-0.2\%$	0/0 $\to$ 0/0
E posterior (3 KL scales)	5	$+0.0260$	0.0256	$+101.7\%$	$+0.0223$	0.0223	$+100.0\%$	301/303 $\to$ 0/0
H episode token, uniform	3	$+0.4481$	0.4885	$+91.7\%$	$+0.4339$	0.4766	$+91.1\%$	3/107 $\to$ 0/0
I episode token, learned prior	3	$+0.4876$	0.4907	$+99.4\%$	$+0.4710$	0.4810	$+97.9\%$	68/217 $\to$ 0/0

Table 2: Exact-accounting re-estimation: head-valid versus both-valid marginals from the same contingency tables, for the 17 of 67 runs whose artifacts store both. Run-level

N_{\mathrm{BV}}

-weighted mean of per-condition quantities, then mean over seeds. Last column: per-condition excursions above

C

and above

C_{K,\mathrm{tight}}

, pooled over the family.

Claim	Evidence	Evidential status
$\|\Delta(c)\|\leq C_{K}(c)$	Algebraic bound; no violation in 1275 exact-accounting robot conditions or in the testbed evaluations	Established : identity, not an empirical finding
Auxiliary latent outside the CFM construction	Two robot environments, three seeds each; TOST at $\delta=2\%$ ; $\hat{d}=1.16/1.66\%$	Supported as finite-model equivalence ; the exact zero is the population prediction under (A1)–(A2), (A3)–(A5)
Shared source noise	v2 eight-seed mean $-0.5\%$ (sd $3.4$ , four signs each way); testbed factorial at ten seeds per cell: interaction range 90.1 pp, permutation $p=10^{-4}$	Map dependence preregistered-and-confirmed ; seed-unstable on the robots; quote with seed and representation map
Endpoint consistency coordinates without a data-derived partition	$\rho=50.3\%$ in v2 and $63.2\%$ in v3; valid-pair rate $0.768\to 0.525$ in v2	Supported in the tested robot regimes , with the validity cost part of the result
Partition tokens reach the robot-side high-capacity corner	Robot-side tokens reach $\rho=91.1$ – $100\%$ ; an amortized posterior reaches the same region on the balanced testbed	Supported as a robust sufficient robot-side route , not as a necessary condition in general
Benign multimodality raises residual false alarms	Residual surcharge $+1.57$ pp, CI $[+0.70,+2.46]$ ; observation-side control $-0.01$ pp	Preregistered and supported on this benchmark and operating point

Table 3: Evidence status of the headline claims. A status applies only within the regime stated in the evidence column.

Figure 3: Mechanism summary for BimodalBypass, averaged over training seeds. The first two columns give capacity and coordination effect; the remaining columns report modal entropy, valid-pair rate, and cross-branch residual. Each column has its own scale. The near-origin posterior family illustrates why the normalized effect must be read with capacity:

\rho=100\%

accompanies

C=0.022

#	Family	Shared	$n$	$\mathrm{obs}$	$\mathrm{BL}$	$\Delta$	$C$	$\rho$	Valid	Res. (mm)	Site
A	independent, $\beta 1\,\lambda 0$	—	3	0.4826	0.4839	$+0.0014$	0.4474	$+0.3\%$	0.7729	84.79	I
B	shared source noise, $\beta 0\,\lambda 0$	noise	3	0.4362	0.4369	$+0.0007$	0.4185	$+0.2\%$	0.7679	82.42	I ‡
C	continuous $z_{s}$ , dim-0 anchor	—	3	0.4921	0.4941	$+0.0020$	0.4857	$+0.4\%$	0.7758	87.02	I
D	continuous $z_{s}$ prior, $d{=}16$	$z{\sim}\mathcal{N}$	3	0.4843	0.4834	$-0.0009$	0.4708	$-0.2\%$	0.8526	85.18	I
E	continuous $z_{s}$ posterior (3 KL)	$z{\sim}q$	5	0.0000	$=C$	$+(0.0001$ – $0.0625)$	0.0001–0.0625	$+100.0\%$	0.9292	15.78	II
F	endpoint consistency $\lambda 0.3$ (+B)	loss	3	0.4235	0.4492	$+0.0257$	0.4363	$+5.9\%$	0.7830	74.17	III

Table 4: Mechanism study in the BimodalBypass (v2) robot environment, using per-condition estimates;

n

is the number of training runs and

K=2

. Sites I–IV denote the floor, near-origin boundary, interior, and high-capacity corner. Families H and I use exact both-valid marginals; the remaining rows use the stored convention stated in the supplement.

Family	$n$	$\mathrm{obs}$	$\mathrm{BL}$	$\Delta$ (per seed)	$C$	$\rho$	Valid	Resid. (mm)
A independent	3	0.4811	0.4811	$0.0000$ ( $-0.0005/{-}0.0016/{+}0.0020$ )	0.4700	$0.0\%$	0.7382	98.5
B shared source noise	3	0.4567	0.4652	$+0.0085$ ( $+0.0079/{+}0.0103/{+}0.0074$ )	0.4555	$+1.9\%$	0.7501	95.7
D continuous $z_{s}$ prior	3	0.4723	0.4748	$+0.0025$ ( $+0.0019/{+}0.0019/{+}0.0037$ )	0.4636	$+0.5\%$	0.6474	96.9
G endpoint consistency $\lambda{=}3$	3	0.1709	0.4458	$+0.2748$ ( $+0.3097/{+}0.2090/{+}0.3057$ )	0.4319	$+63.2\%$	0.4449	60.8
H episode token, uniform draw	2	0.0784	0.4708	$+0.3923$ ( $+0.4232/{+}0.3614$ )	0.4661	$+84.1\%$ (88.6/79.6)	0.7056	38.9
K oracle	1	0.0000	0.4994	$+0.4994$	0.4964	$+100.0\%^{\dagger}$	0.8704	13.79

Table 5: RotatingBypass (v3) results at the coverage-matched threshold

\tau=0.18

. Values are per-condition estimates averaged over seeds. The

\rho

column averages seed-level ratios; † denotes a head-valid estimate slightly above one.

Figure 4: Run-level normalized effects

\rho=\Delta/C

. Points are shown against zero, the measured structural-null range, and the prespecified equivalence band

\delta=\pm 2\%

. Error bars are 95% condition-bootstrap intervals; TOST decisions use the corresponding 90% intervals. Shared source noise is shown separately because it couples the sampled flows and is not covered by Lemma 2 .

v2 family	$\bar{\rho}$	sd	v3 family	$\bar{\rho}$	sd
A independent (6)	$+0.27\%$	0.27	A independent (6)	$+0.00\%$	0.20
B shared noise (8)	$-0.54\%$	3.41	B shared noise (8)	$+0.48\%$	2.09
G consistency $\lambda 3$ (8)	$+45.3\%$	8.72	G consistency $\lambda 3$ (6)	$+41.9\%$	1.79
$\lambda{=}1$ rung (5)	$+20.2\%$	5.18	episode token (6)	$+78.5\%$	3.26
$\beta{=}1,\lambda{=}3$ control (4)	$+0.58\%$	0.48	continuous $z_{s}$ prior (6)	$+0.48\%$	0.19
episode token, learned (8) bv	$+97.2\%$	1.36

Table 6: Preregistered seed expansion: seed-level occupancy

\rho

(head-valid caliber unless noted; seed count in parentheses). Between-seed sd in brackets.

Configuration	seed	$\mathrm{obs}$	Valid	$q_{L}$ / $p_{L}$	Entropy (nats)	$\mathrm{BL}$	$\Delta$	$C$	$\rho$ (pool/p.-c.)	Resid.
$\beta_{\mathrm{KL}}{=}1$ , fb 0.1	0	0.0000	0.8804	0.992/0.986	0.046/0.074	0.0158	$+0.0158$	0.0158	104.1%/100.0%	17.61
$\beta_{\mathrm{KL}}{=}1$ , fb 0.1	1	0.0000	0.9087	0.992/0.988	0.048/0.067	0.0163	$+0.0163$	0.0163	102.1%/100.0%	15.95
$\beta_{\mathrm{KL}}{=}0.01$	0	0.0000	0.9620	0.990/0.991	0.057/0.051	0.0170	$+0.0170$	0.0170	100.2%/100.0%	14.76
$\beta_{\mathrm{KL}}{=}0.01$	1	0.0000	0.9010	0.032/0.039	0.143/0.166	0.0625	$+0.0625$	0.0625	100.5%/100.0%	15.56
$\beta_{\mathrm{KL}}{=}1$ , fb 0.5	0	0.0000	0.9937	1.000/1.000	0.000/0.000	0.0001	$+0.0001$	0.0001	100.0%/100.0%	15.04
(ref.) window token $K{=}2$	0	0.0000	0.9800	0.502/0.501	0.693/0.693	0.5000	$+0.5000$	0.5000	100.0%/100.0%	22.51

Table 7: Amortized-posterior runs under exact both-valid accounting, with a window-token reference. Each posterior run has

\mathrm{obs}=0

and

\Delta=C

, so the normalized effect is 100% despite capacities close to zero. The pooled ratio is shown only to illustrate the aggregation effect.

Figure 5: Endpoint-consistency sweep. (a) Paths through the

(C,\Delta)

plane for the

\beta\times\lambda

grid. With shared source noise (

\beta=0

), coordination increases while capacity changes little; with independent source noise (

\beta=1

), the path remains near the floor. (b) Valid-pair rate for the same runs. (c) Signed normalized effects on the non-robotic testbed for

K\in\{2,4,8\}

, with three seeds per cell.

Figure 6: Two shared-variable constructions. An auxiliary random input that is irrelevant to the CFM regression target is ignored at the population optimum (top). A low-cardinality symbol fitted from training trajectories can be learned by both branches and coordinate their samples (bottom). The symbol provides a shared partition; it need not coincide with a semantic task label.

Figure 7: Effect of mode count on the balanced non-robotic testbed. (a) Normalized effects for five mechanism families across

K\in\{2,3,4,8\}

, averaged over three seeds; error bars show the seed range and the shaded line marks the prespecified

85\%

criterion. The unsupervised token degrades at

K=4

and

8

, whereas the low-KL posterior remains above the criterion. (b) All 186 runs in relative-capacity–effect coordinates. The construction keeps capacity near its maximum; unequal target probabilities in Fig. 10 provide the complementary capacity sweep.

Figure 8: Robot-environment and balanced-testbed runs in one attainable region. (a) Raw coordinates for

K=2

; (b) the high-capacity region magnified; (c) both axes normalized by

\max C_{K}=1-1/K

to compare

K\in\{2,3,4,8\}

. The balanced testbed stays near maximum capacity by construction, whereas the robot runs span the capacity axis and include the collapsed-posterior cluster. Pronounced negative effects occur only in map-dependent testbed families.

Family	$K{=}2$	$K{=}3$	$K{=}4$	$K{=}8$
amortized posterior, $\beta_{\mathrm{KL}}{=}0.01$	100.0%	—	97.1%	94.3%
amortized posterior, $\beta_{\mathrm{KL}}{=}1$ , fb 0.5	86.4%	—	75.6%	63.5%
oracle token, uniform test draw	82.4%	87.3%	84.9%	80.1%
oracle token, learned prior	83.4%	—	84.5%	80.4%
unsupervised k-means token	77.8%	78.1%	48.7%	51.2%

Table 8: Normalized coordination effect

\rho

across mode counts on the balanced testbed. Bold entries meet the prespecified high-capacity criterion

\rho\geq 85\%

and

C_{K}\geq 0.80\max C_{K}

. All rows have

C_{K}\geq 0.992\max C_{K}

by construction.

Figure 9: Environment and representation-map checks. (a) Best single-coordinate AUROC for mode prediction in BimodalBypass and RotatingBypass; the fixed-coordinate shortcut present in v2 is substantially reduced in v3. (b) Coefficient of variation of the mode gap in the two robot environments and the non-robotic testbed. (c) Normalized effect

\Delta/C_{K}

under three representation maps. The floor and symbol-based families vary by less than 0.7 percentage points, whereas shared source noise and the

\beta=0

consistency family change sign over ranges of 90.6 and 99.7 points.

Family	nullspace	orthogonal	polar	Range
independent / continuous $z_{s}$ prior	$+0.02\%$ / $+0.10\%$	$+0.37\%$ / $+0.30\%$	$+0.15\%$ / $+0.17\%$	0.2–0.4 pp
unsupervised token / oracle token	$+77.8\%$ / $+82.4\%$	$+77.5\%$ / $+82.4\%$	$+77.9\%$ / $+83.0\%$	0.4–0.7 pp
shared source noise	$\mathbf{-22.7\%}$	$\mathbf{-9.2\%}$	$\mathbf{+67.8\%}$	90.6 pp
$\beta{=}0$ consistency, $\lambda{=}3$	$\mathbf{-7.3\%}$	$\mathbf{+10.9\%}$	$\mathbf{+92.4\%}$	99.7 pp

Table 9: Occupancy

\rho

by representation map (

K{=}2

; family pairs quoted first/second).

Figure 10: Unequal-probability testbed (153 runs, null-space map, $\tau=0.55$ , NFE 20). (a) Relative capacity as the target prior becomes more imbalanced. Uniform token draws retain high generated diversity even when it no longer matches the target frequencies. (b) Normalized coordination and marginal distortion under uniform and learned-marginal token draws. Matching the token marginal improves both. (c) Amortized-posterior behavior over KL weight and target skew; each cell reports modal entropy and relative capacity. Strong skew with a nonzero KL term reproduces the low-capacity, zero-mismatch regime observed in the robot experiments.

KL pressure	symm. .50	$p{=}.65$	$p{=}.80$	$p{=}.92$	tight .50	tight .80
$\beta_{\mathrm{KL}}=0$	.963 / 1.000	—	.904 / .998	.960 / .999	.963 / .999	.962 / .999
$\beta_{\mathrm{KL}}=0.01$	.992 / 1.000	.997 / 1.000	.994 / 1.000	.006 / .017	.997 / .999	.601 / .695
$\beta_{\mathrm{KL}}=0.1$	.997 / 1.000	—	.521 / .624	.106 / .189	.993 / .999	.762 / .820
$\beta_{\mathrm{KL}}=1$ (fb 0.1)	.994 / 1.000	—	.259 / .368	—	—	—

Table 10: The collapse switch: amortized posterior across KL pressure

\times

target skew (cells:

C_{K}/\max C_{K}

/ modal entropy

\div\ln K

; “tight” = reduced separability).

Figure 11: Residual decomposition in BimodalBypass. (a) Per-condition median residuals for same-mode and cross-mode sample pairs, grouped by mechanism family. (b) Full, lateral, and label-orthogonal residual components for one uncoordinated run; the separation is concentrated in the coordinate used by the mode assignment. (c) Cross-mode contribution to the both-valid residual mean, one point per run. Stronger coordination reduces this contribution from roughly

85\%

to nearly zero, while the cross-mode magnitude itself remains set by the task geometry.

Quantity	v2 (10 runs)	v3 (9 runs)
same-mode floor $\mathbb{E}[D\mid\mathrm{same}]$	18.0–24.0 mm	25.0–33.5 mm
cross-mode residual $\mathbb{E}[D\mid\mathrm{cross}]$	151.4–159.5 mm	174.7–183.9 mm
both-valid mean $\mathbb{E}[D]$	77.3–88.5 mm	87.0–105.2 mm
$\mathbb{E}[D]$ / same-mode floor	3.32–4.68 $\times$	2.96–3.96 $\times$
cross-mode contribution share	82.8–89.7%	81.2–87.0%

Table 11: Residual decomposition for uncoordinated families; ranges are over runs.

Object	Mechanism	$\mathrm{obs}$	$\Delta$	Valid-pair rate (uncoupled)
cross-representation (family B)	shared source noise	0.4362	$+0.0039$	0.768 (0.773)
ensemble b1_l0 (ten pairs)	shared $x_{0}$	0.1434	$+0.3632$	0.9305 (0.9296)
ensemble b0_l0	shared $x_{0}$	0.1859	$+0.3259$	0.9330 (0.9334)
ensemble b0_l3	shared $x_{0}$	0.1079	$+0.4003$	0.8353 (0.8303)
ensemble, mixed objectives	shared $x_{0}$	0.0551	$+0.4130$	0.8656 (0.8587)

Table 12: Effect of shared initial noise across representations and across ensemble members.

\Delta

is reported on the pooled accounting used for this comparison; per-condition values are 0.0018 lower on average.

$K$	fail	succ	coord. residual	uncoord. residual	$\max\|z\|$	RND	ACE
50	369	482	0.4878	0.5226	0.4808	0.5137	0.3080
75	369	482	0.5740	0.6212	0.5352	0.5756	0.3590
100	369	312	0.6200	0.6856	0.5285	0.5662	0.5458
125	369	99	0.6139	0.7309	0.5237	0.6066	0.5269
150	369	48	0.7667	0.9000	0.3333	0.4000	0.8667

Table 13: Fixed-horizon detection AUROC (stratified, EWMA-max aggregation, monitor seed 0) as the truncation length

K

varies. Failures are 250-step timeouts, so all 369 remain at every

K

; successes shorter than

K

drop out.

Signal	Family	Stratified AUROC
uncoordinated residual, 16-sample mean	cross-representation	0.8780
uncoordinated residual	cross-representation	0.8479
coordinated residual, 16-sample mean	cross-representation	0.8401
coordinated residual	cross-representation	0.8209
random-network distillation ( Römer et al., 2025 )	observation side	0.7524
$\max\|z\|$	observation side	0.7066

Table 14: Secondary episode-level analysis on 851 test episodes (369 timeout failures), stratified over 20 perturbation–severity cells, for the first monitor seed. Values are affected by episode length and are used only for relative ordering.

Signal	FA multimodal	FA unimodal	Gap (pp)	95% CI (pp)
coordinated residual	9.75%	8.19%	$+1.57$	$[+0.70,+2.46]$
uncoordinated residual	8.52%	6.98%	$+1.54$	$[+0.71,+2.46]$
$\max\|z\|$	5.41%	5.42%	$-0.01$	$[-1.17,+1.30]$
random-network distillation	5.33%	6.38%	$-1.05$	$[-2.17,+0.14]$
action-chunk entropy	5.23%	5.63%	$-0.39$	$[-1.38,+0.51]$

Table 15: False-alarm rates on successful test episodes, split by the prespecified multimodality flag. Thresholds are calibrated separately for each signal at 5%; intervals use episode-level cluster bootstrap.

	Token partition			Local partition
Arm	$\Delta$	$C$	$\rho$	$\Delta$	$C$	$\rho$
coordinated	$\mathbf{+0.000135}$	0.0055	$+2.5\%$	$+0.000020$	0.0281	$+0.1\%$
independent tokens	$-0.000007$	0.0055	$-0.1\%$	$+0.000004$	0.0282	$+0.0\%$
shuffled pairing	$-0.000003$	0.0055	$-0.1\%$	$-0.000072$	0.0281	$-0.3\%$
uncoordinated network	$+0.000012$	0.0068	$+0.2\%$	$+0.000049$	0.0309	$+0.2\%$
prior entropy $>0.30$ nats

Table 16: Coordination on the LIBERO-Plus test split. Each test step is a condition; estimates use 32 samples per branch, both-valid marginals, and a three-seed mean. The last two rows retain only steps whose token-prior entropy exceeds 0.30 nats.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Round	Environment	Runs	Evaluation
Mechanism atlas	BimodalBypass (v2)	68 total	75 conditions, 500 pairs
	RotatingBypass (v3)	68 total	36 conditions, 240 pairs
Threshold / NFE sensitivity	v2 and v3	(no retraining)	108 re-evaluations
Ensemble protocols	v2 only	4 families, 5 members	4 protocols, 10 pairs (main family)
Testbed, balanced	RotatingRing	186 $\times$ 50k steps	96,000 samples/run; 135 re-evaluations
Testbed, skewed prior	RotatingRing	153 (51 cells $\times$ 3 seeds)	75 conditions/run; 15.1 GPU-hours

Appendix

Table S1: Run inventory. “Runs” counts trained models; evaluation columns count what each run was scored on. The original mechanism atlas comprises 68 training runs, of which 67 carry per-condition coordinates and appear in Fig. 1 (52 v2, 15 v3). The preregistered seed expansion adds 49 robot runs and 84 testbed runs (design, seed lists, and analysis script committed before training and before unblinding, with the registration and its two revision records archived alongside the artifacts); the expansion runs are reported in Table 6 and Section 6.3 and do not enter Fig. 1 .

Family	$\mathrm{obs}$	$\mathrm{BL}$	$\Delta$	$C$	$\rho$	Marg.
A independent $\beta 1\lambda 0$ (3s)	0.4826	0.4839	$+0.0014$	0.4474	0.3%	head
A seed0/1/2			$-0.0006/{+}0.0029/{+}0.0017$	0.4439/0.4495/0.4488	0.1/0.7/0.4%	head
B shared noise $\beta 0\lambda 0$ (3s)	0.4362	0.4369	$+0.0007$	0.4185	0.2%	head
B seed0/1/2			$-0.0160/{+}0.0015/{+}0.0166$	0.4565/0.3864/0.4126	$-3.5$ /0.4/4.0%	head
C $z_{s}$ dim-0 anchor (3s)	0.4921	0.4941	$+0.0020$	0.4857	0.4%	both
D continuous $z_{s}$ prior $d16$ (3s)	0.4843	0.4834	$-0.0009$	0.4708	$-0.2$ %	both

Appendix

Table S2: BimodalBypass (v2, 75 conditions,

\tau=0.11

), per-condition accounting;

\Delta=\mathrm{BL}-\mathrm{obs}

. ∗ Per-seed occupancies 100.05%/99.90%/100.02%: micro-excursions within the head-valid approximation error of Section 4.3 . ∗∗ Families H and I are quoted in the exact both-valid accounting (ratio of means), the only accounting under which they can be re-estimated from the shipped artifacts: H 91.1% (per seed 91.7/84.1/96.9) and I 97.9% (97.3/97.9/98.6), with all

75\times 6

conditions satisfying

y_{i}\leq C_{i}

. Under head-valid marginals the same families read 91.7% and 99.4%, and a minority of family-I conditions micro-exceed the bound (15/13/40 conditions, 68 in total, maximum

+0.0463

); family E contributes 301 excursions above

C

and 303 above the fixed-marginal bound across its five runs. All of them vanish under exact both-valid accounting (Section 4.5 ).

Family	$\mathrm{obs}$	$\mathrm{BL}$	$\Delta$ (per seed)	$C$	$\rho$	Valid
A independent (3 seeds)	0.4811	0.4811	$0.0000$ ( $-0.0005/{-}0.0016/{+}0.0020$ )	0.4700	0.3%	0.7382
B shared source noise (3 seeds)	0.4567	0.4652	$+0.0085$ ( $+0.0079/{+}0.0103/{+}0.0074$ )	0.4555	1.9%	0.7501
D continuous $z_{s}$ prior (3 seeds)	0.4723	0.4748	$+0.0025$ ( $+0.0019/{+}0.0019/{+}0.0037$ )	0.4636	0.5%	0.6474
G endpoint consistency $\lambda{=}3$ (3 seeds)	0.1709	0.4458	$+0.2748$ ( $+0.3097/{+}0.2090/{+}0.3057$ )	0.4319	63.2%	0.4449
H episode uniform token (2 seeds)	0.0784	0.4708	$+0.3923$ ( $+0.4232/{+}0.3614$ )	0.4661	84.1% (88.6/79.6)	0.7056
K oracle (1 seed)	0.0000	0.4994	$+0.4994$	0.4964	100.0% †	0.8704

Appendix

Table S3: RotatingBypass (v3, 36 conditions,

\tau=0.18

), per-condition accounting;

\Delta=\mathrm{BL}-\mathrm{obs}

. † 100.6% under head-valid marginals. All v3 pooled baselines are 0.4999–0.5001 (the environment is fully branch-symmetric, leaving no global preference for pooled marginals to exploit), yet pooled-accounting effects can deviate by up to 0.09; this table is per-condition throughout (Section 4.3 ). The

\rho

column averages per-seed occupancies (G: 65.0/56.8/67.9

\to

63.2%); mean effect over mean bound gives 63.6% (ratio nonlinearity; either is usable when the aggregation convention is stated).

Run	step 1	step 25k	step 50k
prior $d16$ seed0	0.3791 / 0.4002	0.00280 / 0.00227	0.00146 / 0.00137
prior $d16$ seed1	0.4466 / 0.4674	0.00293 / 0.00256	0.00135 / 0.00127
prior $d16$ seed2	0.3694 / 0.4217	0.00283 / 0.00267	0.00177 / 0.00167
prior, three-seed mean	0.398 / 0.430	0.00286 / 0.00250	0.00153 / 0.00144
posterior (range, 5 runs)	0.368–0.442 / 0.382–0.525	0.0158–0.0328 / 0.0192–0.0435	0.0125–0.0215 / 0.0175–0.0269
$z$ -dim-0 anchor (3 runs)	0	0	0

Appendix

Table S4: Sensitivity of the velocity fields to

z_{s}

(relative change under a redrawn

z_{s}

; joint/position branch) at three checkpoints, from training histories. A validation-batch companion metric agrees in magnitude (prior 0.00128–0.00196; posterior 0.0122–0.0370).

	$\lambda{=}0$	$\lambda{=}0.1$	$\lambda{=}0.3$	$\lambda{=}1$	$\lambda{=}3$
effect $\Delta$ (pooled)
shared $\beta{=}0$	$-0.0124$	$-0.0062$	$+0.0222$	$+0.0799$	$+0.2442$
partial $\beta{=}0.1$	$-0.0117$	$-0.0094$	$+0.0088$	$+0.0474$	$+0.1519$
independent $\beta{=}1$	$+0.0024$	$+0.0006$	$+0.0038$	$+0.0020$	$+0.0126$
valid-pair rate
shared $\beta{=}0$	0.9088	0.8728	0.8569	0.4975	0.4576

Appendix

Table S5: The full

\beta\times\lambda

grid (v2, seed 0, 15 runs): pooled

\Delta=\mathrm{BL}-\mathrm{obs}

, valid-pair rate, residual (mm).

seed	$\mathrm{obs}$	95% CI	Valid	$q_{L}$	$p_{L}$	$\mathrm{BL}$	$\Delta$	95% CI	$C$	$\rho$	$\Delta_{\mathrm{pc}}$	$\rho_{\mathrm{pc}}$	$H_{q}$ / $H_{T}$	Res.
0	0.1900	[.1853, .1949]	0.4576	0.211	0.386	0.4342	$+0.2442$	[ $+.2382,+.2500$ ]	0.3974	61.5%	$+0.2404$	61.7%	0.515/0.667	54.72
1	0.2782	[.2735, .2831]	0.6338	0.164	0.375	0.4160	$+0.1378$	[ $+.1323,+.1433$ ]	0.3583	38.5%	$+0.1366$	38.8%	0.446/0.662	58.81
2	0.2454	[.2389, .2516]	0.4849	0.562	0.611	0.4863	$+0.2410$	[ $+.2370,+.2448$ ]	0.4838	49.8%	$+0.2355$	49.5%	0.686/0.668	59.44
mean	0.2379		0.5254			0.4455	$+0.2077$		0.4132	49.9%	$+0.2042$	50.3%	0.549/0.666	57.66

Appendix

Table S6: Three seeds of

\beta{=}0,\lambda{=}3

(v2). In the mean row the occupancy can be the mean of per-seed occupancies (pooled 49.9%, per-condition 50.0%) or mean effect over mean bound (50.3%); the text quotes the per-condition 50.3% with per-seed 61.7/38.8/49.5%.

$\lambda_{\mathrm{cons}}$	$\lambda_{\mathrm{rot}}$	$q\!\leftrightarrow\!$ pos	$q\!\leftrightarrow\!$ rot	pos $\leftrightarrow$ rot	3-way	Pos. (mm)	Rot. ( ∘ )
0	0	$+0.0501$ [ $+0.0316$ , $+0.0633$ ]	$+0.0227$ [ $+0.0064$ , $+0.0371$ ]	$+0.0291$ [ $+0.0070$ , $+0.0476$ ]	0.307	93.90	13.65
0	1	$+0.0403$ [ $+0.0241$ , $+0.0522$ ]	$+0.0223$ [ $+0.0084$ , $+0.0355$ ]	$+0.0356$ [ $+0.0170$ , $+0.0523$ ]	0.302	94.00	13.46
3	0	$+0.3353$ [ $+0.3210$ , $+0.3465$ ]	$+0.0018$ [ $-0.0131$ , $+0.0165$ ]	$+0.0279$ [ $+0.0087$ , $+0.0461$ ]	0.430	60.97	13.05
3	1	$+0.2671$ [ $+0.2499$ , $+0.2843$ ]	$+0.0227$ [ $+0.0105$ , $+0.0337$ ]	$+0.0166$ [ $+0.0009$ , $+0.0316$ ]	0.408	63.40	12.63

Appendix

Table S7:

\lambda_{\mathrm{cons}}\times\lambda_{\mathrm{rot}}

ablation (v3, seed 0,

\tau=0.18

; the orientation branch is mode-assigned by nearest mean-yaw match;

\Delta=\mathrm{BL}-\mathrm{obs}

. Random three-way agreement 0.25; teacher-forced training residuals: position 10.7–17.6 mm, orientation 1.45–1.88 ∘ ).

$\tau$	v2 $\Delta$	v2 valid	v3 $\Delta$	v3 valid
0.05	$+0.1913$	0.729	$+0.2150$	0.999
0.08	$+0.2141$	0.609	$+0.2150$	0.999
0.11	$+0.2442$	0.458	$+0.2161$	0.994
0.14	$+0.2995$	0.255	$+0.2396$	0.880
0.17	$+0.4427$	0.050	$+0.3006$	0.585
0.20	—	—	$+0.3936$	0.154

Appendix

Table S8:

\lambda{=}3

family under threshold variation (pooled

\Delta

; valid-pair rate).

Flow Matching

Flow matching learns continuous-time vector fields that transport simple noise distributions into data for efficient generative modeling.

See all 661 papers

Beyond Multimodal Alignment: Shared Physical Representations Across Sensors and Action Orders
Aug 19, 2026Kaizhen Tan, Xin Xu, Siru Tao +4World Models Cross-Modal Alignment
New York University, New York, NY, USA · Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA
MOSS: Mode-Oriented Source Shaping for Robot Flow Matching Policies
Oct 15, 2026Zirun Ma, Jindou Jia, Tuo An +3Robot Policy Learning Robotic Manipulation
MARS Lab, Nanyang Technological University · Tsinghua University
Multi-Agent Flow Matching with Decoupled Generative Guidance
Sep 29, 2026Ruoyu Lin, Magnus Egerstedt, Fabio PasqualettiConstrained Generative Modeling Flow Matching
Department of Electrical Engineering and Computer Science, University of California, Irvine, Irvine, CA 92697 USA · University of North Carolina at Chapel Hill, Chapel Hill, NC 27599 USA

Aug 19, 2026J/K move · Enter open · S save

Beyond Multimodal Alignment: Shared Physical Representations Across Sensors and Action Orders

Kaizhen Tan, Xin Xu, Siru Tao, Yixiao Li+3

New York University, New York, NY, USA · Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA

World Models Cross-Modal Alignment System Identification Compositional Generalization Multimodal Sensor Fusion

Open paper AskPDF arXiv

Multimodal world models are often evaluated by whether different sensors produce similar representations. However, similar representations do not necessarily imply that the models make the same physical predictions, or that those representations can be reused when actions are combined in a new order. We study both questions through the physical responses predicted by a model. We first use the Cluster Haptic dataset to ask whether audio and acceleration can independently recover the behavior of the same surface from different observations. Predictions from the two sensors are substantially closer for the same surface than for different surfaces, with a

4.5\times

gap on average, while both also outperform an average-surface prediction. We then show that this agreement alone does not determine how familiar actions should compose. In a controlled elastoplastic system, shared step dynamics fit observed programs less accurately than a whole-program predictor but generalize better to unseen action orders, with the ranking reversing on both held-out transitions across three independent initializations. Fusing free-decay and hysteresis observations further improves prediction, with diagonal Gaussian beliefs yielding the lowest errors. Together, these results distinguish cross-sensor consistency, multimodal fusion, and generalization to new action orders as separate questions in evaluating multimodal physical representations.

Abstract

Flow Matching

Explore similar work

Beyond Multimodal Alignment: Shared Physical Representations Across Sensors and Action Orders

MOSS: Mode-Oriented Source Shaping for Robot Flow Matching Policies

Multi-Agent Flow Matching with Decoupled Generative Guidance