BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series

Yue Li; Kangqi Ni; Zhen Tan; Tianlong Chen

cs.LGOct 4, 2026

BeliefGraph-JEPA: Structured Latent World Models for Action-Conditioned Time Series

Authors: Yue Li, Kangqi Ni, Zhen Tan, Tianlong Chen

Organizations: Carnegie Mellon University · University of North Carolina at Chapel Hill · Stevens Institute of Technology

Abstract

Action-conditioned time-series forecasting requires accounting for how future actions and exogenous forcings influence multiple targets through partially observed effects with different delays and persistence. Direct conditioning leaves the evolution and target-specific influence of these effects implicit in the predictor, while static relational graphs specify connections without tracking evolving effects. This motivates representing future-driver influence through structured latent states that evolve over the forecast horizon and route information to individual targets. We introduce BeliefGraph-JEPA, a structured latent world model that factorizes driver influence into typed latent-effect states. These states are rolled forward under future drivers and routed through a graph to target-specific nodes, forming the predictive base of a joint-embedding predictive architecture. A capacity-controlled residual supplements this base with direct driver information. On four multi-target clinical, agricultural, environmental, and industrial systems, the framework outperforms a range of pretrained and supervised known-future-covariate baselines. Matched controls isolate latent dynamics, future rollout, graph routing, and residual capacity; future rollout and graph-first residual routing improve forecasting across all four systems.

Figures & tables

Figure 1: Direct-to-predictor conditioning versus BeliefGraph-JEPA. (a) Drivers condition predictions without explicit typed latent-effect states. (b) Drivers update typed latent-effect states, which are rolled to each horizon and routed to targets. Graph-base prediction of that trajectory is the predictive base; a capacity-controlled residual recovers omitted details.

Figure 2: BeliefGraph-JEPA: structured latent-effect rollout as the predictive base. (a) Past observations and past drivers initialize a history state $H_{t}^{X}$ and a family-effect state $Z_{t}^{f}$ . (b) Future drivers roll typed latent-effect states; a shared relation-aware GNN, guided by the typed driver–effect–target prior, produces the horizon-specific graph trajectory. (c) $z_{t+h}^{G}$ is the predictive base; a capacity-controlled residual $\alpha^{R}\Delta z_{t+h}^{R}$ recovers omitted details. (d) JEPA, graph-base, and capacity losses supervise the aligned latent.

Dataset	Targets	Actions	Exogenous
VitalDB	MBP, HR, SpO 2 , BIS (4)	propofol, remifentanil rates	—
Greenhouse	indoor T, RH, CO 2 , PAR (4)	heating, vents, screens, light, CO 2 dosing	outdoor weather (8)
Wastewater	DO, NH 4 , PO 4 (3)	metal-salt dosing; process phases	inflow, water temperature
PreDist †	supply/return T, heat power, flow (5)	valve, setpoint; pump, mode, 3-way valve	outdoor T, grid supply T

Table 1: Multi-target forecasting protocols. † PreDist uses an instantaneous-effect head for algebraic heat power and flow (Appendix E ).

	Greenhouse		Wastewater		VitalDB		PreDist †
Method	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE
Persistence	$0.1552$	$0.2603$	$0.2911$	$0.4548$	$0.0655$	$0.1285$	$0.0677$	$0.1115$
Chronos-2	$0.1572$	$0.2620$	$0.2349$	$0.3836$	$0.0640$	$0.1244$	$0.0598$	$0.1004$
Chronos-2-Cov	$0.1464$	$0.2521$	$0.1765$	$0.3118$	$0.0639$	$0.1253$	$0.0554$	$0.0986$
TimesFM 2.5	$0.1562$	$0.2579$	$0.2231$	$0.3584$	$0.0611$	$0.1177$	$0.0563$	$0.0951$
DAG	$0.1552_{\scriptscriptstyle\pm 0.0011}$	$0.2441_{\scriptscriptstyle\pm 0.0038}$	$0.1761_{\scriptscriptstyle\pm 0.0022}$	$0.2957_{\scriptscriptstyle\pm 0.0006}$	$0.0629_{\scriptscriptstyle\pm 0.0001}$	$0.1173_{\scriptscriptstyle\pm 0.0001}$	$0.0512_{\scriptscriptstyle\pm 0.0004}$	$0.0835_{\scriptscriptstyle\pm 0.0011}$

Table 2: Test nMAE / nRMSE (

\downarrow

) on the four multi-target systems, mean

\pm

sample std over three seeds. Best in each column in bold, second-best underlined. External baselines, a cumulative ladder up to FR , then unstructured-readout and GR controls. † Uses the instantaneous-effect head (Appendix E ).

	Greenhouse		Wastewater		VitalDB		PreDist †
Method	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE
FR	$0.1392_{\scriptscriptstyle\pm 0.0028}$	$0.2030_{\scriptscriptstyle\pm 0.0020}$	$0.1304_{\scriptscriptstyle\pm 0.0018}$	$0.2230_{\scriptscriptstyle\pm 0.0025}$	$0.0587_{\scriptscriptstyle\pm 0.0002}$	$0.1040_{\scriptscriptstyle\pm 0.0002}$	$0.0536_{\scriptscriptstyle\pm 0.0021}$	$0.0816_{\scriptscriptstyle\pm 0.0029}$
HB	$0.1370_{\scriptscriptstyle\pm 0.0014}$	$0.2009_{\scriptscriptstyle\pm 0.0011}$	$\underline{0.1277}_{\scriptscriptstyle\pm 0.0011}$	$0.2205_{\scriptscriptstyle\pm 0.0028}$	$0.0588_{\scriptscriptstyle\pm 0.0002}$	$0.1038_{\scriptscriptstyle\pm 0.0002}$	$0.0518_{\scriptscriptstyle\pm 0.0016}$	$0.0798_{\scriptscriptstyle\pm 0.0024}$
FR-GR-GO	$0.1381_{\scriptscriptstyle\pm 0.0009}$	$0.2014_{\scriptscriptstyle\pm 0.0035}$	$0.1354_{\scriptscriptstyle\pm 0.0014}$	$0.2262_{\scriptscriptstyle\pm 0.0018}$	$0.0593_{\scriptscriptstyle\pm 0.0003}$	$0.1044_{\scriptscriptstyle\pm 0.0002}$	$0.0534_{\scriptscriptstyle\pm 0.0007}$	$0.0813_{\scriptscriptstyle\pm 0.0011}$
FR-GR-U	$\underline{0.1319}_{\scriptscriptstyle\pm 0.0009}$	$\underline{0.1944}_{\scriptscriptstyle\pm 0.0004}$	$0.1280_{\scriptscriptstyle\pm 0.0004}$	$\underline{0.2172}_{\scriptscriptstyle\pm 0.0010}$	$\mathbf{0.0578}_{\scriptscriptstyle\pm 0.0001}$	$\underline{0.1033}_{\scriptscriptstyle\pm 0.0002}$	$\underline{0.0507}_{\scriptscriptstyle\pm 0.0025}$	$\underline{0.0777}_{\scriptscriptstyle\pm 0.0037}$
RB-GR	$0.1403_{\scriptscriptstyle\pm 0.0019}$	$0.2086_{\scriptscriptstyle\pm 0.0017}$	$0.1326_{\scriptscriptstyle\pm 0.0009}$	$0.2249_{\scriptscriptstyle\pm 0.0016}$	$0.0583_{\scriptscriptstyle\pm 0.0002}$	$0.1037_{\scriptscriptstyle\pm 0.0002}$	$0.0535_{\scriptscriptstyle\pm 0.0018}$	$0.0832_{\scriptscriptstyle\pm 0.0023}$

Table 3: Routing and bottleneck diagnostics.

A. Retraining
	Greenhouse		Wastewater
Method	nMAE	nRMSE	nMAE	nRMSE
Domain graph ( FR-GR )	$\mathbf{0.1285}_{\scriptscriptstyle\pm 0.0016}$	$\mathbf{0.1928}_{\scriptscriptstyle\pm 0.0033}$	$\mathbf{0.1245}_{\scriptscriptstyle\pm 0.0018}$	$\mathbf{0.2157}_{\scriptscriptstyle\pm 0.0028}$
Learned-FC ( LFC )	$0.1357_{\scriptscriptstyle\pm 0.0013}$	$0.2029_{\scriptscriptstyle\pm 0.0010}$	$0.1256_{\scriptscriptstyle\pm 0.0007}$	$\underline{0.2170}_{\scriptscriptstyle\pm 0.0011}$
Shuffled-slot	$\underline{0.1309}_{\scriptscriptstyle\pm 0.0043}$	$\underline{0.1935}_{\scriptscriptstyle\pm 0.0055}$	$\underline{0.1254}_{\scriptscriptstyle\pm 0.0007}$	$0.2189_{\scriptscriptstyle\pm 0.0014}$
B. Inference-time prior swap

Table 4: Graph-structure controls.

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Grid	Context, horizon
VitalDB	10 s	15 min, 6 min
Greenhouse	5 min	12 h, 3 h
Wastewater	2 min	6 h, 3 h
PreDist	10 min	12 h, 6 h

Appendix

Table 6: Multi-target grids and forecast windows.

Dataset	Grid	Context, horizon	Actions
CGMacros	5 min	12 h, 2 h	sparse meal events
Shanghai Diabetes	15 min	12 h, 2 h	sparse meal, injection, and oral-drug events

Appendix

Table 7: Single-target glucose protocols.

	CGMacros		Shanghai Diabetes
Method	MAE	RMSE	MAE	RMSE
Persistence	$17.25$	$28.11$	$18.71$	$29.55$
Chronos-2	$15.42$	$25.38$	$18.96$	$30.56$
Chronos-2-Cov	$15.91$	$26.31$	$18.75$	$30.37$
TimesFM 2.5	$14.89$	$24.59$	$17.25$	$27.72$
DAG	$13.37_{\scriptscriptstyle\pm 0.12}$	$21.52_{\scriptscriptstyle\pm 0.21}$	$\underline{15.88}_{\scriptscriptstyle\pm 0.26}$	$25.05_{\scriptscriptstyle\pm 0.45}$

Appendix

Table 8: Test MAE / RMSE (

\downarrow

) in mg/dL on the two single-target glucose datasets. Foundation-model rows are single-run; remaining rows are mean

\pm

sample std over three seeds. The HO – FR ladder and its GR variants.

	MAE			RMSE
Method	$z^{G}$	$\hat{z}$	$\Delta$	$z^{G}$	$\hat{z}$	$\Delta$
RB-GR	$17.10_{\scriptscriptstyle\pm 0.13}$	$16.24_{\scriptscriptstyle\pm 0.31}$	$0.86_{\scriptscriptstyle\pm 0.19}$	$25.81_{\scriptscriptstyle\pm 0.22}$	$25.01_{\scriptscriptstyle\pm 0.39}$	$0.81_{\scriptscriptstyle\pm 0.22}$
FR-GR	$16.78_{\scriptscriptstyle\pm 0.18}$	$16.52_{\scriptscriptstyle\pm 0.19}$	$0.25_{\scriptscriptstyle\pm 0.04}$	$25.62_{\scriptscriptstyle\pm 0.18}$	$25.41_{\scriptscriptstyle\pm 0.13}$	$0.21_{\scriptscriptstyle\pm 0.05}$

Appendix

Table 9: Shanghai residual gain. Graph-base error at

\alpha^{R}{=}0

versus the full prediction

\hat{z}=z^{G}+\alpha^{R}\Delta z^{R}

on the same RB-GR and FR-GR checkpoints as Table 8 . Residual gain

\Delta=\mathrm{err}(z^{G})-\mathrm{err}(\hat{z})

is computed per seed. Mean

\pm

sample std over three seeds.

	indoor $T$		RH		CO 2		PAR
Method	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
Persistence	$1.489$	$2.273$	$5.51$	$8.09$	$81.5$	$151.5$	$73.8$	$143.2$
Chronos-2	$1.618$	$2.432$	$5.68$	$8.32$	$78.0$	$153.6$	$71.3$	$128.4$
Chronos-2-Cov	$1.450$	$2.290$	$5.18$	$7.89$	$76.7$	$151.7$	$67.7$	$125.0$
TimesFM 2.5	$1.537$	$2.317$	$5.57$	$8.08$	$77.0$	$148.6$	$77.1$	$138.1$
DAG	$1.397_{\scriptscriptstyle\pm 0.026}$	$2.040_{\scriptscriptstyle\pm 0.035}$	$5.61_{\scriptscriptstyle\pm 0.08}$	$7.83_{\scriptscriptstyle\pm 0.08}$	$87.7_{\scriptscriptstyle\pm 0.5}$	$152.4_{\scriptscriptstyle\pm 3.1}$	$70.5_{\scriptscriptstyle\pm 1.8}$	$118.2_{\scriptscriptstyle\pm 2.4}$

Appendix

Table 10: Per-target test MAE / RMSE (

\downarrow

) on Greenhouse, physical units. Same methods as Table 2 .

	DO		NH 4		PO 4
Method	MAE	RMSE	MAE	RMSE	MAE	RMSE
Persistence	$0.813$	$1.223$	$0.768$	$1.105$	$0.549$	$0.825$
Chronos-2	$0.604$	$0.988$	$0.693$	$1.066$	$0.459$	$0.712$
Chronos-2-Cov	$0.484$	$0.828$	$0.474$	$0.785$	$0.338$	$0.572$
TimesFM 2.5	$0.567$	$0.920$	$0.664$	$0.995$	$0.439$	$0.668$
DAG	$0.533_{\scriptscriptstyle\pm 0.011}$	$0.830_{\scriptscriptstyle\pm 0.009}$	$0.431_{\scriptscriptstyle\pm 0.011}$	$0.688_{\scriptscriptstyle\pm 0.012}$	$0.307_{\scriptscriptstyle\pm 0.006}$	$0.492_{\scriptscriptstyle\pm 0.008}$

Appendix

Table 11: Per-target test MAE / RMSE (

\downarrow

) on Wastewater, physical units. Same methods as Table 2 .

	MBP		HR		SpO 2		BIS
Method	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
Persistence	$5.948$	$10.745$	$3.714$	$7.126$	$0.235$	$1.050$	$4.892$	$7.087$
Chronos-2	$6.063$	$10.455$	$3.470$	$6.416$	$\underline{0.229}$	$1.025$	$4.693$	$6.995$
Chronos-2-Cov	$5.992$	$10.463$	$3.533$	$6.648$	$0.230$	$1.036$	$4.676$	$7.035$
TimesFM 2.5	$5.671$	$9.679$	$\underline{3.334}$	$6.205$	$\mathbf{0.214}$	$0.952$	$4.564$	$6.800$
DAG	$5.608_{\scriptscriptstyle\pm 0.020}$	$9.335_{\scriptscriptstyle\pm 0.032}$	$3.526_{\scriptscriptstyle\pm 0.017}$	$6.318_{\scriptscriptstyle\pm 0.026}$	$0.280_{\scriptscriptstyle\pm 0.003}$	$1.050_{\scriptscriptstyle\pm 0.004}$	$4.638_{\scriptscriptstyle\pm 0.011}$	$6.864_{\scriptscriptstyle\pm 0.025}$

Appendix

Table 12: Per-target test MAE / RMSE (

\downarrow

) on VitalDB, physical units. Same methods as Table 2 .

	Supply $T$		HC ret. $T$		Net ret. $T$		Heat power		Flow
Method	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE	MAE	RMSE
Persistence	$2.641$	$4.164$	$2.470$	$3.732$	$2.450$	$3.958$	$21.85$	$37.40$	$0.617$	$1.064$
Chronos-2	$2.295$	$3.740$	$2.163$	$3.427$	$2.144$	$3.504$	$19.95$	$34.15$	$0.559$	$0.954$
Chronos-2-Cov	$1.772$	$3.061$	$2.168$	$3.521$	$2.117$	$3.630$	$\underline{18.83}$	$35.93$	$\underline{0.524}$	$0.984$
TimesFM 2.5	$2.182$	$3.591$	$2.024$	$3.205$	$2.014$	$3.313$	$18.84$	$32.18$	$0.527$	$\underline{0.904}$
DAG	$1.671_{\scriptscriptstyle\pm 0.022}$	$2.588_{\scriptscriptstyle\pm 0.058}$	$1.938_{\scriptscriptstyle\pm 0.025}$	$2.940_{\scriptscriptstyle\pm 0.045}$	$1.969_{\scriptscriptstyle\pm 0.030}$	$3.107_{\scriptscriptstyle\pm 0.045}$	$\mathbf{17.60}_{\scriptscriptstyle\pm 0.14}$	$\mathbf{30.30}_{\scriptscriptstyle\pm 0.28}$	$\mathbf{0.492}_{\scriptscriptstyle\pm 0.003}$	$\mathbf{0.840}_{\scriptscriptstyle\pm 0.006}$

Appendix

Table 13: Per-target test MAE / RMSE (

\downarrow

) on PreDist † , physical units. Same methods as Table 2 .

	nMAE	Supply $T$	HC ret. $T$	Net ret. $T$	Heat	Flow
FR-GR , shared $z$	$0.0582_{\scriptscriptstyle\pm 0.0020}$	$1.447_{\scriptscriptstyle\pm 0.078}$	$1.823_{\scriptscriptstyle\pm 0.073}$	$2.104_{\scriptscriptstyle\pm 0.138}$	$28.52_{\scriptscriptstyle\pm 3.13}$	$0.730_{\scriptscriptstyle\pm 0.049}$
FR-GR	$\mathbf{0.0495}_{\scriptscriptstyle\pm 0.0003}$	$\mathbf{1.290}_{\scriptscriptstyle\pm 0.030}$	$1.834_{\scriptscriptstyle\pm 0.006}$	$\mathbf{1.928}_{\scriptscriptstyle\pm 0.069}$	$\mathbf{18.85}_{\scriptscriptstyle\pm 0.58}$	$\mathbf{0.562}_{\scriptscriptstyle\pm 0.021}$

Appendix

Table 14: PreDist test error of FR-GR with a shared JEPA latent versus with the instantaneous-effect head. Pooled nMAE on

[-1,1]

; per-target MAE in physical units. Mean

\pm

sample std over seeds

\{42,43,44\}

. The headed row is the PreDist FR-GR entry of Tables 2 and 13 .

Setting	Value
Width $d$ / heads / FFN	$128$ / $4$ / $4d$
History encoder	2-layer causal Transformer, Pre-LN, GELU
GNN / predictor / residual adapter	2 relation-aware layers; 2-layer Transformer decoder
Dropout / positional encoding	$0.1$ ; learned embeddings on the state encoder; sinusoidal $\Delta t$ on driver events
EMA target decay	$0.996$
Optimizer	AdamW, weight decay $10^{-4}$ , gradient clip $1.0$

Appendix

Table 15: Shared FR-GR architecture and optimization. PreDist uses the large-batch row of Table 16 ; Wastewater uses the 50-epoch row.

Dataset	Batch	LR	LR warm.	Stride	Train $N$	Params	Time
VitalDB	8	$3{\times}10^{-4}$	0	30	45,425	2.0M	4.0 h
Greenhouse	8	$3{\times}10^{-4}$	0	36	2,144	2.9M	0.4 h
Wastewater	32	$1.2{\times}10^{-3}$	3	90	4,093	2.9M	0.4 h
PreDist	64	$1{\times}10^{-3}$	1	36	87,475	3.1M	2.1 h

Appendix

Table 16: Dataset-specific FR-GR training settings. Train windows are the indexed training set. Wall-clock is one seed, train and validation. Wastewater uses

50

epochs, batch

32

, learning rate

1.2{\times}10^{-3}

, min LR

4{\times}10^{-5}

, and a 3-epoch warmup; remaining datasets use

30

epochs. PreDist uses a larger batch, learning rate

10^{-3}

, and min LR

3{\times}10^{-5}

Dataset	$H$	Interval $I$	Range	$\delta$
VitalDB	$36$ ( $6$ min)	$[18,36)$ ( $3$ min)	$[0,1]$	$0.15$
Greenhouse	$36$ ( $3$ h)	$[18,36)$ ( $90$ min)	$[0,1]$	$0.15$
Wastewater	$90$ ( $3$ h)	$[45,90)$ ( $90$ min)	$[-1,1]$	$0.30$
PreDist	$36$ ( $6$ h)	$[18,36)$ ( $3$ h)	$[-1,1]$	$0.30$

Appendix

Table 17: Second-half continuous-action offset.

\delta

is the per-channel magnitude; values are clipped to the listed range.

Dataset	$\alpha^{R}$	$\rho_{R}$
Greenhouse	$0.107_{\scriptscriptstyle\pm 0.001}$	$0.0302_{\scriptscriptstyle\pm 0.0019}$
Wastewater	$0.104_{\scriptscriptstyle\pm 0.001}$	$0.0391_{\scriptscriptstyle\pm 0.0032}$
VitalDB	$0.050_{\scriptscriptstyle\pm 0.002}$	$0.0142_{\scriptscriptstyle\pm 0.0009}$
PreDist	$0.069_{\scriptscriptstyle\pm 0.001}$	$0.0143_{\scriptscriptstyle\pm 0.0016}$

Appendix

Table 18: Converged residual scale

\alpha^{R}

and reliance

\rho_{R}

(Eq. 6 ) of FR-GR .

	Greenhouse		Wastewater		VitalDB		PreDist †
Setting	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE
FR-GR	$0.1285_{\scriptscriptstyle\pm 0.0016}$	$0.1928_{\scriptscriptstyle\pm 0.0033}$	$0.1245_{\scriptscriptstyle\pm 0.0018}$	$0.2157_{\scriptscriptstyle\pm 0.0028}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0495_{\scriptscriptstyle\pm 0.0003}$	$0.0768_{\scriptscriptstyle\pm 0.0004}$
$\alpha^{R}$ init $0.3$	$0.1296_{\scriptscriptstyle\pm 0.0014}$	$0.1930_{\scriptscriptstyle\pm 0.0017}$	$0.1246_{\scriptscriptstyle\pm 0.0019}$	$0.2158_{\scriptscriptstyle\pm 0.0027}$	$0.0579_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0000}$	$0.0502_{\scriptscriptstyle\pm 0.0009}$	$0.0773_{\scriptscriptstyle\pm 0.0009}$
$\lambda_{G}=0.01$	$0.1272_{\scriptscriptstyle\pm 0.0004}$	$0.1915_{\scriptscriptstyle\pm 0.0009}$	$0.1242_{\scriptscriptstyle\pm 0.0004}$	$0.2151_{\scriptscriptstyle\pm 0.0008}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0497_{\scriptscriptstyle\pm 0.0009}$	$0.0768_{\scriptscriptstyle\pm 0.0010}$

Appendix

Table 19: Sensitivity to residual-scale initialization and graph-base supervision. FR-GR uses

\alpha^{R}

init

0.1

and

\lambda_{G}=0.1

	Greenhouse		Wastewater		VitalDB		PreDist †
Setting	$\alpha^{R}$	$\rho_{R}$	$\alpha^{R}$	$\rho_{R}$	$\alpha^{R}$	$\rho_{R}$	$\alpha^{R}$	$\rho_{R}$
FR-GR	$0.107_{\scriptscriptstyle\pm 0.001}$	$0.0302_{\scriptscriptstyle\pm 0.0019}$	$0.104_{\scriptscriptstyle\pm 0.001}$	$0.0391_{\scriptscriptstyle\pm 0.0032}$	$0.050_{\scriptscriptstyle\pm 0.002}$	$0.0142_{\scriptscriptstyle\pm 0.0009}$	$0.069_{\scriptscriptstyle\pm 0.001}$	$0.0143_{\scriptscriptstyle\pm 0.0016}$
$\alpha^{R}$ init $0.3$	$0.305_{\scriptscriptstyle\pm 0.002}$	$0.0274_{\scriptscriptstyle\pm 0.0030}$	$0.287_{\scriptscriptstyle\pm 0.002}$	$0.0383_{\scriptscriptstyle\pm 0.0012}$	$0.084_{\scriptscriptstyle\pm 0.005}$	$0.0141_{\scriptscriptstyle\pm 0.0010}$	$0.138_{\scriptscriptstyle\pm 0.002}$	$0.0156_{\scriptscriptstyle\pm 0.0007}$
$\lambda_{G}=0.01$	$0.108_{\scriptscriptstyle\pm 0.001}$	$0.0306_{\scriptscriptstyle\pm 0.0029}$	$0.105_{\scriptscriptstyle\pm 0.001}$	$0.0401_{\scriptscriptstyle\pm 0.0029}$	$0.050_{\scriptscriptstyle\pm 0.002}$	$0.0148_{\scriptscriptstyle\pm 0.0006}$	$0.072_{\scriptscriptstyle\pm 0.003}$	$0.0156_{\scriptscriptstyle\pm 0.0011}$

Appendix

Table 20: Converged residual scale

\alpha^{R}

and reliance

\rho_{R}

(Eq. 6 ).

	Greenhouse		Wastewater		VitalDB		PreDist †
Setting	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE
FR-GR ( $L_{A}=L_{X}$ )	$0.1285_{\scriptscriptstyle\pm 0.0016}$	$0.1928_{\scriptscriptstyle\pm 0.0033}$	$0.1245_{\scriptscriptstyle\pm 0.0018}$	$0.2157_{\scriptscriptstyle\pm 0.0028}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0495_{\scriptscriptstyle\pm 0.0003}$	$0.0768_{\scriptscriptstyle\pm 0.0004}$
$L_{A}=2L_{X}$	$0.1282_{\scriptscriptstyle\pm 0.0023}$	$0.1918_{\scriptscriptstyle\pm 0.0014}$	$0.1254_{\scriptscriptstyle\pm 0.0013}$	$0.2173_{\scriptscriptstyle\pm 0.0017}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0500_{\scriptscriptstyle\pm 0.0010}$	$0.0771_{\scriptscriptstyle\pm 0.0007}$

Appendix

Table 21: Sensitivity to the latent-effect history window. FR-GR uses context-aligned belief initialization with

L_{A}=L_{X}

; the ablation adds one additional context length of pre-context driver history.

	Greenhouse		Wastewater		VitalDB		PreDist †
Setting	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE	nMAE	nRMSE
FR-GR	$0.1285_{\scriptscriptstyle\pm 0.0016}$	$0.1928_{\scriptscriptstyle\pm 0.0033}$	$0.1245_{\scriptscriptstyle\pm 0.0018}$	$0.2157_{\scriptscriptstyle\pm 0.0028}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0495_{\scriptscriptstyle\pm 0.0003}$	$0.0768_{\scriptscriptstyle\pm 0.0004}$
shared $\tau=8$	$0.1279_{\scriptscriptstyle\pm 0.0019}$	$0.1918_{\scriptscriptstyle\pm 0.0038}$	$0.1244_{\scriptscriptstyle\pm 0.0011}$	$0.2150_{\scriptscriptstyle\pm 0.0021}$	$0.0580_{\scriptscriptstyle\pm 0.0003}$	$0.1032_{\scriptscriptstyle\pm 0.0001}$	$0.0497_{\scriptscriptstyle\pm 0.0016}$	$0.0773_{\scriptscriptstyle\pm 0.0021}$

Appendix

Table 22: Robustness to family-timescale initialization. FR-GR uses the family-specific

\tau_{f}

values of Appendix G ; the ablation initializes every family with the same

\tau=8

. Forecast accuracy is essentially unchanged.

Dataset	Family	Prior $\tau_{f}$	$\hat{\tau}$ ( FR-GR )	$\hat{\tau}$ ( $\tau=8$ )
Greenhouse	thermal	$24$	$21.4_{\scriptscriptstyle\pm 0.6}$	$7.0_{\scriptscriptstyle\pm 0.1}$
	ventilation	$6$	$5.7_{\scriptscriptstyle\pm 0.2}$	$7.4_{\scriptscriptstyle\pm 0.5}$
	light	$2$	$2.0_{\scriptscriptstyle\pm 0.0}$	$8.5_{\scriptscriptstyle\pm 0.1}$
	screen	$4$	$4.1_{\scriptscriptstyle\pm 0.1}$	$8.1_{\scriptscriptstyle\pm 0.1}$
	CO 2	$6$	$6.3_{\scriptscriptstyle\pm 0.2}$	$8.2_{\scriptscriptstyle\pm 0.3}$
Wastewater	dosing	$8$	$9.2_{\scriptscriptstyle\pm 0.5}$	$9.0_{\scriptscriptstyle\pm 1.0}$

Appendix

Table 23: Learned implied family timescales

\hat{\tau}_{f}=-1/\log\lambda_{f}

under family-specific and shared initialization. The two initializations do not generally converge to the same persistence solution despite nearly identical forecasting accuracy in Table 22 .

Method	$\lambda_{\mathrm{fast}}$	$\lambda_{\mathrm{slow}}$	Rank	MAE	MAE fast	MAE slow
Truth	$0.40$	$0.75$	—	—	—	—
Init	$0.575$	$0.575$	—	—	—	—
DC id	—	—	—	$0.063$	$0.077$	$0.049$
IE id	$0$	$0$	No	$0.063$	$0.077$	$0.049$
RB id	$0.619$	$0.627$	No	$0.063$	$0.077$	$0.049$
RB-GR id	$0.629$	$0.637$	No	$0.063$	$0.077$	$0.049$

Appendix

Table 24: Learned persistence and test MAE on the matched-dynamics system (seed

42

). Rank is

\lambda_{\mathrm{slow}}>\lambda_{\mathrm{fast}}

. IE id fixes

\lambda_{f}{=}0

. DC id has no family persistence. FR id ( HB ) is the hard bottleneck of Section 5.2 . MAE is on the physical target scale.

Method	$I_{\mathrm{fast}\to\mathrm{fast}}$	$I_{\mathrm{fast}\to\mathrm{slow}}$	$I_{\mathrm{slow}\to\mathrm{slow}}$	$I_{\mathrm{null}}$
Truth	$0.600$	$0$	$0.250$	$0$
DC id	$0.000$	$0.000$	$0.000$	$0.000$
IE id	$0.000$	$0.000$	$0.000$	$0.000$
RB id	$0.000$	$0.000$	$0.000$	$0.000$
RB-GR id	$0.000$	$0.000$	$0.000$	$0.000$
FR id ( HB )	$0.371$	$0.010$	$0.000$	$0.001$

Appendix

Table 25: Peak absolute response to a unit future pulse (horizon

32

, zero context). Truth peaks are

0.60

(

u_{\mathrm{fast}}{\to}y_{\mathrm{fast}}

) and

0.25

(

u_{\mathrm{slow}}{\to}y_{\mathrm{slow}}

Method	$\lambda_{\mathrm{fast}}$	$\lambda_{\mathrm{slow}}$	Rank	MAE	MAE fast	MAE slow
Truth	$0.40$	$0.75$	—	—	—	—
Init	$0.575$	$0.575$	—	—	—	—
DC id	—	—	—	$0.044$	$0.056$	$0.034$
IE id	$0$	$0$	No	$0.044$	$0.055$	$0.034$
RB id	$0.622$	$0.628$	No	$0.044$	$0.055$	$0.034$
RB-GR id	$0.632$	$0.639$	No	$0.044$	$0.055$	$0.034$

Appendix

Table 26: Learned persistence and test MAE on the saturating

\tanh

system (seed

42

). Rank is

\lambda_{\mathrm{slow}}>\lambda_{\mathrm{fast}}

. IE id fixes

\lambda_{f}{=}0

. DC id has no family persistence. The ladder uses

40

epochs; the last row is FR-GR id at

80

epochs. MAE is on the physical target scale.

Method	$I_{\mathrm{fast}\to\mathrm{fast}}$	$I_{\mathrm{fast}\to\mathrm{slow}}$	$I_{\mathrm{slow}\to\mathrm{slow}}$	$I_{\mathrm{null}}$
Truth	$0.537$	$0$	$0.245$	$0$
DC id	$0.000$	$0.000$	$0.000$	$0.001$
IE id	$0.000$	$0.000$	$0.000$	$0.000$
RB id	$0.000$	$0.000$	$0.000$	$0.000$
RB-GR id	$0.000$	$0.000$	$0.000$	$0.000$
FR id ( HB )	$0.000$	$0.000$	$0.000$	$0.000$

Appendix

Table 27: Peak absolute response to a unit future pulse on the saturating

\tanh

system (horizon

32

, zero context). Truth peaks are

0.537

(

u_{\mathrm{fast}}{\to}y_{\mathrm{fast}}

) and

0.245

(

u_{\mathrm{slow}}{\to}y_{\mathrm{slow}}

). The ladder uses

40

epochs; the last row is FR-GR id at

80

epochs.

Explore similar work

Orthogonal JEPA: Factorized Predictive States for Latent World Models
Aug 20, 2026Taoyong Cui, Pheng Ann Heng, Wanli OuyangJoint-Embedding Predictive Architectures Next-State Prediction
The Chinese University of Hong Kong (CUHK)
Phys-JEPA: Physics-Informed Latent World Models for Multivariate Time-Series Forecasting
Jun 15, 2026Weizhi Nie, Weichao Liu, Honglin Guo +1Multivariate Time Series Forecasting Joint-Embedding Predictive Architectures
Tianjin University
UWM-JEPA: Predictive World Models That Imagine in Belief Space
May 25, 2026Santosh Kumar Radha, Oktay GoktasLatent World Models World Models
AgentField AI, Toronto, Canada

Aug 20, 2026, cs.LGJ/K move · Enter open · S save

Orthogonal JEPA: Factorized Predictive States for Latent World Models

Taoyong Cui, Pheng Ann Heng, Wanli Ouyang

The Chinese University of Hong Kong (CUHK)

Joint-Embedding Predictive Architectures Next-State Prediction Latent World Models World Models Regularization Single-Cell Rna

Open paper AskPDF arXiv

World models construct latent states that support prediction, planning, and reasoning about an underlying system. Joint-embedding predictive architectures (JEPAs) offer a direct way to learn such states by predicting targets in representation space instead of reconstructing every detail of the observation. Standard JEPAs, however, organize all predictable content through one target embedding and one prediction pathway. In complex systems, this monolithic state can allocate redundant capacity to dominant signals while providing weak or conflicting gradients to less dominant predictive structure. We introduce \method, a latent world-modeling framework based on orthogonal predictive factorization. Learned basis matrices analyze each target state into multiple components, and a dedicated prediction branch estimates each component from a shared context representation. Predictive regression preserves the factor magnitudes required for state synthesis, an orthogonality objective discourages repeated directions, factor-activity regularization maintains variation in projected targets, and online variance regularization discourages coordinate-wise encoder collapse. Predicted components are synthesized into a complete latent state that can be used by a readout, decoder, planner, or autoregressive rollout. The same predictive-state mechanism applies when the target is temporally future, spatially hidden, or another partial observation of the same system. Experiments on controlled vision, single-cell transcriptomics, longitudinal health records, continuous control, and molecular dynamics evaluate representation quality, forecasting, planning, and long-horizon stability.

Abstract

Explore similar work

Orthogonal JEPA: Factorized Predictive States for Latent World Models

Phys-JEPA: Physics-Informed Latent World Models for Multivariate Time-Series Forecasting

UWM-JEPA: Predictive World Models That Imagine in Belief Space