Directed Temporal Representations for Offline Visual Control

Chenyang Yuan; Haoyu Wang; Zhuo Sun; Xiaoyuan Cheng

cs.LGOct 6, 2026

Directed Temporal Representations for Offline Visual Control

Authors: Chenyang Yuan, Haoyu Wang, Zhuo Sun, Xiaoyuan Cheng

Organizations: University of Sheffield · Shanghai Jiao Tong University · Shanghai University of Finance and Economics · University College London

Abstract

Predictive world models provide compact visual representations for control. Control requires a latent geometry aligned with temporal reachability rather than predictive similarity alone. We introduce Directed Temporal Representations for Control (DTRC), which learns such a geometry from offline visual trajectories on top of frozen LeWorldModel (LeWM) features. DTRC constructs a directed temporal quasimetric over the learned control representation. Short-range temporal offsets calibrate the distance scale. Bootstrapped targets extend temporal reachability across longer horizons. Action-conditioned consistency aligns the representation with local transition dynamics. The resulting distance estimates temporal reaching cost, and its change across a transition defines goal-relative temporal progress. We use this progress signal as a temporal critic for direct goal-conditioned policy learning. Model-assisted targets provide an additional training-time refinement under behavior-support and dynamics-agreement constraints. Across ten visual control tasks, DTRC achieves strong goal-conditioned control performance relative to planning and direct-policy baselines. Held-out diagnostics on the four LeWM tasks show consistent short-range temporal calibration, task-dependent long-range and directional structure, and positive transition-level progress. Temporal supervision improves the same flow-policy parameterization across all four LeWM tasks, while the resulting policy acts directly without iterative trajectory search at test time.

Figures & tables

Figure 1: Overview of directed temporal representation learning and policy training. (a) Offline visual trajectories provide temporal supervision on top of frozen LeWM features. (b) DTRC learns a control representation with a directed temporal quasimetric. A symmetric metric component defines the base geometry, and an order-sensitive residual represents directional state–goal cost. (c) Temporal distance reduction measures goal-relative progress. The resulting temporal advantage weights recorded continuations for direct policy learning.

Figure 2: Goal-conditioned control on the four LeWM tasks. Bars report mean success rate over five evaluation seeds; error bars denote sample standard deviation.

Method	PointMaze	Cube-Double	Scene	AntMaze-Medium	HumanoidMaze-Medium	Puzzle-3x3
LeWM	$13.3\pm 1.2$	$50.7\pm 2.3$	$56.7\pm 6.4$	$32.0\pm 6.0$	$82.0\pm 2.0$	$24.7\pm 1.2$
GCBC	$28.7\pm 3.1$	$44.7\pm 2.3$	$48.7\pm 5.0$	$28.0\pm 3.5$	$83.3\pm 3.1$	$35.3\pm 3.1$
GC-IQL	$16.0\pm 2.0$	$48.0\pm 5.3$	$51.3\pm 9.0$	$30.0\pm 5.3$	$84.0\pm 2.0$	$31.3\pm 5.0$
HILP	$22.7\pm 4.6$	$43.3\pm 5.8$	$50.0\pm 7.2$	$28.7\pm 3.1$	$81.3\pm 4.2$	$40.0\pm 6.9$
GC-IDM	$28.0\pm 5.3$	$38.7\pm 4.6$	$48.7\pm 10.3$	$30.0\pm 3.5$	$88.0\pm 2.0$	$33.3\pm 2.3$
DTRC (ours)	$11.3\pm 3.1$	$54.7\pm 8.3$	$53.3\pm 4.2$	$31.3\pm 4.6$	$84.7\pm 5.0$	$40.0\pm 5.3$

Table 1: Goal-conditioned control on six additional visual OGBench tasks. Results report mean success rate (%)

\pm

sample standard deviation over five evaluation seeds. Best results are shown in bold and second-best results are underlined.

Figure 3: Temporal structure learned by DTRC on PushT. We evaluate 20,000 held-out state–future-goal pairs, with 200 pairs at each temporal offset from 1 to 100 transitions. (a) Forward temporal distance as a function of recorded temporal offset. The solid curve and shaded region show the median and interquartile range in 5-step bins; the dashed line denotes equality with the recorded offset ( $\rho=0.845$ ). (b) Forward distance $d(e_{t},e_{g})$ and reverse distance $d(e_{g},e_{t})$ for the same state–goal pairs. The gray curve shows the symmetric contribution of the same DTRC model. (c) Distribution of one-step temporal progress $P_{t,g}^{(1)}=d(e_{t},e_{g})-d(e_{t+1},e_{g})$ . Positive progress occurs for $81.41\%$ of the evaluated pairs.

Task	MAE 1:8	$\rho_{1:100}$	Reverse $>$ Forward	Positive progress
	(steps)		(%)	(%)
PushT	1.55	0.845	92.76	81.41
TwoRoom	1.51	0.800	57.65	65.78
Reacher	1.62	0.510	50.28	59.41
Cube	0.74	0.221	52.46	72.35

Table 2: Temporal representation diagnostics across the four LeWM tasks. MAE 1:8 measures the mean absolute difference between forward temporal distance and recorded offset over temporal offsets 1–8.

\rho_{1:100}

denotes Spearman’s rank correlation between forward temporal distance and recorded offset over offsets 1–100. Reverse

>

Forward reports the fraction of state–goal pairs satisfying

d(e_{g},e_{t})>d(e_{t},e_{g})

. Positive progress reports the fraction satisfying

P_{t,g}^{(1)}>0

Configuration	PushT	TwoRoom	Reacher	Cube
Flow BC	$74.7\pm 4.6$	$94.7\pm 4.2$	$68.0\pm 3.5$	$91.3\pm 1.2$
+ temporal supervision	$84.0\pm 5.3$	$100.0\pm 0.0$	$100.0\pm 0.0$	$97.3\pm 3.1$

Table 3: Effect of temporal supervision on policy learning. Success rates (%) are reported as mean

\pm

sample standard deviation over five evaluation seeds. Both variants use the same flow-policy parameterization. Temporal supervision adds the learned DTRC control representation and temporal-advantage weighting without model-assisted targets. All variants use the same 50 start–goal pairs per task and seed, a goal offset of 25 recorded transitions, and a 50-step interaction budget.

Figure 4: Qualitative organization of the learned representation on PushT. We project frozen LeWM features $z$ , learned control representations $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ onto their first two principal components. Points are colored by normalized time within the offline trajectory. The projections are used only for visualization; all temporal distances and quantitative diagnostics are computed in the full learned representation.

Figure 5: Task-dependent geometry of the learned control representation. Recorded trajectories are shown in the environment (top) and in the first two principal components of the DTRC control representation (bottom) for TwoRoom, Reacher, and PointMaze. Gray points show the offline representation distribution, and colored curves show recorded trajectories. Stars denote trajectory goals. The PCA projections are used only for qualitative visualization.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Network	Input	Hidden widths	Output
Representation $\psi$	$z$	$512,512$	$128$
Symmetric head $S$	$e$	$256$	$64$
Order-sensitive head $A_{\theta}$	$e$	$256$	$64$
Transition head $T_{\xi}$	$[e,a]$	$256$	$128$
Dynamics member $f_{m}$	$[z,a]$	$512,512$	$192$
Policy velocity $v_{\omega}$	$[x_{s},\operatorname{Emb}(s),c_{t}]$	$512,512,512$	$kd_{a}$

Appendix

Table 4: Trainable network architectures. Hidden widths exclude output layers. The policy condition has dimension

2d_{z}+2d_{e}+64=704

\operatorname{Emb}(s)

denotes the

64

-dimensional flow-time embedding.

Hyperparameter	Value
Optimization
Training iterations $U$	$200{,}000$
Minibatch size	$1024$
Optimizer	Adam
Learning rate	$3\times 10^{-4}$ (constant)
Adam moment coefficients	$(0.9,0.999)$

Appendix

Table 5: Default training and inference hyperparameters for DTRC.

Configuration	PushT	TwoRoom	Reacher	Cube
Flow BC	$74.7\pm 4.6$	$94.7\pm 4.2$	$68.0\pm 3.5$	$91.3\pm 1.2$
Flow BC + temporal supervision	$84.0\pm 5.3$	$100.0\pm 0.0$	$100.0\pm 0.0$	$97.3\pm 3.1$
DTRC w/o behavior support	$88.0\pm 4.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$95.3\pm 1.2$
DTRC w/o directional residual	$90.0\pm 5.3$	$100.0\pm 0.0$	$100.0\pm 0.0$	$\mathbf{98.7\pm 2.3}$
DTRC (full)	$\mathbf{90.7\pm 5.0}$	$\mathbf{100.0\pm 0.0}$	$\mathbf{100.0\pm 0.0}$	$96.0\pm 2.0$

Appendix

Table 6: Ablation study on the four LeWM tasks. Success rates (%) are reported as mean

\pm

sample standard deviation over five evaluation seeds. All variants use the same 50 start–goal pairs per task and seed, a goal offset of 25 recorded transitions, and a 50-step interaction budget.

	Goal offset
Method	25	50	100
LeWM	$43.2\pm 1.6$	$17.6\pm 4.1$	$10.7\pm 1.9$
GCBC	$44.8\pm 1.4$	$16.4\pm 4.6$	$9.1\pm 0.8$
GC-IQL	$43.4\pm 2.2$	$16.9\pm 3.8$	$9.8\pm 1.0$
HILP	$44.3\pm 2.2$	$17.2\pm 2.7$	$9.0\pm 0.9$
GC-IDM	$44.5\pm 2.2$	$16.8\pm 3.3$	$8.3\pm 0.3$

Appendix

Table 7: Goal-conditioned control across goal offsets. Task-averaged success rates (%) over six OGBench tasks.

Panel of Figure 6 — Figure 6: Learned representation spaces on TwoRoom and Reacher. From left to right, each row shows frozen LeWM features $z$ , the learned control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors denote normalized offline episode time. PCA is fitted independently in each space.

Figure 7: Learned representation spaces on OGBench-Cube. From left to right, the panels show frozen LeWM features $z$ , the learned control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors denote normalized offline episode time. PCA is fitted independently in each space.

Panel of Figure 8 — Figure 8: Learned representation spaces on PointMaze and AntMaze-Medium. For each task, the panels show frozen LeWM features $z$ , the learned control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors indicate normalized offline episode time. PCA is fitted independently in each space, so orientations and scales are not directly comparable across panels or tasks.

Panel of Figure 9 — Figure 9: Learned representation spaces on Cube-Double and Scene. The visualization follows the same construction as Figure 8 . Each space is projected independently onto its first two principal components and colored by normalized offline episode time.

Panel of Figure 10 — Figure 10: Learned representation spaces on HumanoidMaze-Medium and Puzzle-3x3. From left to right, each task shows frozen LeWM features $z$ , the learned control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . PCA is fitted independently in each space.

Panel of Figure 11 — Figure 11: Additional temporal diagnostics on TwoRoom and Reacher. Each task is evaluated on 20,000 held-out state–future-goal pairs with balanced temporal offsets from 1 to 100 transitions. The panels show temporal structure across recorded offsets, directional state–goal cost, and one-step temporal progress. The shared symmetric term is computed from the same trained DTRC model and is not a separately trained baseline.

Figure 12: Additional temporal diagnostics on OGBench-Cube. The evaluation uses the same held-out, offset-balanced protocol as the main text. The panels show temporal structure across recorded offsets, directional state–goal cost, and one-step temporal progress.

Panel of Figure 13 — Figure 13: Temporal diagnostics on PointMaze and AntMaze-Medium. The panels show temporal structure across recorded offsets, directional state–goal cost, and one-step temporal progress. Forward, reverse, and shared symmetric curves are computed from the same trained DTRC model.

Panel of Figure 14 — Figure 14: Temporal diagnostics on Cube-Double and Scene. The evaluation follows the same held-out, offset-balanced protocol used for the main-text representation diagnostics. Recorded offset is used as a reference for trajectory duration rather than as ground-truth optimal reaching cost.

Panel of Figure 15 — Figure 15: Temporal diagnostics on HumanoidMaze-Medium and Puzzle-3x3. Each task is evaluated on the same offset-balanced cohort used throughout the additional temporal diagnostics.

Panel of Figure 16 — Figure 16: Pairwise decomposition of the learned temporal distance on PushT and TwoRoom. Rows index source states and columns index goal states by normalized episode time. From left to right, each row shows the symmetric metric contribution, the order-sensitive directed contribution, and their total temporal distance. Each task uses its own distance scale.

Panel of Figure 17 — Figure 17: Pairwise decomposition of the learned temporal distance on Reacher and OGBench-Cube. Rows index source states and columns index goal states by normalized episode time. From left to right, each row shows the symmetric metric contribution, the order-sensitive directed contribution, and their total temporal distance. Each task uses its own distance scale.

Panel of Figure 18 — Figure 18: Pairwise decomposition of the learned temporal distance on PointMaze and AntMaze-Medium. Rows index source states and columns index goal states by normalized episode time. From left to right, each row shows the symmetric metric contribution, the order-sensitive directed contribution, and their total temporal distance. Each task uses its own distance scale.

Panel of Figure 19 — Figure 19: Pairwise decomposition of the learned temporal distance on Cube-Double and Scene. The visualization follows the same construction as Figure 18 . Symmetric and directed contributions are evaluated separately before forming the total temporal distance.

Panel of Figure 20 — Figure 20: Pairwise decomposition of the learned temporal distance on HumanoidMaze-Medium and Puzzle-3x3. Rows and columns index source and goal states by normalized episode time. Each task uses its own distance scale.

Panel of Figure 21 — Figure 21: Goal-relative directional landscapes on PushT and TwoRoom. For a fixed recorded goal, the panels show the goal observation, forward temporal distance from each state to the goal, reverse temporal distance from the goal to each state, and the reverse–forward difference. PCA is used only for visualization; temporal distances are evaluated in the full learned representation.

Panel of Figure 22 — Figure 22: Goal-relative directional landscapes on Reacher and OGBench-Cube. The visualization follows the same construction as Figure 21 . The forward and reverse landscapes are evaluated for the same fixed goal, and the final panel shows their difference.

Panel of Figure 23 — Figure 23: Goal-relative directional landscapes on PointMaze and AntMaze-Medium. For a fixed recorded goal, the panels show the goal observation, forward temporal distance from each state to the goal, reverse temporal distance from the goal to each state, and the reverse–forward difference. PCA is used only for visualization.

Panel of Figure 24 — Figure 24: Goal-relative directional landscapes on Cube-Double and Scene. Forward and reverse temporal distances are evaluated for the same fixed goal, and the final panel shows their difference. Distances are computed in the full learned representation.

Panel of Figure 25 — Figure 25: Goal-relative directional landscapes on HumanoidMaze-Medium and Puzzle-3x3. The visualization follows the same construction as Figure 23 .

Panel of Figure 26 — Figure 26: Evolution of the learned representation on PushT and TwoRoom. Columns correspond to training checkpoints and rows show the control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors indicate normalized offline episode time.

Panel of Figure 27 — Figure 27: Evolution of the learned representation on Reacher and OGBench-Cube. Columns correspond to training checkpoints and rows show the control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors indicate normalized offline episode time.

Panel of Figure 28 — Figure 28: Evolution of the learned representation on PointMaze and AntMaze-Medium. Columns correspond to training checkpoints and rows show the control representation $e$ , symmetric coordinates $S(e)$ , and directed potentials $A(e)$ . Colors indicate normalized offline episode time. PCA is fitted independently at each checkpoint, so orientation and scale are not directly comparable across columns.

Panel of Figure 29 — Figure 29: Evolution of the learned representation on Cube-Double and Scene. The visualization follows the same checkpoint and PCA protocol as Figure 28 .

Panel of Figure 30 — Figure 30: Evolution of the learned representation on HumanoidMaze-Medium and Puzzle-3x3. Columns correspond to training checkpoints and rows show $e$ , $S(e)$ , and $A(e)$ . PCA is fitted independently at each checkpoint.

Explore similar work

May 25, 2026cs.AI

Back to Parsimonious Latents: Learning Task-Centric World Models from Visual Foundations

World models enable agents to predict future dynamics conditioned on actions, making the choice of latent representation central to planning and control. Such representations are often either learned directly from pixels with limited semantic structure or inherited from frozen visual foundation models with excessive task-irrelevant detail, yielding state spaces that are poorly matched to downstream planning and control. This is especially challenging in reward-free offline settings, where the model must learn from fixed trajectories without reward supervision or online interaction. To address this, we propose TC-WM, a framework for turning foundation-model embeddings into compact, task-sufficient world representations. The key design is to treat the pretrained embedding space as a semantic scaffold rather than as the final state space: TC-WM linearly projects high-dimensional visual embeddings into a compact latent as the dynamic space, aligns a subspace with the agent's physical state via contrastive learning, and reconstructs embeddings to preserve useful visual structure. This combines the generality of foundation features with the controllability of task-centric dynamics. Theoretically, we show that TC-WM suffices to identify the underlying task-centric latent factors up to a simple transformation. Empirically, TC-WM enables test-time planning across diverse environments (e.g., Robomimic and D4RL), achieving better world-modeling quality and more precise control than state-of-the-art approaches.

Minghao Fu, Fan Feng, Nicklas Hansen +1

University of California, San Diego

World Models Frozen Vision-Language Models Latent Variable State Space

Ask PDF

Aug 6, 2026cs.AI

TaskSense: Focusing on What Matters in World Models

World models for visual control typically learn compact latent states by reconstructing observations, implicitly encouraging representations to preserve information across the entire visual input. However, task-relevant content often occupies only a small fraction of the observation, while background clutter and distractors consume valuable representational capacity. This mismatch between visual reconstruction and control objectives biases latent representations to model task-irrelevant visual content, diluting learning signals for control-relevant features and severely degrading downstream performance under visual distractions. We introduce TaskSense, a task-centric world modeling framework that enforces task relevance before latent encoding through a differentiable stochastic spatial attention mechanism conditioned on the previous latent state. To steer attention toward control-relevant regions, we augment training with an auxiliary inverse-dynamics objective. Rather than reconstructing the full observation, the world model reconstructs only the attended regions, encouraging latent representations to preserve task-relevant information while discarding irrelevant visual content. The decoder is further conditioned on the sampled attention map, enabling consistent reconstruction despite stochastic attention. Compared with the DreamerV3 baseline, TaskSense maintains competitive performance on the DeepMind Control Suite while consistently outperforming DreamerV3 on the Distracting Control Suite, demonstrating substantially improved robustness to visual distractions. Qualitative analysis further confirms that the learned attention, guided by inverse-dynamics supervision, consistently localizes control-relevant regions while suppressing irrelevant visual content.

SM Mazharul Islam, Manfred Huber

1The University of Texas at Arlington

Latent World Models Visuomotor Control World Models

Ask PDF

Sep 29, 2026cs.AI

Beyond a single latent space: a dual-latent world model for long-horizon planning

Latent world models often struggle with long-horizon planning despite accurate short-term predictions. Recursive rollouts accumulate errors, while distance concentration in high-dimensional latent spaces can weaken goal discrimination. We introduce the Dual-Latent World Model (Dual-WM), which separates local execution and long-range planning through distinct state representations and dynamics models. The low-level model predicts action-conditioned transitions, while the high-level model uses learned macro-actions to plan over longer temporal spans. We also propose Long-Horizon Representation Learning with Weighted Rollout (LoRe), which supervises self-generated predictions at both levels. An analysis of recursive error propagation motivates exponential horizon weights with separate decay rates for the two temporal scales. During planning, the high-level model generates latent subgoals that the low-level model refines into actions for precise execution. We evaluate from-scratch Dual-WM on five goal-conditioned visual control tasks against the task-wise strongest baselines without actor-guided proposals. At goal offsets of 50 and 100 environment steps, mean success increases from 75.9% to 84.4% and from 61.4% to 69.5%, respectively. At offset 100, Dual-WM outperforms these baselines on all five tasks and improves mean success over LeWM by 30.8 percentage points. Ablations and supporting analyses provide evidence of more informative representations for goal evaluation and greater consistency under recursive prediction. These results highlight the value of separating temporal roles and training across multiple horizons for reliable latent planning. Our core implementation is available at https://github.com/DeLin1001/Dual-WM-Official.

Delin Zhao, Zhengrong Yue, Shaobin Zhuang +6

Nanjing University · Shenzhen University of Advanced Technology · Shanghai Jiao Tong University +1

Latent World Models Long-Horizon Planning Latent Space Planning

Ask PDF