Temporal Action Representation Learning for Tactical Resource Control and Subsequent Maneuver Generation

Hoseong Jung; Sungil Son; Daesol Cho; Jonghae Park; Changhyun Choi; H. Jin Kim

cs.ROFeb 21, 2026

Temporal Action Representation Learning for Tactical Resource Control and Subsequent Maneuver Generation

Authors: Hoseong Jung, Sungil Son, Daesol Cho, Jonghae Park, Changhyun Choi, H. Jin Kim

Organizations: Seoul National University, Republic of Korea · Seoul National University and Life Assistant Robotics, Republic of Korea · Georgia Institute of Technology, USA

Abstract

Autonomous robotic systems should reason about resource control and its impact on subsequent maneuvers, especially when operating with limited energy budgets or restricted sensing. Learning-based control is effective in handling complex dynamics and represents the problem as a hybrid action space unifying discrete resource usage and continuous maneuvers. However, prior works on hybrid action space have not sufficiently captured the causal dependencies between resource usage and maneuvers. They have also overlooked the multi-modal nature of tactical decisions, both of which are critical in fast-evolving scenarios. In this paper, we propose TART, a Temporal Action Representation learning framework for Tactical resource control and subsequent maneuver generation. TART leverages contrastive learning based on a mutual information objective, designed to capture inherent temporal dependencies in resource-maneuver interactions. These learned representations are quantized into discrete codebook entries that condition the policy, capturing recurring tactical patterns and enabling multi-modal and temporally coherent behaviors. We evaluate TART in two domains where resource deployment is critical: (i) a maze navigation task where a limited budget of discrete actions provides enhanced mobility, and (ii) a high-fidelity air combat simulator in which an F-16 agent operates weapons and defensive systems in coordination with flight maneuvers. Across both domains, TART consistently outperforms hybrid-action baselines, demonstrating its effectiveness in leveraging limited resources and producing context-aware subsequent maneuvers.

Figures & tables

Fig. 1: Snapshots of tactical decision-making in an air combat scenario. (a) A discrete action (e.g., weapon release) both constrains the set of feasible follow-up maneuvers ( causal dependency ) and (b) gives rise to multiple valid maneuver modes ( multi-modality ). TART is designed to capture these temporal dependencies and multi-modal outcomes, conditioning the policy to select context-appropriate maneuvers.

Fig. 2: Overview of TART: (1) The agent interacts with the environment and collects a set of trajectories. (2) A mutual information objective guides the clustering of given trajectories into multiple tactical modes through contrastive learning (Sec. III-B). (3) The resulting distinct modes are then mapped to discrete vectors via vector quantization (VQ). The continuous actor distinguishes between the modes and generates multi-modal maneuver distributions accordingly (Sec. III-C).

Fig. 3: Overviews and difficulty settings of the evaluation environments. (a)–(c) Maze Navigation: Easy (trivially solvable), Medium (complex), Hard (dynamic obstacles). (d)–(f) Air-to-Air Combat: Easy (fixed-maneuver opponent), Medium (unarmed evasive opponent), Hard (armed pursuing opponent).

Task/ Scenario		TART	PADDPG [ 6 ]	PDQN [ 8 ]	HPPO [ 9 ]	HyAR [ 10 ]	TART w/o $\mathcal{L}_{\text{NCE}}$	TART w/o $\mathcal{L}_{\text{VQ}}$
Maze Navigation	Easy	$\mathbf{97.2\pm 1.3}$	$88.2\pm 6.9$	$85.8\pm 6.1$	$87.2\pm 4.6$	$94.5\pm 3.2$	$92.8\pm 3.3$	$95.4\pm 1.9$
Maze Navigation	Medium	$\mathbf{90.8\pm 4.2}$	$74.2\pm 6.1$	$68.8\pm 6.2$	$74.6\pm 9.0$	$80.6\pm 8.5$	$89.0\pm 3.5$	$88.2\pm 5.3$
	Hard	$\mathbf{72.8\pm 9.9}$	$38.8\pm 10.5$	$38.4\pm 13.5$	$46.2\pm 6.7$	$60.4\pm 9.1$	$69.2\pm 5.7$	$67.0\pm 3.2$
Air-to-Air Combat	Easy	$\mathbf{94.8\pm 3.1}$	$79.6\pm 5.6$	$81.2\pm 5.9$	$87.8\pm 4.6$	$86.6\pm 4.8$	$92.4\pm 5.1$	$91.8\pm 2.2$
Air-to-Air Combat	Medium	$\mathbf{90.8\pm 4.2}$	$68.6\pm 5.9$	$74.2\pm 4.3$	$76.2\pm 4.5$	$77.6\pm 6.3$	$87.0\pm 6.2$	$89.8\pm 2.6$
	Hard	$\mathbf{76.8\pm 5.0}$	$61.8\pm 7.8$	$57.2\pm 7.4$	$65.4\pm 3.0$	$64.4\pm 7.2$	$73.6\pm 6.2$	$70.8\pm 2.1$

TABLE I: Performance ( Success Rate ) for different PAMDP methods over resource-limited environments. Results are averaged over five random seeds,

\pm

indicates the standard deviation across seeds, and bold entries denote the best performance.

Fig. 4: Experimental results across the designed environments and metrics. Values are averaged over five seeds and black bars indicate standard deviation. For failed episodes, TTG and TTE are set to their maximum values (100 and 1800, respectively), while SPE is reported only for successful episodes. Hatched bars indicate the results discussed in the text and exhibiting superior performance.

Fig. 5: Representative qualitative results in (a)-(b) Maze Navigation and (c)-(d) Air-to-Air Combat. (a) Heatmaps in Easy , Medium , and Hard scenarios, where the triangle marks the optimal path. Red indicates higher, while white indicates lower visitation frequency. (b) Deadlock examples, where the agent employs the Penetration action to navigate through cluttered corridors. (c) An offensive maneuver: the agent consecutively launches shots using the Missile action. (d) A defensive maneuver: the agent neutralizes the opponent’s missile with a Defense action and responds with a counter shot.

Robot Policy Learning

Robot policy learning develops controllers that map sensory observations, task goals, and interaction history to actions for embodied robots.

See all 372 papers

Oct 27, 2025

TARC: Time-Adaptive Robotic Control

Most robotic systems rely on fixed-frequency discrete-time controllers, creating a trade-off between the efficiency of low-frequency control and the responsiveness of high-frequency feedback. As a result, systems typically default to high control rates for robustness, at the cost of wasted inference and unnecessary actuation. Addressing this, we introduce Time-Adaptive Robotic Control (TARC), a reinforcement learning framework in which the policy jointly predicts a control action and its duration of application. TARC learns temporally extended actions by optimizing task performance under soft or hard constraints on the number of control switches, enabling adaptive modulation of control rates. We evaluate TARC on two robotic hardware platforms: a high-speed RC car and the Unitree Go1 quadruped, and on a vision-language action model in simulation, where each query incurs a costly transformer forward pass. Across all settings, TARC matches the performance of high-frequency discrete-time controllers while operating at less than half their control frequency. Unlike fixed-rate controllers, TARC adapts its control frequency online, allocating high-frequency feedback only when required.

Arnav Sukhija, Lenart Treven, Jin Cheng +3

ETH Zurich

Robotic Control RL Control Adaptive Control Real-Time Robot Control

Ask PDF

Jul 13, 2026

EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation

Learning effective action representations is critical for robotic manipulation, where raw control trajectories are often noisy, redundant, and difficult to model directly. Existing methods mainly encode the structure of the action stream itself, treating the role of actions in the environment as implicit. Yet manipulation is about changing the world: the same action segment can induce different outcomes under different scene contexts, making action semantics inherently environment-dependent. We propose EDAR, an Environment-Dependent Action Representation that grounds action tokens in both executable control structure and expected visual consequences. By coupling motor commands with their environment-conditioned effects, EDAR encourages the learned action space to capture interaction semantics rather than merely command-level patterns. Experiments on simulated and real-robot manipulation benchmarks demonstrate that EDAR improves downstream policy learning, especially in long-horizon manipulation. These results highlight the importance of grounding action representations in executable control structure and environment-conditioned visual change.

Yuecheng Xu, Tong Yang, Jingkai Jia +3

College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · TeleAI, China Telecom

Latent Action Learning Robot Policy Learning Long-Horizon Robotic Manipulation Robotic Manipulation Robot Learning Action Representation Learning

Ask PDF

Oct 16, 2026

Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control

Planning while moving faces an historical trade-off between deliberation and reactivity. A planner running concurrently with execution needs time to compute, so the robot must commit to an open-loop action chunk for that duration: a longer commitment buys a better plan at the cost of feedback frequency. Existing controllers fix this action commitment, and consequently the computation budget, before deployment. However, the optimal computation time is highly state-dependent (due to factors such as unmodeled dynamics, safety constraints, and marginal optimisation gains), i.e. dynamic, therefore offline static tuning risks leaving systems blind during critical maneuvers or unable to find global effective solutions. We cast the identification of the optimal computation budget as a semi-Markov decision process (SMDP) in which a single decision defines both the robot action commitments and the time allocated to the solver. To solve this, we train a reinforcement-learning policy conditioned on the current state, the executing plan, and inexpensive planner statistics. Across ground-robot navigation, agile quadrotor flight, and contact-rich manipulation, the learned policy changes its budget with the demands of the scenario, discovering efficient reasoning patterns that outperform fixed-budget baselines. https://lorenzomagnino.github.io/learning-robot-metareasoning-review/

Lorenzo Magnino, Antonio Marino, Michael Amir +2

University of Cambridge — Cambridge, United Kingdom. · College of Engineering and Computing at George Mason University

Real-Time Robot Control Robotic Control Robot Policy Learning Robotic RL

Ask PDF

Abstract

Robot Policy Learning

Explore similar work

TARC: Time-Adaptive Robotic Control

EDAR: Learning Environment-Dependent Action Representations for Robotic Manipulation

Learning Robot Metareasoning: Adaptive Computation Budgets for Real-Time Control