cs.LGAug 16, 2024

TRAM: Test-Time Risk Adaptation with Mixture of Agents

Authors: Mohamad Chehade, Amrit Singh Bedi, Souradip Chakraborty, Amy Zhang, Hao Zhu

Organizations: UT Austin · University of Central Florida · MIT

Abstract

Deployed reinforcement learning agents often face safety requirements that are specified only after training: new hazard maps, revised risk thresholds, or behavioral alignment constraints. We study zero-update deployment-time adaptation in which a fixed library of risk-neutral source policies must be reused under a newly specified reward-risk tradeoff. We propose TRAM (Test-time Risk Adaptation with Mixture of Agents), a source-scored composition rule that evaluates each source under the target reward and an occupancy-based deployment risk, then selects actions using risk-adjusted source scores. Unlike training-time risk-sensitive methods tied to a fixed surrogate such as return variance, TRAM supports spatial barrier exposure, divergence to a reference behavior, and local volatility risks specified at test time. We make the surrogate nature of the method explicit: TRAM is not claimed to solve the full occupancy-control problem of the stitched policy, but it admits a measurable source-hull mismatch term connecting source-scored risk to realized risk. Experiments in gridworlds, MuJoCo Reacher, Safety-Gymnasium, and an inference-time LLM alignment setting show that TRAM improves deployment risk while preserving reward and requires no parameter updates at test time.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. SafeAdapt: Provably Safe Policy Updates in Deep Reinforcement Learning

    Apr 10, 2026Maksim Anisimov, Francesco Belardinelli, Matthew WickerSafe RLContinual RL

  2. Efficient Test-Time Adaptation For Robot Policies

    Oct 14, 2026Motasem Alfarra, Pietro Mazzaglia, Markus Peschl +2Robot Policy AdaptationTest-Time RL

  3. Position: Deployed Reinforcement Learning should be Continual

    Jun 1, 2026Parnian Behdin, Kevin Roice, Golnaz MesbahiNon-Stationary RLReinforcement Learning