Online investment services must translate asset opportunities into investment outcomes for users' accounts, yet the same market opportunity can warrant different account actions. We study how supervision from future asset opportunities can strengthen account-level trading decisions and propose Forecast-Augmented Relative Policy Optimization (FRPO). FRPO separately supervises asset opportunities and account decisions through a jointly learned representation: a Financial State Head (FSH) learns future relative-return targets to provide asset-level opportunity supervision, while Same-Path Group-Relative Optimization compares trading trajectories under the same market path and initial account to provide account-level relative-outcome supervision. An Adaptive Gate coordinates auxiliary supervision using predictive variance and gradient alignment. FSH is used only during training; inference uses the encoder and policy for trading. Across DJIA30, TOPIX Core30, and three investment-style portfolios, ten-seed evaluations show that FRPO achieves the highest mean cumulative and annualized returns among the compared methods in all five settings. These results support strengthening investment agents' trading decisions through joint supervision from asset opportunities and trading outcomes.
Figures & tables
Figure 1. From asset opportunities to portfolio actions. (a) Illustrative accounts link the same favorable asset outlook to different actions under different cash balances and exposures. (b) Same-path relative outcomes and future-state targets jointly supervise the shared encoder; predictive variance and gradient alignment weight auxiliary learning. The account and representation artwork is schematic. (c) Ten-seed mean cumulative returns exceed the strongest non-FRPO method in each setting (Tables 1 and 2 ). Solid arrows show forward flow; dashed arrows show training supervision. Three panels connect motivation, method, and performance. The illustrative accounts share a favorable Asset A outlook; ample cash and low exposure accompany a Buy example, while limited cash and high exposure accompany Hold. FRPO uses a shared encoder with separate policy and training-only FSH branches. Same-path relative returns and future-state targets supervise the encoder; FSH predictive variance and gradient alignment control the auxiliary weight. A common-scale horizontal bar chart compares ten-seed mean cumulative returns against the strongest non-FRPO method on DJIA30, TOPIX Core30, Mainstream, Defensive, and High-Volatility Growth.
Figure 2. FRPO architecture with shared predictive and decision supervision. Market observations and cash/holdings enter one encoder. Its representation supports a continuous portfolio policy and the upper Financial State Head (FSH). Centered, historical-volatility-normalized, clipped future-return targets supervise FSH through heteroscedastic Gaussian NLL; predictive variance and gradient alignment determine a detached auxiliary weight. The right inset compares full-horizon net returns on a shared market path and initial account, while later accounts diverge. Each group-relative advantage supervises every step of its trajectory. Solid arrows show forward flow, dashed arrows training, and dotted arrows loss weighting. Input artwork and trajectory traces are schematic. Inference uses the encoder and policy alone. An open horizontal diagram shows illustrated historical market charts and a current account ledger merging before input projection and a shared Transformer encoder. Shared features continue right into a continuous portfolio policy and branch upward into a training-only Financial State Head. Predicted distributions and future-state targets meet at a Gaussian NLL objective. Predictive variance and gradient alignment determine the detached auxiliary weight. Short dashed curves show auxiliary supervision of the encoder and group-relative supervision of the policy. A narrow right inset contains one shared external price path and identical initial account, three different trading and account trajectories, and all three full-horizon returns collected for group-relative advantages. Each advantage applies at every decision step. Input illustrations and trajectory traces are conceptual, not measured experimental results. FSH predictions do not enter the policy.
Method
DJIA30
TOPIX Core30
CR (%)
AR (%)
Sharpe
Sortino
Calmar
CR (%)
AR (%)
Sharpe
Sortino
Calmar
Technical and Market Baselines
MACD
-8.47
-3.27
-0.200
-0.160
-0.100
19.23
7.13
0.570
0.550
0.660
SMA
5.63
2.08
0.110
0.090
0.060
-2.36
-0.93
-0.070
-0.070
-0.030
ZMR
8.07
2.96
0.190
0.160
0.100
21.96
8.08
0.670
0.730
0.880
B&H
51.65
16.93
0.690
0.750
0.460
36.36
12.90
0.700
0.930
0.480
Table 1. Performance on two large market universes. Technical and market baselines are deterministic price-only backtests; learned methods report ten-seed averages. CR and AR are cumulative and annualized returns (%); higher is better for all metrics. Red and blue denote the best and second-best values in each dataset and metric.
Table 2. Performance on three heterogeneous six-asset portfolios. Reporting conventions follow Table 1 .
TOPIX Core30
DJIA30
Method
CR (%)
AR (%)
Sharpe
Sortino
Calmar
CR (%)
AR (%)
Sharpe
Sortino
Calmar
FRPO
50.52
15.76
0.592
0.923
0.524
64.93
19.59
0.695
0.855
0.535
TD3
30.77
10.48
0.397
0.583
0.395
48.16
15.40
0.639
0.711
0.448
SAC
27.64
8.88
0.333
0.508
0.284
37.28
12.19
0.474
0.541
0.319
A2C
26.93
8.98
0.313
0.465
0.285
43.26
13.88
0.557
0.639
0.391
PPO
19.51
6.76
0.324
0.482
0.227
51.87
15.78
0.565
0.691
0.453
Table 6. Temporal robustness: equal-weight means over the overlapping splits w00 , w01 , and w02 , with ten seeds per split. Aggregation follows Equation 13 . CR and AR are percentages; higher is better. Colors follow Table 1 .
Figure 5. Cumulative-return retention relative to the 1-bp setting as per-side transaction costs increase across the five evaluation settings. Grouped bars show cumulative-return retention as per-side transaction costs increase from one to fifty basis points in five markets.
Reinforcement learning agents for portfolio management are typically trained and deployed as static policies, with no mechanism for policy adjustments using price forecasts at inference time. We propose FinPILOT (Financial Plugin Inference-time Learning for Optimal Trading), a plugin inference-time policy optimization framework inspired by Model Predictive Control (MPC). Under the price-taker assumption, portfolio actions do not affect future market prices, allowing a multi-step price forecaster to provide imagined market trajectories without iterative action-conditioned rollouts. At each decision step, FinPILOT uses the forecaster's predicted price trajectory to construct an allocation-based imagined return objective and optimizes the pretrained policy at inference time before executing one step of the trade. Our framework is compatible with any pretrained policy and adapts the policy using the forecaster's predictions without modifying the original RL training pipeline. Evaluated with five pretrained policies (PPO, SAC, A2C, TD3, DDPG) across DJ30 equities and foreign exchange (FX) over three rolling held-out test years (30 settings in total), FinPILOT improves Total Return in 28 of 30 settings and Sharpe ratio in 27 of 30, with gains spanning both stochastic and deterministic policies. Direct single- and multi-period portfolio optimizers supplied with the same forecasts do not reproduce these aggregate gains under our evaluation protocol. Further, using controlled forecasts at calibrated quality levels, we show that both the magnitude and consistency of FinPILOT's performance increase as forecast error is reduced.
Eun Go, Rohan Deb, Arindam Banerjee
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign
Financial prediction typically relies on task-specific regression, ranking, or policy heads, separating the language model from the numerical object ultimately evaluated. We investigate whether a causal language model can instead represent forecasts and decisions directly through constrained token generation. FinATOM introduces a unified, head-free interface for three-step stock-return forecasting and dynamic five-ETF allocation. The forecasting model autoregressively emits volatility-standardized return tokens and is trained with ordinal and ranking supervision followed by a one-epoch token-level policy stage. The allocation model generates normalized long-only weights; supervised fine-tuning imitates a causal mean--variance anchor, and DAPO-augmented GRPO optimizes realized 21-day Sharpe subject to anchor consistency. In 2023--2025 ETF tests, the allocation policy improves pooled gross Sharpe from 1.428 to 1.529 and net Sharpe under a 5-bp transaction-cost model from 1.394 to 1.494. The multimodal allocation input attains the highest three-period mean Sharpe of 1.540, with its clearest advantage in 2025. On FinTexTS, the SFT and policy strategies achieve 73.52%/2.68 and 73.72%/2.69 cumulative-return/Sharpe, respectively. These results support the feasibility of direct language-model token generation for financial numerical prediction and decision-making, while motivating broader tests across assets, regimes, and random seeds.
Sparse tangent portfolio optimization aims to learn an interpretable, low-cardinality portfolio in the tangency direction of the mean-variance frontier. However, the associated cardinality-constrained formulation is NP-hard, and standard predict-then-optimize pipelines often misalign forecasting accuracy with downstream portfolio quality. We propose an end-to-end decision-focused learning framework that reformulates Sharpe ratio maximization as a Disciplined Parametrized Programming (DPP)-compliant convex programming layer and replaces discrete selection with a smooth top-k operator enforcing an exact cardinality k. This enables gradient flow through prediction, asset selection, and re-optimization, allowing the predictive model to directly optimize portfolio performance. Across four major equity markets, our method achieves competitive and often superior out-of-sample Sharpe ratios compared with historical and prediction-focused baselines, with particularly strong gains in larger asset universes. Our code is publicly available.
Haeun Jeon, Seunghoon Choi, Hyunglip Bae +2
Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Department of Big Data Convergence, and Bigdata Research Lab, Chonnam National University, Gwangju, Republic of Korea · Department of Industrial Engineering, Ulsan National Institute of Science and Technology, Ulsan, Republic of Korea