Dynamic Context Scheduling: Learning Beyond the Static Universe

Martin Mráz; André Biedenkapp

cs.AIAug 21, 2026

Dynamic Context Scheduling: Learning Beyond the Static Universe

Authors: Martin Mráz, André Biedenkapp

Organizations: University of Freiburg

Abstract

We study dynamic context scheduling as a training instrument for contextual re- inforcement learning. Rather than treating intra-episode context variation as a deployment reality, we treat it as a controlled shaping mechanism. Thereby, context evolves within each training episode according to a predetermined schedule, expos- ing the policy to a richer and more temporally structured region of the environment parameter space. We introduce DYNAMICCARLENV, a framework that wraps contextual environments with pluggable schedule families, such as sinusoidal off- sets or cosine annealing. Across CartPole, BipedalWalker and VehicleRacing with CARL contextualization, we show that dynamic schedules match or outperform static context baselines in the out-of-distribution (OOD) regimes. Interestingly, for the more complex BipedalWalker and VehicleRacing environments we also achieve higher in-distribution (ID) evaluation performance. Preliminary findings indicate that automatic search for multi-stage curricula can successfully discover schedules that improve generalization, performing comparably to extensive grid search over single-stage schedulers.

Figures & tables

Figure 1: Dynamic context wrapper around CARLEnv. Blue components are user-controlled. The Context Scheduler produces the per-step context

c_{t}

given the Schedule Parameters (e.g., amplitude, period, drift rate, dwell times, change points). The wrapper applies

c_{t}

through a Physics Adapter , controls what the policy observes through an Observation Adapter (none,

c_{0}

, or

c_{t}

), and logs

(s_{t},a_{t},r_{t+1},c_{t},c_{\mathrm{vis}})

via the Context and State Tracker . The base contextualized environment still receives

c_{0}

at reset.

Mode	Observation	Policy input
None	$s_{t}$	State only; dynamics change silently.
Live	$[s_{t},\;c_{t}^{*}]$	State concatenated with current context. ∗ Context normalized; see text.

Table 1: Context observability modes.

Figure 2: IQM episode reward (bars: Q1–Q3) at the final checkpoint across all 16 evaluation contexts for CartPole. Dashed lines show the random and oracle policy baselines. For dynamic schedulers we show the two best representatives by combined IQM: one observing only states, one also observing the context value.

Condition	CartPole	Walker	CarRacing (lateral)
Static observed	$429.0\;[401.5,\;451.4]$	$31.0\;[-34.9,\;63.8]$	$-85.9\;[-93.1,\;-76.2]$
Static blind	$410.0\;[383.1,\;436.5]$	$91.9\;[60.6,\;113.0]$	$103.6\;[-59.0,\;378.9]$
Best sched. observed	$\mathbf{468.2}\;[454.7,\;484.3]$	$\mathbf{151.4}\;[122.2,\;185.3]$	$-35.2\;[-67.3,\;-7.4]$
Best sched. blind	$462.9\;[430.9,\;494.1]$	$131.1\;[94.9,\;167.6]$	$\mathbf{592.3}\;[549.1,\;625.7]$

Table 2: Combined IQM score at the last checkpoint , computed from the per-seed avg_combined metric (average over all ID and OOD contexts). Values: IQM [Q1, Q3]. Bold marks the best scheduler per environment.

Condition	CartPole	Walker (1.5 M)
Static observed	$429.0\;[401.5,\;451.4]$	$\phantom{0}77.6\;[\phantom{0}54.2,\;107.8]$
Static blind	$410.0\;[383.1,\;436.5]$	$108.0\;[\phantom{0}72.3,\;127.5]$
Best sched. observed	$\textbf{468.2}\;[454.7,\;484.3]$	$119.3\;[\phantom{0}95.7,\;136.6]$
Best sched. blind	$462.9\;[430.9,\;494.1]$	$114.8\;[\phantom{0}88.6,\;131.9]$
Optuna #1 (T168 / T118)	$459.3\;[445.9,\;480.9]$	$\textbf{125.0}\;[\phantom{0}72.5,\;161.7]$
Optuna #2 (T218 / T117)	$459.3\;[441.1,\;470.0]$	$115.6\;[\phantom{0}96.0,\;157.4]$

Table 3: Multi-stage Optuna retrain results at the last checkpoint . Walker values are evaluated at the matched 1.5 M-step budget (half the full single-scheduler training run); single-scheduler baselines are re-evaluated at the same budget for a fair comparison. Values: IQM [Q1, Q3]. Top-3 retrained trials shown per environment.

Panel of Figure 3 — Figure 3: CartPole Optuna multi-stage retrain results.

Figure 4: IQM score (left) and 4D state-space coverage (right) per scheduler for CartPole pole-length at pool sizes 3 and 7 (20 seeds each). Points show individual seeds; bars show mean

\pm

SEM. ANOVA on coverage:

p>0.05

Appendix figures & tables60 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Trajectory examples for all eleven schedule families, simulated over three episodes of 500 steps each (episode boundaries marked by dashed vertical lines). All values are normalised to

[-1,+1]

relative to the training context range so that qualitative structure can be compared across context variables with different physical scales. Dotted horizontal lines mark the range boundaries (

\pm 1

) and midpoint (

0

). Multiple curves per panel show how the qualitative behaviour changes with the key hyperparameters of each family (amplitude, step size, drift rate, etc.).

Hyperparameter	CartPole	BipedalWalker	CarRacing
Total timesteps	150 000	3 000 000	1 500 000
Rollout steps ( $n$ )	2 048	2 048	2 048
Minibatch size	64	64	64
Learning rate	$3\times 10^{-4}$	$3\times 10^{-4}$	$3\times 10^{-4}$
Discount $\gamma$	0.99	0.99	0.99
GAE $\lambda$	0.95	0.95	0.95

Appendix

Table 4: PPO hyperparameters across environments.

Context	Training range	Eval ID	Eval OOD-low	Eval OOD-high
Pole length (m)	$[0.35,\,0.75]$	$\{0.37,0.50,0.60,0.71\}$	$\{0.06\ldots 0.12\}$	$\{0.92\ldots 1.32\}$

Appendix

Table 5: CartPole context ranges and evaluation splits.

Observation	Min	Max
Cart Position	-4.8	4.8
Cart Velocity	$-\infty$	$\infty$
Pole Angle	$\sim$ -0.418 rad (-24 ∘ )	$\sim$ 0.418 rad (24 ∘ )
Pole Angular Velocity	$-\infty$	$\infty$

Appendix

Table 6: Observation Space Bounds of CartPole

	Eval ID	Eval OOD-low	Eval OOD-high
COM_X (m)	$\{-0.4,-0.2,0.0,+0.2,+0.4\}$	$\{-1.05,\ldots,-0.65\}$	$\{+0.9,\ldots,+1.7\}$

Appendix

Table 7: BipedalWalker evaluation splits (COM_X, m).

Index	Description	Min	Max
0	Hull Angle	$-\pi$	$\pi$
1	Hull Angular Velocity	-5.0	5.0
2	Horizontal Speed	-5.0	5.0
3	Vertical Speed	-5.0	5.0
4	Hip 1 (Joint Angle)	$-\pi$	$\pi$
5	Hip 1 Speed	-5.0	5.0

Appendix

Table 8: BipedalWalker Observation Space Bounds

	Eval ID	Eval OOD-low	Eval OOD-high
COM_X / COM_Y (m)	$\{-0.5,\ldots,+0.5\}$	$\{-1.5,\,-1.3,\,-1.1,\,-0.9\}$	$\{+0.9,\,+1.1,\,+1.3,\,+1.5\}$

Appendix

Table 9: CarRacing evaluation splits (COM_X and COM_Y, m). Both axes share the same ranges.

Panel of Figure 6 — Figure 6: Visualisation of the payload-extended environments. Both payloads are novel additions to the Carl benchmark introduced in this work. Shifting the payload position changes the center-of-mass of the agent, altering its dynamics in a physically grounded and continuously parameterisable way.

Figure 7: Fraction of scheduler families (last checkpoint IQM) that outperform the static baseline per environment and mode. Numbers above bars indicate the exact count over the total number of families evaluated.

Figure 8: Heatmap of scheduler family performance relative to the static baseline. Colour encodes the static-anchored score: yellow ( $=0$ ) matches the static baseline, green ( $=1$ ) is the best scheduler in that column, red indicates worse than static. Raw IQM values (last checkpoint) are shown in each cell. Families are sorted by mean score across all columns (right margin).

Figure 9: Blind

-

observed IQM gap per scheduler family and environment, normalised by the environment’s IQM range (max

-

min across all schedulers). Positive values indicate blind schedulers outperform observed ones; negative values indicate the reverse. Top panel: last checkpoint; bottom panel: best anytime.

Condition	CartPole	Walker	CarRacing (lateral)
Static observed	$449.7\;[440.3,\;462.8]$	$134.2\;[\phantom{0}89.5,\;167.9]$	$358.5\;[183.6,\;450.8]$
Static blind	$463.4\;[453.4,\;474.0]$	$146.9\;[118.3,\;181.4]$	$617.7\;[272.5,\;803.1]$
Best sched. observed	$\mathbf{479.6}\;[469.3,\;491.0]$	$\mathbf{193.0}\;[156.7,\;233.4]$	$634.1\;[520.7,\;659.4]$
Best sched. blind	$479.3\;[465.5,\;495.9]$	$175.5\;[151.9,\;200.5]$	$\mathbf{778.9}\;[718.5,\;813.5]$

Appendix

Table 10: Combined IQM score of the best anytime checkpoint , computed from the per-seed avg_combined metric (average over all ID and OOD contexts). Values: IQM [Q1, Q3]. Bold marks the best scheduler per environment.

Figure 10: CartPole / pole length / pool-7 — combined IQM training progression per scheduler family (10 seeds). Each line shows the best representative configuration for that family, selected by final combined IQM. Shaded bands indicate Q1–Q3 across seeds. The dashed line marks the static pool-7 baseline.

Figure 11: 4D state-space coverage vs. final eval score across all sequential seeds (pool sizes 3 and 7 for dynamic schedulers, pool sizes 1–61 for identity), coloured by scheduler. Dynamic and static seeds are interleaved throughout the coverage axis. The dashed line shows the least-squares fit (

r=0.21

Figure 12: Last-checkpoint IQM score (A) and final 4D coverage (B) vs. pool size for CartPole pole-length (10 seeds). Error bars show SEM.

Figure 13: IQM training curves (shading: Q1–Q3) for identity pools 1–61; sinusoidal pool-7 shown dashed for reference.

	Identity			Sinusoidal			Cosine annealing
Pool	S1	S2	S3	S1	S2	S3	S1	S2	S3
1	6.45	0.57	0.27	6.48	0.52	0.29	6.68	0.71	0.27
3	6.93	0.75	0.32	6.68	0.61	0.28	6.52	0.53	0.26
5	6.65	0.77	0.29	6.70	0.69	0.29	6.80	0.62	0.32
7	6.65	0.64	0.29	6.39	0.77	0.29	6.61	0.61	0.30
15	6.67	0.64	0.29	6.69	0.58	0.27	6.61	0.62	0.31

Appendix

Table 11: Fraction of the

12^{4}

CartPole hypergrid visited per 50 k-step training window (mean over 10 seeds, %). Coverage in stages 2–3 is below

1\%

for all conditions; virtually no cells are newly discovered after stage 1.

Scheduler	Pool	$\Delta$ IQM score	$\Delta$ Coverage
Identity	3	$-7.1$	$-0.47\%$
Identity	5	$+1.8$	$+0.22\%$
Identity	7	$+11.8$	$+0.12\%$
Identity	15	$+28.3$	$-0.02\%$
Sinusoidal	5	$+0.4$	$-0.25\%$
Sinusoidal	15	$-20.7$	$-0.25\%$

Appendix

Table 12: Score and coverage difference (parallel

-

sequential) per condition. Positive = parallel preferred; negative = sequential preferred.

Figure 14: Sequential vs. parallel IQM score (A) and 4D coverage (B) for CartPole. Points on the dashed diagonal indicate no difference. Colour encodes scheduler family; marker shape encodes pool size.

Figure 15: BipedalWalker full-run coverage per scheduler across four state-space projections (4 seeds

\times

4 context ranges, aggregated). A : inter-leg phase (

30^{2}

p=0.12

). B : 4D joint angles (

12^{4}

p=0.34

). C : 3D posture (

15^{3}

p=0.68

). D : leg-1 phase (

30^{2}

p=0.31

	Coverage (%)
Scheduler	A: inter-leg	B: 4D joints	C: posture	D: leg phase	Return
Identity	75.1	12.5	48.7	25.7	142
Cosine annealing	71.7	11.6	47.0	25.8	150
Lévy walk	74.3	11.4	47.7	25.4	163
Sinusoidal	73.5	11.0	45.7	26.2	177
ANOVA $p$	0.12	0.34	0.68	0.31	—

Appendix

Table 13: Full-run coverage (%) per scheduler across four projections and mean rollout return, averaged over pool sizes 1, 3, 7, 15 (4 seeds each). Bold marks the highest value per column.

Figure 16: BipedalWalker 4D joint-space coverage per scheduler across three equal training stages (

\approx\!333

k steps each). Shared

y

-axis shows the progressive contraction of the visited state space. ANOVA

p>0.05

for all stages.

Panel of Figure 17 — Figure 17: Effect of context normalization on static and dynamic PPO training (CartPole, pole length context). The dynamic curves aggregate performance across a preliminary set of non-stationary schedules. Lines show mean evaluation score across seeds; shaded regions indicate standard deviation.

Figure 18: CartPole combined IQM (best anytime, pool-7). Per-scheduler values in Table 15 .

Figure 19: BipedalWalker combined IQM (last checkpoint, pool-3). Per-scheduler values in Table 16 .

Figure 20: BipedalWalker combined IQM (best anytime, pool-3). The gap relative to the last-checkpoint figure above is most pronounced for the static observed baseline, which collapses late in training.

Figure 21: CarRacing COM_X last-checkpoint IQM (pool-3).

Figure 22: CarRacing COM_X best-anytime IQM (pool-3).

Figure 23: CarRacing COM_Y last-checkpoint IQM (pool-3).

Figure 24: CarRacing COM_Y best-anytime IQM (pool-3).

Condition	CartPole	Walker (1.5 M)
Static observed	$449.7\;[440.3,\;462.8]$	$143.6\;[114.5,\;167.9]$
Static blind	$463.4\;[453.4,\;474.0]$	$136.3\;[101.8,\;160.4]$
Best sched. observed	$\textbf{479.6}\;[469.3,\;491.0]$	$\textbf{176.9}\;[135.6,\;197.0]$
Best sched. blind	$479.3\;[465.5,\;495.9]$	$140.3\;[108.7,\;164.9]$
Optuna #1 (T168 / T118)	$470.1\;[450.9,\;492.1]$	$171.4\;[148.6,\;196.2]$
Optuna #2 (T218 / T117)	$469.3\;[447.4,\;482.9]$	$165.2\;[149.2,\;177.3]$

Appendix

Table 14: Multi-stage Optuna retrain results at the best-anytime checkpoint . Walker values at the matched 1.5 M-step budget. Values: IQM [Q1, Q3]. Top-3 retrained trials shown per environment.

Figure 25: Learning curves (IQM

\pm

IQR) for the top-5 retrained CartPole trials vs. static baseline.

Figure 26: Frequency of each scheduler mode (constant, sinusoidal, cosine annealing) at each stage position across all 20 retrained CartPole trials. Stage 2 is active in nearly all configurations; Stage 1 is most often constant.

Figure 27: Hyperparameter distributions for active stages across all 20 retrained CartPole trials. Top row: sinusoidal amplitude fraction vs. period; bottom row: cosine annealing neighbourhood radius vs.

T_{0}

. Only trials where the respective mode is active are shown.

Figure 28: Rank shift between the 4-seed Optuna search ranking and the 10-seed retrain ranking for CartPole. Left: scatter of Optuna score vs. retrain IQM

\pm

IQR; the dashed line marks the static pool-7 baseline. Right: bump chart showing the rank change for each trial after retraining with more seeds.

Panel of Figure 29 — Figure 29: BipedalWalker Optuna retrain: scheduler patterns and final scores.

Figure 30: Learning curves (IQM

\pm

IQR) for the top-5 retrained Walker trials vs. static baseline.

Figure 31: Rank shift between the 3-seed Optuna ranking and the 8-seed retrain ranking for BipedalWalker.

Figure 32: Scheduler mode frequency per stage across all retrained Walker trials.

Figure 33: Scheduler parameter distributions per stage for all retrained Walker trials (active schedulers only).

Scheduler	Mode	Last IQM [Q1, Q3]	Best IQM [Q1, Q3]
Static Baseline	Observed	$429.0\;[401.5,\;451.4]$	$449.7\;[440.3,\;462.8]$
Static Baseline	Blind	$410.0\;[383.1,\;436.5]$	$463.4\;[453.4,\;474.0]$
Cont. Incrementer	Observed	$449.8\;[411.5,\;473.2]$	$477.6\;[449.6,\;494.7]$
Cont. Incrementer	Blind	$409.9\;[381.7,\;437.5]$	$473.4\;[455.0,\;487.8]$
Cosine Annealing	Observed	$458.2\;[444.6,\;468.8]$	$477.9\;[469.4,\;484.6]$
Cosine Annealing	Blind	$411.0\;[384.5,\;452.6]$	$455.0\;[444.6,\;468.6]$

Appendix

Table 15: CartPole / length / pool-7. Per-scheduler-type IQM [Q1, Q3] of avg_combined over 10 seeds.

Scheduler	Mode	Last IQM [Q1, Q3]	Best IQM [Q1, Q3]
Static Baseline	Observed	$31.0\;[-34.9,\;63.8]$	$134.2\;[89.5,\;167.9]$
Static Baseline	Blind	$91.9\;[60.6,\;113.0]$	$146.9\;[118.3,\;181.4]$
Cont. Incrementer	Observed	$113.6\;[97.6,\;130.2]$	$154.5\;[140.1,\;179.6]$
Cont. Incrementer	Blind	$121.5\;[62.1,\;178.3]$	$157.5\;[121.1,\;196.8]$
Cosine Annealing	Observed	$106.5\;[75.9,\;142.5]$	$153.7\;[134.7,\;180.4]$
Cosine Annealing	Blind	$50.0\;[-36.0,\;121.8]$	$100.6\;[10.3,\;172.1]$

Appendix

Table 16: BipedalWalker / COM_X / pool-3. Per-scheduler-type IQM [Q1, Q3] of avg_combined over 8 seeds.

Scheduler	Mode	Last IQM [Q1, Q3]	Best IQM [Q1, Q3]
Static Baseline	Observed	$-81.0\;[-93.1,\;-69.4]$	$446.7\;[160.9,\;606.1]$
Static Baseline	Blind	$521.9\;[377.7,\;656.7]$	$\mathbf{833.5}\;[775.6,\;873.4]$
Cont. Incrementer	Observed	$-28.9\;[-35.4,\;-17.4]$	$327.5\;[221.3,\;436.2]$
Cont. Incrementer	Blind	$262.8\;[94.8,\;466.2]$	$703.5\;[530.4,\;833.7]$
Cosine Annealing	Observed	$-59.6\;[-86.5,\;-12.9]$	$594.7\;[350.5,\;745.6]$
Cosine Annealing	Blind	$\mathbf{602.5}\;[409.6,\;664.3]$	$762.0\;[617.0,\;771.4]$

Appendix

Table 17: CarRacing / longitudinal / pool-3. Per-scheduler-type IQM [Q1, Q3] of avg_combined over 5 seeds.

Scheduler	Mode	Last IQM [Q1, Q3]	Best IQM [Q1, Q3]
Static Baseline	Observed	$-85.9\;[-93.1,\;-76.2]$	$358.5\;[183.6,\;450.8]$
Static Baseline	Blind	$103.6\;[-59.0,\;378.9]$	$617.7\;[272.5,\;803.1]$
Cont. Incrementer	Observed	$-70.5\;[-83.7,\;-45.1]$	$380.3\;[110.8,\;643.4]$
Cont. Incrementer	Blind	$153.8\;[-16.6,\;391.1]$	$564.0\;[443.6,\;656.1]$
Cosine Annealing	Observed	$-40.1\;[-82.5,\;-10.1]$	$395.8\;[273.8,\;560.3]$
Cosine Annealing	Blind	$533.2\;[458.5,\;618.7]$	$762.3\;[720.9,\;796.3]$

Appendix

Table 18: CarRacing / lateral / pool-3. Per-scheduler-type IQM [Q1, Q3] of avg_combined over 5 seeds.

Figure 34: CartPole / pole length / pool-7 — combined IQM (all contexts, 10 seeds). Each family’s best representative is selected by final combined IQM.

Figure 35: CartPole / pole length / pool-7 — ID IQM (in-distribution contexts only, 10 seeds).

Figure 36: CartPole / pole length / pool-7 — OOD-low IQM (short-pole contexts below training range, 10 seeds).

Figure 37: CartPole / pole length / pool-7 — OOD-high IQM (long-pole contexts above training range, 10 seeds).

Figure 38: BipedalWalker / COM_X / pool-3 — combined IQM (all contexts, 8 seeds).

Figure 39: BipedalWalker / COM_X / pool-3 — ID IQM (in-distribution contexts only, 8 seeds).

Figure 40: BipedalWalker / COM_X / pool-3 — OOD-low IQM (centre-of-mass shifted below training range, 8 seeds).

Figure 41: BipedalWalker / COM_X / pool-3 — OOD-high IQM (centre-of-mass shifted above training range, 8 seeds).

Figure 42: CarRacing / COM_X (longitudinal CoM) / pool-3 — combined IQM (all contexts, 5 seeds).

Figure 43: CarRacing / COM_X / pool-3 — ID IQM (in-distribution contexts only, 5 seeds).

Figure 44: CarRacing / COM_X / pool-3 — OOD-low IQM (longitudinal CoM shifted below training range, 5 seeds).

Figure 45: CarRacing / COM_X / pool-3 — OOD-high IQM (longitudinal CoM shifted above training range, 5 seeds).

Figure 46: CarRacing / COM_Y (lateral CoM) / pool-3 — combined IQM (all contexts, 5 seeds).

Figure 47: CarRacing / COM_Y / pool-3 — ID IQM (in-distribution contexts only, 5 seeds).

Figure 48: CarRacing / COM_Y / pool-3 — OOD-low IQM (lateral CoM shifted below training range, 5 seeds).

Figure 49: CarRacing / COM_Y / pool-3 — OOD-high IQM (lateral CoM shifted above training range, 5 seeds).

Contextual RL

See all 19 papers

When does learning pay off? A study on DRL-based dynamic algorithm configuration for carbon-aware scheduling
Apr 2, 2026Andrea Mencaroni, Robbert Reijnen, Yingqian Zhang +1RL Control Learning to Optimize
Department of Industrial Systems Engineering and Product Design, Ghent University, Ghent, Belgium · Industrial Systems Engineering (ISyE), Flanders Make, Kortrijk, Belgium · Eindhoven Artificial Intelligence Systems Institute, Eindhoven University of Technology, The Netherlands +1
Curriculum Generation under Structured Parametric Environments for Robust Navigation Policies
Aug 9, 2026Prishita RayCurriculum RL Robot Policy Generalization
Cornell University
Self Paced Gaussian Contextual Reinforcement Learning
Mar 24, 2026Mohsen Sahraei Ardakani, Rui SongContextual RL Reinforcement Learning
Department of Electrical and Computer Engineering North Carolina State University Raleigh, NC · Department of Statistics North Carolina State University Raleigh, NC

Apr 2, 2026J/K move · Enter open · S save

When does learning pay off? A study on DRL-based dynamic algorithm configuration for carbon-aware scheduling

Andrea Mencaroni, Robbert Reijnen, Yingqian Zhang, Dieter Claeys

Department of Industrial Systems Engineering and Product Design, Ghent University, Ghent, Belgium · Industrial Systems Engineering (ISyE), Flanders Make, Kortrijk, Belgium · Eindhoven Artificial Intelligence Systems Institute, Eindhoven University of Technology, The Netherlands+1

RL Control Learning to Optimize Combinatorial Optimization

Open paper AskPDF arXiv

Deep reinforcement learning (DRL) has recently emerged as a promising tool for Dynamic Algorithm Configuration (DAC), enabling evolutionary algorithms to adapt their parameters online rather than relying on static tuned configurations. While DRL can learn effective control policies, training is computationally expensive. This cost may be justified if learned policies generalize, allowing the training effort to transfer across instance types and problem scales. Yet, for real-world optimization problems, it remains unclear whether this promise holds in practice and under which conditions the investment in learning pays off. In this work, we investigate this question in the context of the carbon-aware permutation flow-shop scheduling problem. We develop a DRL-based DAC framework and train it exclusively on small, simple instances. We then deploy the learned policy on both similar and more complex unseen instances and compare its performance against a static tuned baseline, which provides a fair point of comparison. Our findings show that the proposed method provides a strong dynamic algorithm control policy that can be effectively transferred to different unseen problem instances. Notably, on simple and cheap to compute instances, similar to those observed during training and tuning, DRL performs comparably with the statically tuned baseline. However, as instance characteristics diverge and computational complexities increase, the DRL-learned policy continuously outperforms static tuning. These results confirm that DRL can acquire robust and generalizable control policies which are effective beyond the training instance distributions. This ability to generalize across instance types makes the initial computational investment worthwhile, particularly in settings where static tuning struggles to adapt to changing problem scenarios.

Abstract

Contextual RL

Explore similar work

When does learning pay off? A study on DRL-based dynamic algorithm configuration for carbon-aware scheduling

Curriculum Generation under Structured Parametric Environments for Robust Navigation Policies

Self Paced Gaussian Contextual Reinforcement Learning