cs.LGMar 31, 2026

Learning to Play Blackjack: A Curriculum Learning Perspective

Authors: Amirreza Alasti, Efe Erdal, Yücel Celik, Theresa Eimer

Organizations: Leibniz University Hannover, Hannover, Germany · Leibniz AI Academy, Hannover, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Reinforcement Learning (RL) agents often struggle with efficiency and performance in complex environments. We propose a novel framework that uses a Large Language Model (LLM) to dynamically generate a curriculum over available actions, enabling the agent to incorporate each action individually. We apply this framework to the game of Blackjack, where the LLM creates a multi-stage training path that progressively introduces complex actions to a Tabular Q-Learning and a Deep Q-Network (DQN) agent. Our evaluation in a realistic 8-deck simulation over 10 independent runs demonstrates significant performance gains over standard training methods. The curriculum-based approach increases the DQN agent's average win rate from 43.97% to 47.41%, reduces the average bust rate from 32.9% to 28.0%, and accelerates the overall workflow by over 74%, with the agent's full training completing faster than the baseline's evaluation phase alone. These results validate that LLM-guided curricula can build more effective, robust, and efficient RL agents.

Figures & tables

Figure 1. The adaptive, LLM-guided curriculum learning framework. (1) Generation & Adaptation: An LLM generates an initial curriculum stage. After a training phase, the agent’s performance summary is fed back to the LLM, which adapts the curriculum by deciding whether to advance the agent to the next stage. (2) Training Loop: The agent’s training is constrained by the current curriculum stage, which masks unavailable actions in the environment to focus exploration. A flowchart showing an adaptive loop. In phase 1, an LLM generates a curriculum. In phase 2, an RL agent trains in an environment constrained by that curriculum. A feedback arrow shows that the agent's performance summary from phase 2 is sent back to the LLM in phase 1, which then updates the curriculum.
Figure 2. A statistical summary of agent performance over 10 independent runs in the 8-deck curriculum environment. The results highlight the DQN agent’s significantly higher peak performance distribution (left) and confirm that this peak is most frequently achieved at Stage 4 (right). The high stage completion rate for both agents (middle) indicates the curriculum was well-paced and successfully navigated in most runs. Statistical summary charts of agent performance across 10 runs.
Figure 3. Win rate progression for baseline DQN and Tabular agents trained without a curriculum. The DQN agent’s performance is highly volatile, illustrating the instability and exploration challenges that arise when the full action space is introduced at once. Win rate progression chart for baseline agents showing high volatility.
Figure 4. Win rate progression for DQN and Tabular agents trained with the LLM-guided curriculum. The curriculum provides a stable learning trajectory for the DQN agent, preventing the performance volatility seen in baseline training and allowing it to converge to a high and consistent win rate. Win rate progression chart for curriculum-trained agents showing a stable learning trajectory.
Figure 5. Deeper analysis of agent performance, showing a positive correlation between the number of curriculum stages completed and the best achieved win rate for the DQN agent (left). However, the agent’s final performance is often lower than its peak (middle), reinforcing the finding, quantified in the summary (right), that the optimal policy is achieved at an intermediate stage. Charts showing correlation between completed stages and win rates.
StageAvailable ActionsDQN (%)Tabular (%)
1: Hit/Stand0,142.2439.74
2: +Double0,1,242.3738.72
3: +Split (no Double)0,1,345.5438.78
4: Full Basic (0–3)0,1,2,349.4238.15
5: +Insurance0,1,2,3,548.6640.00
6: +Surrender0,1,2,3,446.5737.78
Table 1. Win rate by curriculum stage (8-deck). The values represent the best observed win rate for a single run at each stage.
Figure 6. A strategy heatmap for an agent trained without a curriculum. Rows are the player’s sum, columns are the dealer’s up card. Cell colors represent the chosen action. The strategy appears less structured than the curriculum-trained agent. Strategy heatmap showing unstructured decision boundaries for the baseline agent.
Figure 7. A strategy heatmap for an agent trained with a curriculum. Rows are the player’s sum, columns are the dealer’s up card. Cell colors represent the chosen action. The strategy appears more structured than the no curriculum-trained agent. Strategy heatmap showing structured decision boundaries for the curriculum agent.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8. Performance progression of the DQN agent through a 5-stage curriculum in an 8-deck game. DQN agent performance progression through 5 curriculum stages.
Figure 9. Comparative analysis in a 1-deck environment. Comparative analysis chart for 1-deck environment.
Figure 10. Comparative analysis in a 4-deck environment. Comparative analysis chart for 4-deck environment.
Figure 11. Comparative analysis in an 8-deck environment. Comparative analysis chart for 8-deck environment.
Figure 12. Comparative analysis in an infinite-deck environment. Comparative analysis chart for infinite-deck environment.
Figure 13. Heatmap visualization of the DQN agent’s learned policy at Stage 3 ("Introducing Splits"). Heatmap of learned policy at Stage 3.
More in this topic

Curriculum RL

Curriculum reinforcement learning organizes tasks, examples, or training conditions to help agents learn complex behaviors efficiently.

See all 121 papers

Explore similar work

CardsList

  1. Anticipating the Consequences of Curriculum Decisions with Large Language Models

    Oct 3, 2026Octavio Pappalardo, Nathan Herr, Tim RocktäschelCurriculum LearningCurriculum RL

    University College London

  2. What Training Data Teaches RL Memory Agents: An Empirical Study of Curriculum Effects in Memory-Augmented QA

    May 21, 2026Xinjie He, Zhiyuan Lin, Su Liu +4Memory-Augmented Neural NetworksCurriculum RL

    Columbia University · Independent Researcher · Johns Hopkins University +1

  3. Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training

    Feb 24, 2026Zhengyao Gu, Jonathan Light, Raul Astudillo +7Curriculum RLCurriculum Learning

    University of Illinois Chicago · Caltech · RPI +2