Bitboard version of Tetris AI

Xingguo Chen; Pingshou Xiong; Zhenyu Luo; Mengfei Hu; Xinwen Li; Yongzhou Lü; Guang Yang; Chao Li

cs.AIMar 24, 2026

Bitboard version of Tetris AI

Authors: Xingguo Chen, Pingshou Xiong, Zhenyu Luo, Mengfei Hu, Xinwen Li, Yongzhou Lü, Guang Yang, Chao Li,

Organizations: School of Computer Science, Nanjing University of Posts and Telecommunications, Nanjing, Jiangsu, 210023, China

Abstract

The efficiency of game engines and policy optimization algorithms is crucial for training reinforcement learning (RL) agents in complex sequential decision-making tasks, such as Tetris. Existing Tetris implementations suffer from low simulation speeds, suboptimal state evaluation, and inefficient training paradigms, limiting their utility for large-scale RL research. To address these limitations, this paper proposes a high-performance Tetris AI framework based on bitboard optimization and improved RL algorithms. First, we redesign the Tetris game board and tetrominoes using bitboard representations, leveraging bitwise operations to accelerate core processes (e.g., collision detection, line clearing, and Dellacherie-Thiery Features extraction) and achieve a 53-fold speedup compared to OpenAI Gym-Tetris. Second, we introduce an afterstate-evaluating actor network that simplifies state value estimation by leveraging Tetris afterstate property, outperforming traditional action-value networks with fewer parameters. Third, we propose a buffer-optimized Proximal Policy Optimization (PPO) algorithm that balances sampling and update efficiency, achieving an average score of 3,829 on 10x10 grids within 3 minutes. Additionally, we develop a Python-Java interface compliant with the OpenAI Gym standard, enabling seamless integration with modern RL frameworks. Experimental results demonstrate that our framework enhances Tetris's utility as an RL benchmark by bridging low-level bitboard optimizations with high-level AI strategies, providing a sample-efficient and computationally lightweight solution for scalable sequential decision-making research.

Figures & tables

Method	Avg. Removed Lines	Training Samples	Feature	Generator
BCTS Thiery and Scherrer (2009a)	3,000	$6.5\times 10^{7}$	DT features	Random
CBMPI Gabillon et al. (2013) ; Scherrer et al. (2015)	4,300	$8\times 10^{6}$	DT features + RBF heights	Random
dSiLU-TD( $\lambda$ ) Elfwing et al. (2018)	4,900	200,000	Bertsekas features + neural networks	Random
STEW Lichtenberg and Şimşek (2019)	4,800	238,000	DT features	7-Bag random
Buffered PPO (Ours)	3,850	61,440	DT features	Random

Table 1: Records in 10

\times

10 Tetris AI.

Panel of Figure 1 — Figure 1: Tetris Game Board with All Pieces

Panel of Figure 2 — Figure 2: llustration of the Afterstate in Tetris Game

Feature	DT-10 weight	DT-20 weight
Landing height	-2.18	-2.68
Eroded piece cells	2.42	1.38
Row transitions	-2.17	-2.41
Column transitions	-3.31	-6.32
Holes	0.95	2.03
Board well	-2.22	-2.71

Table 2: DT-10 and DT-20 weights Gabillon et al. (2013) ; Scherrer et al. (2015) .

Panel of Figure 3 — Figure 3: The O-Piece and Its Rotation(s)

Panel of Figure 4 — Figure 4: The I-Piece and Its Rotations

Panel of Figure 5 — Figure 5: The S-Piece and Its Rotations

Panel of Figure 6 — Figure 6: The Z-Piece and Its Rotations

Panel of Figure 7 — Figure 7: The L-Piece and Its Rotations

Panel of Figure 8 — Figure 8: The J-Piece and Its Rotations

Panel of Figure 9 — Figure 9: The T-Piece and Its Rotations

Panel of Figure 10 — Figure 10: Collision Detection

Figure 11: The feedforward process of the Actor evaluating afterstates

Board Size	Weights	Gabillon et al. (2013) Gabillon et al. (2013)	Our bitboard impl
10 $\times$ 10(10,000 games)	DT-10	5,000(-)	5,152.02(5,137.51)
10 $\times$ 10(10,000 games)	DT-20	4,300(-)	4,188.61(4,153.42)
10 $\times$ 20(10 games)	DT-10	29,000,000(-)	54,377,543(54,353,156)
10 $\times$ 20(10 games)	DT-20	51,000,000(-)	75,093,204(65,000,224)

Table 3: Scores with different weights on different board sizes (Mean(SD))

Implementation	Time for 10,000 samples
OpenAI Gym Kauten (2019)	12.92 seconds
Bitboard Tetris (ours)	0.24 seconds
Bitboard Tetris via Python (ours)	0.59 seconds

Table 4: Time comparison of Tetris implementations for 10,000 samples

Figure 12: Comparison of Action-Value Evaluating Actor and Afterstate Evaluating Actor

Param	Value	Param	Value	Param	Value
$\gamma$	0.99	$\tau_{0}$	0.5	$\alpha_{\theta}$	3e-4
$\lambda$	0.99	$\tau_{k}$	0.00025	$\alpha_{\omega}$	3e-4
$epoch$	10	$\epsilon$	0.1	$episode$	12,500

Table 5: Trajectory-Based PPO hyperparameters

Figure 13: Trajectory-Based PPO Algorithm

Parameter	Value	Parameter	Value	Parameter	Value
$\gamma$	0.99	$\epsilon$	0.2	$batchSize$	2,048
$\lambda$	0.99	$\alpha_{\theta}$	3e-4	$miniBatchSize$	256
$epoch$	10	$\alpha_{\omega}$	3e-4	$totalSteps$	61,440

Table 6: Buffer-based PPO hyperparameters

Algorithm	Sampling	Policy Updates	Total	Update Ratio
Trajectory PPO	10,536s	436s	10,972s	3.97%
Buffer PPO	112s	54s	166s	32.53%

Table 7: Time cost comparison of PPO variants

Algorithm	Total Steps	Avg. Converged Score
Trajectory PPO	69,046,726	3,840.30
Buffer PPO (optimized)	61,440	3,829.04

Table 8: Step cost comparison of PPO variants

Feature	Weight
Landing height	-0.51
Eroded piece cells	0.16
Row transitions	-0.40
Column transitions	-0.75
Holes	-0.18
Board well	-0.39

Table 9: Best weights obtained from PPO training

Algorithm	Score
Trajectory PPO	32,110,545.18(30,425,374.19)
Buffer PPO	13,936,917.72(13,198,381.24)

Table 10: Scores of two PPO algorithms on the 10

\times

20 game board (Mean(SD))

Block Generation Rule	CBMPI	Trajectory PPO	Buffer PPO
Random (Baseline)	4,300	4,965.44 (4884.14)	3,590.90 (3545.05)
7-Bag	251,501.20	173,489.37 (177,212.68)	198,206.23 (198,903.07)
Adversarial Sequence	75	34(0)	53(0)

Table 11: Scores of agents under different block generation rules (Mean(SD))

Game-Playing Agents

Game-playing agents use search, learning, and reasoning to choose actions in games with strategic, uncertain, or changing environments.

See all 200 papers

Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX
Oct 2, 2025Waris Radji, Thomas Michel, Hector PiteauRL Benchmarks Reinforcement Learning
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189-CRIStAL, France · Independent Researcher
Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served
Mar 2, 2026Haochuan WangLLM Inference Efficiency LLM Agent Evaluation
Massachusetts Institute of Technology
JAXtari: High-Throughput and Easy-to-Modify Arcade Learning Environment
Oct 16, 2026Quentin Delfosse, Raban Emunds, Paul Seitz +5GPU-Accelerated RL RL Benchmarks
Department of Computer Science, Technical University Darmstadt, Germany · Google Intrinsic AI Research · Hessian Center for Artificial Intelligence (hessian.AI) +2

Oct 2, 2025J/K move · Enter open · S save

Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX

Waris Radji, Thomas Michel, Hector Piteau

Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189-CRIStAL, France · Independent Researcher

RL Benchmarks Reinforcement Learning GPU-Accelerated RL RL for Game AI

Open paper AskPDF arXiv

Reinforcement learning (RL) research requires diverse, challenging environments that are both tractable and scalable. While modern video games may offer rich dynamics, they are computationally expensive and poorly suited for large-scale experimentation due to their CPU-bound execution. We introduce Octax, a high-performance suite of classic arcade game environments implemented in JAX, based on CHIP-8 emulation, a predecessor to Atari, which is widely adopted as a benchmark in RL research. Octax provides the JAX community with a long-awaited end-to-end GPU alternative to Atari games, offering image-based environments, spanning puzzle, action, and strategy genres, all executable at massive scale on modern GPUs. Our JAX-based implementation achieves orders-of-magnitude speedups over traditional CPU emulators. We demonstrate Octax's capabilities by training RL agents across multiple games, showing significant improvements in training speed and scalability compared to existing solutions. The environment's modular design enables researchers to easily extend the suite with new games or generate novel environments using large language models, making it an ideal platform for large-scale RL experimentation. Our open-source framework is available at https://github.com/riiswa/octax/.

Abstract

Game-Playing Agents

Explore similar work

Octax: Accelerated CHIP-8 Arcade Environments for Reinforcement Learning in JAX

Same Pieces, Different Servers: A Tetris Benchmark for AI Agents as Served

JAXtari: High-Throughput and Easy-to-Modify Arcade Learning Environment