cs.LGNov 15, 2026

Sample-Efficient Ensemble Reinforcement Learning at Scale

Authors: Yoav Feigenbaum, Dmitrii Krylov, Roy Fox

Organizations: Department of Computer Science, University of California, Irvine

Abstract

Ensemble methods such as MeanQ have been shown to improve the sample efficiency of relatively small model-free deep reinforcement learning (RL) architectures, including DQN and similarly sized variants such as Rainbow. However, to our knowledge, the effects of ensembling on large-scale architectures such as Bigger Better Faster (BBF) have not been studied. In this work, we present MeanQ-BBF, a hybrid model-free algorithm that combines the principles of MeanQ and BBF. Through statistically rigorous evaluation on the Atari 100K benchmark, we show that in contrast to naive scaling of the network size of BBF, which degrades performance, MeanQ-BBF is able to leverage the increase in model capacity to improve performance over BBF. Moreover, MeanQ-BBF does this in a computationally efficient way by organizing the added parameters into largely independent learners. Finally, we discuss the performance-cost tradeoff in the context of practical sample-efficient RL.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Representation Learning Enables Scalable Multitask Deep Reinforcement Learning

    Jun 4, 2026Johan Obando-Ceron, Lu Li, Scott Fujimoto +3Reinforcement LearningRepresentation Learning for RL

  2. Accelerating Q-learning through Efficient Value-Sharing across Actions

    Jun 29, 2026Prabhat Nagarajan, Brett Daley, Martha White +1Deep Q-LearningValue Function Estimation