We study packet routing in queuing systems where queues act as selfish, myopic agents that use no-regret learning algorithms to select servers with unknown service rates. To model the small buffers used in data centers, we assume that each server has a single-packet buffer. Even such a tiny buffer already captures the features of buffering: routers can exploit multiple servers while accepted packets remain in service, but packets can occupy servers for multiple rounds, complicating future service and learning. We first analyze a single queue routing packets to
m buffered servers with service rates
μ1,…,μm. This setting is equivalent to an instance of the recharging bandits problem with geometric recharge. We prove that any no-regret learner achieves a long-run average throughput of at least the Hamacher sum of the server rates, and that this bound is tight. Consequently, a single no-regret queue is stable if and only if its arrival rate is below the Hamacher sum of the server rates. Moreover, comparing against an optimal policy, we show that any no-regret learner is at least
0.77-competitive, which is also tight. We then study multi-queue systems where
n queues with arrival rates
λ1,…,λn, each at most
λmax, compete for service from
m buffered servers. We show that the smallest overprovisioning factor that guarantees the system to be stable under high-probability no-regret learning is
1−λmax1. Equivalently, in terms of the total arrival and service rates, the condition
∑i=1nλi<(1−λmax)∑j=1mμj is sufficient, and sometimes necessary, for stability under no-regret learning. In particular, for
λmax=1/2, this improves the upper bound of Abel et al. (EC 2026) from
3 to
2, matching their lower bound and resolving their open question.