Overview

A/B testing studies whether changing a product, policy, or interface causes measurable differences in outcomes such as clicks, revenue, or user retention. Researchers use randomized assignment and methods such as CUPED, regression adjustment, and stratification to improve precision, while sequential tests, bandits, and factorial designs help allocate traffic across many options. Work also addresses longer-term effects, marketplace interference, and whether offline or LLM-based predictions can stand in for outcomes measured with real users. The central tension is making experiments faster and cheaper without compromising valid inference, especially when effects vary across users or the experimental setting violates standard assumptions.

Latest papers

CardsList

33 papers4 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

  1. SimAB: Simulating A/B Tests with Persona-Conditioned AI Agents for Rapid Design Evaluation

    Mar 1, 2026Tim Rieder, Marian Schneider, Mario Truss +6A/b TestingUser Simulation

  2. Calibrate Globally, Measure Everywhere: Scaling LLM-Based Prevalence Measurement Across A/B Experiments

    Feb 18, 2026Zehao Xu, Tony Paek, Kevin O'Sullivan +1A/b Testing