A/b Testing

A/B testing studies whether changing a product, policy, or interface causes measurable differences in outcomes such as clicks, revenue, or user retention. Researchers use randomized assignment and methods such as CUPED, regression adjustment, and stratification to improve precision, while sequential tests, bandits, and factorial designs help allocate traffic across many options. Work also addresses longer-term effects, marketplace interference, and whether offline or LLM-based predictions can stand in for outcomes measured with real users. The central tension is making experiments faster and cheaper without compromising valid inference, especially when effects vary across users or the experimental setting violates standard assumptions.