Autoregressive Generation

Autoregressive generation models how to build sequences one token, event, or chunk at a time, using prior outputs to create text, audio, images, motion, graphs, molecules, and other structured data. Common approaches include next-token transformers, hierarchical or coarse-to-fine prediction, compact learned tokenizations, and hybrids that combine autoregression with diffusion, flow matching, or reward-based optimization. Researchers use conditioning, structured representations, and training on imperfect rollouts to improve controllability, validity, and long-horizon consistency across benchmarks ranging from language and music to 3D geometry and scientific simulation. A central tension is that sequential dependencies support coherent, flexible outputs but can make generation slow and errors accumulate, motivating parallel decoding and better measures of quality beyond next-step prediction.