Next-Action Prediction

Next-action prediction asks what a person, process, or embodied or language-model agent will do next, supporting anticipation, assistance, monitoring, and planning. Methods range from temporal video transformers using gaze, affordance, trajectory, and object-interaction cues to event-log n-grams and neural sequence models, symbolic state models, and agents with retrieval or learned memory. Evaluation spans egocentric action benchmarks such as Ego4D and EPIC-Kitchens, process logs, interaction tasks, and robot settings, with measures of action accuracy, prediction horizon, robustness, and timing. Key challenges include using causal temporal history rather than static cues, adapting across people, tasks, and viewpoints, and determining when complex models outperform simple frequency baselines; long-horizon and personalized prediction remain difficult.