Multimodal Trajectory Prediction

Multimodal trajectory prediction aims to forecast the multiple possible future paths of moving agents (e.g., vehicles, pedestrians) by integrating diverse data sources like sensor readings, maps, and even language descriptions. Current research emphasizes improving prediction accuracy and diversity through advanced deep learning architectures, including transformers, convolutional neural networks, and recurrent networks, often incorporating attention mechanisms and probabilistic modeling to handle uncertainty. This field is crucial for advancing autonomous systems, particularly in robotics and self-driving cars, by enabling safer and more efficient navigation in complex and unpredictable environments. The development of robust and reliable multimodal trajectory prediction is essential for the safe deployment of these technologies.

Papers

October 4, 2024

MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents
Junpeng Yue, Xinru Xu, Börje F. Karlsson, Zongqing Lu
Embodied Agent MLLM Training Multimodal Retrieval MLLM Attention Hybrid Retriever Multimodal Trajectory Prediction MLLM Security

September 27, 2024

Multimodal Trajectory Prediction for Autonomous Driving on Unstructured Roads using Deep Convolutional Network
Lei Li, Zhifa Chen, Jian Wang, Bin Zhou, Guizhen Yu, Xiaoxuan Chen
Autonomous Driving Deep Convolutional Neural Network Human Driven Vehicle Multimodal Trajectory Prediction Open Pit

July 22, 2024

MuTT: A Multimodal Trajectory Transformer for Robot Skills
Claudius Kienle, Benjamin Alt, Onur Celik, Philipp Becker, Darko Katic, Rainer Jäkel, Gerhard Neumann
Skill Learning Robot Skill Robot Programming Multimodal Trajectory Prediction

July 8, 2024

MapsTP: HD Map Images Based Multimodal Trajectory Prediction for Automated Vehicles
Sushil Sharma, Arindam Das, Ganesh Sistu, Mark Halton, Ciarán Eising
Autonomous Vehicle Trajectory Prediction Automated Driving Ego Vehicle Multimodal Trajectory Prediction

May 12, 2024

Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising
Yao Liu, Quan Z. Sheng, Lina Yao
Trajectory Prediction Diffusion Probabilistic Model Pedestrian Trajectory Prediction Human Trajectory Multimodal Trajectory Prediction

May 10, 2024

Hierarchical Learned Risk-Aware Planning Framework for Human Driving Modeling
Nathan Ludlow, Yiwei Lyu, John Dolan
Vehicle Trajectory Human Driving Behavior Multimodal Trajectory Prediction

March 27, 2024

Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction
Inhwan Bae, Junoh Lee, Hae-Gon Jeon
Language Model Multimodal Trajectory Prediction

February 27, 2023

LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints
Mengmeng Liu, Hao Cheng, Lin Chen, Hellward Broszio, Jiangtao Li, Runjiang Zhao, Monika Sester, Michael Ying Yang
Autonomous Driving Trajectory Prediction Lane Detection Trajectory Prediction Model Multimodal Trajectory Prediction

February 21, 2023

Multimodal Trajectory Prediction: A Survey
Renhao Huang, Hao Xue, Maurice Pagnucco, Flora Salim, Yang Song
Timely Survey Trajectory Prediction Synthetic Trajectory Future Prediction Multimodal Trajectory Prediction

December 14, 2022

Learning and Predicting Multimodal Vehicle Action Distributions in a Unified Probabilistic Model Without Labels
Charles Richter, Patrick R. Barragán, Sertac Karaman
LeArning Abstract Variational Inference Action Space Label Information Action Representation Multimodal Trajectory Prediction

December 9, 2022

Reliable Multimodal Trajectory Prediction via Error Aligned Uncertainty Optimization
Neslihan Kose, Ranganath Krishnan, Akash Dhamasia, Omesh Tickoo, Michael Paulitsch
Uncertainty Estimation Structural Uncertainty Multimodal Trajectory Prediction Vehicle Motion Prediction

August 31, 2022

Class-Aware Attention for Multimodal Trajectory Prediction
Bimsara Pathiraja, Shehan Munasinghe, Malshan Ranawella, Maleesha De Silva, Ranga Rodrigo, Peshala Jayasekara
Autonomous Driving Multimodal Phenomenon Class Enhanced Attentive Response Multimodal Trajectory Prediction

June 17, 2022

Diverse Multiple Trajectory Prediction Using a Two-stage Prediction Network Trained with Lane Loss
Sanmin Kim, Hyeongseok Jeon, Junwon Choi, Dongsuk Kum
Trajectory Prediction Multi Stage Real World Autonomous Multimodal Trajectory Prediction Ground Truth Trajectory

November 26, 2021

Jointly Learning Agent and Lane Information for Multimodal Trajectory Prediction
Jie Wang, Caili Guo, Minan Guo, Jiujiu Chen
Learning Agent Multimodal Trajectory Prediction Map Prediction Lane Attention Lane Boundary

Multimodal Trajectory Prediction

Papers

MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents

Multimodal Trajectory Prediction for Autonomous Driving on Unstructured Roads using Deep Convolutional Network

MuTT: A Multimodal Trajectory Transformer for Robot Skills

MapsTP: HD Map Images Based Multimodal Trajectory Prediction for Automated Vehicles

Modeling Pedestrian Intrinsic Uncertainty for Multimodal Stochastic Trajectory Prediction via Energy Plan Denoising

Hierarchical Learned Risk-Aware Planning Framework for Human Driving Modeling

Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction

LAformer: Trajectory Prediction for Autonomous Driving with Lane-Aware Scene Constraints

Multimodal Trajectory Prediction: A Survey

Learning and Predicting Multimodal Vehicle Action Distributions in a Unified Probabilistic Model Without Labels

Reliable Multimodal Trajectory Prediction via Error Aligned Uncertainty Optimization

Class-Aware Attention for Multimodal Trajectory Prediction

Diverse Multiple Trajectory Prediction Using a Two-stage Prediction Network Trained with Lane Loss

Jointly Learning Agent and Lane Information for Multimodal Trajectory Prediction