Theoretical Understanding
Theoretical understanding in artificial intelligence currently focuses on rigorously analyzing the capabilities and limitations of various models, aiming to bridge the gap between empirical observations and formal guarantees. Research emphasizes developing theoretical frameworks for explaining model behavior, particularly in areas like large language models (LLMs), diffusion models, and graph neural networks, often employing techniques from information theory, optimization, and statistical learning theory to analyze model performance and generalization. These theoretical advancements are crucial for improving model design, enhancing reliability, and addressing concerns about robustness, fairness, and explainability, ultimately impacting the trustworthiness and responsible deployment of AI systems across diverse applications.
Papers
A Theoretical Understanding of Shallow Vision Transformers: Learning, Generalization, and Sample Complexity
Hongkang Li, Meng Wang, Sijia Liu, Pin-yu Chen
On Second-Order Derivatives of Rigid-Body Dynamics: Theory & Implementation
Shubham Singh, Ryan P. Russell, Patrick M. Wensing
Theory on Forgetting and Generalization of Continual Learning
Sen Lin, Peizhong Ju, Yingbin Liang, Ness Shroff
Toward a Theory of Causation for Interpreting Neural Code Models
David N. Palacio, Alejandro Velasco, Nathan Cooper, Alvaro Rodriguez, Kevin Moran, Denys Poshyvanyk
On the Theories Behind Hard Negative Sampling for Recommendation
Wentao Shi, Jiawei Chen, Fuli Feng, Jizhi Zhang, Junkang Wu, Chongming Gao, Xiangnan He