Fundamental Limitation
Fundamental limitations in artificial intelligence research currently focus on identifying and addressing bottlenecks in model capabilities and performance. Active research areas include exploring the limitations of large language models (LLMs) in reasoning, particularly compositional abilities and handling complex tasks; analyzing the inherent quadratic time complexity of transformer architectures and the challenges of developing subquadratic alternatives; and investigating the impact of data quality and size on model performance and safety. Understanding these limitations is crucial for improving the reliability, safety, and efficiency of AI systems and for developing more robust and generalizable models across various applications.
Papers
An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4
Hui Huang, Yingqi Qu, Xingyuan Bu, Hongli Zhou, Jing Liu, Muyun Yang, Bing Xu, Tiejun Zhao
Exploring the Limitations of Large Language Models in Compositional Relation Reasoning
Jinman Zhao, Xueyan Zhang