Capability Evolution
Capability evolution in artificial intelligence focuses on understanding and enhancing the abilities of various AI models, particularly large language models (LLMs), across diverse tasks. Current research emphasizes evaluating these capabilities through novel benchmarks and frameworks, often analyzing model performance under incomplete information or with limited data, and exploring the role of factors like data quality and model architecture (e.g., transformers, state space models). This research is crucial for responsible AI development, informing the creation of more robust and reliable systems with applications ranging from robotics and software engineering to education and scientific research.
Papers
Prism: A Framework for Decoupling and Assessing the Capabilities of VLMs
Yuxuan Qiao, Haodong Duan, Xinyu Fang, Junming Yang, Lin Chen, Songyang Zhang, Jiaqi Wang, Dahua Lin, Kai Chen
CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks
Jie Feng, Jun Zhang, Tianhui Liu, Xin Zhang, Tianjian Ouyang, Junbo Yan, Yuwei Du, Siqi Guo, Yong Li