Large Model
Large models, encompassing large language models (LLMs) and their multimodal counterparts (MLLMs), are rapidly advancing artificial intelligence by leveraging massive datasets and immense computational power to achieve state-of-the-art performance across diverse tasks. Current research emphasizes efficient fine-tuning techniques, including model compression and low-rank adaptation, to address the challenges of deploying these resource-intensive models, as well as improving their robustness and uncertainty quantification. These advancements are driving progress in various fields, from improved search engines and medical image analysis to novel applications in robotics, finance, and agriculture. The development of robust evaluation benchmarks and the exploration of the interplay between large and small models are also key areas of focus.
Papers
Large Concept Models: Language Modeling in a Sentence Representation Space
The LCM team, Loïc Barrault, Paul-Ambroise Duquenne, Maha Elbayad, Artyom Kozhevnikov, Belen Alastruey, Pierre Andrews, Mariano Coria, Guillaume Couairon, Marta R. Costa-jussà, David Dale, Hady Elsahar, Kevin Heffernan, João Maria Janeiro, Tuan Tran, Christophe Ropers, Eduardo Sánchez, Robin San Roman, Alexandre Mourachko, Safiyyah Saleem, Holger Schwenk
Adaptive Principal Components Allocation with the $\ell_{2,g}$-regularized Gaussian Graphical Model for Efficient Fine-Tuning Large Models
Jingjing Zheng, Yankai Cao
Towards Efficient Model-Heterogeneity Federated Learning for Large Models
Ruofan Jia, Weiying Xie, Jie Lei, Haonan Qin, Jitao Ma, Leyuan Fang
Enhancing Multi-Agent Consensus through Third-Party LLM Integration: Analyzing Uncertainty and Mitigating Hallucinations in Large Language Models
Zhihua Duan, Jialin Wang