Hidden CoST
Hidden CoST research focuses on optimizing the trade-off between the performance and resource consumption of various computational models and algorithms. Current efforts concentrate on developing cost-effective alternatives to expensive models like GPT-4, exploring efficient architectures for specific applications (e.g., IoT security, automatic speech recognition), and improving the efficiency of existing methods through techniques such as active learning, ensemble selection, and early stopping. This work is significant because it addresses the critical need for resource-efficient solutions in diverse fields, ranging from AI model training and deployment to resource-constrained IoT devices and automated machine learning.
Papers
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost
Sungjun Cho, Seonwoo Min, Jinwoo Kim, Moontae Lee, Honglak Lee, Seunghoon Hong
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models
Bowen Shen, Zheng Lin, Yuanxin Liu, Zhengxiao Liu, Lei Wang, Weiping Wang