Resource Intensive LLM

Resource-intensive Large Language Models (LLMs) present a significant challenge due to their high computational demands during both training and inference. Current research focuses on improving efficiency through techniques like retrieval-augmented generation (RAG), developing linear-scaling architectures for streaming applications, and creating hybrid systems that combine smaller, less expensive models with larger LLMs only when necessary. These efforts aim to make LLMs more accessible and sustainable, impacting both scientific research by enabling broader experimentation and practical applications by reducing deployment costs and improving user experience.

Papers

November 8, 2024

GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Caching
Sajal Regmi, Chetan Phakami Pun
Language Understanding Query Information Low Latency Forward Caching Query Embeddings Resource Intensive LLM Semantic Cache

October 2, 2024

July 23, 2024

ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency
Yuhang Yao, Han Jin, Alay Dilipbhai Shah, Shanshan Han, Zijian Hu, Yide Ran, Dimitris Stripelis, Zhaozhuo Xu, Salman Avestimehr, Chaoyang He
Large Language Model New Framework End to End LLM Inference Local Inference Resource Intensive LLM

February 19, 2024

All Language Models Large and Small
Zhixun Chen, Yali Du, David Mguni
Language Model Resource Intensive LLM

January 16, 2024

A Survey of Resource-efficient LLM and Multimodal Foundation Models
Mengwei Xu, Wangsong Yin, Dongqi Cai, Rongjie Yi, Daliang Xu, Qipeng Wang, Bingyang Wu, Yihao Zhao, Chen Yang, Shihe Wang, Qiyang Zhang, Zhenyan Lu, Li Zhang, Shangguang Wang, Yuanchun Li, Yunxin Liu, Xin Jin, Xuanzhe Liu
Large Language Model Timely Survey Foundation Model Multimodal Model Large Model Multimodal Foundation Model Model Architecture Resource Intensive LLM

November 16, 2023

More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering
Bingsheng Yao, Guiming Chen, Ruishi Zou, Yuxuan Lu, Jiachen Li, Shao Zhang, Yisi Sang, Sijia Liu, James Hendler, Dakuo Wang
Medical LLM Context Learning Prompt Engineering NLI Model NLI Datasets Resource Intensive LLM

April 14, 2023

nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales
Yiqun Yao, Siqi fan, Xiusheng Huang, Xuezhi Fang, Xiang Li, Ziyi Ni, Xin Jiang, Xuying Meng, Peng Han, Shuo Shang, Kang Liu, Aixin Sun, Yequan Wang
Language Model New Benchmark Large Model Ghanaian Musical Scale Scaling Behavior Hyperparameter Space Prediction Loss Resource Intensive LLM

Resource Intensive LLM

Papers

GPT Semantic Cache: Reducing LLM Costs and Latency via Semantic Embedding Caching

Open-RAG: Enhanced Retrieval-Augmented Reasoning with Open-Source Large Language Models

Efficient Streaming LLM for Speech Recognition

ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency

All Language Models Large and Small

A Survey of Resource-efficient LLM and Multimodal Foundation Models

More Samples or More Prompts? Exploring Effective In-Context Sampling for LLM Few-Shot Prompt Engineering

nanoLM: an Affordable LLM Pre-training Benchmark via Accurate Loss Prediction across Scales