Large Language Model

Large language models (LLMs) are sophisticated AI systems designed to process and generate human-like text, aiming to improve various natural language processing tasks. Current research focuses on enhancing LLM safety, efficiency (through techniques like quantization and optimized decoding), and fairness, as well as improving their ability to perform complex reasoning and handle diverse instructions. These advancements are significant because they address critical limitations in current LLMs and pave the way for broader applications across diverse fields, including healthcare, legal tech, and autonomous systems.

Papers

November 25, 2024

Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability
Jatin Nainani, Sankaran Vaidyanathan, AJ Yeung, Kartik Gupta, David Jensen
Large Language Model Strong Generalization Inherent Interpretability Model Generalization Mechanistic Interpretability Interpretable Algorithm Mechanistic Study Circuit Representation Learning
SAGEval: The frontiers of Satisfactory Agent based NLG Evaluation for reference-free open-ended text
Reshmi Ghosh, Tianyi Yao, Lizzy Chen, Sadid Hasan, Tianwei Chen, Dario Bernal, Huitian Jiao, H M Sajjad Hossain
Large Language Model Language Generation New Frontier Robust Evaluation Free Text Explanation Informed Agent NLG Evaluation

November 24, 2024

Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown
Lifu Tu, Rui Meng, Shafiq Joty, Yingbo Zhou, Semih Yavuz
Large Language Model Text Generation Factual Claim Self Information Multi Role
Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
Chao Fang, Man Shi, Robin Geens, Arne Symons, Zhongfeng Wang, Marian Verhelst
Large Language Model LLM Inference Memory Access Activation Compression Variable Length
Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language models
Olivia Ma, Jonathan Passerat-Palmbach, Dmitrii Usynin
Large Language Model Language Model High Efficiency Differential Privacy Parameter Efficient Fine Tuning PRIvacy Leakage Memorisation Profile
LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting
Lingzheng Zhang, Lifeng Shen, Yimin Zheng, Shiyuan Piao, Ziyue Li, Fugee Tsung
Large Language Model Natural Language Time Series Mixture Component State of the Art Forecasting Real World Time Series Time Series Generation Linear Learner
LoRA-Mini : Adaptation Matrices Decomposition and Selective Training
Ayush Singh, Rajdeep Aher, Shivank Garg
Large Language Model Natural Language Processing Parameter Efficient Fine Tuning LLM Fine Tuning Selective Learning
A Method for Building Large Language Models with Predefined KV Cache Capacity
Zhonghua Yi, Ge Niu, Lei Wang, Wei Tang, Liqiu Zhang
Large Language Model Practical Method Key Value Cache Capacity Loss Decoder Only Architecture Key Value Sequence
TableTime: Reformulating Time Series Classification as Zero-Shot Table Understanding via Large Language Models
Jiahao Wang, Mingyue Cheng, Qingyang Mao, Qi Liu, Feiyang Xu, Xin Li, Enhong Chen
Large Language Model Latent Space Human Understanding Time Series Classification Representation Space Multivariate Time Series Classification Graphic Tablet Tabular Time Series
LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
Xiaoye Qu, Daize Dong, Xuyang Hu, Tong Zhu, Weigao Sun, Yu Cheng
Large Language Model Mixture of Expert Visual Perspective Sparsity Increase Random Sparsification LLaMa LlamaCare LLaMA Model

November 23, 2024

Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities
Ashwin Ramachandran, Sunita Sarawagi
Large Language Model Natural Language Community Need Relative Probability Calibration Method Text to SQL Benchmark Word Level Rescaling Sequence Likelihood
Multi-label Sequential Sentence Classification via Large Language Model
Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu
Large Language Model Language Model Sentence Level
Do LLMs Agree on the Creativity Evaluation of Alternative Uses?
Abdullah Al Rabeyah, Fabrício Góes, Marco Volpe, Talles Medeiros
Large Language Model Greater Public Use Accurate Model Creativity Assessment
Reassessing Layer Pruning in LLMs: New Insights and Methods
Yao Lu, Hao Cheng, Yujie Fang, Zeyu Wang, Jiaheng Wei, Dongwei Xu, Qi Xuan, Xiaoniu Yang, Zhaowei Zhu
Large Language Model NCD Method Layer Selection Layer Pruning Pruned Model New Insight
ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain
Haochen Zhao, Xiangru Tang, Ziran Yang, Xiao Han, Xuanzhi Feng, Yueqing Fan, Senhao Cheng, Di Jin, Yilun Zhao, Arman Cohan, Mark Gerstein
Large Language Model LLM Safety State of the Art Large Chemical Knowledge
Large Language Model with Region-guided Referring and Grounding for CT Report Generation
Zhixuan Chen, Yequan Bie, Haibo Jin, Hao Chen
Large Language Model Computed Tomography Radiology Report Generation Report Generation Chest Computed Tomography CT Volume Region Word Alignment
Towards Robust Evaluation of Unlearning in LLMs via Data Transformations
Abhinav Joshi, Shaswati Saha, Divyaksh Shukla, Sriram Vema, Harsh Jhamtani, Manas Gaur, Ashutosh Modi
Large Language Model Large Corpus Machine Unlearning Harmful Unlearning Data Transformation Robust Evaluation LLM Unlearning

Large Language Model

Papers

Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability

SAGEval: The frontiers of Satisfactory Agent based NLG Evaluation for reference-free open-ended text

Investigating Factuality in Long-Form Text Generation: The Roles of Self-Known and Self-Unknown

Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format

Efficient and Private: Memorisation under differentially private parameter-efficient fine-tuning in language models

LeMoLE: LLM-Enhanced Mixture of Linear Experts for Time Series Forecasting

LoRA-Mini : Adaptation Matrices Decomposition and Selective Training

A Method for Building Large Language Models with Predefined KV Cache Capacity

TableTime: Reformulating Time Series Classification as Zero-Shot Table Understanding via Large Language Models

LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training

Text-to-SQL Calibration: No Need to Ask -- Just Rescale Model Probabilities

Multi-label Sequential Sentence Classification via Large Language Model

Do LLMs Agree on the Creativity Evaluation of Alternative Uses?

Reassessing Layer Pruning in LLMs: New Insights and Methods

ChemSafetyBench: Benchmarking LLM Safety on Chemistry Domain

Large Language Model with Region-guided Referring and Grounding for CT Report Generation

Towards Robust Evaluation of Unlearning in LLMs via Data Transformations

PPLqa: An Unsupervised Information-Theoretic Quality Metric for Comparing Generative Large Language Models

Efficient Pruning of Text-to-Image Models: Insights from Pruning Stable Diffusion

Sycophancy in Large Language Models: Causes and Mitigations