Large Language Model

Large language models (LLMs) are sophisticated AI systems designed to process and generate human-like text, aiming to improve various natural language processing tasks. Current research focuses on enhancing LLM safety, efficiency (through techniques like quantization and optimized decoding), and fairness, as well as improving their ability to perform complex reasoning and handle diverse instructions. These advancements are significant because they address critical limitations in current LLMs and pave the way for broader applications across diverse fields, including healthcare, legal tech, and autonomous systems.

Papers

October 17, 2024

Learning to Route with Confidence Tokens
Yu-Neng Chuang, Helen Zhou, Prathusha Kameswara Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, Xia Hu
Large Language Model LeArning Abstract Token Prediction Confidence Score Human Generated Route Token Level Uncertainty
SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs
Yizhao Gao, Zhichen Zeng, Dayou Du, Shijie Cao, Hayden Kwok-Hay So, Ting Cao, Fan Yang, Mao Yang
Large Language Model Fine Tuning Sparse Attention Block Sparsity
Breaking Chains: Unraveling the Links in Multi-Hop Knowledge Unlearning
Minseok Choi, ChaeHun Park, Dohyun Lee, Jaegul Choo
Large Language Model Unlearning Framework Side Chain Unlearned Model Multi Hop Knowledge Internet Link
Roadmap towards Superhuman Speech Understanding using Large Language Models
Fan Bu, Yuhao Zhang, Xidong Wang, Benyou Wang, Qun Liu, Haizhou Li
Large Language Model Full Model Spoken Language Understanding Large Language Requirement Based Roadmap
Atomic Calibration of LLMs in Long-Form Generations
Caiqi Zhang, Ruihan Yang, Zhisong Zhang, Xinting Huang, Sen Yang, Dong Yu, Nigel Collier
Large Language Model Text Generation Calibration Performance LLM Fine Tuning Confidence Calibration Long Form Generation Confidence Elicitation
Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation
Hyungjoo Chae, Namyoung Kim, Kai Tzu-iunn Ong, Minju Gwak, Gwanwoo Song, Jihoon Kim, Sunghwan Kim, Dongha Lee, Jinyoung Yeo
Large Language Model Autonomous Agent World Model Long Horizon Task Web Agent
Quamba: A Post-Training Quantization Recipe for Selective State Space Models
Hung-Yueh Chiang, Chi-Chih Chang, Natalia Frumkin, Kai-Chiang Wu, Diana Marculescu
Large Language Model Full Model Long Sequence Selective State Space Accurate Quantization Per Tensor
Anchored Alignment for Self-Explanations Enhancement
Luis Felipe Villa-Arenas, Ata Nizamoglu, Qianli Wang, Sebastian Möller, Vera Schmitt
Large Language Model Model Alignment Instruction Datasets Explanation Quality Self Explanation Alignment Algorithm Human Annotated Rationale
AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations
Qian Tao, Wenyuan Yu, Jingren Zhou
Large Language Model Quantization Operator Layer Wise Quantization Technique KV Cache
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces
Ahmed Oumar El-Shangiti, Tatsuya Hiraoka, Hilal AlQuabeh, Benjamin Heinzerling, Kentaro Inui
Large Language Model Language Model Comparative Study Geometric Analysis Numerical Data Numerical Reasoning Linear Subspace
Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models
Jiatao Li, Xinyu Hu, Xunjian Yin, Xiaojun Wan
Large Language Model Retrieval Augmented Generation Faithful Generation Knowledge Intensive Task
AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning
Hao Sun, Jiayi Wu, Hengyi Cai, Xiaochi Wei, Yue Feng, Bo Wang, Shuaiqiang Wang, Yan Zhang, Dawei Yin
Large Language Model Adaptive Importance Agent System Switch List Cloud Collaborative Learning
SLM-Mod: Small Language Models Surpass LLMs at Content Moderation
Xianyang Zhan, Agam Goyal, Yilun Chen, Eshwar Chandrasekharan, Koustuv Saha
Large Language Model Language Model Content Moderation

October 16, 2024

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models
Linhao Luo, Zicheng Zhao, Chen Gong, Gholamreza Haffari, Shirui Pan
Large Language Model Knowledge Graph Unstructured Text Reasoning Path Graph Reasoning
Language Models as Semiotic Machines: Reconceptualizing AI Language Systems through Structuralist and Post-Structuralist Theories of Language
Elad Vromen
Large Language Model Language Model Human Cognition Cognitive Model Information Approach AI Generated Code
Hypothesis Testing the Circuit Hypothesis in LLMs
Claudia Shi, Nicolas Beltran-Velez, Achille Nazaret, Carolina Zheng, Adrià Garriga-Alonso, Andrew Jesson, Maggie Makar, David M. Blei
Large Language Model Transformer Model Hypothesis Testing Large Scale Circuit Biological Circuit Circuit Discovery
Agent Skill Acquisition for Large Language Models via CycleQD
So Kuroki, Taishi Nakamura, Takuya Akiba, Yujin Tang
Large Language Model Language Model Continual Adaptation Linguistic Benchmark
LLM Chain Ensembles for Scalable and Accurate Data Annotation
David Farr, Nico Manzonelli, Iain Cruickshank, Kate Starbird, Jevin West
Large Language Model Ensemble Method Data Annotation LLM Ensemble
POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization
Batuhan K. Karaman, Ishmam Zabir, Alon Benhaim, Vishrav Chaudhary, Mert R. Sabuncu, Xia Song
Large Language Model Full Model Faithful Generation Large Language Preference Optimization Model Completion
Leveraging LLMs for Translating and Classifying Mental Health Data
Konstantinos Skianis, A. Seza Doğruöz, John Pavlopoulos
Large Language Model Mental Health Mental Health Care Depression Severity Linguistic Marker

Large Language Model

Papers

Learning to Route with Confidence Tokens

SeerAttention: Learning Intrinsic Sparse Attention in Your LLMs

Breaking Chains: Unraveling the Links in Multi-Hop Knowledge Unlearning

Roadmap towards Superhuman Speech Understanding using Large Language Models

Atomic Calibration of LLMs in Long-Form Generations

Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigation

Quamba: A Post-Training Quantization Recipe for Selective State Space Models

Anchored Alignment for Self-Explanations Enhancement

AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations

The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces

Evaluating Self-Generated Documents for Enhancing Retrieval-Augmented Generation with Large Language Models

AdaSwitch: Adaptive Switching between Small and Large Agents for Effective Cloud-Local Collaborative Learning

SLM-Mod: Small Language Models Surpass LLMs at Content Moderation

Graph-constrained Reasoning: Faithful Reasoning on Knowledge Graphs with Large Language Models

Language Models as Semiotic Machines: Reconceptualizing AI Language Systems through Structuralist and Post-Structuralist Theories of Language

Hypothesis Testing the Circuit Hypothesis in LLMs

Agent Skill Acquisition for Large Language Models via CycleQD

LLM Chain Ensembles for Scalable and Accurate Data Annotation

POROver: Improving Safety and Reducing Overrefusal in Large Language Models with Overgeneration and Preference Optimization

Leveraging LLMs for Translating and Classifying Mental Health Data