Large Language Model

Large language models (LLMs) are sophisticated AI systems designed to process and generate human-like text, aiming to improve various natural language processing tasks. Current research focuses on enhancing LLM safety, efficiency (through techniques like quantization and optimized decoding), and fairness, as well as improving their ability to perform complex reasoning and handle diverse instructions. These advancements are significant because they address critical limitations in current LLMs and pave the way for broader applications across diverse fields, including healthcare, legal tech, and autonomous systems.

Papers

October 8, 2024

KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server
Wenhao Wang, Xiaoyu Liang, Rui Ye, Jingyi Chai, Siheng Chen, Yanfeng Wang
Large Language Model Knowledge Distillation Differential Privacy Private Data PRIvacy Leakage
Enhancing Temporal Modeling of Video LLMs via Time Gating
Zi-Yuan Hu, Yiwu Zhong, Shijia Huang, Michael R. Lyu, Liwei Wang
Large Language Model Video Understanding Temporal Attention Temporal Modeling Gating Mechanism Video Benchmark Video LLM
Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMs
Yi Jiang, Qingyang Shen, Shuzhong Lai, Shunyu Qi, Qian Zheng, Lin Yao, Yueming Wang, Gang Pan
Large Language Model Autism Spectrum Disorder Real World Clinical NextGen Communication COPILOT
Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models
Siqi Wang, Zhengyu Chen, Bei Li, Keqing He, Min Zhang, Jingang Wang
Large Language Model Comparative Study Mixture of Expert Scaling Law Dense Model Positive Scaling
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models
Ranchi Zhao, Zhen Leng Thai, Yifan Zhang, Shengding Hu, Yunqi Ba, Jie Zhou, Jie Cai, Zhiyuan Liu, Maosong Sun
Large Language Model Language Model Training Corpus Explicit in Document Tagging Data Engineering
Vector-ICL: In-context Learning with Continuous Vector Representations
Yufan Zhuang, Chandan Singh, Liyuan Liu, Jingbo Shang, Jianfeng Gao
Large Language Model Context Learning Different Context Vector Representation Continuous Representation Language Modeling Objective Context Vector
Stereotype or Personalization? User Identity Biases Chatbot Recommendations
Anjali Kantharuban, Jeremiah Milbauer, Emma Strubell, Graham Neubig
Large Language Model Stereotype Content Personalized Recommendation Implicit Communication
Chain-of-Thoughts for Molecular Understanding
Yunhui Jang, Jaehyung Kim, Sungsoo Ahn
Large Language Model Chain of Thought Molecular Information Molecule Task
Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition
Zheyang Xiong, Ziyang Cai, John Cooper, Albert Ge, Vasilis Papageorgiou, Zack Sifakis, Angeliki Giannou, Ziqian Lin, Liu Yang, Saurabh Agarwal, Grigorios G Chrysos, Samet Oymak, Kangwook Lee, Dimitris Papailiopoulos
Large Language Model Context Learning Different Context First Order Superposition Task Vector Multiple Task
ParallelSpec: Parallel Drafter for Efficient Speculative Decoding
Zilin Xiao, Hongming Zhang, Tao Ge, Siru Ouyang, Vicente Ordonez, Dong Yu
Large Language Model Speculative Decoding Speculative Execution
Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?
Fırat Öncel, Matthias Bethge, Beyza Ermis, Mirco Ravanelli, Cem Subakan, Çağatay Yıldız
Large Language Model Language Model Deep Learning Model Traditional Deep Learning Perplexity Analysis Supplementary Material

October 7, 2024

Chain and Causal Attention for Efficient Entity Tracking
Erwan Fagnou, Paul Caillon, Blaise Delattre, Alexandre Allauzen
Large Language Model Language Model Attention Mechanism Side Chain Causal Attention Entity Tracking
Fill In The Gaps: Model Calibration and Generalization with Synthetic Data
Yang Ba, Michelle V. Mancenido, Rong Pan
Large Language Model Strong Generalization Machine Learning Model Synthetic Data Synthetic Data Generation Filling Gap Large Scale Synthetic Model Calibration Efficient Calibration
What makes your model a low-empathy or warmth person: Exploring the Origins of Personality in LLMs
Shu Yang, Shenzhe Zhu, Ruoxuan Bao, Liang Liu, Yu Cheng, Lijie Hu, Mengdi Li, Di Wang
Large Language Model Full Model Personality Trait Visual Creation Social Determinant
Superficial Safety Alignment Hypothesis
Jianwei Li, Jung-Eun Kim
Large Language Model Safety Alignment
From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency
Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, Jingzhao Zhang
Large Language Model Chain of Thought Attention Layer Sparse Attention Reasoning Performance Sequential Dependency
Aligning LLMs to Be Robust Against Prompt Injection
Sizhe Chen, Arman Zharmagambetov, Saeed Mahloujifar, Kamalika Chaudhuri, Chuan Guo
Large Language Model Adversarial Prompt Prompt Injection Attack Prompt Injection
Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback
Sanjiban Choudhury, Paloma Sodhi
Large Language Model LLM Agent Good Teacher Teacher Model Student Model AI Feedback Self Improvement
Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models
Fei Wang, Ninareh Mehrabi, Palash Goyal, Rahul Gupta, Kai-Wei Chang, Aram Galstyan
Large Language Model Data Generation Safety Alignment Data Curation Model Safety
PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs
Mengzhao Chen, Yi Liu, Jiahao Wang, Yi Bin, Wenqi Shao, Ping Luo
Large Language Model Dynamic Quantization Per Tensor

Large Language Model

Papers

KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server

Enhancing Temporal Modeling of Video LLMs via Time Gating

Copiloting Diagnosis of Autism in Real Clinical Scenarios via LLMs

Scaling Laws Across Model Architectures: A Comparative Analysis of Dense and MoE Models in Large Language Models

DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models

Vector-ICL: In-context Learning with Continuous Vector Representations

Stereotype or Personalization? User Identity Biases Chatbot Recommendations

Chain-of-Thoughts for Molecular Understanding

Everything Everywhere All at Once: LLMs can In-Context Learn Multiple Tasks in Superposition

ParallelSpec: Parallel Drafter for Efficient Speculative Decoding

Adaptation Odyssey in LLMs: Why Does Additional Pretraining Sometimes Fail to Improve?

Chain and Causal Attention for Efficient Entity Tracking

Fill In The Gaps: Model Calibration and Generalization with Synthetic Data

What makes your model a low-empathy or warmth person: Exploring the Origins of Personality in LLMs

Superficial Safety Alignment Hypothesis

From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency

Aligning LLMs to Be Robust Against Prompt Injection

Better than Your Teacher: LLM Agents that learn from Privileged AI Feedback

Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models

PrefixQuant: Static Quantization Beats Dynamic through Prefixed Outliers in LLMs