Large Language Model

Large language models (LLMs) are sophisticated AI systems designed to process and generate human-like text, aiming to improve various natural language processing tasks. Current research focuses on enhancing LLM safety, efficiency (through techniques like quantization and optimized decoding), and fairness, as well as improving their ability to perform complex reasoning and handle diverse instructions. These advancements are significant because they address critical limitations in current LLMs and pave the way for broader applications across diverse fields, including healthcare, legal tech, and autonomous systems.

Papers

October 2, 2024

A Two-Stage Proactive Dialogue Generator for Efficient Clinical Information Collection Using Large Language Model
Xueshen Li, Xinlong Hou, Nirupama Ravi, Ziyi Huang, Yu Gan
Large Language Model Medical Dialogue Doctor Patient Structured Clinical Proactive Dialogue
CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL
Mohammadreza Pourreza, Hailong Li, Ruoxi Sun, Yeounoh Chung, Shayan Talaei, Gaurav Tarlok Kakkar, Yu Gan, Amin Saberi, Fatma Ozcan, Sercan O. Arik
Large Language Model Text to SQL Preference Optimization Reasoning Path Fine Tuned Large Language Model Din SQL
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads
Yuxiang Huang, Binhang Yuan, Xu Han, Chaojun Xiao, Zhiyuan Liu
Large Language Model Long Context Human Head Long Context LLM GPU Memory Eviction Prediction
Investigating on RLHF methodology
Alexey Kutalev, Sergei Markoff
Large Language Model Language Model Direct Preference Optimization RLHF V
Quantifying Generalization Complexity for Large Language Models
Zhenting Qi, Hongyin Luo, Xuliang Huang, Zhuokai Zhao, Yibo Jiang, Xiangjun Fan, Himabindu Lakkaraju, James Glass
Large Language Model Complex Reasoning Task Task Complexity
Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing
Yilmazcan Ozyurt, Stefan Feuerriegel, Mrinmaya Sachan
Large Language Model Knowledge Tracing Semantic Annotation Question Representation Knowledge Concept
SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks
Tianhao Li, Jingyu Lu, Chuangxin Chu, Tianyu Zeng, Yujia Zheng, Mei Li, Haotian Huang, Bin Wu, Zuoxian Liu, Kai Ma, Xuejing Yuan, Xingkai Wang, Keyan Ding, Huajun Chen, Qiang Zhang
Large Language Model Comprehensive Benchmark Scientific Research Safety Alignment Scientific Domain Implicit Target Span
Auto-Demo Prompting: Leveraging Generated Outputs as Demonstrations for Enhanced Batch Prompting
Longyu Feng, Mengze Hong, Chen Jason Zhang
Large Language Model Structured Output Noisy Demonstration Shot Prompting
Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective
Zeyu Gan, Yong Liu
Large Language Model Synthetic Data Theoretical Understanding Synthetic Data Generation
CreDes: Causal Reasoning Enhancement and Dual-End Searching for Solving Long-Range Reasoning Problems using LLMs
Kangsheng Wang, Xiao Zhang, Hao Liu, Songde Han, Huimin Ma, Tianyu Hu
Large Language Model Multi Step Reasoning Causal Reasoning Causality Extraction
U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models
Tung-Yu Wu, Pei-Yu Lo
Large Language Model U Shaped Network Emergent Ability Quantifying Emergence Inverse Scaling
FactAlign: Long-form Factuality Alignment of Large Language Models
Chao-Wei Huang, Yun-Nung Chen
Large Language Model Fact Verification Factual Accuracy
VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment
Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, Nicolas Le Roux
Large Language Model Proximal Policy Optimization Complex Reasoning Task LLM Reasoning Credit Assignment State of the Art Reinforcement Value Network
Mind Scramble: Unveiling Large Language Model Psychology Via Typoglycemia
Miao Yu, Junyuan Mao, Guibin Zhang, Jingheng Ye, Junfeng Fang, Aoxiao Zhong, Yang Liu, Yuxuan Liang, Kun Wang, Qingsong Wen
Large Language Model Human Like Cognitive Learning Psychological Experiment
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding
Yanming Liu, Xinyue Peng, Jiannan Cao, Shi Bo, Yanxin Shen, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, Tianyu Du
Large Language Model Coreference Resolution Long Span Coreference Information Contextual Understanding Bridging Gap
On The Adaptation of Unlimiformer for Decoder-Only Transformers
Kian Ahrabian, Alon Benhaim, Barun Patra, Jay Pujara, Saksham Singhal, Xia Song
Large Language Model Adaptation Concern Retrieval Augmentation Context Length Decoder Only Transformer
Intent Detection in the Age of LLMs
Gaurav Arora, Shreya Jain, Srujana Merugu
Large Language Model Intent Detection Generative Large Language Model Task Oriented Dialogue System Speech Based Age Sentence Transformer Intent Label
Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?
Xi Chen, Kaituo Feng, Changsheng Li, Xunhao Lai, Xiangyu Yue, Ye Yuan, Guoren Wang
Large Language Model Low Rank Grand Challenge Low Rank Constraint Low Rank Training
Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging
Tingfeng Hui, Zhenyu Zhang, Shuohuan Wang, Yu Sun, Hua Wu, Sen Su
Large Language Model Mixture of Expert
Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression
Jingcun Wang, Yu-Guang Chen, Ing-Chao Lin, Bing Li, Grace Li Zhang
Large Language Model Compression Technique Joint Representation Weight Matrix Cross Layer

Large Language Model

Papers

A Two-Stage Proactive Dialogue Generator for Efficient Clinical Information Collection Using Large Language Model

CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL

Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads

Investigating on RLHF methodology

Quantifying Generalization Complexity for Large Language Models

Automated Knowledge Concept Annotation and Question Representation Learning for Knowledge Tracing

SciSafeEval: A Comprehensive Benchmark for Safety Alignment of Large Language Models in Scientific Tasks

Auto-Demo Prompting: Leveraging Generated Outputs as Demonstrations for Enhanced Batch Prompting

Towards a Theoretical Understanding of Synthetic Data in LLM Post-Training: A Reverse-Bottleneck Perspective

CreDes: Causal Reasoning Enhancement and Dual-End Searching for Solving Long-Range Reasoning Problems using LLMs

U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models

FactAlign: Long-form Factuality Alignment of Large Language Models

VinePPO: Unlocking RL Potential For LLM Reasoning Through Refined Credit Assignment

Mind Scramble: Unveiling Large Language Model Psychology Via Typoglycemia

Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding

On The Adaptation of Unlimiformer for Decoder-Only Transformers

Intent Detection in the Age of LLMs

Fira: Can We Achieve Full-rank Training of LLMs Under Low-rank Constraint?

Upcycling Instruction Tuning from Dense to Mixture-of-Experts via Parameter Merging

Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression