Language Model

Language models are computational systems designed to understand and generate human language, primarily aiming to improve tasks like translation, question answering, and text summarization. Current research focuses on enhancing efficiency (e.g., through novel learning rate schedules and optimized architectures), improving alignment with human preferences (via preference optimization and reward modeling), and addressing biases and limitations (including techniques for mitigating toxicity and enhancing robustness). These advancements have significant implications for various fields, impacting natural language processing research and enabling the development of more powerful and reliable AI applications.

Papers

November 19, 2024

Information Theory of Meaningful Communication
Doron Sivan, Misha Tsodyks
Language Model Speech Analysis Information Theory Entropy Solution
Neurosymbolic Graph Enrichment for Grounded World Models
Stefano De Giorgis, Aldo Gangemi, Alessandro Russo
Language Model Structured Representation Semantic Structure Abstract Meaning Representation Visual Metaphor
RedPajama: an Open Dataset for Training Large Language Models
Maurice Weber, Daniel Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
Large Language Model Language Model Open Dataset Open Source Language Model Large Language Model Training Sleep Datasets
HouseLLM: LLM-Assisted Two-Phase Text-to-Floorplan Generation
Ziyang Zong, Zhaohuan Zhan, Guang Tan
Language Model Floor Plan
CoMeDi Shared Task: Models as Annotators in Lexical Semantics Disagreements
Zhu Liu, Zhen Hu, Ying Liu
Language Model Full Model Shared Task Individual Annotator Opinion Disagreement Similarity Score Lexical Semantics Disagreement Analysis Framework

November 18, 2024

Vision Language Models Are Few-Shot Audio Spectrogram Classifiers
Satvik Dixit, Laurie M. Heller, Chris Donahue
Language Model Vision Language Model
FedCoLLM: A Parameter-Efficient Federated Co-tuning Framework for Large and Small Language Models
Tao Fan, Yan Kang, Guoqiang Ma, Lixin Fan, Kai Chen, Qiang Yang
Language Model Domain Knowledge Domain Specific Task Federated Tuning
Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty
Huashan Sun, Yang Gao
Language Model Continual LEArning Catastrophic Forgetting Anti Forgetting Rationale Alignment Memory Retrieval Replay Based Continual Learning
Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality
Viktoriia Chekalina, Anton Razzigaev, Elizaveta Goncharova, Andrey Kuznetsov
Language Model Knowledge Graph Jina Embeddings Entity Linking
Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactions
Robin Carpentier, Benjamin Zi Hao Zhao, Hassan Jameel Asghar, Dali Kaafar
Language Model Differential Privacy Text Sanitization
PALMS: Parallel Adaptive Lasso with Multi-directional Signals for Latent Networks Reconstruction
Zhaoyu Xing, Wei Zhong
Language Model Compressed Sensing Intermediate Latent Large Scale Network Large Network Adaptive Lasso Network Reconstruction Directional Communication
Rethinking Thinking Tokens: Understanding Why They Underperform in Practice
Sreeram Vennam, David Valente, David Herel, Ponnurangam Kumaraguru
Language Model Complex Reasoning Chain of Thought Practice Mode Thinking Token
SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models
Muhammad Fadhil Ginting, Dong-Ki Kim, Sung-Kyun Kim, Bandi Jai Krishna, Mykel J. Kochenderfer, Shayegan Omidshafiei, Ali-akbar Agha-mohammadi
Language Model Task Planning Domain Knowledge Regulatory Compliance Field Robotics Robot Goal
Steering Language Model Refusal with Sparse Autoencoders
Kyle O'Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, Forough Poursabzi-Sangde
Language Model Sparse Autoencoders Steering Technique Phi 3 Refusal Training
LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models
Yungi Kim, Hyunsoo Ha, Seonghoon Yang, Sukyung Lee, Jihoo Kim, Chanjun Park
Large Language Model Language Model Lightweight High Large Scale Datasets Data Pipeline
MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis
Yingjie Zhou, Zicheng Zhang, Jiezhang Cao, Jun Jia, Yanwei Jiang, Farong Wen, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai
Language Model Sentiment Analysis Text to Image Multimodal Phenomenon Human Emotion Fine Grained Emotion

November 17, 2024

Debiasing Watermarks for Large Language Models via Maximal Coupling
Yangxinyu Xie, Xiang Li, Tanwi Mallick, Weijie J. Su, Ruixun Zhang
Language Model Machine Generated Token Bias Novel Watermarking
LLäMmlein: Compact and Competitive German-Only Language Models from Scratch
Jan Pfister, Julia Wunderle, Andreas Hotho
Language Model Training Data Scratch Project Embracing CompAct LLaVA HD Decoder Only Model
SRA-MCTS: Self-driven Reasoning Aurmentation with Monte Carlo Tree Search for Enhanced Code Generation
Bin Xu, Yiguan Lin, Yinghao Li, YangGao
Large Language Model Language Model Code Generation Search Query Reasoning Path Augmented Data Minimax McTs Self Reasoning
SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization
Hongrui Jia, Chaoya Jiang, Haiyang Xu, Wei Ye, Mengfan Dong, Ming Yan, Ji Zhang, Fei Huang, Shikun Zhang
Language Model Context Learning Large Multimodal Model Direct Preference Optimization Multimodal Context Context Demonstration

Language Model

Papers

Information Theory of Meaningful Communication

Neurosymbolic Graph Enrichment for Grounded World Models

RedPajama: an Open Dataset for Training Large Language Models

HouseLLM: LLM-Assisted Two-Phase Text-to-Floorplan Generation

CoMeDi Shared Task: Models as Annotators in Lexical Semantics Disagreements

Vision Language Models Are Few-Shot Audio Spectrogram Classifiers

FedCoLLM: A Parameter-Efficient Federated Co-tuning Framework for Large and Small Language Models

Reviving Dormant Memories: Investigating Catastrophic Forgetting in Language Models through Rationale-Guidance Difficulty

Addressing Hallucinations in Language Models with Knowledge Graph Embeddings as an Additional Modality

Preempting Text Sanitization Utility in Resource-Constrained Privacy-Preserving LLM Interactions

PALMS: Parallel Adaptive Lasso with Multi-directional Signals for Latent Networks Reconstruction

Rethinking Thinking Tokens: Understanding Why They Underperform in Practice

SayComply: Grounding Field Robotic Tasks in Operational Compliance through Retrieval-Based Language Models

Steering Language Model Refusal with Sparse Autoencoders

LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models

MEMO-Bench: A Multiple Benchmark for Text-to-Image and Multimodal Large Language Models on Human Emotion Analysis

Debiasing Watermarks for Large Language Models via Maximal Coupling

LLäMmlein: Compact and Competitive German-Only Language Models from Scratch

SRA-MCTS: Self-driven Reasoning Aurmentation with Monte Carlo Tree Search for Enhanced Code Generation

SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization