Language Model

Language models are computational systems designed to understand and generate human language, primarily aiming to improve tasks like translation, question answering, and text summarization. Current research focuses on enhancing efficiency (e.g., through novel learning rate schedules and optimized architectures), improving alignment with human preferences (via preference optimization and reward modeling), and addressing biases and limitations (including techniques for mitigating toxicity and enhancing robustness). These advancements have significant implications for various fields, impacting natural language processing research and enabling the development of more powerful and reliable AI applications.

Papers

November 15, 2024

SlimLM: An Efficient Small Language Model for On-Device Document Assistance
Thang M. Pham, Phat T. Nguyen, Seunghyun Yoon, Viet Dac Lai, Franck Dernoncourt, Trung Bui
Language Model Participation Constraint HELP Request Advanced Language Model Device Use Case
AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference
Janghwan Lee, Jiwoong Park, Jinseok Kim, Yongjik Kim, Jungju Oh, Jinwook Oh, Jungwook Choi
Language Model LLM Inference Low Bit Floating Point

November 14, 2024

Adaptive Decoding via Latent Preference Optimization
Shehzaad Dhuliawala, Ilia Kulikov, Ping Yu, Asli Celikyilmaz, Jason Weston, Sainbayar Sukhbaatar, Jack Lanchantin
Language Model Adaptive Importance Preference Optimization Discrete Latent Variable Temperature Sampling
On the Limits of Language Generation: Trade-Offs Between Hallucination and Mode Collapse
Alkis Kalavasis, Anay Mehrotra, Grigoris Velegkas
Language Model Language Generation Continuum Limit Content Hallucination Mode Collapse Consistent Generation
Accelerating Knowledge Graph and Ontology Engineering with Large Language Models
Cogan Shimizu, Pascal Hitzler
Language Model Knowledge Graph Top Level Ontology
LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models
Zhengyi Wang, Jonathan Lorraine, Yikai Wang, Hang Su, Jun Zhu, Sanja Fidler, Xiaohui Zeng
Large Language Model Language Model 3D Mesh LLaMA Model 3D Mesh Generation Isomorphic Mesh
BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency
Akari Haga, Akiyo Fukatsu, Miyu Oba, Arianna Bisazza, Yohei Oseki
Language Model Mixed Effect Human Language Data Efficient Child Speech BabyLM Challenge Variation Space
Enhancing Financial Domain Adaptation of Language Models via Model Augmentation
Kota Tanabe, Masanori Hirano, Kazuki Matoya, Kentaro Imajo, Hiroki Sakaji, Itsuki Noda
Large Language Model Language Model Domain Adaptation Full Model Soft Augmentation Financial Domain Financial Dataset

November 13, 2024

Language Models for Music Medicine Generation
Emmanouil Nikolakakis, Joann Ching, Emmanouil Karystinaios, Gabrielle Sipin, Gerhard Widmer, Razvan Marinescu
Language Model Music Driven Dance Music Therapy
Can sparse autoencoders be used to decompose and interpret steering vectors?
Harry Mayne, Yushi Yang, Adam Mahdi
Language Model X Vector Steering Technique Sparse Autoencoders Steering Vector Multi Property Steering
Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers
Clément Dumas, Chris Wendler, Veniamin Veselovsky, Giovanni Monea, Robert West
Language Model Transformer Megatron Decepticons Latent Representation Multilingual Language Model Thought Reasoning Language Agnostic Ultrasound Tongue
A Comparative Study of Discrete Speech Tokens for Semantic-Related Tasks with Large Language Models
Dingdong Wang, Mingyu Cui, Dongchao Yang, Xueyuan Chen, Helen Meng
Language Model Comparative Study Speech Feature Semantic Task Discrete Token Speech Discrete Token Discrete Lexical Symbol
Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models
Somanshu Singla, Zhen Wang, Tianyang Liu, Abdullah Ashfaq, Zhiting Hu, Eric P. Xing
Language Model Prompt Optimization Self Alignment Alignment Tuning
Dynamic Subset Tuning: Expanding the Operational Range of Parameter-Efficient Training for Large Language Models
Felix Stahlberg, Jared Lichtarge, Shankar Kumar
Language Model NLP Task Prompt Tuning Parameter Efficient Range Query Parameter Selection Meta Tuning
Neural Topic Modeling with Large Language Models in the Loop
Xiaohao Yang, He Zhao, Weijie Xu, Yuanyuan Qi, Jueqing Lu, Dinh Phung, Lan Du
Language Model Neural Topic Topic Modeling Topic Model Neural Topic Model
Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering
Farouq Sammour, Jia Xu, Xi Wang, Mo Hu, Zhenyu Zhang
Language Model Prompt Engineering Responsible AI Safety Critical Safety Engineering Product Safety Idiom

November 12, 2024

Language Models as Causal Effect Generators
Lucius E.J. Bynum, Kyunghyun Cho
Language Model Causal Inference Causal Structure Causal Model Structural Causal Model Novel Causal Model
Derivational Morphology Reveals Analogical Generalization in Large Language Models
Valentin Hofmann, Leonie Weissweiler, David Mortensen, Hinrich Schütze, Janet Pierrehumbert
Language Model Analogical Reasoning Morphological Inflection Linguistic Generalization Nominal Phrase
Towards Low-bit Communication for Tensor Parallel LLM Inference
Harry Dong, Tyler Johnson, Minsik Cho, Emad Soroush
Language Model LLM Inference Multiplier Free Quantization Inference Efficiency Low Bit Product Quantization Tensor Parallelism
Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders
Xiaofeng Zhu, Jaya Krishna Mandivarapu
Language Model Retrieval Augmented Generation Text Generation Knowledge Based Hallucination Detection Content Generation Prompt Customization Dual Decoder

Language Model

Papers

SlimLM: An Efficient Small Language Model for On-Device Document Assistance

AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference

Adaptive Decoding via Latent Preference Optimization

On the Limits of Language Generation: Trade-Offs Between Hallucination and Mode Collapse

Accelerating Knowledge Graph and Ontology Engineering with Large Language Models

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models

BabyLM Challenge: Exploring the Effect of Variation Sets on Language Model Training Efficiency

Enhancing Financial Domain Adaptation of Language Models via Model Augmentation

Language Models for Music Medicine Generation

Can sparse autoencoders be used to decompose and interpret steering vectors?

Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers

A Comparative Study of Discrete Speech Tokens for Semantic-Related Tasks with Large Language Models

Dynamic Rewarding with Prompt Optimization Enables Tuning-free Self-Alignment of Language Models

Dynamic Subset Tuning: Expanding the Operational Range of Parameter-Efficient Training for Large Language Models

Neural Topic Modeling with Large Language Models in the Loop

Responsible AI in Construction Safety: Systematic Evaluation of Large Language Models and Prompt Engineering

Language Models as Causal Effect Generators

Derivational Morphology Reveals Analogical Generalization in Large Language Models

Towards Low-bit Communication for Tensor Parallel LLM Inference

Trustful LLMs: Customizing and Grounding Text Generation with Knowledge Bases and Dual Decoders