Language Model

Language models are computational systems designed to understand and generate human language, primarily aiming to improve tasks like translation, question answering, and text summarization. Current research focuses on enhancing efficiency (e.g., through novel learning rate schedules and optimized architectures), improving alignment with human preferences (via preference optimization and reward modeling), and addressing biases and limitations (including techniques for mitigating toxicity and enhancing robustness). These advancements have significant implications for various fields, impacting natural language processing research and enabling the development of more powerful and reliable AI applications.

Papers

November 11, 2024

A Text Classification Model Combining Adversarial Training with Pre-trained Language Model and neural networks: A Case Study on Telecom Fraud Incident Texts
Liu Zhuoxian, Shi Tuo, Hu Xiaofeng
Language Model Neural Network Adversarial Training Fraud Detection Text Classification Model Incident Report Language Model Pre Training
Reverse Prompt Engineering
Hanqing Li, Diego Klabjan
Large Language Model Language Model Automatic Prompt Engineering Zero Shot Language
Model Fusion through Bayesian Optimization in Language Model Fine-Tuning
Chaeyun Jang, Hyungi Lee, Jungtaek Kim, Juho Lee
Language Model Fine Tuning Pre Trained Model Bayesian Optimization Language Model Fine Tuning Multi Objective Bayesian Optimization Model Fusion
What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performance
Hong Meng Yam, Nathan J Paek
Language Model Model Performance Efficient Training Sample Efficient Compositional Ability Child Speech Reader Model

November 10, 2024

CriticAL: Critic Automation with Language Models
Michael Y. Li, Vivek Vajipey, Noah D. Goodman, Emily B. Fox
Language Model Critic Model Model Criticism Scientific Model
Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents
Yu Gu, Boyuan Zheng, Boyu Gou, Kai Zhang, Cheng Chang, Sanjari Srivastava, Yanan Xie, Peng Qi, Huan Sun, Yu Su
Language Model Medical LLM World Model Language Agent Web Agent Model Based Planning
Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models
Sultan Alrashed, Dmitrii Khizbullin, David R. Pugh
Large Language Model Language Model Machine Translation Large Corpus Multilingual Large Language Model Machine Translated Arabic Language
LLM Vocabulary Compression for Low-Compute Environments
Sreeram Vennam, Anish Joishy, Ponnurangam Kumaraguru
Language Model Tensor Compiler LLM Compression Story Point Byte Pair Encoding Low Computational

November 9, 2024

ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese
Yikang Liu, Yeting Shen, Hongao Zhu, Lilong Xu, Zhiheng Qian, Siyuan Song, Kejia Zhang, Jialong Tang, Pei Zhang, Baosong Yang, Rui Wang, Hai Hu
Language Model Chinese Character Chinese Corpus Chinese Language Minimal Pair
Concept Bottleneck Language Models For protein design
Aya Abdelsalam Ismail, Tuomas Oikarinen, Amy Wang, Julius Adebayo, Samuel Stanton, Taylor Joren, Joseph Kleinhenz, Allen Goodman, Héctor Corrada Bravo, Kyunghyun Cho, Nathan C. Frey
Language Model Concept Bottleneck Model Protein Language Model Protein Design Protein Language
Zyda-2: a 5 Trillion Token High-Quality Dataset
Yury Tokpanov, Paolo Glorioso, Quentin Anthony, Beren Millidge
Language Model K TOKEN Token Classification Quality Prediction
Learning Mixtures of Experts with EM
Quentin Fruytier, Aryan Mokhtari, Sujay Sanghavi
Language Model Machine Learning Model Gradient Descent Expert Knowledge Expectation Maximization Learning Mixture Kullback Leibler

November 8, 2024

The Empirical Impact of Data Sanitization on Language Models
Anwesan Pal, Radhika Bhargava, Kyle Hinsz, Jacques Esterhuizen, Sudipta Bhattacharya
Language Model Natural Language Processing Clean Data Data Redaction
NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts
Yen-Ting Lin, Chao-Han Huck Yang, Zhehuai Chen, Piotr Zelasko, Xuesong Yang, Zih-Ching Chen, Krishna C Puvvada, Szu-Wei Fu, Ke Hu, Jun Wei Chiu, Jagadeesh Balam, Boris Ginsburg, Yu-Chiang Frank Wang
Language Model Full Model Mixture of Expert Speech to Text Multi Task Optimization Post OCR Task Expert
Using Language Models to Disambiguate Lexical Choices in Translation
Josh Barua, Sanjay Subramanian, Kayo Yin, Alane Suhr
Language Model View Translation Additional Disambiguation Task Lexical Knowledge
Autonomous Industrial Control using an Agentic Framework with Large Language Models
Javal Vyas, Mehmet Mercangöz
Language Model Multi Agent Autonomous System Agent System Decomposed Automation Correction Autonomous Operation Autonomous Control
Towards Multi-Modal Mastery: A 4.5B Parameter Truly Multi-Modal Small Language Model
Ben Koska, Mojmír Horváth
Language Model Multi Task Learning Edge Inference Performing Model
Evaluating Large Language Model Capability in Vietnamese Fact-Checking Data Generation
Long Truong To, Hung Tuan Le, Dat Van-Thanh Nguyen, Manh Trong Nguyen, Tri Thien Nguyen, Tin Van Huynh, Kiet Van Nguyen
Language Model Data Generation Language Data
WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models
Shengda Fan, Xin Cong, Yuepeng Fu, Zhong Zhang, Shuyan Zhang, Yuanwei Liu, Yesai Wu, Yankai Lin, Zhiyuan Liu, Maosong Sun
Language Model Real World Office Workflow Process Data
Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning
Mohammad Ghiasvand Mohammadkhani
Large Language Model Language Model Chain of Thought Mathematical Reasoning Structured Prompting

Language Model

Papers

A Text Classification Model Combining Adversarial Training with Pre-trained Language Model and neural networks: A Case Study on Telecom Fraud Incident Texts

Reverse Prompt Engineering

Model Fusion through Bayesian Optimization in Language Model Fine-Tuning

What Should Baby Models Read? Exploring Sample-Efficient Data Composition on Model Performance

CriticAL: Critic Automation with Language Models

Is Your LLM Secretly a World Model of the Internet? Model-Based Planning for Web Agents

Fineweb-Edu-Ar: Machine-translated Corpus to Support Arabic Small Language Models

LLM Vocabulary Compression for Low-Compute Environments

ZhoBLiMP: a Systematic Assessment of Language Models with Linguistic Minimal Pairs in Chinese

Concept Bottleneck Language Models For protein design

Zyda-2: a 5 Trillion Token High-Quality Dataset

Learning Mixtures of Experts with EM

The Empirical Impact of Data Sanitization on Language Models

NeKo: Toward Post Recognition Generative Correction Large Language Models with Task-Oriented Experts

Using Language Models to Disambiguate Lexical Choices in Translation

Autonomous Industrial Control using an Agentic Framework with Large Language Models

Towards Multi-Modal Mastery: A 4.5B Parameter Truly Multi-Modal Small Language Model

Evaluating Large Language Model Capability in Vietnamese Fact-Checking Data Generation

WorkflowLLM: Enhancing Workflow Orchestration Capability of Large Language Models

Gap-Filling Prompting Enhances Code-Assisted Mathematical Reasoning