Neural Language Model
Neural language models (NLMs) are computational systems designed to understand and generate human language, aiming to capture the statistical regularities and underlying structure of text. Current research focuses on improving NLM efficiency (e.g., through optimized training schedules and low-rank adaptation), enhancing their ability to represent complex linguistic structures (e.g., using transformer architectures and exploring the role of tokenization), and mitigating biases and improving interpretability. NLMs have significant implications for various fields, including natural language processing, cognitive science, and even areas like healthcare through applications such as clinical text analysis and improved speech recognition.
Papers
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Zhaochen Su, Zecheng Tang, Xinyan Guan, Juntao Li, Lijun Wu, Min Zhang