Generative Language Model
Generative language models (GLMs) are artificial intelligence systems designed to produce human-like text, aiming to improve tasks like text summarization, question answering, and creative writing. Current research focuses on enhancing GLMs' accuracy, addressing biases and hallucinations, and improving efficiency through techniques like retrieval-augmented generation (RAG), fine-tuning with smaller, specialized models, and optimizing model architectures (e.g., transformers). These advancements have significant implications for various fields, including education (automated scoring), scientific discovery (catalyst design), and addressing societal challenges (mitigating harmful outputs), but also raise concerns about ethical implications and potential biases.
Papers
How do you know that? Teaching Generative Language Models to Reference Answers to Biomedical Questions
Bojana Bašaragin, Adela Ljajić, Darija Medvecki, Lorenzo Cassano, Miloš Košprdić, Nikola Milošević
AI Safety in Generative AI Large Language Models: A Survey
Jaymari Chua, Yun Li, Shiyi Yang, Chen Wang, Lina Yao
Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning
Yiming Wang, Pei Zhang, Baosong Yang, Derek F. Wong, Zhuosheng Zhang, Rui Wang
Slaves to the Law of Large Numbers: An Asymptotic Equipartition Property for Perplexity in Generative Language Models
Raghu Mudumbai, Tyler Bell