Transformer Based Language Model
Transformer-based language models are deep learning architectures designed to process and generate human language, aiming to understand and replicate the nuances of natural language understanding and generation. Current research focuses on improving model interpretability, addressing contextualization errors, and exploring the internal mechanisms responsible for tasks like reasoning and factual recall, often using models like BERT and GPT variants. These advancements are significant for both the scientific community, furthering our understanding of neural networks and language processing, and for practical applications, enabling improvements in machine translation, question answering, and other NLP tasks.
Papers
A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
Daking Rai, Yilun Zhou, Shi Feng, Abulhair Saparov, Ziyu Yao
Multi-Peptide: Multimodality Leveraged Language-Graph Learning of Peptide Properties
Srivathsan Badrinarayanan, Chakradhar Guntuboina, Parisa Mollaei, Amir Barati Farimani