Pre Trained Language Model
Pre-trained language models (PLMs) are large neural networks trained on massive text datasets, aiming to capture the statistical regularities of language for various downstream tasks. Current research focuses on improving PLM efficiency through techniques like parameter-efficient fine-tuning and exploring their application in diverse fields, including scientific text classification, mental health assessment, and financial forecasting, often leveraging architectures like BERT and its variants. The ability of PLMs to effectively process and generate human language has significant implications for numerous scientific disciplines and practical applications, ranging from improved information retrieval to more sophisticated AI assistants.
Papers
A Close Look into the Calibration of Pre-trained Language Models
Yangyi Chen, Lifan Yuan, Ganqu Cui, Zhiyuan Liu, Heng Ji
Leveraging Pre-trained Models for Failure Analysis Triplets Generation
Kenneth Ezukwoke, Anis Hoayek, Mireille Batton-Hubert, Xavier Boucher, Pascal Gounet, Jerome Adrian
When Language Model Meets Private Library
Daoguang Zan, Bei Chen, Zeqi Lin, Bei Guan, Yongji Wang, Jian-Guang Lou
Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change
Zhaochen Su, Zecheng Tang, Xinyan Guan, Juntao Li, Lijun Wu, Min Zhang
Beyond Prompting: Making Pre-trained Language Models Better Zero-shot Learners by Clustering Representations
Yu Fei, Ping Nie, Zhao Meng, Roger Wattenhofer, Mrinmaya Sachan
Exploiting prompt learning with pre-trained language models for Alzheimer's Disease detection
Yi Wang, Jiajun Deng, Tianzi Wang, Bo Zheng, Shoukang Hu, Xunying Liu, Helen Meng
DORE: Document Ordered Relation Extraction based on Generative Framework
Qipeng Guo, Yuqing Yang, Hang Yan, Xipeng Qiu, Zheng Zhang
Assessing Phrase Break of ESL speech with Pre-trained Language Models
Zhiyi Wang, Shaoguang Mao, Wenshan Wu, Yan Xia
RoChBert: Towards Robust BERT Fine-tuning for Chinese
Zihan Zhang, Jinfeng Li, Ning Shi, Bo Yuan, Xiangyu Liu, Rong Zhang, Hui Xue, Donghong Sun, Chao Zhang
COST-EFF: Collaborative Optimization of Spatial and Temporal Efficiency with Slenderized Multi-exit Language Models
Bowen Shen, Zheng Lin, Yuanxin Liu, Zhengxiao Liu, Lei Wang, Weiping Wang
Dial2vec: Self-Guided Contrastive Learning of Unsupervised Dialogue Embeddings
Che Liu, Rui Wang, Junfeng Jiang, Yongbin Li, Fei Huang