Ensemble BERT

Ensemble BERT methods combine multiple BERT-based models to improve performance on various natural language processing tasks, addressing limitations of single models in accuracy, robustness, and interpretability. Research focuses on optimizing ensemble techniques, including weighted averaging of predictions, diverse model architectures within the ensemble, and innovative training strategies like Multi-CLS BERT which achieves efficiency gains. These advancements enhance the capabilities of BERT for applications such as sentiment analysis, patent document similarity assessment, and hate speech detection, ultimately improving the reliability and effectiveness of NLP systems in diverse domains.

Papers

August 9, 2024

Ensemble BERT: A student social network text sentiment classification model based on ensemble learning and BERT architecture
Kai Jiang, Honghao Yang, Yuexian Wang, Qianru Chen, Yiming Luo
Sentiment Analysis BERT Model Ensemble Learning Student Friendly Knowledge BERT Architecture Ensemble Neural Network Ensemble BERT

January 6, 2024

Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Method
Liqiang Yu, Bo Liu, Qunwei Lin, Xinyu Zhao, Chang Che
Semantic Similarity Text Analysis Patent Text Ensemble BERT

August 23, 2023

Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models
Nancy Tyagi, Aidin Shiri, Surjodeep Sarkar, Abhishek Kumar Umrawal, Manas Gaur
Natural Language Processing Deep Ensemble Model Ensembling Ensemble BERT

October 10, 2022

Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling
Haw-Shiuan Chang, Ruei-Yao Sun, Kathryn Ricci, Andrew McCallum
BERT Model Model Ensembling Efficient Alternative Ensemble BERT

September 14, 2022

BERT-based Ensemble Approaches for Hate Speech Detection
Khouloud Mnassri, Praboda Rajapaksha, Reza Farahbakhsh, Noel Crespi
Hate Speech Hate Speech Detection Twitter Dataset Ensemble BERT

September 7, 2022

AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles
Beatrice Portelli, Simone Scaboro, Emmanuele Chersoni, Enrico Santus, Giuseppe Serra
Generative Model Transformer Megatron Decepticons Transformer Based Text Classification Continuum Limit Entity Extraction Entity Normalization AI Lab Ensemble BERT

May 24, 2022

RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder
Shitao Xiao, Zheng Liu, Yingxia Shao, Zhao Cao
Language Model Dense Retrieval Auto Encoder Masked Auto Encoder Ensemble BERT

Ensemble BERT

Papers

Ensemble BERT: A student social network text sentiment classification model based on ensemble learning and BERT architecture

Semantic Similarity Matching for Patent Documents Using Ensemble BERT-related Model and Novel Text Processing Method

Simple is Better and Large is Not Enough: Towards Ensembling of Foundational Language Models

Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling

BERT-based Ensemble Approaches for Hate Speech Detection

AILAB-Udine@SMM4H 22: Limits of Transformers and BERT Ensembles

RetroMAE: Pre-Training Retrieval-oriented Language Models Via Masked Auto-Encoder