BERT Embeddings

BERT embeddings, derived from the Bidirectional Encoder Representations from Transformers model, are dense vector representations of text capturing semantic meaning and contextual information. Current research focuses on improving their utility in various NLP tasks, including information retrieval, sentiment analysis, and question answering, often employing techniques like multitask learning, generative adversarial networks, and novel fine-tuning strategies to enhance performance and interpretability. The widespread adoption of BERT embeddings reflects their significant impact on numerous applications, from improving the accuracy of text classification systems to facilitating more nuanced understanding of textual data in diverse domains.

Papers

October 10, 2024

Robust AI-Generated Text Detection by Restricted Embeddings
Kristian Kuznetsov, Eduard Tulchinskii, Laida Kushnareva, German Magai, Serguei Barannikov, Sergey Nikolenko, Irina Piontkovskaya
Transformer Based Robust Version Text Detection BERT Embeddings Spurious Feature Robust Classifier

October 3, 2024

Intrinsic Evaluation of RAG Systems for Deep-Logic Questions
Junyi Hu, You Zhou, Jie Wang
BERT Embeddings Conjunctive Query Intrinsic Evaluation

August 12, 2024

August 11, 2024

Multitask Fine-Tuning and Generative Adversarial Learning for Improved Auxiliary Classification
Christopher Sun, Abishek Satish
BERT Model Generative Adversarial BERT Embeddings BERT Architecture Auxiliary Classifier Multitask Finetuning

July 28, 2024

Exploring Genre and Success Classification through Song Lyrics using DistilBERT: A Fun NLP Venture
Servando Pizarro Martinez, Moritz Zimmermann, Miguel Serkan Offermann, Florian Reither
Natural Language Processing Genre Classification Financial Success BERT Embeddings Song Lyric

July 24, 2024

A Novel Two-Step Fine-Tuning Pipeline for Cold-Start Active Learning in Text Classification Tasks
Fabiano Belém, Washington Cunha, Celso França, Claudio Andrade, Leonardo Rocha, Marcos André Gonçalves
Active Learning Long Form Novel Text Classification Task BERT Embeddings Start Active Learning Text Representation Method

March 7, 2024

Automating the Information Extraction from Semi-Structured Interview Transcripts
Angelina Parfenova
Information Extraction Qualitative Analysis BERT Embeddings Interview Data Qualitative Methodology Semi Structured Interview

February 5, 2024

Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features
Simone Bombari, Marco Mondelli
Study Feature Attention Layer Random Feature BERT Embeddings Lexical Sensitivity Random Feature Attention

January 30, 2024

Detecting Racist Text in Bengali: An Ensemble Deep Learning Framework
S. S. Saruar, Nusrat, Sadia
Novel Dataset Bangla Text BERT Embeddings Neural Network Ensemble Deviant Racist Behaviour

January 17, 2024

BERTologyNavigator: Advanced Question Answering with BERT-based Semantics
Shreya Rajpal, Ricardo Usbeck
Knowledge Graph Relation Extraction BERT Based BERT Embeddings Question Complexity

December 17, 2023

Bengali Intent Classification with Generative Adversarial BERT
Mehedi Hasan, Mohammad Jahid Ibna Basher, Md. Tanvir Rouf Shawon
MAESTRO Dataset Intent Classification BERT Embeddings

November 22, 2023

Detecting out-of-distribution text using topological features of transformer-based language models
Andres Pollano, Anupam Chaudhuri, Anj Simmons
Distribution Detection Distribution Data Transformer Based Language Model Topological Feature Attention Map BERT Embeddings

November 8, 2023

DACBERT: Leveraging Dependency Agreement for Cost-Efficient Bert Pretraining
Martin Kuo, Jianyi Zhang, Yiran Chen
Language Model Ticket BERT BERT Based BERT Embeddings Pre Trained BERT Dependency Aware Incident

October 7, 2023

Hybrid Recommendation System using Graph Neural Network and BERT Embeddings
Shashidhar Reddy Javaji, Krutika Sarode
Graph Neural Network Recommender System Personalized Recommendation BERT Embeddings Hybrid Recommendation

August 15, 2023

MultiSChuBERT: Effective Multimodal Fusion for Scholarly Document Quality Prediction
Gideon Maillette de Buy Wenniger, Thomas van Dongen, Lambert Schomaker
Text Embeddings BERT Embeddings Quality Prediction Multimodal Prediction

June 6, 2023

LEACE: Perfect linear concept erasure in closed form
Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, Stella Biderman
Language Model BERT Embeddings Concept Erasure Concept Removal Closed Form

May 23, 2023

All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations
Yuxin Ren, Qipeng Guo, Zhijing Jin, Shauli Ravfogel, Mrinmaya Sachan, Bernhard Schölkopf, Ryan Cotterell
Transformer Megatron Decepticons Meaningful Representation Learned Representation Alpha Invariance Invertible Neural Network Built Road BERT Embeddings

April 20, 2023

Is Cross-modal Information Retrieval Possible without Training?
Hyunjin Choi, Hyunjae Lee, Seongho Joe, Youngjune L. Gwon
Training Data Jina Embeddings Image Text Retrieval Cross Modal Representation BERT Embeddings Cross Modal Information Retrieval

April 17, 2023

New Product Development (NPD) through Social Media-based Analysis by Comparing Word2Vec and BERT Word Embeddings
Princessa Cintaqia, Matheus Inoue
Sentiment Analysis BERT Embeddings

BERT Embeddings

Papers

Robust AI-Generated Text Detection by Restricted Embeddings

Intrinsic Evaluation of RAG Systems for Deep-Logic Questions

LOLgorithm: Integrating Semantic,Syntactic and Contextual Elements for Humor Classification

Improving Structural Diversity of Blackbox LLMs via Chain-of-Specification Prompting

Multitask Fine-Tuning and Generative Adversarial Learning for Improved Auxiliary Classification

Exploring Genre and Success Classification through Song Lyrics using DistilBERT: A Fun NLP Venture

A Novel Two-Step Fine-Tuning Pipeline for Cold-Start Active Learning in Text Classification Tasks

Automating the Information Extraction from Semi-Structured Interview Transcripts

Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features

Detecting Racist Text in Bengali: An Ensemble Deep Learning Framework

BERTologyNavigator: Advanced Question Answering with BERT-based Semantics

Bengali Intent Classification with Generative Adversarial BERT

Detecting out-of-distribution text using topological features of transformer-based language models

DACBERT: Leveraging Dependency Agreement for Cost-Efficient Bert Pretraining

Hybrid Recommendation System using Graph Neural Network and BERT Embeddings

MultiSChuBERT: Effective Multimodal Fusion for Scholarly Document Quality Prediction

LEACE: Perfect linear concept erasure in closed form

All Roads Lead to Rome? Exploring the Invariance of Transformers' Representations

Is Cross-modal Information Retrieval Possible without Training?

New Product Development (NPD) through Social Media-based Analysis by Comparing Word2Vec and BERT Word Embeddings