Sentence Embeddings

Follow this topic Ask about this topic

Momentum

3 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 23

All topics

Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings
Sep 23, 2026Jiaqi DengSemantic Correspondence Cross-Encoder Reranking
Independent Researcher
TEMPS: Temporal Sentence Embeddings for Temporal Information Retrieval
Sep 23, 2026Mourad Hassani, Julien Romero, Amel Bouzeghoub +1Sentence Embeddings Temporal IR
SAMOVAR, Télécom SudParis, Institut Polytechnique de Paris, France · Aldebaran Care, France
Improving Cross-Lingual Token Representations by Adding a Pinch of SALT
Sep 9, 2026Guillem RamírezSentence Embeddings Cross-Lingual Representation Alignment
ILCC, University of Edinburgh
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
Jul 30, 2026Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand +1Contrastive Learning Denoising Autoencoders
Department of Computer Science and Information Technology Institude for Advanced Studies in Basic Sciences (IASBS) Zanjan, Iran, 45137-66731
From transcription to semantic corpus analysis: unsupervised learning of sentence representations for ancient languages
Jul 27, 2026Th{é}otime de la SelleSemantic Textual Similarity Sentence Embeddings
ISC, HiSoMA, CNRS · Institut des Sources Chrétiennes, HiSoMA, CNRS, Lyon, France
Multilingual Sentence Embeddings for Linguistic-Integrated Reliability Audit
Jul 20, 2026Ummugul Bezirhan, Ji Yoon Jung, Matthias von DavierSentence Embeddings Educational Assessment
Boston College Chestnut Hill, Massachusetts, USA
Riemannian Geometry for Pre-trained Language Model Embeddings
Jul 8, 2026Szczepan Konior, Alexandre Quemy, Przemysław Klocek +2Sentence Embeddings Geometric Representation Learning
IBM Automation and AI, Krakow, Poland · Hother, Krakow, Poland · University of Warsaw +1
Beyond Multilingual Averages: MTEB-PT, a Benchmark for Portuguese Sentence Encoders
Jul 5, 2026Lucas Hideki Takeuchi Okamura, Alexandre Alcoforado, Anna Helena Reali CostaSentence Embeddings Text Embedding Evaluation
Escola Politécnica, Universidade de São Paulo, São Paulo, Brazil
Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment
Jun 19, 2026Aaron Marker, Oscar Kjell, Vasudha Varadarajan +1Sentence Embeddings Mental Health
Vanderbilt University · Lund University · Carnegie Mellon University
Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders
Jun 19, 2026Wonseok Shin, Songkuk KimText Embeddings Sentence Embeddings
Yonsei University
Correlation Is Not Enough: Embedding Human Metadata for Individual Causal Discovery
Jun 8, 2026Suraj Biswas, Saurabh Gupta, Pritam MukherjeeContrastive Learning Sentence Embeddings
¹ Assessli Research · ² Dots-In Research, Bengaluru, India
Modeling semantic association in self-paced reading with language model embeddings
Jun 5, 2026Sara Møller Østergaard, Kenneth Enevoldsen, Afra Alishahi +1Sentence Embeddings Text Embedding Evaluation
Department of Computational Cognitive Science, Tilburg University · Center for Humanities Computing, Aarhus University
Principles of Concept Representation in Sentence Encoders
Jun 5, 2026Isabelle Mohr, John Dujany, Jonathan Souquet +1Sentence Embeddings Compositional Reasoning
Idiap Research Institute · Merck KGaA
When Does Complexity Conditioning Help a Frozen Sentence Embedding? A Controlled Study of Per-Sentence and Pair-Level Difficulty Adaptation
Jun 2, 2026Suhwan HwangSentence Embeddings Learning to Rank
Model-Based Quality Assessment for Massively Multilingual Parallel Data
May 29, 2026Abdelaziz M. A. Ibrahim, Zihao Li, Jörg Tiedemann +1Sentence Embeddings MT Evaluation
University of Jyväskylä · University of Helsinki · ELLIS Institute Finland +1
Bridging the Gap Between Natural Language and Market Dynamics via High-Dimensional Representation Learning
May 28, 2026Yujin Jeong, Noelle Jung, Brian Y. C. LeungSentence Embeddings Time Series Forecasting
Mike
Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation
May 28, 2026M. Ali Bayram, Banu Diri, Savaş YıldırımMultilingual Language Models Sentence Embeddings
Department of Computer Engineering Yıldız Technical University Istanbul, Turkey · Department of Computer Engineering Istanbul Bilgi University Istanbul, Turkey
Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing
May 1, 2026Leonid BedratyukRepresentation Learning Neural Representation Geometry
Lost in State Space: Probing Frozen Mamba Representations
Apr 30, 2026Bhagyashree Wagh, Akash SinghRepresentation Geometry Representation Collapse
University of Washington · University of Florida
MIPIC: Matryoshka Representation Learning via Self-Distilled Intra-Relational and Progressive Information Chaining
Apr 27, 2026Phung Gia Huy, Hai An Vu, Minh-Phuc Truong +4Representation Learning Sentence Embeddings
1Hanoi University of Science and Technology · University of Oregon · 3Monash University
Finding Meaning in Embeddings: Concept Separation Curves
Apr 23, 2026Paul Keuren, Marc Ponsen, Robert Ayoub BagheriSentence Embeddings Text Embedding Evaluation
Utrecht University · Statistics Netherlands
FLiP: Towards understanding and interpreting multimodal multilingual sentence embeddings
Apr 20, 2026Santosh Kesiraju, Bolaji Yusuf, Šimon Sedláček +2Linear Probing Sentence Embeddings
Comparison of Modern Multilingual Text Embedding Techniques for Hate Speech Detection Task
Apr 16, 2026Evaldas Vaiciukynas, Paulius Danenas, Linas Ablonskis +5Sentence Embeddings Text Embedding Evaluation
Department of Information Systems, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania · Centre of Information Systems Design Technologies, Faculty of Informatics, Kaunas University of Technology, Kaunas, Lithuania

Sep 23, 2026, cs.CLJ/K move · Enter open · S save

Computation Over Geometry: Meaning Identity Is Computed, Not Shipped in the Embeddings

Jiaqi Deng

Independent Researcher

Semantic Correspondence Cross-Encoder Reranking Sentence Embeddings Representation Geometry in Language Models

Open paper AskPDF arXiv

Meaning identity (whether two sentences say the same thing after wording changes) is treated in retrieval and RAG as a geometric fact about independently encoded sentence vectors. We show that, for frozen off-the-shelf encoders and language models, it is not: identity is computed when both sentences share one forward pass, and is not a property of the embedding geometry those systems ship. On overlap-matched PAWS-X, purpose-built encoders (BGE, E5, GTE, MiniLM, E5-Mistral-7B) reach English confirm AUC only 0.55-0.65 (dense peak 0.70). Independently encoded last-token states of Llama 3, Mistral, and Qwen do no better; late fusion of the two vectors stays near chance. The same probe on a joint forward pass reaches 0.90-0.96 from 1.5B to 32B, collapses under partner shuffle, is mid-depth, saturates near 0.94 by 3B, and appears more weakly in GPT-2 XL (0.76). The gap holds beyond Llama-style models on other causal LMs, bidirectional encoders (DeBERTa, RoBERTa), and encoder-decoders (Flan-T5, T5, BART). Fixed or linear readers over frozen independent encodings never unlock identity; nonlinear pair readers recover part of it only on the full 49k-pair PAWS train split (0.68-0.87). Off-the-shelf rerankers split: BGE-reranker-large reaches 0.94, while MS-MARCO and Jina stay at 0.55-0.64. Independently trained families compute the same relation and a 1.5B joint reader can distill it from unlabelled teacher scores, while no linear function of the teachers own independent vectors can. Bi-encoders can be fine-tuned to fit PAWS (0.87-0.93), but transfer and STS-B suffer. Cosine compares wording neighbourhoods; identity is a cheap computed operator, not a property of either sentence vector.

Momentum

Related topics

Latest papers 23