Giga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text Embeddings

Egor Kolodin; Egor Krasnoperov; Evgeniy Kosarev; Fyodor Minkin

cs.CLAug 24, 2026

Giga-Embeddings: Mixture-of-Experts Encoders for High-Throughput Text Embeddings

Authors: Egor Kolodin, Egor Krasnoperov, Evgeniy Kosarev, Fyodor Minkin

Organizations: 1MIPT · 2SaluteDevices · 3MSU

Abstract

We introduce Giga-Embeddings, a family of text embedding models designed to combine strong retrieval quality with efficient serving. Its largest member is a sparse 10B-parameter Mixture-of-Experts encoder with approximately 1.8B active parameters per token. Across English, Russian, multilingual, and code MTEB benchmarks, this model achieves the strongest aggregate performance within the family on all four evaluated suites. In our vLLM benchmark with 1024-token inputs, it processes 114.5k tokens per second, providing 25 percent higher throughput than the dense 3B model and 1.56-2.65x the throughput of the evaluated external systems. The family also includes a dense 3B encoder and a distilled 480M encoder for tighter compute and memory budgets. We train the compact model using a dimension-agnostic objective that aligns teacher and student similarity distributions. The resulting 480M model scores 70.98 on Russian MTEB, surpassing FRIDA while using 42 percent fewer parameters. We release all three model checkpoints.

Figures & tables

Setting	Pretrain	Finetune	Multitask
Global batch size	16,384	1,024	1,024
Candidates/sample	in-batch	1 pos. + 7 neg.	1 pos. + 7 neg.
In-batch negatives	yes	no	no

Table 1: Stage-specific training settings. Each fine-tuning and multitask sample contains one positive and seven negatives, giving 8,192 candidate texts per global batch.

Figure 1: Three-stage embedding training pipeline. All models undergo contrastive pre-training, retrieval fine-tuning, and multitask fine-tuning. Similarity-distribution distillation is applied only to the 480M student during the fine-tuning and multitask stages.

Table 3

Model (vLLM)	512 tokens	1024 tokens	2048 tokens	Relative
Qwen3 Embedding 4B	70.1k	73.2k	71.2k	0.64 $\times$
F2LLM-v2-8B	43.2k	43.4k	42.6k	0.38 $\times$
Nemotron 8B	42.6k	43.2k	41.7k	0.38 $\times$
Ours: 3B-0826	87.9k	91.5k	90.4k	0.80 $\times$
Ours: 10B-A1.8B-0826	112.6k	114.5k	102.3k	1.00 $\times$

Table 4: Token throughput (thousands of tokens per second) with vLLM at fixed input lengths. Relative throughput is measured at 1024 tokens and normalized to our 10B-A1.8B-0826 model.

Model	English	Russian	Code
480M, no distillation	69.43	70.86	72.65
480M, distilled	69.52	70.98	72.87

Table 5: Distillation ablation for the 480M model on the English, Russian, and Code MTEB suites (%).

Model	Params	Dim.	Russian	English	Multilingual	Code
sergeyzh/BERTA	128M	768	69.39	—	—	—
google/embeddinggemma-300m	308M	768	65.19	69.67	61.15	68.76
codefuse-ai/F2LLM-v2-0.6B	596M	1024	65.97	69.97	62.74	77.41
Qwen/Qwen3-Embedding-0.6B	596M	1024	64.29	70.47	64.34	75.42
ai-forever/FRIDA	823M	1536	70.95	—	—	—
Ours: 480M-0826	480M	1024	70.98	69.52	56.97	72.87

Table 6: Comparison with compact models on MTEB benchmark suites (%). Public baselines use official MTEB Leaderboard meanTask values retrieved on 2026-08-08; our result is from Table 3 . Bold marks the best score in each column, and a dash denotes an unavailable score.

Model	Params	Dim.	Russian	English	Multilingual	Code
codefuse-ai/F2LLM-v2-1.7B	1.72B	2048	68.52	71.63	65.21	78.76
codefuse-ai/F2LLM-v2-4B	4.02B	2560	69.46	72.41	67.06	80.15
Qwen/Qwen3-Embedding-4B	4.02B	2560	70.13	74.61	69.45	80.07
Qwen/Qwen3-Embedding-8B	7.57B	4096	71.42	75.23	70.58	80.69
codefuse-ai/F2LLM-v2-8B	7.57B	4096	70.57	72.86	68.09	80.16
codefuse-ai/F2LLM-v2-14B	13.99B	5120	70.90	73.08	68.74	80.75

Table 7: Comparison with larger models on MTEB benchmark suites (%). Public baselines use official MTEB Leaderboard meanTask values retrieved on 2026-08-08; our results are from Table 3 . Bold marks the best score in each column.

Text Embedding Models

Text embedding models map words, sentences, or documents to vectors for semantic search, retrieval, clustering, and downstream prediction.

See all 126 papers

BitNet Text Embeddings
Jun 24, 2026Zhen Li, Xin Huang, Liang Wang +8LLM Quantization Embedding Compression
Peking University · Microsoft Research
The Embedder's Dilemma: LLMs Are Better, but at What Cost?
Aug 13, 2026Adnan El Assadi, Niklas Muennighoff, Jinhyuk LeeLLM Inference Efficiency LLM Evaluation
Harvard University · Stanford University · Independent Researcher
Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems
May 22, 2026Stefano Cirillo, Domenico Desiato, Giuseppe Polese +1Retrieval-Augmented Generation Text Embedding Evaluation
University of Salerno · Fisciano, Salerno, Italy · University of Bari +1

Jun 24, 2026J/K move · Enter open · S save

BitNet Text Embeddings

Zhen Li, Xin Huang, Liang Wang, Nan Yang+7

Peking University · Microsoft Research

LLM Quantization Embedding Compression Text Embedding Models Low-Bit Quantization

Open paper AskPDF arXiv

LLM-based text embedders have substantially improved retrieval and semantic representation quality, but their deployment remains costly: large backbone models slow down embedding inference, while high-dimensional full-precision embeddings impose substantial storage and bandwidth overhead on large-scale indexes. In this paper, we present BITEMBED, an extreme low-bit framework for LLM-based text embedding that jointly targets encoding efficiency and vector storage. BITEMBED converts pretrained LLM backbones into BitNet-style embedding encoders with ternary weights, quantized activations, and lightweight normalization refinement. The converted model is adapted to representation learning through continual contrastive pre-training, followed by supervised contrastive fine-tuning with both similarity-distribution distillation and attention-relation distillation from a full-precision teacher. Beyond quantizing the backbone, BITEMBED further trains output embeddings to support multiple storage precisions meeting different storage needs in various scenarios. Experiments on MMTEB (eng, v2) with Qwen3-0.6B and Gemma3-270M show that BITEMBED is largely comparable to full precision teacher embedders. Moreover, BITEMBED flexibly obtains text embeddings of various precisions, achieving a trade-off between performance and storage cost.

Abstract

Text Embedding Models

Explore similar work

BitNet Text Embeddings

The Embedder's Dilemma: LLMs Are Better, but at What Cost?

Benchmarking Google Embeddings 2 against Open-Source Models for Multilingual Dense Retrieval and RAG Systems