Transformer sentence encoders provide strong retrieval quality, but their inference cost becomes a bottleneck when embeddings must be produced on CPUs or refreshed frequently. We introduce StaRSE, a compact Russian static sentence encoder that removes Transformer inference at deployment time: token directions are stored as sign bits, token norms are stored separately, and sentences are encoded by mean pooling over norm-scaled sign directions. StaRSE is initialized from ruBERT-base, fine-tuned on Russian adjacent-sentence and task-shaped pairs, and post-trained with a contrastive stage that directly optimizes the sign-projected token table. On RuMTEB, StaRSE reaches a task mean of 51.16, 7.98 points above the strongest included static baseline. The packaged model occupies 11.3 MiB and has a mean CPU latency of 0.08 ms per sentence, compared with an estimated 55.7 MiB 16-bit parameter footprint and 1.43 ms mean CPU latency for the closest-quality contextual encoder. Ablations examine adjacent-sentence supervision, token norms, and direct sign-projected post-training.
Figures & tables
Figure 1: Quality on RuMTEB ( Snegirev et al., 2025 ) versus mean end-to-end single-sentence encoding latency in milliseconds (log scale; lower is better). Static encoders are measured on CPU and contextual encoders on GPU, so the two device groups represent different deployment regimes. Marker area shows encoder footprint in MiB on a log scale: contextual footprints are estimated as parameter count times two bytes, while static footprints are measured packages. Labeled blue points form the nondominated frontier of the displayed score–latency measurements under this mixed-device diagnostic. The star marks StaRSE.
Model (dims)
Mean
Class.
Cluster.
Multilabel
PairClass.
Rerank
Retrieval
STS
Sentence Transformers (STs)
ruBERT-tiny-turbo (312)
53.76
55.08
49.62
32.05
56.27
54.94
48.82
71.73
ruBERT-mini-FRIDA (312)
63.22
64.28
59.71
47.19
60.36
65.62
64.70
72.14
BERTA (768)
69.39
71.22
64.78
56.46
65.74
69.72
73.79
73.72
FRIDA (1536)
70.95
73.71
67.49
52.23
66.45
71.50
76.38
74.35
GigaEmbeddings (2048)
74.16
76.66
67.48
57.07
79.57
74.02
81.41
75.75
Table 1: RuMTEB (v 1.1 ) comparison across the 23 Russian tasks. Columns report the benchmark mean and task-type means on the 0 – 100 scale; bold marks the best row within each model block. Lexical rankers are defined only for reranking and retrieval, so their other cells are not applicable.
Model (dims)
nDCG@ 10
Sentence Transformers (STs)
USER2-small (384)
51.69
USER2-base (768)
54.17
USER-bge-m3 (1024)
58.53
KaLM-v1.5 (896)
53.75
jina-embeddings-v3 (1024)
49.67
Table 2: MLDR-RU long-document retrieval on the test split. Scores are nDCG@ 10 percentages; bold marks the best row within each block.
Max doc tokens
BM25
StaRSE
StaRSE+BM25
USER2-small
128
28.34
32.11
30.84
42.31
512
30.09
33.63
34.17
46.97
1024
32.47
36.88
36.05
48.22
2048
34.07
40.90
38.38
48.67
4096
68.12
52.01
69.93
53.70
8192
69.97
51.66
73.07
51.69
Table 3: MLDR-RU length scaling on 10,000 Russian documents and 200 test queries. Scores are nDCG@ 10 percentages. Rows show how retrieval quality changes as more document tokens are retained; bold marks the best model at each length.
Model (dims)
Size (MiB)
RuMTEB
CPU (ms)
GPU (ms)
Sentence Transformers (STs)
ruBERT-tiny-turbo (312)
55.7
53.76
1.43
1.68
ruBERT-mini-FRIDA (312)
61.5
63.22
2.97
3.01
BERTA (768)
245
69.39
19.36
3.80
FRIDA (1536)
1571
70.95
110.87
9.61
GigaEmbeddings (2048)
6580
74.16
773.61
43.26
Table 4: RuMTEB score, encoder footprint, and mean end-to-end single-sentence latency on a fixed Cultura RU Edu input. Contextual footprints are estimated from 16-bit parameter counts; static footprints are measured packages. Static-model GPU latency is omitted because no protocol-matched optimized run is available. Bold marks the best reported value within each block and column.
Tokenizer
Vocab
Valid tok/word
BM25 nDCG@ 10
ai-forever/ruBert-base
120,138
1.342
52.46
deepvk/USER-bge-m3
46,166
1.696
51.02
cointegrated/rubert-tiny2
83,828
1.303
50.81
DeepPavlov/rubert-base-cased-conversational
100,792
1.280
50.57
cointegrated/rubert-tiny
29,564
2.171
50.27
deepvk/USER2-small
50,368
2.153
49.88
Table 5: Tokenizer diagnostics used for vocabulary selection. BM25 is used as a tokenizer-only retrieval proxy over 12 RusBEIR tasks. BM25 nDCG@ 10 is reported on the 0 – 100 scale. Valid tok/word counts retained analyzer tokens per sampled lexical word and is affected by filtering and out-of-vocabulary coverage.
Initialization source
PCA
STS dev
Input embeddings
token-level
0.76810
Input embeddings
sentence-level
0.73663
Averaged contextual embeddings
sentence-level
0.73349
Per-token last-hidden state (Model2Vec)
token-level
0.64753
Table 6: Matched initialization screening on the Russian STS Benchmark validation split. All rows use the ruBERT-base vocabulary with the same post-processing. Scores are cosine Spearman; bold marks the selected initialization.
Variant
RuSTS dev
RuMTEB
Distillation baselines
BERTA-distilled static student
74.7
42.0
USER2-small-distilled static student
69.2
44.2
Positive-pair construction
Self-pairs
71.1
37.2
Raw adjacent Cultura
76.1
48.2
Table 7: Data-construction and distillation ablations. Scores are percentages; bold marks the best result in each column.
Setting
RuMTEB
Δ
Training route to a sign-coded token table
Post-hoc signing after contrastive fine-tuning; no post-training
49.33
–
Contrastive continuation, then post-hoc signing
50.09
+0.76
Optimized sign-projected post-training, final StaRSE recipe
51.16
+1.83
Inference-time negative controls on the final checkpoint
StaRSE inference: token norms kept; real-valued pooled sentence vector
51.16
–
Table 8: Ablations for the sign-coded StaRSE token table. Scores are full-RuMTEB means; deltas are relative to the first row in each block.