Skip to main content
Back to timeline
arXivSource publication:

StaRSE packs Russian sentence vectors into 11.3 MiB: no Transformer at deployment, 0.08 ms per sentence on CPU

Synopsis

Russian sentence retrieval gains an option that runs no Transformer at deployment: StaRSE stores token directions as sign bits with token norms kept separately, and builds sentence vectors by mean-pooling norm-scaled sign directions. On RuMTEB it reaches a task mean of 51.16, 8.0 points above the strongest included static baseline, Potion multilingual at 43.18, while running from a measured 11.3 MiB package at 0.08 ms mean CPU latency per sentence; the closest-quality contextual encoder, ruBERT-tiny-turbo, has an estimated 55.7 MiB 16-bit parameter footprint and 1.43 ms mean CPU latency. The final hyperparameters were chosen by an exploratory sweep over the full RuMTEB mean, so the score is not a held-out estimate.

Source-provided article image: StaRSE: Compact Russian Sentence Embeddings with a Sign-Coded Static Encoder
arXiv

Interpretation

Russian retrieval can now produce sentence vectors from a lookup table plus pooling, with no Transformer forward pass at deployment. Russian static representations were mostly word-level, multilingual static sentence embedders were not tuned for Russian morphology and vocabulary coverage, and high-quality sentence vectors relied on a contextual encoder pass per sentence. On RuMTEB, StaRSE reaches a task mean of 51.16, 8.0 points above the strongest included static baseline, Potion multilingual at 43.18, and 2.6 points below ruBERT-tiny-turbo at 53.76.

The compression sits at the token table: each token keeps only the sign of its direction, with its strength stored as a separate scalar norm, and the pooled sentence vector stays real-valued. Signing an already-trained continuous table after the fact breaks the match between the training objective and the representation used at inference. In the same 512-dimensional architecture, signing immediately after contrastive fine-tuning gives a RuMTEB of 49.33, continuing ordinary contrastive training before signing gives 50.09, and quantization-aware post-training through the sign-projected table reaches 51.16.

At inference the sign-coded table also depends on token norms; dropping them or signing the pooled sentence vector instead both lose clear ground. Negative controls on the final checkpoint separate two places to compress: the token directions, or the pooled sentence vector. Keeping token norms and returning a real-valued pooled vector gives 51.16; dropping norms for unweighted signs falls to 42.74; keeping norms but signing the pooled sentence vector gives 47.33; dropping both gives 39.06.

Perspective

The demonstrated scope is Russian sentence vectors: the Russian tasks of RuMTEB, the MLDR Russian long-document retrieval diagnostic, and encoder latency and package measurements on a fixed-length Cultura RU Edu sentence including tokenization, pooling and normalization; the efficiency numbers cover the encoder, not a complete retrieval service.

The final sign-projected hyperparameters were selected by an exploratory Optuna sweep over the full RuMTEB mean, so the main score is selection-biased rather than a held-out estimate; the large contrastive runs are single-seed with no measured run-to-run noise; the data-construction and distillation blocks use different training budgets, and the continuous initializer and final sign-projected stage use different source mixtures, so those comparisons are diagnostic.