Hugging Face has released two compact multilingual retrieval models, bekko-embedding-v1-a8m and bekko-embedding-v1-a25m, designed for efficient document retrieval. These models, shortened to bekko-a8m and bekko-a25m, have 7.67M and 24.93M active parameters respectively, making them significantly smaller than many existing multilingual embedding models. The smaller model, bekko-a8m, achieves an average score of 56.2 on the MMTEB Multilingual v2 suite, outperforming models like BGE-M3 and multilingual-e5 models, despite having fewer active parameters. The larger model, bekko-a25m, scores even higher, reaching 57.5 on the 18 retrieval tasks of the same benchmark. Both models are available under the MIT license and can be tested via a web demo without any installation. Source: huggingface

The models are built by pruning a 22-layer mmBERT-small encoder to 4 layers (a8m) and 13 layers (a25m), then training them on about 1.1 billion multilingual pairs. They support over 100 languages and are cross-lingual by design, allowing a Japanese query to retrieve an English document without translation. Both models handle up to 8,192 tokens and offer embedding dimensions of 384, which can be truncated to 256, 128, or 64. Truncating the larger model to 256 dimensions results in a 1.3% drop in retrieval quality on HAKARI-Bench, while truncating to 128 dimensions causes a 6.2% drop. Source: huggingface

Hugging Face noted that the active parameters in a model are the weights used in the transformer computation for every token, excluding the token embedding lookup table. The total parameter count in multilingual models often includes the token embedding table for all languages, which is only looked up during inference. For example, multilingual-e5-small has 118M total parameters, of which 21.6M are active, while bekko-a8m has 106M total parameters, with 7.67M active. This distinction is crucial for understanding inference costs, as active parameters alone do not determine throughput, but smaller models generally run faster. Source: huggingface