Hugging Face has released mDenseOn and mLateOn, two open-source multilingual retrieval models with 307M parameters. These models are trained on a 2.8B-pair translate-train corpus, one of the largest open multilingual retrieval training sets to date. The models extend the open English data recipe used in DenseOn and LateOn to eight additional languages. According to Hugging Face, mLateOn achieves the best results on BEIR, target-language MIRACL, and MLDR benchmarks, while both models remain competitive in code retrieval. The results show that mLateOn generalizes better to languages and scripts not seen during retrieval training. The company also translated fine-tuning datasets and augmented them with organic and code data, creating 16.3M samples across nine natural languages and code. Everything, including models, datasets, and training code, is open-source. Source: huggingface
Hugging Face previously released DenseOn and LateOn, which set new state-of-the-art results on BEIR benchmarks with 149M parameters. The company emphasized that closed training data is the main bottleneck, not model architecture or scale, and that an open, curated data recipe can close the gap with private labs. The best example was the comparison between modernbert-embed-base, trained on open Nomic Embed data, and gte-modernbert-base, trained on closed GTE data. Both models used the same ModernBERT backbone, but gte-modernbert scored almost 2.5 points higher on average BEIR. Hugging Face's English release demonstrated that the gap can be closed openly, and decontamination experiments confirmed the gains came from genuine generalization rather than benchmark leakage. Source: huggingface
The company explained that the challenge with multilingual retrieval is the dominance of English in training data. Collecting large, clean, and diverse corpora for other languages would be expensive and result in uneven quality, especially since non-native speakers cannot reliably assess data in those languages. Instead, Hugging Face extended the validated English recipe through translate-train: machine-translating curated English data into eight target languages to create multilingual data and cross-lingual pairs. This approach led to the training of mDenseOn and mLateOn, two 307M-parameter models built on the mmBERT-base backbone. Source: huggingface
