What changed
Hugging Face introduced a new retrieval and question-answering (RAG) approach using BM25 and Seb, which outperforms traditional 8B vector RAG methods in quality, speed, and cost. The method uses simple BM25 for fast candidate retrieval and Seb for semantic judgment, eliminating the need for expensive embedding models and vector databases.
What this means in practice
The new approach is designed for production search and RAG pipelines, offering significant cost and latency advantages. For example, indexing the entire King James Bible (31,102 verses) with BM25 takes just 77.0 ms wall time, compared to minutes of GPU/API embedding calls for 8B models. The BM25 + Seb pipeline achieves 100% accuracy on 160 Bible QA queries, with a median latency of 46.9 ms and a cost of $0.00396 per 1,000 queries. In contrast, the 8B embedding + reranker approach has a median latency of 280–390 ms and a cost of $0.060–$0.200 per 1,000 queries.
Limitations and availability
The BM25 + Seb approach is currently available as a production-ready solution, with metered cost and latency data provided by the Seb API. The method is suitable for static datasets and production search, with the potential to improve ranking quality in real-world scenarios. The supplied announcement does not specify the exact hardware requirements or licensing conditions for the Seb model. The approach is particularly effective for tasks where exact phrasing and subtle entity distinctions are important, such as Bible QA and product catalog searches.
