What changed

On 6 November, OpenRouter launched a new version of its Auto Router, which scored 65.2% on the AutoBench Router Benchmark. This is lower than the previous version, which scored 79.1% in October. The new router, along with two new routers (Conifer and dari), was tested on the same benchmark, with none surpassing the AutoBench Router.

What this means in practice

The new Auto Router, when set to high cost tier, scores 65.2%, which is below the previous version's 79.1%. The router's default and other settings score at or below random selection, with the new router agreeing with the previous version on only 1–12% of picks. The AutoBench Router, which is live and available, outperforms all other routers, with quality scores of 88.6%, Balanced at 87.8%, and Cost at 86.8%. The AutoBench Router is accessible via the Botscanner API and offers a $1 free credit for new accounts.

Limitations and availability

The new Auto Router, despite being rebuilt, does not outperform the previous version. It is available under the same slug (openrouter/auto), but its performance is notably lower. The AutoBench Router, which is the top-performing option, is live and accessible at botscanner.ai/router. The new routers, Conifer and dari, also show limited improvement over random selection, with Conifer scoring 74.8% and dari scoring 79.1% on smaller model pools. The AutoBench Router remains the best option for quality and cost efficiency.

Source facts for OpenRouter's New Auto Router Scores Lower Than Previous Version: None gets close to the AutoBench Router. On 6 November OpenRouter launched the next generation of Auto Router (openrouter/auto). Conifer and dari join the board and still lose to always calling one mid-priced model.
Original AINewsTrend explainer based on the cited source. This graphic summarizes source statements; it is not an independent benchmark.

Sources and further reading

  1. huggingface: OpenRouter's new Auto router is worse than the old one