Hugging Face’s chief policy officer, Irene Solaiman, said embedded AI evaluators are not a simple solution, highlighting reproducibility and standardization as critical issues. During a discussion at TechCrunch Disrupt last week, Solaiman emphasized that the methods behind AI model evaluations are complicated and still maturing.
Testing needs to come together into a more consistent science if the industry is to rely on them in the future. Solaiman noted that trust in current evaluations is a weighted term, with reproducibility being a major factor.
"Trust is such a weighted term," said Solaiman. "In terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation?"
Reproducibility in model evaluations has long remained one of the stickier aspects of benchmarking — and remains a topic of significant debate in the industry.
Outside of an individual test’s validity, there’s also the added question of how that test compares to other evaluations. "They are an evolving science — where we don’t know how to meaningfully compare them to each other," Solaiman said.
The idea of embedded evaluators — in which supposedly "neutral" third-party observers are placed inside AI organizations — has been popularized over the last several months, as AI labs grapple with a rash of safety incidents that have spurred concerns over the potential harms of their technology. Hugging Face was famously the target of one of those incidents.
Instead of active government regulation to identify and mitigate model safety concerns, labs have pushed for the third-party solution. However, doubts remain over how neutral or effective those evaluators will actually be.
Anthropic’s decision to hire Accenture as its first evaluator spurred a good amount of backlash, with some critics mocking the idea that the consulting firm was qualified to do the task at hand.
Evaluations are now moving from merely a test of the model itself to a test of the entire system around the model — including the harness and agent.
The harness — or the tool the user is using to access the model — specifically, has a lot to do with a model’s performance, and a weaker model with a better harness could easily outperform a frontier model, Solaiman shared.
Source: techcrunch
