Microsoft CEO Satya Nadella has criticized AI labs like OpenAI and Anthropic for banning distillation, a process where smaller models learn from the outputs of larger ones. Nadella argues that these companies use public data for training while prohibiting the practice of distillation, which allows knowledge transfer between models. This, he says, creates an imbalance in the AI ecosystem, where economic value is concentrated with infrastructure providers rather than knowledge creators.
Nadella described the situation as 'ironic,' noting that while these companies train on public data under fair use, they also ban distillation, which could allow smaller models to benefit from the knowledge of larger ones. He further highlighted that companies pay for AI twice—once with money and again through the 'exhaust' of corrections, ratings, and usage data that reveal internal company knowledge. This data, he argues, can be used by AI providers to build competing products, giving them an unfair advantage.
In a blog post, Nadella emphasized that the issue is not just about data usage but also about control over the learning process. He pointed out that Microsoft has the infrastructure to help companies manage their own learning loops, offering an alternative to the current system.
Source: thedecoder
