Tokenization Challenges Hinder Multilingual LLMs
A 2026 study highlights tokenization's role in poor performance of low-resource language models, with 340+ languages struggling due to suboptimal segmentation.
ARTICLES BY MAYA CHEN
LATEST NEWS
-
-
HuggingFace Introduces Token-In, Token-Out for RL Training
HuggingFace outlines a method to maintain token integrity during reinforcement learning, ensuring gradients are computed on the exact tokens the model generated. This approach resolves issues with token mismatch during multi-turn interactions.
-
Anthropic Study Reveals Men Use AI Coding Agents More Than Twice as Often as Women in Social Science Research
Anthropic's study found men use AI coding agents more than twice as often as women in social science research, with economists leading at 39% and education researchers at 4%.
-
AI Search Agents Rely on Memory, Not Web Research
A study reveals leading AI search agents often confirm existing knowledge rather than research the web, with models like GPT-5.4 and Kimi-K2.6 scoring high on static benchmarks.
-
Review Paper Argues Code Is How AI Agents Think and Act
A new review paper by Meta, Stanford, and the University of Illinois Urbana-Champaign highlights code as the foundation for AI agents’ reasoning and actions, citing real-world examples like Claude Code and Codex.
-
Training AI Chatbots to Be Helpful Impairs Their Ability to Simulate Human Behavior
A large-scale study shows that making AI chatbots helpful reduces their ability to mimic human behavior, with the effect worsening across generations.
-
Terence Tao Says AI Could Enable Division of Labor in Math
Mathematician Terence Tao suggests AI could transform math research by enabling collaboration, as seen in industry and natural sciences. He argues AI can fill skill gaps in math teams, though challenges remain in verification.
-
Developers Refuse to Work Without AI Tools
In February 2026, METR found most developers would not work without AI, even for limited tasks, raising concerns about long-term code quality.