What changed
Apple researchers, including William L. Tong and Aryo Lotfi, proposed a new technique called weighted state souping to improve the performance of state space models (SSMs) on long, crowded contexts. This method allows models to learn query-dependent weights for combining cached states, resulting in significant improvements in long-context performance across various tasks such as document question answering and retrieval.
What this means in practice
The technique addresses the limitations of both state souping and vanilla SSMs, which previously struggled with long contexts. By adapting the method into a sparse variant, the researchers achieved near-constant encoding costs relative to context length while maintaining or improving performance. This could be particularly beneficial for applications involving large collections of documents, tools, and structured data, which are common in modern agentic workloads.
Limitations and availability
The supplied announcement does not specify the exact implementation details, hardware requirements, or licensing conditions for this method. The research was accepted at the Workshop on Personalized, Aligned, Long-term Memory for AI Systems (PALM) at NeurIPS 2026, but it remains unclear whether the method is available for use or further development. The paper is available for viewing and citing, but no details on deployment or integration are provided.
