What changed
Hugging Face introduced MotionGraphicsBench, an expert-curated benchmark for evaluating AI models' ability to animate static vector graphics. The benchmark includes 100 tasks, each involving a static SVG and a motion edit prompt, with models required to animate the supplied elements. The evaluation was conducted on 32 proprietary and open-weight LLMs, with only four models exceeding a 50% pass rate at their highest reasoning-effort settings.
What this means in practice
The benchmark focuses on tasks that reflect real-world designer workflows, where a designer provides a static design and asks for motion to be added while preserving the design. Each task includes a motion edit prompt that describes the desired motion in terms of spatial and temporal relationships rather than exact keyframes. The verification programs used in the benchmark are deterministic and human-readable, allowing for detailed analysis of model performance. The benchmark's verification programs achieved 93.3% agreement with human judgments on a held-out set of 180 animations, compared to 83.3% for a VLM baseline.
Limitations and availability
The benchmark is available at https://huggingface.co/datasets/figma-ai/motion-graphics-bench, and the full paper is accessible at https://arxiv.org/abs/2611.10592. The tasks and animations can be explored at https://motion-graphics-bench.github.io/. The benchmark is designed to evaluate motion generation independently of graphic creation, addressing the limitations of previous benchmarks that did not measure these specific capabilities. The supplied announcement does not specify the exact hardware or license conditions required to use the benchmark.
