What changed
Google AI & Infrastructure team announced Physical-Engineering's Last Exam (PLE), a new benchmark and open competition developed in collaboration with Mercor. PLE is designed to test frontier models' ability to complete tasks in mechanical, electrical, civil, and structural engineering.
What this means in practice
The release includes 29 public tasks, grading tools, and an open competition leaderboard at ple-bench.com. PLE is intended to address the gap in existing benchmarks that do not evaluate AI agents' ability to produce safe, code-compliant, and cost-effective designs under a licensed engineer's review and control. The benchmark includes 85 long-horizon tasks across four core disciplines: mechanical (30 tasks), electrical (27 tasks), civil (18 tasks), and structural (10 tasks) engineering.
Each task involves working inside a containerized project folder built on Mercor’s Archipelago environment, which mirrors what a practicing engineer receives on the job. The agent must reason through files such as PDFs, spreadsheets, emails, 2D CAD drawings, 3D BIM models, and document revisions, find governing building codes and physical limits, run calculations, and generate production-ready deliverables.
Limitations and availability
All frontier models tested on PLE scored below 50%, with the highest-performing model on its native harness reaching a mean score of 48.3%. The test results highlight a wide gap between current models and real-world engineering requirements. The benchmark was developed using two methods: expert-authored tasks (44 tasks) and co-authored tasks (41 tasks), with engineers validating and increasing the difficulty of every task assembled by the agent.
To participate, users can download the 29 public tasks, grading scripts, and evaluation container from the GitHub repository. They can also submit their own agent setup for evaluation on the private test set at ple-bench.com, with 13 Google models running free on the servers.
