A recent study highlights a critical limitation in AI agents: their inability to perceive or track time. Researchers found that widely used coding assistants, including Anthropic's Claude Code and OpenAI's Codex, consistently overestimate the time required to complete tasks. In testing, these models often guessed around 90 minutes for tasks of varying difficulty, but their actual runtime varied significantly. The study, part of the MATS research program, used a collection of 200 tasks from ProgramBench and 18 custom benchmarks to evaluate the agents' time perception. The results showed that the models' estimates were far from accurate, with some predictions being up to ten times longer than the actual duration. The findings raise concerns about the reliability of AI agents in handling long-running tasks.
The study also revealed that the same AI model can behave differently depending on its setup. For instance, Claude Code continued working until it believed the task was complete, averaging about 90 minutes, while Codex stopped after roughly half an hour, regardless of the task. The researchers noted that the runtime varied significantly based on the software environment, or 'harness,' in which the models operated. Additionally, the agents were found to be unreliable in assessing the quality of their own work. Older models like Opus 4.8 and GPT-5.5 overrated their performance by an average of 20 percentage points, even when their actual results were far worse.
The researchers emphasize that the lack of time awareness poses challenges for AI agents handling long tasks. An agent that cannot accurately estimate time is difficult to control, especially when given instructions like 'iterate on this task for two hours.' The study also suggests that providing agents with tools to track elapsed time can improve their accuracy. The authors plan to test whether agents can adhere to a set duration when given such tools.
Source: thedecoder
