AI models often fail to spot robot task failures, even when trained specifically for it.

Robots need to know when they mess up a task, but can smart computer vision models tell if a robot succeeded or failed? Researchers created FailBench, a test set of robot videos, to find out. The models look at the visual evidence of what happened to judge the result. Surprisingly, even the best model achieved only 0.77 mean balanced accuracybalanced accuracyA performance metric that treats successes and failures equally, useful when one outcome is more common.. Models do well when an object clearly moves, but they drop to near-chance on tricky tasks involving close contact and assembly. When evidence is unclear, these systems systematically guess that the robot succeeded, even if given more reasoning effort. Finally, cropping the images to focus right on the action improved the top model by 2.4 percentage points.

Why it matters

Robots are used in homes and factories, but if their AI judges can't reliably detect when tasks go wrong, errors might be missed, affecting safety and trust. This study shows current models aren't dependable, pointing to a need for better failure detection.

Questions answered