Robots can now be judged by watching their performance videos, not just counting successes.

Testing robot software in the real world is slow, exhausting, and often unhelpful. People have to reset objects by hand, watch every test, and run the same task again and again. These tests can give conflicting rankings and only score robots with a simple pass or fail, missing how well the robot actually moved. To fix this, researchers built a system called R2S-Eval. First, it tunes a computer simulator to closely match the real-world setup. The robot then runs its tests inside this calibrated simulation, creating videos of its attempts without wearing down physical machines. Next, an artificial intelligence that understands sight and language watches these video clips, compares pairs of robot attempts, and ranks which behavior is better. The team found that R2S-Eval saves massive physical effort, matches human judgments, stays stable across tests, and spots movement quality differences that pass-fail scores completely miss.

Why it matters

This makes robot testing cheaper and more reliable by replacing repeated physical trials with simulated videos. Researchers and engineers get clearer insights into how well robots actually perform, beyond just pass or fail.

Questions answered