A safeguard can score well on tests yet still leave a deployed system dangerous.

Companies test safety tools on AI systems by counting how often they block harmful prompts. But that only shows how the safety tool handled the specific tests it saw. In the real world, an attacker can keep trying new tricks or find another path around the guard. The researchers looked at safety reports to see if these tools truly keep deployed systems safe. They found a major problem. If an attacker succeeds just once, the whole system is unsafe, and these successful attacks show up repeatedly in the records. However, a safety tool getting a great test score does not prove the system is safe. A high score only measures one local check, not what the rest of the system still allows. Truly safer AI requires protecting the entire deploymentdeploymentThe live, publicly accessible version of the system, as opposed to a test setting., not just chasing higher test scores.

Why it matters

Users, regulators, and safety teams rely on these numbers. A model provider can report low refusal and attack-success rates, but an attacker who adapts can still get harmful help. That means benchmark numbers can make a service look safer than it is, and safety decisions should rest on whether the whole deployed system still helps attackers.

Questions answered