Even with AI help, most document pages still leak personal information after redaction.

When people black out private information in pictures of documents, missing just one detail leaves the entire page unsafe. Real document images, like scans or screenshots, have messy layouts and visual noise that make this hard. Researchers built a test called LeakageBench to see if modern tools can safely redact these files. The system checks whether tools successfully find private data and prevent leaks across entire pages, rather than just grading individual words. They tested standard scanners, commercial detectors, and advanced visual models. They found that giving a model like GPT-5.5 extra coding tools helped it locate private items better. However, it still leaked sensitive information on 0.968 of the tested pages. Better detection tools still fail to make most documents safe.

Why it matters

Companies redacting PII from scans and screenshots often assume a page is safe if most identifiers are hidden. This benchmark shows that missing even one makes the entire document a privacy risk, affecting anyone whose data appears in these images.

Questions answered