A new scoring method separates correct-but-wordy answers from those that invent facts.

Checking if artificial intelligence gives the right response to an open question is hard. An answer can be written in many ways, or fail in different ways, like leaving out details, contradicting facts, or adding made-up information. Older testing tools often miss these differences. To solve this, researchers built a new system that sorts answers into eight ranked levels. This separates wordy but accurate answers from answers with made-up facts. The team also created two new datasets and a testing tool called Context-Aware Precision, or CAP. CAP works by turning answers into statements and using logic models to check them in both directions against a reference. In tests checking if scores follow the ranked levels properly, CAP beat standard tools.

Why it matters

When AI gives free-form answers, we can't tell if they're mostly right or contain dangerous lies. This framework helps developers and users measure answer quality precisely, making AI more reliable for real-world use.

Questions answered