Language models give probability predictions that can be tricked into guaranteed profit.

People use language models to help make big choices that depend on predicting future events. Users often assume these predictions come from a sensible, logically consistent view of the world. To test this, researchers checked if model predictions fit together without contradicting each other. They used stock return events and tested them with a classic math idea called a Dutch bookDutch bookA set of bets that guarantees profit, revealing inconsistent probabilities.. If bets placed against a model's stated chances can guarantee a profit, the model's logic is broken. A computer program calculated the largest guaranteed profit possible to score this flaw. Because this method tests internal logic, it works without knowing what actually happened. The researchers found substantial flaws in the models' forecasts. Contradictions grew worse with complex logical links, and irrelevant details made them ten times worse.

Why it matters

People relying on language models for decisions like finances or safety might receive inconsistent advice. This could lead to poor choices if the model's probabilities aren't logically sound.

Questions answered