Input errors affect language models in ways output tests don't capture.

Language models face messy inputs like typos, scrambled words, and corrupted text. Usually, people only test robustness by checking the final generated text. The researchers tested six realistic and synthetic input changes to see how disturbances move through decoder-only models. They looked at three levels: final outputs, internal hidden states, and attention heads, which are components that help models focus on related words. They found that different input disturbances create unique internal patterns that output scores alone miss. These patterns are only partly consistent across models. Attention heads tied to copying were specifically linked to recovering information during token swaps and shuffling. Overall, judging a model's toughness using only one metric can be misleading, so researchers must check multiple internal levels.

Why it matters

If we only check what language models say, we might miss serious flaws. This matters because AI is used in important areas where hidden errors could have big consequences.

Questions answered