Language models use dedicated attention heads to track groups of entities mentioned together.

When we speak, we use words like "they" to point back to multiple people or things mentioned earlier. This is called plural reference, and large language models, or LLMs, must learn how to do it. Researchers wanted to know how these models track and link these items. They looked inside LLMs using methods that test how internal parts cause specific decisions. They discovered a clear chain of specialized components called attention heads. First, certain heads capture reference clues from the text. Next, other heads group the right items together into a plural set. Then, they pass this data along to parts that select the final pronoun. Finally, models match human habits: they prefer plural pronouns when grouped items are similar and joined by "and".

Why it matters

Coreference resolution is key for AI to understand context in text. Improving how models handle plural references can make virtual assistants and translation tools more accurate and natural.

Questions answered