Original title: When AI Becomes Hard to Understand: Cognitive Demands in Real-World Human-AI Conversations
Yingcan Carol Wang, Iman Munire Bilal, Qamar Zaman · September 2026
People use artificial intelligence to help make major choices about health and money. Yet, researchers still do not fully know when an AI becomes hard to understand. To find out, a team studied over 84,000 real dialogues. They tracked confusion by seeing how often people had to ask for clarificationA follow-up question that shows the first answer was not fully understood. or repeat their prompts. They discovered that traits like reply length, readability, and vocabulary variety do not act alone. Instead, their effects depend on how they mix together. Using a rich vocabulary actually reduced repeated questions when the AI gave short answers. However, as replies grew longer, that helpful benefit faded. This means AI replies need a careful balance of features to stay clear and manageable.
Imagine you are giving a friend directions to a new café. A short text with a couple of distinctive landmarks makes the route easy to follow. But if the text is already long, adding more landmarks makes it harder, not easier, to remember. In this paper, the same thing happens with assistant responses: using more varied words helps when the response is short, but the benefit fades as the response grows longer.
People increasingly rely on AI for financial and health decisions, where a misunderstood answer can have real consequences. This work shows that there is no simple rule like 'shorter is better' or 'use simpler words' — the same wording can help in a short response and lose its benefit in a long one. That means designers must tailor response complexity to the user, the task, and the conversation.
The analysis identifies associations, not causal effects, so it doesn't show that changing response characteristics would reduce difficulty. It also doesn't specify how to set the complexity budget for a particular user, task, or interaction.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
When AI Becomes Hard to Understand: Cognitive Demands in Real-World Human-AI Conversations — Yingcan Carol Wang, Iman Munire Bilal, Qamar Zaman
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.