Original title: Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?
Nirav Patel, Emily Wenger, Christopher Buccafusco · September 2026
Courts often decide cases by asking if a person acted reasonably. Because reasonableness is vague and depends on context, people worry about using artificial intelligence to make legal decisions. To test this, researchers asked humans and twenty-six chatbots to answer questions across twenty-five legal reasonableness situations. They wanted to see if the computer models answer like everyday human jurors. The researchers found that chatbot answers generally track human responses, but important differences showed up. Chatbots give more uniform answers than humans do, and they sometimes treat flexible standards as fixed rules. The models also tend to favor the government and corporations more than human respondents do. Finally, the chatbots give answers that align closer to humans who are white, male, older, and more educated. The authors conclude that more systematic research is needed to confirm or reject these early findings.
Imagine a group of friends deciding whether someone's joke was appropriate. Normally, each friend considers the context—who said it, where, and to whom—and gives a slightly different opinion. Now suppose you asked a single person who always gives the same answer regardless of context, and that answer tends to side with authority figures and match the views of the oldest, wealthiest friend. In this paper, the chatbots are like that single person: they treat the flexible standard of reasonableness as a fixed rule, and their judgments are biased toward the government and toward the demographics that already hold more power.
If AI chatbots are ever used in legal decisions—as 'silicon jurors'—their tendency to favor the government and corporations, and to align with privileged demographics, could systematically disadvantage others. Even now, when researchers use AI to simulate human judgments, these biases could distort findings about what is 'reasonable.' The law's reliance on reasonableness means these biases could affect everything from negligence cases to police conduct.
The study is exploratory, with only 25 scenarios and 26 LLMs. The findings are suggestive but not definitive. The abstract does not specify the human sample size or demographics, so the comparison may not be generalizable. The biases observed may not hold across different legal domains or chatbot versions.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments? — Nirav Patel, Emily Wenger, Christopher Buccafusco
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.