Original title: Which inference is at risk? Assessment validity reasoning and generative AI
Joshua Weidlich · September 2026
Generative artificial intelligence has disrupted university grading because it makes it hard to know what a student's work actually proves about their skills. In this paper, researchers created a clear way to rethink tests and assignments instead of just trying to catch cheaters or block technology. Their process works step by step: teachers first name the exact skill they want to test and the rules for artificial intelligence use. Next, teachers identify specific threats to grading, adjust their assessment design, and gather local evidence to back up their results. The researchers found this structured approach gives educators a practical language to defend what student grades mean, helping universities move past generic tool-blocking toward thoughtful assignment redesign.
Imagine you are a coach trying to pick players for a team. You watch each person run a hundred meters and record their time. That time normally tells you how fast they can run—the core skill. But now everyone is allowed to use jet-powered sneakers during the race. Suddenly the time does not tell you about running ability; it tells you about equipment choice, sneaker quality, and who remembered to charge the battery. In this paper the assessment is the race, generative AI is the jet-powered sneakers, and the five threats are different ways the sneakers break what the race time can tell the coach.
Universities are scrambling to adapt assessments to generative AI, but most responses focus on catching or blocking AI use. This paper argues the real problem is deeper: AI changes what a student's work actually proves. Without rethinking the intended meaning of assessments, institutions risk making invalid judgments about student ability, which affects grades, degrees, and career paths.
As a conceptual article, this work does not provide empirical evidence that the proposed Validity Reasoning Matrix improves assessment validity in practice. The reasoning sequence and matrix are theoretical and await testing in real course and program contexts. The abstract does not specify how to operationalize threat diagnosis or weight proportionate design responses.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Which inference is at risk? Assessment validity reasoning and generative AI — Joshua Weidlich — Assessment & Evaluation in Higher Education — doi:10.1080/02602938.2026.2734795
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.