Original title: LLM-Assisted Grading for Object-Oriented Programming: A C++ Dataset and Evaluation Study
Adrian-Gabriel Diaconu, Alexandru Guzu, A. Ulmamei · September 2026
Grading computer science homework is hard, especially in object-oriented programming, where code must have good internal structure, not just the right final answer. Researchers wanted to see if AI could grade these assignments fairly. They tested four open-weight models on real student code using a single prompt. To work, an automated grader must follow directions, give accurate marks, stay consistent, and avoid junk scores. The models struggled. All four models graded much more strictly than the university system. One model completely broke down, giving a score of zero to all 109 submissions across three exams. Another model gave wildly different marks to the exact same homework when students were graded in a different batch. A third model got stuck in loops, wasting nearly thirty percent of its total energy producing nothing. The researchers warn that simple accuracy numbers hide these serious failures in real-world grading.
Imagine a teacher grading a stack of essays. Normally, you read each one and assign a score. Now suppose that teacher sometimes just writes '0' on every essay in a whole pile without reading them. That is what one model did for three entire exams. In this paper, the researchers tested AI graders and found that one model stopped grading entirely on 109 submissions, which is like a teacher who gives up and marks everyone the same without looking.
If you are a teacher hoping to use a local AI to grade programming assignments, this paper shows that most models are not ready. The best model still disagrees with the auto-grader by about 3 points on a 30-point scale, and one model gave every student a zero on three entire exams. Even worse, the same model can give the same submission wildly different grades on different runs. Until these problems are fixed, using an LLM as a grader could be unfair to students.
The dataset is limited to 271 C++ solutions from one university's OOP exams, so results may not generalize to other languages, problem types, or student populations. The reference grades come from an auto-grader, not human experts, so agreement with it is not the same as correctness. The study only tested four specific open-weight models under one prompt and one endpoint, so other models or prompts could perform differently. The negative result about declaration-level reconstruction means the auto-grader's internal logic is not fully transparent, limiting what can be learned from its grades.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
LLM-Assisted Grading for Object-Oriented Programming: A C++ Dataset and Evaluation Study — Adrian-Gabriel Diaconu, Alexandru Guzu, A. Ulmamei — Electronics — doi:10.3390/electronics15184331
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper.