Original title: Teachers' collaborative evaluation of AI-generated content: Insights from a professional development workshop
Idit Gat, Maya Usher, Miri Barak · September 2026
Generative AI tools can create school materials, but they might spread misconceptions or fuzzy logic. This poses a real problem for subjects like science that require exact thinking. Researchers wanted to see how science teachers judge AI-made content and what happens when they critique it together in a training workshop. Teachers evaluated open-ended science questions written by ChatGPT, looking at simpler questions alongside complex ones that push for deeper thought. Overall, teachers gave simpler recall questions much better scores, noting they matched the school curriculum far better. They viewed the complex questions as confusing and overly wordy. Even so, teachers admitted these harder questions had real value because they could spark deeper conceptual thought and analysis. In their reflections, teachers stressed that evaluating AI together helps them improve tests, talk through their subject, redefine their roles, and share teaching experience.
You are a chef tasting two soup samples from a new recipe book. One soup is simple and familiar, so you rate it highly. The other soup has unusual spices and flavors—you find it confusing and give it a lower score, even though it might be more interesting and complex. In this paper, the teachers are the chefs, the simple soup is the basic questions from the AI, and the complex soup is the deeper thinking questions. The recipe book is the AI that generated the content.
As schools adopt AI tools, teachers must judge AI-generated content for quality. This study shows that even experienced educators struggle with complex AI questions, revealing gaps between AI outputs and teaching goals. It underscores the need for professional development that trains teachers to critically evaluate AI, especially for tasks that demand deeper thinking.
The study is limited to 60 middle-school science teachers in a single workshop, so results may not generalize to other subjects, grade levels, or longer contexts. It only used ChatGPT for open-ended questions on thinking skills; other AI tools or question types might yield different patterns. The study does not measure whether these evaluations improve actual student learning or teaching practices.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Teachers' collaborative evaluation of AI-generated content: Insights from a professional development workshop — Idit Gat, Maya Usher, Miri Barak — EdArXiv — doi:10.35542/osf.io/fwa6n_v1
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.