Original title: AI literacy over tool design: a mixed-methods study of scaffolded versus unrestricted generative AI in programming education
Sepinoud Azimi · September 2026
Schools often try to control how students use artificial intelligence. Researchers tested whether designing a strictly limited AI tool actually helps people learn programming better than letting them use any AI they want. They split students into two groups. One group used everyday AI tools freely. The other group used an AI Study Coach. This coach never wrote code; it only gave step-by-step clues, capped how many clues a student could ask for, and asked for written reflections. Surprisingly, both groups got similar grades on their assignments. The custom coach gave learners more confidence, but they struggled to budget their clues. Meanwhile, unrestricted students liked their tools but worried about relying on them too much. Overall, tool design mattered less than self-control. Students who understood how AI works and set personal rules performed best. The study suggests teaching AI literacy directly instead of just restricting tools.
Imagine learning to cook by using a recipe app. One version of the app gives you step-by-step hints but never shows you the final dish, and limits how many hints you can use. Another version lets you search for any recipe and watch videos. Both groups of learners end up cooking equally well. But the ones who understand why ingredients react the way they do, and set their own rules for when to follow a recipe exactly, become the best cooks. In this paper, the recipe app is the AI tool, and the understanding of cooking chemistry is AI literacy.
For educators, this means that restricting or scaffolding AI tools may be less effective than teaching students how to use them wisely. Students who understand AI's workings and set their own rules perform better, regardless of tool design. The study suggests shifting from tool control to AI literacy as a core skill.
This is a pilot study with only 33 students in one master's-level course, so results may not generalize. The seven-week duration may not capture long-term effects. The study does not prove that AI literacy causes better performance, only that they are correlated. The scaffolded tool design is specific to this implementation, and other designs might yield different results.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
AI literacy over tool design: a mixed-methods study of scaffolded versus unrestricted generative AI in programming education — Sepinoud Azimi
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.