Original title: Beyond the Device: What the Evidence Actually Shows About Digital Mathematics Learning
Anifowose Olawale Saheed · August 2026
Schools spend heavily on educational technology, but past research shows confusing and conflicting results about whether it helps students learn mathematics. To make sense of this, researchers reviewed empirical studies published mainly between 2013 and 2026 across wealthy and low-resource settings. They examined how different tools and teaching methods actually work in classrooms. They found that technology gives a positive but modest boost to math achievement. How teachers use a tool matters more than the tool itself. Using devices for problem-solving and collaborative work helps students far more than simply replacing paper worksheets with digital screens. Furthermore, real learning benefits take time to appear. The largest, best-designed studies detect gains only in the second year of use, meaning shorter studies miss these improvements entirely. Finally, unguided access to generative artificial intelligenceComputer systems that can create new content, like text or answers, that look like a human made them. can artificially boost immediate performance while hiding a drop in actual student learning.
Imagine teaching someone to drive. For the first year, they make no progress and you think the lessons are worthless. Then in the second year, suddenly everything clicks and they pass the test. In this paper, the largest digital math studies only show benefits after two years, meaning most evaluations end before the improvement appears.
For decades, schools have spent billions on digital math tools hoping to boost achievement. But the evidence shows only a small average gain, and it only appears after two years and with proper teacher support. Worse, unguided use of generative AI can actually hide learning loss, threatening to undermine basic skills.
This paper is a narrative synthesis, not a new meta-analysis, so it cannot resolve disagreements between existing syntheses. The five findings are based on 'principal' evidence from 2013-2026, but the literature remains inconclusive—especially for generative AI, which is still emerging. The paper does not propose a specific instructional configuration; it argues that configuration matters, without naming which one works best.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Beyond the Device: What the Evidence Actually Shows About Digital Mathematics Learning — Anifowose Olawale Saheed — EdArXiv — doi:10.35542/osf.io/5dgtn_v1
Abstract licence: CC BY
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.