AI Tutoring Study of 2,379 Students Finds Lower Final Grades

A trial that undercuts the AI tutoring pitch

In the past year, the most common argument for putting AI tutors in schools has gone like this: one-on-one tutoring is among the most effective things education research has measured, and AI could provide it to every student at close to no extra cost. A new randomized controlled trial challenges that argument directly. Researchers gave students in a large U.S. public university access to a generative AI tutor built into their courses, across multiple disciplines. The study included 2,379 undergraduates and 30 instructors. Students who had access to the tutor did not do better. In sections of the same course, they finished with final grades 0.37 standard deviations lower, and their participation on the learning management system dropped by 0.90 standard deviations.17

The working paper is dated October 2026. Its authors call it, as far as they know, the first large-scale randomized experiment on AI tutoring at a big U.S. public university.26 It lands in a body of research that has mostly been described as encouraging, which makes the result hard to dismiss. The fair conclusion is not that AI tutoring fails. It is that whether it helps depends far more on design and on how it is deployed than the 2026 marketing suggests.

What the university trial found

The numbers are bad for the tutor, and they are also odd. The paper estimates a drop in final grades of 0.27 to 0.37 standard deviations. The strongest evidence comes from the exact-match sample, where the gap was about 4 points on a 0–100 scale.26 Only about 15% of students offered the tool actually used it, and users averaged roughly four sessions. The authors say that low usage raises questions about how the harm happened, including whether simply being given the assistant changed how students studied.26

The clearest effects were on behavior. Students with access viewed course pages less, were active on fewer days, and did much less of the online work instructors had set up. In surveys, users said they asked instructors fewer questions and sent fewer emails.26 Students mostly used the assistant to look things up quickly. Requests for feedback on their own work were rare.26

The effects were not the same for everyone. Estimated grade losses for first-generation students were more than twice as large as for continuing-generation students. The clearest academic harm showed up in humanities courses.26 The authors also say their design cannot show that lower engagement caused the lower grades, only that the two happened together.26 That caution matters. Even so, a tool that pulls students away from the rest of the course and coincides with worse results, especially for first-generation students, should give pause to any university considering a campus-wide rollout.

The case for AI tutors is real, but narrow

The optimistic evidence holds up. It just covers less ground than it is often credited with. The best-known positive result is Kestin and colleagues' Harvard trial, published in Scientific Reports in June 2025. It compared a custom AI tutor with an in-class active-learning lesson in an undergraduate physics course with 194 students. Students using the AI tutor had median learning gains more than twice as large, spent less time on the material, and reported more engagement and motivation.11 The median time on task for the AI group was 49 minutes.11 The AI tutor was not competing against a dull lecture. Active learning is itself considered best practice in physics teaching.14

The Harvard team said the gains depended on building the tutor around research-based teaching practices.11 The new university paper makes the same distinction. It describes the Harvard result as evidence of what a purpose-built tutor can do, but notes that it is uncertain whether the finding carries over beyond one course at a highly selective university.26

In K-12, the strongest positive results come from setups where humans stay closely involved. In an exploratory trial with 165 UK secondary students on the Eedi math platform, expert tutors supervised Google's LearnLM model. They approved 76.4% of its draft messages with no edits or only tiny ones.13 Students helped by the supervised model did at least as well as students with human tutors on every outcome measured. They also solved the first problem in the next unit correctly 66.2% of the time, compared with 60.7% for students with human tutors.13 Reviewers found five factual errors in 3,617 drafted messages and no harmful content.13 The authors' own statistics are more cautious than the headlines: the credible interval for that transfer gain runs from slightly negative to about 12 points.13

Stanford's Tutor CoPilot study took a different approach. The AI coached human tutors instead of replacing them. In a trial with more than 700 tutors and 1,000 students from underserved communities, students whose tutors used the tool were 4 percentage points more likely to master math topics. Gains reached 9 points for students with lower-rated tutors, at a cost of about $20 per tutor per year.20 Researchers linked the improvement to better teaching moves: tutors using CoPilot were more likely to ask students to explain their thinking.22 An economics-education trial reached a similar conclusion, finding that structured AI tutoring combined with peer group work produced the largest gains of any condition.18

Where the research agrees: design decides

Read together, these studies are less contradictory than they look, and the explanation is fairly consistent. The OECD's Digital Education Outlook 2026 reported that students using general-purpose chatbots produced better work, but that advantage disappeared and sometimes reversed on exams taken without the tool. Tools built with a clear teaching purpose showed lasting gains.2 A widely cited example from Türkiye makes the point. GPT-4 raised short-term performance by 48% with a standard interface and by 127% with a tutoring-style interface. But once access was removed, students scored 17% worse than peers who had studied without AI.7

The AEFP Live Handbook's summary of K-12 research describes the same divide. AI tools often improve how students perform while using them, but those gains do not reliably show up as independent skill. Systems that prompt students to explain their reasoning are more likely to support lasting learning than systems that hand over finished answers.8 The new university paper places itself in this line of work. It cites studies where unrestricted chatbot use led to cognitive offloading and weaker retention, and others where guardrails reduced the harm.26

The university result may be the most useful finding in this debate, because it suggests that being "course-integrated" is not enough. A tool can be official, tied to the syllabus, and approved by instructors, and still function as a shortcut that crowds out the slower work of learning.

Where the coverage diverges

The gap between this research and the sales story is large. Industry blogs and statistics roundups describe the evidence as settled, repeating figures like "54% higher test scores" from AI-enhanced learning28 or presenting a ranked list of tutoring products as proof that AI tutors "do improve learning outcomes."21 One roundup cites a 2026 meta-analysis reporting a mean effect size of 0.67.24 These claims sit awkwardly next to independent reviews. A Stanford review of more than 800 papers found only 20 high-quality causal studies, and none of them tested student-facing AI tools in U.S. K-12 schools.8

Reporting on schools reflects that gap. NPR described districts trying chatbots, AI math tutors, and even a humanoid robot, with little evidence and no agreement on how to measure success. One district consultant said he could only measure perception.9 The Christian Science Monitor reported that financially strained districts are spending more on AI products even though most have not been tested in randomized trials against regular instruction. It also noted that Common Sense Media rated Google's Gemini for schools "high risk."4 In January, a Brookings Institution study drawing on interviews in 50 countries concluded that, for now, the risks of generative AI for children outweigh the benefits, warning of a cycle of dependence that weakens thinking.1

Not everyone who is cautious wants schools to pull back. Robin Lake of the Center on Reinventing Public Education has argued that shutting down experimentation carries its own risks, because students need to learn to use the technology well.9 One account of school deployments argues that equity will be the deciding issue: AI tutors could narrow achievement gaps or widen them.23 The university finding that first-generation students lost the most is an early warning on that point.26

What it means for the 2026–27 school year

Adoption is not waiting for the evidence. Microsoft's 2026 education report found that 92% of students and education leaders have used AI for school, while 77% of students said they had received no formal AI training.10 Common Sense Media found that 25% of U.S. teens who use AI for schoolwork submit its answers unchanged.5

The research suggests three practical points. First, "AI tutor" is not one kind of product. A Socratic tutor with human oversight and a convenient answer assistant can have opposite effects. Second, results should be judged on assessments taken without AI, not on homework completed with it. Third, schools and universities should look at what an AI tool replaces, including discussion, office hours, and questions to instructors, and not only at what it adds. The 2,379-student trial shows that adding an AI tutor to a course can make outcomes worse when those things are displaced.