AI Tutoring Research Splits as Maryland Trial Finds Lower Grades

Two trials, opposite results

Two of the most-cited AI tutoring experiments of the past two years point in opposite directions. The difference between them says less about the technology than about how students actually use it.

The first is the Harvard physics study that tutoring advocates often cite. Kestin and colleagues ran a randomized controlled trial with 194 undergraduates. Each student took one lesson in an active-learning classroom and another with a custom AI tutor called PS2 Pal2022. Median learning gains in the AI condition were more than double those in the classroom. Students also needed less time: the median AI session lasted 49 minutes, against a 60-minute class period21. Secondary coverage often puts the effect at 0.73 to 1.3 standard deviations, though those figures come from summaries of the paper rather than the paper itself1116. The comparison group matters. The tutor was measured against active learning, which is already considered best practice in physics teaching, not against a passive lecture20.

The second study arrived this fall and tells a different story. A University of Maryland team randomized access to a course-integrated generative AI tutor for 2,379 undergraduates across 30 instructors and several disciplines24. In the cleanest comparison, sections of the same course, students with access ended up with final grades 0.37 standard deviations lower. That works out to about four points on a 100-point scale14. Their participation on the course's learning management system dropped by 0.90 standard deviations14. First-generation students fared worst. Their estimated grade losses were more than twice those of continuing-generation peers14.

Commentary on the Maryland paper has been blunt, calling the result counterintuitive and a challenge to the optimism around AI tutoring19. The researchers themselves are more careful. They say the data cannot show that reduced engagement caused the lower grades, only that the two showed up together among students who had access14.

Design is the variable that matters

Read together, the two studies make the same argument from opposite sides. How a tool is built, and how students choose to use it, matters more than whether the tool exists at all.

The Maryland data shows students treating the tutor as an answer service. The authors report that student queries mostly asked for direct answers14. Uptake was also low. Only about 15% of students offered the tool used it, averaging roughly four sessions14. That makes the grade drop harder to explain. The researchers suggest that simply being assigned the tutor may have changed how students studied, or how much they used other course materials14.

The Harvard tutor, by contrast, was built on verified content, structured practice, timely feedback and self-pacing, and the authors credit that design for the gains20. The pacing data supports this. Students who found the live class too fast all spent longer than the median with the tutor, and students who found it too slow all finished sooner21.

Other research supports the same split. A widely cited high-school math experiment with nearly 1,000 students tested two GPT-4 setups. During practice, grades rose 48% with a general chatbot and 127% with a tutor that had teacher-designed hints and safeguards27. Once AI access was removed, though, the general-chatbot group scored 17% lower than students who never had it. The safeguards largely prevented that drop27. In UK secondary schools, Google's LearnLM ran inside a supervised tutoring platform. Expert tutors approved 76.4% of its draft messages with zero or minimal edits23. Students it tutored solved new problems on the next topic 66.2% of the time, compared with 60.7% for students tutored only by humans26.

The Maryland authors place their own study among several that found weak or negative effects. One semester-long trial of a retrieval-based chatbot with 500 marketing students found no significant effects on achievement or engagement14. A large NBER experiment with more than 6,000 Tennessee middle schoolers found that AI-coached students worked more slowly and attempted fewer questions. They were more accurate when they did answer, but the delayed-learning gain was only marginally significant27.

Here the reporting splits. Some summaries describe AI tutoring as settled science, citing large effect sizes and meta-analyses1618. The primary research is more mixed. One review of 2026 trials finds gains concentrated in structured subjects such as algebra and grammar. It also finds that AI tutors tie or slightly trail small-group human tutoring when both are properly staffed13. The fair conclusion is that well-designed tutoring with guardrails can work. Handing students a general chatbot inside a course may do harm, and the students hurt most are often those with the least support around them.

The integrity front: students push back on two sides

If tutoring research is about whether AI helps learning, the cheating debate is about whether campuses can trust any unsupervised work. Here students are pushing back in two ways at once.

Some are pushing back against surveillance and false accusations. The Los Angeles Times reported on a UCLA sociology course where students were told to set up a mirror showing their entire desk during online exams. In another course, students took oral video exams with their arms crossed or hands behind their heads1. A Los Angeles lawyer who defends students said AI accusations now make up about 35% of her firm's education cases. She described instructors who reported more than half a class at once1. Students have built their own defenses. Some write every paper in Google Docs so the version history proves their work. Others deliberately make their writing look sloppier so it won't seem machine-written1.

Purdue showed how quickly this can blow up. A computer science professor emailed more than 200 students accusing them of AI cheating and threatened an F to anyone who didn't respond. The email went out on the deadline to drop classes, and more than half the class withdrew10. Many students, including some who admitted using AI, said they weren't upset that cheaters might be caught. They objected to how it was handled. The university dropped the allegations and let students re-enroll10.

Other students are pushing for stricter limits on AI. At the University of Alaska Anchorage, AI cases now make up most of the conduct office's caseload. Of 150 students cited for misconduct, 119 involved AI7. Sophomore Brigitta Taaga, who sits in student government, objected that a draft AI policy didn't go far enough. She said she and fellow high school graduates had avoided colleges whose AI rules seemed weak7. Another student criticized the draft for ignoring privacy and proposed a required AI-literacy course7.

These two kinds of pushback have a common cause. Both groups want clear, consistent rules applied through proper procedures. New survey research supports that diagnosis. A study of 863 Vietnamese undergraduates found that students who saw AI rules as unclear and guidance as thin reported more AI-assisted cheating. Academic pressure showed no such link, although the full model explained only about 5% of the variation4.

Universities step back from detection

Institutions are slowly reaching the same view. Several U.S. universities are moving away from AI detectors. UW–Madison calls the tools unreliable and biased against non-native English speakers2. Cornell says detection is unlikely to be a workable solution5. Harvard College dean David Deming told students that everyone would benefit from leaving the detection business because it damages trust2. Faculty concern remains high, however: 95% of faculty in one national survey fear students will rely too heavily on AI5.

The likely replacement is assessment redesign. Universities are trying oral exams alongside written work5, and the University of Sydney uses a two-lane model that allows AI in some tasks and keeps others secure3. Some instructors allow AI for out-of-class work but base grades on in-class writing done without devices6.

What it adds up to

The idea that students are rescuing education from AI chaos goes too far. More accurately, students are reacting sensibly to the systems they've been handed. When a tool hands over answers, many take them, as the Maryland chat logs show. When rules are vague, cheating rises and accusations become arbitrary. When a tutor is built to make students work through problems, as at Harvard or with LearnLM, students learn more.

The lesson for schools buying AI tools is that access alone is not a policy. The next useful step is to judge tutors by how they're designed and to make integrity rules clear across a whole campus rather than leaving them to each instructor. That is what students on both sides of the integrity debate are asking for.