Why look at the evidence first
Most claims about AI in education come from vendors or from demonstrations. This ebook sets those aside and reads what has been published: randomized trials of AI tutors, a large teacher survey, an international policy review and adoption data. For each, we say what was measured, what was not, and how far the result can be carried to another school.
Two points shape everything that follows. First, the results are mixed: the same type of technology produced clear gains in one study and measurable harm in another. Second, nearly all of the evidence is short-term and none of the trials we cite took place in Nepal. That is a reason to be careful, not a reason to ignore the findings.
What AI in education actually means
In these studies, “AI” mostly means a large language model, the same kind of system behind general chatbots, used in one of three ways.
- As a tutor for students. The student asks questions or works through problems and gets hints, explanations or feedback.
- As a drafting tool for teachers. The teacher asks for a lesson plan, worksheet or adapted material, then reviews and edits it.
- As an administrative helper. Routine tasks and replies, such as answering common student questions, are handled first by software.
The studies show that the third and second uses are easier to measure and to control than the first. The first is where the largest gains and the largest risks both appear.
Three tutoring trials
Three published pieces of research give the clearest picture of AI tutoring. They are different in quality and in what they show.
| Study | Setting | What it reports |
|---|---|---|
| Harvard physics, Scientific Reports, 2025 | Randomized crossover trial; 194 students analysed (233 enrolled); two topics over two weeks | Median post-test score 4.5 with the AI tutor against 3.5 with an in-class active-learning lesson; the authors estimate an effect of about 0.63 standard deviations and report a median time with the tutor of 49 minutes |
| High school math, Turkey, PNAS, 2025 | Field experiment with nearly 1,000 students | Practice performance rose with AI access, but students who used a standard chatbot interface scored 17% lower on a later exam without it; a guardrailed tutor version largely avoided this |
| After-school pilot, Nigeria, World Bank, 2025 | Six-week program in Benin City with teacher support | The authors report gains of about 0.3 standard deviations; this is a blog summary of a pilot, not a peer-reviewed paper, and it gives no sample sizes |
The Harvard result is a narrow one. It covers one course, two topics and two weeks, with a high-achieving student group and expert instructors, and the authors do not claim AI tutoring will always beat active learning.
Why tutor design decided the result
The Harvard tutor was not an open chatbot. According to the paper, it used instructor-written, question-specific prompts, pre-written step-by-step solutions and a structure that guided students through each part of a problem. The authors note that a system prompt alone “could not reliably provide enough structure to scaffold problems with multiple parts.”
The Turkish experiment shows the other side. A tutor that hands over answers raises practice scores and lowers later performance. The PNAS authors describe unrestricted access as a “crutch.” The OECD Digital Education Outlook 2026 reviews the same experiment and reports that access improved short-term performance by 48% with the standard interface and 127% with the tutoring version, but that students performed 17% worse once access was removed.
The practical reading is that the presence of AI is not the variable that matters. What the tool does when a student is stuck, give an answer or ask a question, is.
Teachers: where the time savings are reported
The strongest evidence on teacher workload is a Gallup and Walton Family Foundation survey of 2,232 US public K-12 teachers, fielded between March 18 and April 11, 2025 and published June 24, 2025. It reports that 60% of teachers had used AI tools for work, 32% at least weekly and 28% monthly or less.
- Weekly users estimate they save 5.9 hours a week, which Gallup extends to about six weeks across a 37.4-week school year.
- The most common uses, at least monthly, were preparing to teach (37%), creating worksheets and activities (33%) and modifying materials for student needs (28%).
- Between 57% and 74% of teachers said AI improved the quality of their work at least somewhat, depending on the task, and fewer than 16% said it made it worse. Administrative work scored highest and grading and feedback lowest.
The OECD report adds some figures from other sources: in its 2024 TALIS survey, 37% of teachers used generative AI for work tasks, and it cites a 31% reduction in time spent on lesson and resource planning by secondary science teachers in England. The OECD also reports that seven in ten teachers on average believe AI could let students present others’ work as their own.
What the evidence does not show
- Long-term effects. The OECD notes that most evidence is short-term, and the World Bank team lists long-term effects as unknown.
- Student outcomes from teacher use. The Gallup survey measures teachers’ time and perceived quality. It does not show whether students learned more.
- Self-reported time savings. Gallup says teachers made the estimates themselves, task by task. They are perceptions, not timed measurements.
- Results outside the sampled settings. The Gallup survey covers the United States only, and the 31% planning saving applies to a specific group in England.
- Equity and privacy. The tutoring trials do not examine who lacks devices or connectivity, or what happens to the data students type.
Any school should therefore treat a headline number as a hypothesis to test locally, not as a promise.
Adoption and access: who actually uses AI
General usage data shows how uneven access still is. Pew Research Center (survey of 5,119 US adults, February 2026) found about half of US adults use AI chatbots and roughly one in four use them daily. Microsoft estimates, from aggregated telemetry, that 18.8% of people aged 15 to 64 worldwide used a generative AI product in Q2 2026: 28.8% in the Global North and 16.2% in the Global South. The Stanford 2026 AI Index reports that adoption correlates strongly with GDP per capita.
These three sources use different definitions and should not be compared with each other. We did not find a Nepal-specific figure in them. What they do suggest is that a school in South Asia should expect students and teachers with very different levels of experience, and should plan for connectivity and skills gaps. The OECD makes a similar point and calls for inclusive infrastructure with devices, connectivity, curriculum-aligned resources and professional development.
What this means for a school or training provider
Our own suggestions, not findings from the studies:
- Start with teacher-facing uses. Drafting lesson materials and handling routine administrative work carry less risk to student learning than open student access, and the time-saving evidence is closer to these tasks.
- If you pilot a student tutor, choose one that gives hints and steps instead of answers, keep a teacher in the loop, and limit it to one course or one group.
- Always test students without the tool. Both the Turkish trial and the OECD review show why practice scores alone can mislead.
- Write down your baseline before the pilot (time spent on planning, test results, student questions per week) so that you can tell afterwards whether anything changed.
- Decide before launch what student data the tool may collect, where it is stored and who can see it. Our AI governance checklist is a starting point.
For schools that want a safer rollout plan, see our companion guide, How to Introduce AI in a School or Training Program Safely. For education consultancies handling student inquiries, see AI for Education Consultancies. If you want to talk through a pilot, you can schedule a call.
Sources
- Kestin, Miller, Klales, Milbourne and Ponti, AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting, Scientific Reports, June 2025
- Harvard Gazette, Professor tailored AI tutor to physics course. Engagement doubled, September 5, 2024
- Bastani et al., Generative AI without guardrails can harm learning: Evidence from high school mathematics, Proceedings of the National Academy of Sciences, vol. 122, no. 26, 2025
- World Bank Blogs, From chalkboards to chatbots: Transforming learning in Nigeria, one prompt at a time, January 9, 2025
- Gallup and Walton Family Foundation, report on teacher AI use and time savings, June 24, 2025
- OECD, Digital Education Outlook 2026: Exploring effective uses of generative AI in education, February 3, 2026
- Pew Research Center, Americans and AI 2026, June 17, 2026
- Microsoft On the Issues, The continued state of global AI diffusion in 2026, September 21, 2026
- Stanford HAI, 2026 AI Index Report, 2026
Frequently asked questions
Does AI improve learning?
It depends on how the tool is designed. A Harvard randomized trial found a carefully built AI tutor beat an active-learning lesson on one short physics topic, while a PNAS field experiment found students who practised with an unguarded chatbot did worse once it was removed. The evidence does not support a general claim that AI improves learning.
Does AI save teachers time?
Teachers say it does. In a Gallup survey of 2,232 US public school teachers, those who use AI at least weekly estimated they save 5.9 hours a week. These are self-reported estimates and the survey did not measure student learning.
Can students be harmed by AI tools?
Yes, if the tool simply gives answers. In the PNAS study, students who practised with a standard chatbot interface scored 17% lower on a later test without it than students who never had access. A version with learning guardrails largely avoided that harm.
Is there evidence from Nepal?
We did not find a trial or teacher survey from Nepal in the sources used here. The studies come from the United States, Turkey and Nigeria, so any school in Nepal should treat them as a guide for a small local pilot, not as proof of local results.
What you get:
- What three tutoring trials found, and why tutor design decided the result
- What a large teacher survey reports on time savings and quality
- What the OECD review says about performance versus learning
- The limits of the evidence, including that no cited trial was run in Nepal
- Practical, clearly labelled suggestions for a first pilot