cs.AISep 23, 2026
SaveStudentBench: AI and human tutoring yield equivalent GRE learning gains
Abstract
Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). To support future research, we open-source the de-identified data collected in our studies.
Figures & tables
Figure 1: Learning across GRE domains and tutors. A, Mean pre-to-post test score change within each GRE domain under different tutoring conditions: human tutoring, AI tutoring (results pooled across all AI tutors), or no tutoring control. Error bars depict 95% Student- confidence intervals. Diamonds mark the top performing model in each domain; the dotted line traces the best AI performance across domains. Question counts refer to each assessment (pre-test, post-test). B, Learning gains (post pre test scores) pooled across Quantitative and Verbal, with 95% intervals; denotes the number of sessions under each tutoring condition. Dashed guides indicate the confidence interval for the gain from human tutoring. The inset shows the adjusted AI human difference and 90% confidence interval within pooled-SD equivalence bounds ( ); the interval also lies within the stricter -SD bounds ( ). C, Learning gains for 12 AI tutors, adjusted for pre-test score and section, with 95% HC3 intervals. See Appendix C for learning gains (Table C.1 ) and AI tutor leaderboards for all seven domains (Section C.2 ).
Figure 2: Experimental design of the learning gain study. 1, Students completed either a 27-question Quantitative or Verbal pre -test, with eligibility and effort screening, and then spent five minutes reviewing incorrect answers. 2, Students were assigned to one of three tutoring conditions: AI tutoring (with one of 13 LLM-based tutors; Table A.3 ), human tutoring, or no tutoring control. 3, After tutoring, students completed a post -test with the same length and concepts as the pre-test.
Figure 3: Three evaluations of AI teaching. Rows are ordered by lesson-planning score. A, B, Expert preferences for lesson planning and practice-problem creation, fitted with separate centered Bradley–Terry models; score differences are log-odds. Ratings marked as ties are excluded. Planning uses 9,577 ratings across 381 pre-tests; practice design uses 5,265 across 380. Whiskers are pointwise 95% intervals clustered by pre-test. C, Mean prevalence of six fixed conversation indicators across 1,971 AI and 135 human transcripts, with 95% Student- intervals. The human reference is spoken tutoring; AI sessions use typed chat. Scores should be compared only within panels; blank cells mean not evaluated. Colors identify model families. See Appendices E and F for evaluation details.
Figure 4: Tutoring cost, AI reply time and student engagement. A, B, Mean learning gain versus (A) total AI inference cost per session and (B) median AI reply time. Blue points and dashed lines mark the AI Pareto frontier. The red star uses a $75 per hour human reference. C, Mean student-authored chat messages per session versus the median reply time. Shading shows the pointwise 95% confidence band around the fitted curve. See Appendix D for methods and section-specific results (Figures D.2 and D.3 ).
Figure 5: Cost per percentage point of learning gain. Cost per percentage point (pp) is the mean session cost divided by the mean learning gain. Whiskers show 95% confidence intervals. AI costs reflect inference; human tutoring uses a 1\timesp{<}.05$ ). See Figure D.1 for section-specific rankings of models.
Figure 6: Explaining learning gains. Dots show averages for ten similarly sized groups of sessions per GRE section, ordered by AI reply time ( A ), student messages ( B ) or correct practice ( C ). Values on the vertical axis and regression lines adjust for pre-test score, AI tutor and assessment form. Lines use individual sessions (1,137 Quantitative and 998 Verbal); shading shows pointwise 95% confidence intervals for the fitted means. Tests account for 276 exploratory comparisons (Table B.1 ). See Section 5.2 for definitions and Appendix D.2 for methods.
Appendix figures & tables33 assets
Supplementary material from the paper’s appendix.
Appendix
| Section | Condition | Sessions | Pre-test, mean (SD) | P Q | Q P |
| Quantitative | AI | 1,139 | 50.21 (21.16) | 580 | 559 |
| Quantitative | Human | 61 | 45.17 (18.23) | 27 | 34 |
| Quantitative | Control | 91 | 52.99 (21.22) | 45 | 46 |
| Verbal | AI | 1,000 | 46.35 (19.47) | 499 | 501 |
| Verbal | Human | 79 | 41.44 (15.43) | 57 | 22 |
| Verbal | Control | 99 | 47.29 (19.08) | 50 | 49 |
Table A.1: Starting scores and assessment-form order for the learning comparisons in Figure 1 and the study design in Section 2 . Both forms were used as pre-tests and post-tests; the counts show how often each order was used within each condition. Pre-test scores are percentages correct; parentheses give the sample standard deviation. P Q and Q P denote the pre-test and post-test form order. The human tutoring condition had lower mean pre-test scores in both sections by chance; assignment was random and independent of pre-test score.
| Primary exclusion reason | Quant | Verbal | Total |
| Eligibility and study-protocol checks | |||
| Pre-test score below eligibility range | 225 | 83 | 308 |
| Pre-test score above eligibility range | 266 | 38 | 304 |
| Insufficient assessment effort | 143 | 124 | 267 |
| Incomplete study | 139 | 34 | 173 |
| Assessment score unavailable | 87 | 51 | 138 |
Table A.2: Data-quality exclusions for the study in Section 2 and the learning results in Figure 1 . Each record has one primary reason within its screening stage. No record appears in both stages.
Figure A.1: Student tutoring interface. Lesson planning generates the concepts and time allocations across the top and the initial practice problems on the left. The live tutoring conversation appears on the right, with access to the student’s practice answers. See Figure 2 for the study design.
Figure A.2: Quantitative assessment interface. Pre-tests and post-tests use the same interface to measure unaided performance. The notice states that outside assistance is prohibited and tab switching is recorded. See Figure 1 for learning results.
Figure A.3: Human tutor reviewing a student’s pre-test. The tutor and student review a Quantitative question through screen sharing during a video call. See Figure 2 for the study design.
Figure A.4: Human tutoring by video call. The tutor shares and annotates a Quantitative Comparison problem, illustrating place values and repeating units-digit patterns. See Figure 2 for the study design.
| AI tutor | Endpoint | Quant | Verbal |
|---|---|---|---|
| Gemini 3.1 Pro (high) | gemini-3.1-pro-preview | 91 | 74 |
| Gemini 3.5 Flash (low) | gemini-3.5-flash | 97 | 80 |
| Gemini 3.6 Flash (low) | gemini-3.6-flash | 89 | 79 |
| Gemini 3.7 Flash (med) | gemini-3.7-flash | 87 | 92 |
| Gemma 4 31B (high) | gemma-4-31b-it | 87 | 72 |
| GPT-5.4 mini (off) | gpt-5.4-mini | 90 | 71 |
Table A.3: AI tutors with reasoning effort in parentheses, API endpoints, and Quantitative and Verbal sample sizes for the learning study. See Figure 1 for learning results.
| Component | Main-study system templates | Additional guidance in the detailed variants |
| GRE context | Brief content and format overview | Detailed test structure, pacing, scoring conventions and format-specific strategies |
| Diagnosis and sequencing | AI tutor identifies needs and orders concepts | Explicit question-by-question misconception diagnosis, response-time interpretations and prerequisite sequencing |
| Difficulty calibration | AI tutor selects appropriate concepts and practice | Instructions for interpreting easy versus hard mistakes and using supplied expert ratings |
| Practice progression | Generate practice with worked solutions | Vary surface forms and progress from easier to difficult GRE-level practice |
| Conversational teaching | Teach transferable methods; choose explanation and practice | Attempt before hints, guide students to identify mistakes, request explanations and predictions, and connect conceptual understanding to efficient GRE methods |
Table A.4: Guidance intentionally omitted from the study prompts in Section 2 . The more detailed Quantitative variants were evaluated separately. Both included student-specific pre-test context. Shared application messages still invited practice attempts and responses to students’ mistakes.
| Question | Reporting rule |
| Equivalence | TOST at with -SD bounds; is the larger of the two one-sided values. Each AI tutor is tested separately against human tutoring, without correction across tutors; all six passing tutors are highlighted. |
| Differences and associations | Two-sided tests, or the stated omnibus test. For families of related comparisons, we report the Holm-corrected value; the families and individual tests are specified below. |
Table B.1: Two statistical questions. The same notation is used throughout. See Section 2 for the learning-gain definition and Appendix B for model specifications and comparison families.
| Section | Contrast | Estimate | 95% CI | |
| Quantitative | AI control | 6.86 | [4.02,9.69] | |
| Quantitative | Human control | 6.27 | [1.72,10.82] | .007 |
| Verbal | AI control | 5.47 | [2.46,8.47] | |
| Verbal | Human control | 7.52 | [3.17,11.87] | |
| Combined | AI control | 6.15 | [4.08,8.21] | |
| Combined | Human control | 7.04 | [3.88,10.20] |
Table B.2: Learning-gain differences between each tutoring condition and control, after adjustment for starting score. These complement the mean gains in Figure 1 B . Students in the control condition reviewed their graded pre-test; positive estimates favor tutoring. Values are percentage points with 95% HC3 intervals.
| Original data | After exclusion | |||
| Result | Estimate | Estimate | ||
| Combined AI human gain | .015 | |||
| Quantitative AI human gain | .028 | .034 | ||
| Student messages per 10 seconds longer reply time | ||||
| Correct practice per 10 more student messages | 0.52 | .002 | 0.47 | .006 |
| Gain per 10 more correct practice problems | 5.62 | 5.77 | ||
Table B.3: Main learning results after excluding students who completed both sections. The first two rows report AI human learning gain and equivalence-test values using the same model and fixed margins as the pooled comparison in Section 3 . The final three rows report the Quantitative associations in Section 7 , using 1,137 sessions before and 1,061 after exclusion. Their values use Holm correction across the same 276 tests, applied separately in each cohort. Learning gains are in percentage points.
| AI tutor | ||||||
| Gemini 3.1 Pro (high) | 165 | 6 | 7 | 7.3% | ||
| Gemini 3.5 Flash (low) | 177 | 2 | 4 | 3.3% | ||
| Gemini 3.6 Flash (low) | 168 | 5 | 4 | 5.1% | ||
| Gemini 3.7 Flash (med) | 179 | 7 | 8 | 7.7% | ||
| Gemma 4 31B (high) | 159 | 3 | 7 | 5.9% | ||
| GPT-5.4 mini (off) | 161 | 1 | 8 | 5.3% |
Table B.4: Participation-only exclusions and learning-gain sensitivity by AI tutor. is the retained count; and are restored Quantitative and Verbal counts, with . The fraction is conditional on meeting the other recorded criteria and is not overall randomized attrition. and are changes in unadjusted mean learning gain after restoration (expanded minus retained, in percentage points). A dash denotes a section in which that tutor was not studied.
Figure C.1: Observed and adjusted learning gains across GRE sections. Panels A–B show Quantitative and C–D show Verbal, with all 13 AI tutors and both reference conditions in each section, extending Figure 1 C . Observed means have 95% Student- confidence intervals. To account for differences in starting scores (Table A.1 ), adjusted gains use the same pre-test score distribution for every tutor within a section: we average predicted post-test scores and subtract the section’s mean pre-test score. The regression includes a quadratic pre-test term and uses 95% HC3 intervals. Within each section, panels share the same horizontal scale and are ordered by adjusted gain. See Table A.3 for sample sizes.
Table C.1: Learning gains for the Combined comparison in Figure 1 C . Values are percentage points with 95% HC3 confidence intervals. The regression includes pre-test score, its quadratic term and section. is the number of study sessions. AI tutors are ordered by learning gain.
| Domain | Condition | Gain relative to control [95% CI] |
| Quantitative domains | ||
| Data analysis | AI | 5.7 [1.6, 9.9] |
| Human | 6.8 [0.2, 13.3] | |
| Geometry | AI | 3.1 [ 1.9, 8.2] |
| Human | 1.3 [ 7.3, 9.9] | |
| Arithmetic | AI | 6.2 [2.1, 10.2] |
Table C.2: Learning gains relative to control in all seven GRE domains. These estimates complement the mean domain gains in Figure 1 A and the complete AI tutor rankings in Figure C.2 (Section 3 ). Values are percentage-point differences from a regression with linear domain pre-test score, with pointwise 95% HC3 confidence intervals.
Figure C.2: Complete AI tutor rankings across the seven GRE domains. Bars show mean learning gains in percentage points within each domain for the students analyzed in Figure 1 A and Section 3 . The 13 AI tutors per domain are ordered independently within each domain; human and control results appear below the divider. Each horizontal axis starts one percentage point below its smallest mean, so bar lengths should not be compared across panels. See Table A.3 for sample sizes. The rankings are descriptive; see the domain-outcome data for intervals and all 105 estimates.
Figure C.3: AI tutoring gains relative to control across starting proficiency. All four pre-test score groups (Q1–Q4) are shown for Quantitative and Verbal. Points give the difference in learning gains between pooled AI and control from a regression with linear pre-test score within each group; whiskers are 95% HC3 confidence intervals. Q1–Q4 keep students with tied pre-test scores together. See Table B.1 for the reporting rules for intervals and tests. See Figure 1 B for overall learning gains.
Figure C.4: Quantitative differences among AI tutors vary jointly with subject matter and starting proficiency. All 13 AI tutors are shown across the four pre-test score groups in each Quantitative domain. Each cell gives the AI tutor’s learning gain relative to control in the same domain and score group, in percentage points. These estimates come from regressing post-test domain score on pre-test domain score and tutoring condition. Positive values favor AI tutoring; negative values favor control. Each AI cell contains 12–35 students, and the control groups contain 24, 22, 16, and 29 students, respectively. The title reports a separate test of whether differences among AI tutors vary jointly with domain and quartile, using CR1 uncertainty clustered by study session. The accompanying data include all 364 estimates by AI tutor, domain, and quartile, and their intervals, including Verbal. See Figure 1 C for combined tutor rankings and Figure C.3 for the pooled AI comparison.
Figure D.1: Section-specific inference cost per percentage point of learning gain. Panels show Quantitative and Verbal estimates corresponding to the pooled results in Figure 5 . The human tutor reference is 1\times$ reference; learning gains are in percentage points (pp).
Figure D.2: Learning gain versus tutoring cost and AI reply time by GRE section. Panels show Quantitative (left) and Verbal (right) frontiers corresponding to Figure 4 A, B . Cost and reply time axes are logarithmic. Blue points and dashed lines mark the AI Pareto frontier; gray points show other AI tutors. Red stars give the human reference at $75 per hour. See Section 5 and Appendix D.1 for cost and AI reply time definitions. Frontiers are descriptive and have no uncertainty intervals.
Figure D.3: Student engagement versus AI reply time. Panels show comparisons across AI tutors for Quantitative, Verbal and the 12 shared AI tutors, corresponding to Figure 4 C and Section 5 . For AI reply time, we first take the median duration of complete provider calls in each session, then the median across sessions for each AI tutor. Reply time axes are logarithmic; solid lines fit mean message counts against reply time. These are observational associations.
Figure D.4: The tutoring experience across AI tutors. Combined estimates of pace, dialogue, practice and student ratings for the 12 shared AI tutors. Fractions count how many of the other 11 tutors have significantly longer reply times or lower values on the other measures than the column leader ( ). The practice outcome is the percentage of distinct answered problems receiving final platform credit. See Section 7 for the process comparisons and Appendix D.1 for definitions and adjustment.
| Predictor | Scope | Spearman | |
| Cost | Quantitative | 13 | |
| Cost | Verbal | 13 | |
| Cost | Combined | 12 | |
| Reply time | Quantitative | 13 | |
| Reply time | Verbal | 13 | |
| Reply time | Combined | 12 |
Table D.1: Associations of cost, reply time and student engagement with mean learning gain across AI tutors, corresponding to Figures 4 A, B and D.3 . See Appendix D.1 for cost, reply time and engagement definitions. is the number of AI tutors.
| Section | Predictor / outcome | 95% CI | ||
| Quant. | Reply time / student messages | |||
| Quant. | Student messages / correct practice | |||
| Quant. | Correct practice / learning gain | |||
| Verbal | Reply time / student messages | |||
| Verbal | Student messages / correct practice | |||
| Verbal | Correct practice / learning gain |
Table D.2: Student engagement, correct practice and learning. Selected exploratory associations in Figure 6 , using 1,137 Quantitative, 998 Verbal and 2,135 Combined sessions. See Section 5.2 for the definition of correct practice. Coefficients are outcome changes per ten seconds, student messages or credited problems; gain is in percentage points. Intervals are pointwise 95% HC3 intervals; values follow Table B.1 .
| Criterion | What the reviewer evaluates |
|---|---|
| A. Relevant concepts | Choosing concepts that address the student’s pre-test errors. |
| B. Concept grouping | Teaching related topics together. |
| C. Concept prioritization | Placing high-impact concepts earlier. |
| D. Time allocation | Dividing time according to errors, difficulty and likely benefit. |
| E. Practice alignment | Matching practice to the student’s errors and skill level. |
| F. Appropriate difficulty | Choosing problems or concepts suitable for what the student knows. |
Table E.1: Eight shared criteria for expert review of teaching materials. Letters A–H correspond to Figure E.1 . Figure 3 A pools preference ratings for A–D and H (lesson planning); Figure 3 B pools E–G (practice-problem creation and design). See Section 4.1 for evaluation details.
| Evaluation | Ratings | Pre-tests | Original | Sensitivity |
| Lesson planning | 9,577 | 381 | 9/12 | 8/12 |
| Practice creation/design | 5,265 | 380 | 9/12 | 8/12 |
| All eight shared criteria | 14,842 | 381 | 9/12 | 8/12 |
Table E.2: Expert-ranking sensitivity to repeated reviewers. This checks the expert comparisons in Section 4.1 . Counts show how many version-specific AI tutor entries differ significantly from Opus 5 ( ). The original analysis clusters uncertainty by pre-test; the sensitivity clusters by reviewer and pre-test. Fitted scores are held fixed. Ratings marked as ties are excluded.
Figure E.1: Expert preference by teaching criterion and overall. A–H, the eight criteria in Table E.1 ; I–K, all eight fitted jointly for Quantitative, Verbal and both sections combined, respectively. These rankings expand Figure 3 A, B . Points show centered Bradley–Terry estimates; whiskers are pointwise 95% intervals clustered by student pre-test. For each panel, we report the number of student pre-tests and expert ratings. Ratings marked as ties are excluded. A, 371 pre-tests and 1,899 ratings; B, 370 pre-tests and 1,905 ratings; C, 372 pre-tests and 1,916 ratings; D, 376 pre-tests and 1,956 ratings; E, 377 pre-tests and 1,939 ratings; F, 365 pre-tests and 1,653 ratings; G, 366 pre-tests and 1,673 ratings; and H, 378 pre-tests and 1,901 ratings. Overall fits use 246, 135 and 381 pre-tests, with 8,206, 6,636 and 14,842 ratings, respectively. All panels share the combined AI tutor order; blank cells mean not evaluated. Scores are comparable only within panels.
Figure F.1: How tutors interact with students. Cells show the percentage of sessions matching each rule underlying the ranking in Figure 3 C . Dots show mean prevalence across the six indicators, with 95% Student- intervals across sessions; is the transcript count. The human reference uses spoken sessions, whereas AI sessions use typed chat. See Figure F.2 for Quantitative and Verbal results and Table F.1 for indicator definitions.
| Indicator | Detection rule |
|---|---|
| Scaffolding cues | A tutor response following a student message of at least two words contains evaluative language, or shares at least two substantive words with that message and contains a scaffolding cue. |
| Request for explanation | At least one tutor question contains a request for reasoning or explanation. |
| Interactive questions | At least four tutor messages contain a question mark or an invitation to respond. |
| Early attempt request | At least one of the first five tutor messages asks the student to try or attempt a problem. |
| Long solution after a reply | After a student message of at least two words, a tutor message of at least 140 words includes a solution or calculation cue. |
| Reasoning checks | At least two tutor questions contain a reasoning-check or strategy cue. |
Table F.1: Operational definitions for the six transcript indicators in Figures 3 C , F.1 and F.2 .
Figure F.2: Conversational pedagogy by GRE section. A, Quantitative. B, Verbal. Panels expand the Combined ranking in Figure 3 C using the six rules in Table F.1 . Cells give the percentage of sessions satisfying each rule; counts available transcripts. The right column shows the mean of the six indicators with 95% Student- intervals across sessions. Human tutoring is a spoken reference and AI tutoring uses typed chat.
Figure F.3: How often students disputed a practice answer. A–C show flagged problems per 100 distinct answered problems in Quantitative, Verbal and Combined. Whiskers are 95% session-bootstrap intervals across 1,139 Quantitative, 1,000 Verbal and 2,139 Combined sessions. Horizontal scales differ to show the smaller Verbal rates. Rows share the Combined ordering. The Sonnet row pools its section-specific versions in Combined. These student reports complement the expert practice rankings in Figure 3 B ; they record student disagreements with answers, not independently verified errors.
| Lesson plan / tutor | Opus 4.8 | Gemini 3.5 Flash | GPT-5.5 | All |
| Minimal / Minimal | 15.23 (9) | 17.78 (10) | 13.56 (62) | 14.27 (81) |
| Minimal / Expanded | 14.81 (27) | 8.50 (27) | 12.65 (36) | 12.06 (90) |
| Intermediate / Intermediate | 11.62 (29) | 2.06 (9) | 11.40 (26) | 10.19 (64) |
| Expanded / Minimal | 8.38 (19) | 2.65 (7) | 23.23 (11) | 11.71 (37) |
| Expanded / Expanded | 8.02 (18) | 11.85 (10) | 8.26 (26) | 8.85 (54) |
Table G.1: Prompt-selection evidence for the fixed-instruction design in Section 2 . All five Quantitative pilot settings are shown; these sessions are separate from the main learning results in Figure 1 . Cells give mean percentage-point gain, with the number of sessions in parentheses. The two terms in each row identify the lesson-plan and tutor prompts, respectively. AI tutor columns identify the deployed chat tutor; the lesson-plan generator differed in 41 sessions.
Figure G.1: Learning with minimal and expanded prompts. Supporting the prompt design in Section 2 , points show mean Quantitative learning gain with separate Student- 95% confidence intervals; counts on the right show the number of sessions. Both lesson-plan and tutor prompts are minimal for blue circles and expanded for orange diamonds. A, Three AI tutors in the exploratory prompt-selection pilot. This panel shows the two matched prompt settings; see Table G.1 for all five settings, including mixed and intermediate prompts. B, Two later AI tutors, comparing supplementary cohorts that used expanded prompts with main-study cohorts that used minimal prompts. Differences are descriptive: the pilot informed prompt selection, and the later prompt conditions were not concurrently randomized.