Classroom Final Exam: An Instructor-Tested Reasoning Benchmark

Chongyang Gao; Diji Yang; Shuyan Zhou; Xichen Yan; Luchuan Song; Shuo Li; Kezhen Chen

cs.AIFeb 23, 2026

Classroom Final Exam: An Instructor-Tested Reasoning Benchmark

Authors: Chongyang Gao, Diji Yang, Shuyan Zhou, Xichen Yan, Luchuan Song, Shuo Li, Kezhen Chen

Organizations: Northwestern University · UC Santa Cruz · Duke University · University of Birmingham ·

Abstract

We introduce CFE-Bench (Classroom Final Exam), a multimodal benchmark for evaluating the reasoning capabilities of large language models across more than 20 STEM domains. CFE-Bench is curated from repeatedly used, authentic university homework and exam problems, paired with reference solutions provided by course instructors. CFE-Bench remains challenging for frontier models: the newly released Gemini-3.1-pro-preview achieves 59.69% overall accuracy, while the second-best model, Gemini-3-flash-preview, reaches 55.46%, leaving substantial room for improvement. Beyond aggregate scores, we conduct a diagnostic analysis by decomposing instructor reference solutions into structured reasoning flows. We find that while frontier models often answer intermediate sub-questions correctly, they struggle to reliably derive and maintain correct intermediate states throughout multi-step solutions. We further observe that model-generated solutions typically contain more reasoning steps than instructor solutions, indicating lower step efficiency and a higher risk of error accumulation. Data and code are available at https://github.com/Analogy-AI/CFE_Bench.

Figures & tables

Figure 1: Representative examples from CFE-Bench and variable-based annotation. Top: example text-only and multimodal problems from CFE-Bench . Bottom: the structured annotation for the answer of the text-only example, including the variable name, type, semantic description, and ground-truth value.

Figure 2: Subject distribution of CFE-Bench by modality. We report the field breakdown for the text-only subset ( $305$ ; left) and the multimodal subset ( $144$ ; right).

Setting	Acc (%)	AUC	P	R	F1	FN	FP
S2S	$98.03\pm 0.77$	$0.98\pm 0.01$	$0.98\pm 0.01$	$0.98\pm 0.01$	$0.98\pm 0.01$	$4.75\pm 1.79$	$1.25\pm 0.83$
L2S	$96.72\pm 0.40$	$0.96\pm 0.01$	$0.97\pm 0.00$	$0.97\pm 0.00$	$0.97\pm 0.00$	$1.50\pm 0.50$	$8.50\pm 1.50$
L2L	$89.67\pm 0.75$	$0.89\pm 0.01$	$0.90\pm 0.01$	$0.90\pm 0.01$	$0.90\pm 0.01$	$11.00\pm 2.55$	$20.50\pm 1.12$

Table 1: Performance summary across evaluation settings.

Text subset ( $305$ )
	Model	Variable Accuracy	Question Accuracy
	Gemma-3-27B-it	13.79%	9.84%
	Ministral-3-14B-Reasoning	17.92%	13.11%
	Llama-4-Maverick	24.53%	19.67%
	GPT-oss-120b	41.15%	34.43%
	Qwen3-235B-Instruct	37.41%	32.46%

Table 2: Model performance on CFE-Bench. We report two complementary accuracy metrics for both the text-only and multimodal subsets. Variable Accuracy : For each question containing multiple annotated variables, we compute the proportion of correctly extracted variables, then average this proportion across all questions. Question Accuracy : The proportion of questions for which all variables are correct. The leftmost column uses color coding: green indicates open-weights models and orange indicates proprietary models. Bold underline indicates the best performance and underline indicates the second-best performance within each group (open-weights and proprietary).

Text + Multimodal ( $449$ )
	Model	Variable Accuracy	Question Accuracy
	Llama-4-Maverick	21.82%	16.48%
	Qwen3.5-397B	53.60%	47.44%
	Qwen3.5-plus	53.60%	47.00%
	claude-opus-4.6	54.15%	47.66%
	Grok-4-0709	47.78%	41.65%

Table 3: Combined performance on CFE-Bench (Text + Multimodal).

Figure 3: Unit Execution accuracy for text and multimodal subsets.

Figure 4: Sample-level diagnostics for text subset. The red curve shows unit execution accuracy. The other curves show final-answer accuracy under unit conditioning: Reasoning Prefix , Reasoning Prefix (Questions Only) , Single-Unit Injection , and Single-Unit Injection (Question Only) . Notably, although all curves share the same $y$ -axis scale, the red curve measures unit-level correctness, whereas the remaining curves measure final-answer correctness.

Figure 5: Sample-level diagnostics for multimodal subset.

Figure 6: Reasoning-flow length distribution. Histograms show the frequency of questions at each length, and dashed vertical lines indicate the corresponding mean lengths.

	Text subset ( $n=305$ )			Multimodal subset ( $n=144$ )
Statistic	Solved	Unsolved	Overall	Solved	Unsolved	Overall
# Questions	179	126	305	74	70	144
GT len. (mean $\pm$ std)	$10.60\pm 4.06$	$10.91\pm 4.07$	$10.73\pm 4.07$	$11.04\pm 3.39$	$12.44\pm 5.56$	$11.72\pm 4.63$
Resp. len. (mean $\pm$ std)	$11.91\pm 3.62$	$12.62\pm 4.34$	$12.20\pm 3.95$	$13.19\pm 4.13$	$14.57\pm 5.05$	$13.86\pm 4.65$

Table 4: Reasoning-flow length statistics by outcome. We report the lengths of reasoning flows for ground-truth (GT) and model-response (Resp.).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Question: Short-circuited parallel plate electrodes of area $A$ enclose a lossy dielectric of thickness $s$ with dielectric permittivity $\varepsilon$ and ohmic conductivity $\sigma$ . The lossy dielectric at time $t=0$ has a uniformly distributed free volume charge density $\rho_{0}$ . Neglect fringing field effects. What is the current $i(t)$ flowing through the short circuit?
Step	Question $u^{q}_{i}$	Answer $u^{a}_{i}$
$u^{q}_{1}$ / $u^{a}_{1}$	Write the expression for the total current density in a lossy dielectric in terms of $\sigma$ , $\varepsilon$ , and $E_{x}$ at the electrode ( $x=s$ ).	$\displaystyle\frac{i(t)}{A}=\sigma E_{x}\big\|_{x=s}+\varepsilon\frac{\partial E_{x}}{\partial t}\bigg\|_{x=s}$
$u^{q}_{2}$ / $u^{a}_{2}$	For a uniform free volume charge density $\rho_{f}(t)$ inside the slab of thickness $s$ , state the relation between $E_{x}(x{=}s)$ and $\rho_{f}(t)$ using Gauss’s law.	$\displaystyle E_{x}(x{=}s)=\frac{\rho_{f}(t)\,s}{2\varepsilon}$
$u^{q}_{3}$ / $u^{a}_{3}$	Differentiate the expression from Step 2 with respect to time to obtain $\partial E_{x}/\partial t$ at $x=s$ .	$\displaystyle\left.\frac{\partial E_{x}}{\partial t}\right\|_{x=s}=\frac{s}{2\varepsilon}\,\frac{\partial\rho_{f}}{\partial t}$
$u^{q}_{4}$ / $u^{a}_{4}$	Substitute the expressions for $E_{x}(x{=}s)$ and $\partial E_{x}/\partial t$ from Steps 2 and 3 into the current density expression from Step 1.	$\displaystyle\frac{i(t)}{A}=\sigma\,\frac{\rho_{f}\,s}{2\varepsilon}+\varepsilon\,\frac{s}{2\varepsilon}\,\frac{\partial\rho_{f}}{\partial t}$
$u^{q}_{5}$ / $u^{a}_{5}$	Simplify the expression from Step 4 algebraically.	$\displaystyle\frac{i(t)}{A}=\frac{\sigma\,s}{2\varepsilon}\,\rho_{f}+\frac{s}{2}\,\frac{\partial\rho_{f}}{\partial t}$

Appendix

Table 5: An example reasoning flow for a representative problem. Given a question, the reference solution is decomposed into an ordered sequence of atomic, verifiable reasoning units

R=[u_{1},u_{2},\dots,u_{n}]

. Each unit

u_{i}=\langle u_{i}^{q},u_{i}^{a}\rangle

pairs a sub-question

u_{i}^{q}

with its objectively checkable target answer

u_{i}^{a}

MLLM Reasoning

See all 27 papers

General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks
Apr 13, 2026Junlin Liu, Shengnan An, Shuang Zhou +10LLM Reasoning Language Model Generalization
University of Chinese Academy of Sciences · Meituan
RealMath-Eval: Why SOTA Judges Struggle with Real Human Reasoning
Jun 8, 2026Yiteng Mao, Kenan Xu, Yijia Lyu +3Mathematical Reasoning Benchmarks LLM Evaluation
University of Wisconsin–Madison · East China Normal University · New York University +2
Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks
Apr 21, 2026Jing Jin, Hao Liu, Yan Bai +9Reasoning Evaluation Multimodal Reasoning
1Tsinghua University · 3Central South University · 2Meituan Inc +3

Apr 13, 2026J/K move · Enter open · S save

General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks

Junlin Liu, Shengnan An, Shuang Zhou, Dan Ma+9

University of Chinese Academy of Sciences · Meituan

LLM Reasoning Language Model Generalization

Open paper AskPDF arXiv

Contemporary large language models (LLMs) have demonstrated remarkable reasoning capabilities, particularly in specialized domains like mathematics and physics. However, their ability to generalize these reasoning skills to more general and broader contexts--often termed general reasoning--remains under-explored. Unlike domain-specific reasoning, general reasoning relies less on expert knowledge but still presents formidable reasoning challenges, such as complex constraints, nested logical branches, and semantic interference. To address this gap, we introduce General365, a benchmark specifically designed to assess general reasoning in LLMs. By restricting background knowledge to a K-12 level, General365 explicitly decouples reasoning from specialized expertise. The benchmark comprises 365 seed problems and 1,095 variant problems across eight categories, ensuring both high difficulty and diversity. Evaluations across 26 leading LLMs reveal that even the top-performing model achieves only 62.8% accuracy, in stark contrast to the near-perfect performances of LLMs in math and physics benchmarks. These results suggest that the reasoning abilities of current LLMs are heavily domain-dependent, leaving significant room for improvement in broader applications. We envision General365 as a catalyst for advancing LLM reasoning beyond domain-specific tasks toward robust, general-purpose real-world scenarios. Code, Dataset, and Leaderboard: https://general365.github.io

Abstract

MLLM Reasoning

Explore similar work

General365: Benchmarking General Reasoning in Large Language Models Across Diverse and Challenging Tasks

RealMath-Eval: Why SOTA Judges Struggle with Real Human Reasoning

Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks