ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring

Himanshu Tripathi; Charlottee Crowell; Kaley Newlin; Subash Neupane; Shahram Rahimi; Jason Keith

cs.HCFeb 20, 2026

ACE-TA: An Agentic Teaching Assistant for Grounded Q&A, Quiz Generation, and Code Tutoring

Authors: Himanshu Tripathi, Charlottee Crowell, Kaley Newlin, Subash Neupane, Shahram Rahimi, Jason Keith

Organizations: University of Alabama, Tuscaloosa, Alabama, USA · Brown University, Providence, Rhode Island, USA · Meharry Medical College, Nashville, Tennessee, USA · Iowa State University of Science and Technology, Ames, Iowa, USA

Abstract

We introduce ACE-TA, the Agentic Coding and Explanations Teaching Assistant framework, that autonomously routes conceptual queries drawn from programming course material to grounded Q&A, stepwise coding guidance, and automated quiz generation using pre-trained Large Language Models (LLMs). ACE-TA consists of three coordinated modules: a retrieval grounded conceptual Q&A system that provides precise, context-aligned explanations; a quiz generator that constructs adaptive, multi-topic assessments targeting higher-order understanding; and an interactive code tutor that guides students through step-by-step reasoning with sandboxed execution and iterative feedback.

Figures & tables

Figure 1: ACE-TA multi agent workflow for routed retrieval, quiz construction, and stepwise, feedback driven code tutoring.

Figure 2: Quantitative performance of the conceptual QA module on natural language queries, showing RAGAS (answer relevancy, context precision/recall), BERTScore (precision/recall/F1), and ROUGE (ROUGE1, ROUGE2, ROUGEL) metrics for retrieval and answer fidelity.

Figure 3: Depth of explanation ratings from three SMEs for Gemini 2.5 Pro and ACE-TA conceptual QA module across 20 questions, showing consistently higher depth scores for ACE-TA where higher values reflect more deeper explanations.

Figure 4: (A) Breadth of quiz subtopic coverage across Python topics, measured via penalised Shannon’s entropy. (B) Balance of subtopic coverage, measured via Pielou’s evenness.

Figure 5: Explanation adequacy ratings from three SMEs for quiz master explanations across 10 questions, showing SME specific means and medians with an overall mean of 3.57, indicating generally adequate but improvable explanatory quality.

Model	Output Match Rate			CodeBert (sim_max)			CodeBLEU (codebleu)
	Easy	Medium	Hard	Easy	Medium	Hard	Easy	Medium	Hard
Gemma-3-27B-IT (Q4_0)	1.000	1.000	1.000	0.993	0.994	0.990	0.500	0.544	0.411
Llama-4-Scout-17B-16E-Instruct	1.000	1.000	1.000	0.992	0.995	0.992	0.489	0.510	0.429
GPT-4o-Mini	0.980	1.000	0.960	0.993	0.994	0.992	0.465	0.516	0.409
DolphinCoder-StarCoder2-15B (Q8_0)	1.000	0.980	0.940	0.994	0.994	0.989	0.512	0.487	0.415
DeepSeek-R1-Qwen3-8B-BF16	1.000	0.980	0.940	0.994	0.995	0.989	0.557	0.589	0.404

Table 1: Evaluation of code-generation models across Output Match Rate, CodeBERT similarity, and CodeBLEU for easy, medium, and hard Python coding tasks. Values are color–coded into five ranges: 0.100–0.299 , 0.300–0.499 , 0.500–0.699 , 0.700–0.899 , 0.900–1.000 .

Figure 6: (A) Overall coding robustness with model size and quantization of 7 student LLMs measured by DWPM. (B) CodeBLEU vs. CodeBERT similarity across problems, coloured by difficulty level.

Figure 7: Step clarity ratings from three SMEs across 15 coding tutor questions grouped by easy medium and hard

Feature	Khanmigo	Pensieve Discuss	Jill Watson	Parrot	ACE-TA (Ours)
Guides Instead of Giving Answers	$\checkmark$	$\bullet$	$\bullet$	$\times$	$\checkmark$
Handles Coding Help	$\bullet$	$\checkmark$	$\times$	$\times$	$\checkmark$
Handles Theory Questions	$\checkmark$	$\bullet$	$\checkmark$	$\checkmark$	$\checkmark$
Generates Quizzes	$\bullet$	$\times$	$\times$	$\times$	$\checkmark$
Checks Quiz Answers	$\bullet$	$\times$	$\times$	$\times$	$\checkmark$
Uses Retrieval to Stay Grounded	$\bullet$	$\times$	$\checkmark$	$\checkmark$	$\checkmark$

Table 2: Feature comparison across LLM-based tutoring systems.

\checkmark

= full support,

\bullet

= partial support,

\times

= not supported.

Explore similar work

ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization
Apr 17, 2026Yixu Huang, Xinglei Yu, Zhongyu WeiExecution-Guided Code Generation Language Model Self-Improvement
School of Data Science Fudan University Shanghai, China
Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation
Apr 12, 2026Charles Koutcheme, Juho Leinonen, Arto HellasAI in Education Code Language Models
Aalto University, Espoo, Finland
CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation
Apr 5, 2026Xiaojing Duan, Frederick Nwanganga, Chaoli WangGenerative AI in Education Human-in-the-Loop AI
University of Notre Dame, Notre Dame IN 46556, USA

Apr 17, 2026J/K move · Enter open · S save

ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization

Yixu Huang, Xinglei Yu, Zhongyu Wei

School of Data Science Fudan University Shanghai, China

Execution-Guided Code Generation Language Model Self-Improvement Automated Test Generation LLM Code Generation

Open paper AskPDF arXiv

Large Language Models (LLMs) excel at code generation but remain heavily reliant on large-scale annotated solutions and verification-based supervision, which constrains scalability and hinders sustained self-improvement. Recent solver--verifier frameworks exploit program execution as an automatic supervision signal, but their effectiveness degrades as solvers become moderately strong: verifier-generated tests increasingly confirm semantic correctness rather than exposing the remaining failure modes. We propose ACE, a self-evolving code generation framework based on a solver--adversary architecture that prioritizes active failure discovery through execution-centric supervision. A single LLM alternates between generating candidate programs and producing adversarial unit test inputs optimized to induce execution-level failures, such as runtime errors, exceptions, or non-termination. Supervision is derived solely from execution outcomes: robust programs are selected for supervised fine-tuning, while adversarial tests are optimized via Kahneman--Tversky Optimization using execution-derived preferences. Notably, the entire training loop requires no ground-truth code or external reward models. Experiments on CodeContests, MBPP, and LiveCodeBench demonstrate that ACE consistently outperforms strong solver--verifier baselines, achieving 3--7% absolute gains in pass@1, with larger improvements on out-of-distribution benchmarks, while maintaining competitive or improved inference efficiency.

Abstract

Explore similar work

ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization

Teaching Language Models How to Code Like Learners: Conversational Serialization for Student Simulation

CODE-GEN: A Human-in-the-Loop RAG-Based Agentic AI System for Multiple-Choice Question Generation