Curriculum Brain: Constructing Curriculum Knowledge Graphs as a Substrate for Cognitive Diagnosis
Organizations: AI Ready School
Abstract
Cognitive Diagnostic Models (CDMs) identify which specific skills a student has and has not mastered, the signal a personalized learning path needs and a single aggregate score cannot give. Yet they are rarely deployed. The obstacle is their precondition: the Q-matrix, a mapping from every assessment item to the skills it requires, historically authored by hand. We separate the task into two stages: first construct the curriculum's own knowledge graph, the full space of concepts and skills it contains, independent of any item; then map items against that graph on demand. This paper addresses the first stage only. The item-mapping stage is designed but not implemented here, so the claim that this shifts judgment cost from once per item to once per curriculum is a design rationale rather than a finding. We present Curriculum Brain, a two-repository system pairing a version-controlled knowledge base with an agentic pipeline of eleven single-responsibility agents under a thin deterministic orchestrator. It generates candidate concept-skill mappings from official curriculum documents, checks them against accumulated rules, and compares them with a concept-skill map extracted independently from the textbook, repairing its own failures and escalating to a human only when it cannot resolve a case itself. Across 241 chapter runs (168 distinct chapters), 41.5% produced a Generator output passing both checks without a patch, and 67.6% resolved without escalation. Both are measured against criteria the system itself produced, so both describe internal consistency rather than agreement with an external standard, and both pool two pipeline configurations separated by a single change at run 77; after it the figures are 57.0% and 91.5%. Observed spend was $1.19 per chapter, API spend only, excluding human review. We release both the framework and the resulting curriculum dataset.
Figures & tables
| Agent | Description | Skills used |
|---|---|---|
| Map Extraction | Reads the chapter’s textbook PDF and extracts a concept-skill map, used later only to compare against the Generator’s output, not to produce it. | pdf_reader , kb_access , llm |
| Generator | Produces a candidate concept-skill CSV from the resolved prompt and the curriculum documents. | kb_access , llm , csv_utils |
| Eval | Runs Check 1 (rule compliance) and Check 2 (comparison with the concept-skill map) on a candidate in parallel, returning structured feedback rather than a pass/fail flag. Check 2 is blocking in gate mode and report-only otherwise. | kb_access , llm , csv_utils , diff |
| Doctor | Gate mode only. Surgically patches a candidate that passed Check 1 but failed Check 2, fixing only the coverage gaps. | llm , csv_utils |
| Rules Doctor | Surgically patches a candidate that passed Check 2 but failed Check 1, fixing only the flagged rule violations. | llm , csv_utils |
| Revision | Rewrites the generation prompt itself when a candidate fails both checks (or a single-check repair also fails), for the next attempt only. | llm |
| Skill | Description |
|---|---|
| llm | Shared wrapper around the Vercel AI Gateway (OpenAI-compatible) endpoint that every agent calls through. |
| kb_access | Sole owner of KB folder-structure knowledge; all knowledge base read/write operations. |
| csv_utils | Parses and validates the fixed-schema curriculum CSVs produced by the Generator Agent. |
| diff | Semantically diffs a generated CSV’s concepts/skills against the concept-skill-map for Eval Check 2. |
| pdf_reader | Extracts text from PDF files (textbook chapters, curriculum docs) via pdfplumber. |
| git_sync | Pulls/pushes the KB repo’s git state at the start/end of an orchestrator run. |
| Check 1 (rules) | Check 2 (coverage) | Next step | What the step changes |
|---|---|---|---|
| Pass | Pass | Selection (§ 4.4 ) | Nothing; the candidate joins the run’s pool of passing candidates |
| Pass | Fail | Doctor | Patches only the missing concepts and skills Eval listed; rows that already satisfy the rules are left untouched |
| Fail | Pass | Rules Doctor | Patches only the rule violations Eval flagged; coverage is left untouched |
| Fail | Fail | Revision | Rewrites the generation prompt for the current run; the Generator reruns against the new prompt |
| Subject | Grades | Chapters | Concepts |
|---|---|---|---|
| Mathematics | 1–10 | 128 | 1,620 |
| Environmental Science | 3–5 | 32 | 571 |
| Science | 6–10 | 63 | 1,578 |
| Total | 223 | 3,769 |
| Quantity | Count |
|---|---|
| Chapter CSV files | 223 |
| Concept–skill rows | 7,145 |
| Concept nodes (deduplicated) | 3,769 |
| Prerequisite edges, total | 7,448 |
| L1: within chapter | 4,272 |
| L2: cross-chapter, same grade and subject | 612 |
| Attempt | Runs confirmed |
|---|---|
| 1 | 21 |
| 2 | 69 |
| 3 | 42 |
| 4 | 20 |
| 5 | 7 |
| 6 | 4 |
| Agent | sonnet-4-6 | sonnet-5 | gpt-5.4-mini |
|---|---|---|---|
| Generator | n{=}61$ ) | n{=}165$ ) | n{=}12$ ) |
| Eval | , not reported | n{=}165$ ) | n{=}67$ ) |
| Doctor | , not reported | — | n{=}171$ ) |
| Rules Doctor | , not reported | — | n{=}118$ ) |
| Revision | n{=}5$ ) | — | n{=}210$ ) |
| Judge | — | — | n{=}5$ ) |