cs.CLOct 29, 2026

AI Cosmos: An Agentic Infra for Situated Learning

Authors: AI Cosmos Team, Jinfeng Zhou, Yuxuan Chen, Fangming Chang, Mingyu Su, Yifan Zhu, Yuan Wang, Bo Zhang, 

Organizations: Tsinghua University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large language models (LLMs) can improve task performance during learning, yet these gains do not necessarily reflect learners' ability to thereafter apply the knowledge and skills independently. We present AI Cosmos, an agentic infrastructure for situated learning that organizes learning around two complementary principles: scenario-based participation and cumulative assets of practice. AI Cosmos organizes learners and role-specialized agents within learning scenarios that integrate task-relevant knowledge and, when needed, executable workbenches. An asset-memory loop preserves the processes and outcomes of practice as reusable assets with explicit provenance and dependencies. We evaluate scenario-based interaction through two preregistered randomized controlled studies examining the acquisition of legal reasoning and programming competence, holding the same underlying LLM across conditions. AI Cosmos achieves greater learning gains than a conventional LLM in legal reasoning and higher mastery of unseen programming problems than an LLM-as-Teacher baseline.

Figures & tables

Figure 1: Overview of the Scenario–Asset Framework. AI Cosmos shifts AI-assisted learning from request–artifact exchange to scenario-based participation, and from one-off outputs to cumulative assets of practice, supporting competence development through sustained practice.
Figure 2: Architecture of AI Cosmos. The scenario control plane configures learning environments, the task execution plane coordinates participant interactions and workbench operations, and the asset–memory substrate preserves and reuses validated assets across scenarios.
Figure 3: Reorganizing the interaction as a structured scene improves bar-exam learning over a vanilla assistant. Three arms ( n=37n=37 each) shared a single base model, differing only in interaction architecture. (a) Pre-test scores. Dots are individuals, markers denote mean ±\pm 95%95\% CI. The arms were equivalent at baseline ( F(2,108)=0.88F(2,108)=0.88 , p=0.42p=0.42 ), with AI Cosmos being numerically the lowest. (b) Immediate learning gain (post −- pre). Gains increased monotonically across arms of increasing interaction structure ( 15.7 ⁣→ ⁣18.5 ⁣→ ⁣22.815.7\!\to\!18.5\!\to\!22.8 pp; linear-trend contrast p=0.018p=0.018 ). The bracket highlights the a priori primary contrast, AI Cosmos vs. vanilla ( Δ=+7.0\Delta=+7.0 pp, Cohen’s d=0.56d=0.56 , bootstrap 95%95\% CI [0.10,1.05][0.10,1.05] ; p=0.018p=0.018 ); an ANCOVA adjusting for the pre-test reproduced this finding ( +4.5+4.5 pp, p=0.018p=0.018 , partial η2=0.053\eta^{2}=0.053 ). The LLM-as-Teacher arm did not differ significantly from vanilla ( p=0.12p=0.12 ), locating the performance effect in the architecture rather than the persona prompt. (c) Interaction dynamics. Each point is one participant, and ×\bm{\times} are group centroids. AI Cosmos elicited ∼\sim 1.7×1.7\times more dialogue turns ( 51.051.0 vs. 29.229.2 / 29.629.6 ; F(2,108)=21.8F(2,108)=21.8 , p<10−7p<10^{-7} , η2=0.29\eta^{2}=0.29 , g ⁣≈ ⁣1.2g\!\approx\!1.2 ) while requiring a statistically identical total input ( ∼\sim 7,1007{,}100 characters, p=0.88p=0.88 ). This yielded ∼40%\sim 40\% fewer characters per turn ( 162162 vs. 276276 ; p≈0.001p\approx 0.001 , g=−0.80g=-0.80 ), which is a behavioral signature of active participation replacing one-shot delivery. (d) Cumulative gain over each arm’s own baseline at three time points within the delayed-test subsample ( n=18n=18 per arm; mean, ±\pm , 95%95\% ,CI). AI Cosmos retained the largest net gain at one month ( +18.9+18.9 pp). Note: this subsample analysis is exploratory, and the omnibus was not significant ( p=0.62p=0.62 ).
Figure 4: The advantage concentrates in the applied scenario-based choice, rather than open-ended argumentation. Each test was split into objective (scenario-based choice) and subjective (open-response) sections, each renormalized to 00 – 100100 . (a) Learning gain by arm and section (bars indicate mean ±\pm 95%95\% CI; points indicate individuals). On the objective section gains were ordered 18.2<21.2<26.518.2<21.2<26.5 pp ( F(2,108)=2.89F(2,108)=2.89 , p=0.060p=0.060 , η2=0.051\eta^{2}=0.051 ; ANCOVA-adjusted AI Cosmos vs. vanilla p=0.038p=0.038 ). The subjective section improved equivalently across arms ( 5.65.6 – 7.87.8 pp; F(2,108)=0.23F(2,108)=0.23 , p=0.80p=0.80 ). (b) AI Cosmos −- baseline gain for each section against both controls (point estimate, 95%95\% bootstrap CI; grey indicates vs. vanilla; blue indicates vs. LLM-as-Teacher). On the objective section the advantage over vanilla excludes zero ( +8.3+8.3 pp [1.8,14.9][1.8,14.9] ) and over the teacher baseline is positive but unresolved ( +5.3+5.3 pp [−1.5,12.2][-1.5,12.2] ). On the subjective section, neither contrast significantly differs from zero ( +2.1+2.1 [−4.9,9.3][-4.9,9.3] ; −0.1-0.1 [−7.1,7.1][-7.1,7.1] ). The group ×\times section interaction in a mixed-design ANOVA was not significant ( F(2,108)=0.95F(2,108)=0.95 , p=0.39p=0.39 ). Thus, the localization is descriptive. Inter-rater reliability for subjective scoring was high (Cronbach’s α=0.95\alpha=0.95 – 0.960.96 ).
Figure 5: The advantage over the vanilla assistant is directionally consistent across learner subgroups. AI Cosmos −- baseline difference in immediate learning gain (Cohen’s dd , 95%95\% bootstrap CI) against a, the vanilla assistant and b, the LLM-as-Teacher , for the full sample (diamond; shaded band and dotted line mark the overall estimate) and four strata from median splits on prior knowledge (median =46.7=46.7 ) and AI literacy. Against vanilla, the effect is positive in every stratum and most precisely estimated overall ( d=0.56d=0.56 [0.10,1.05][0.10,1.05] ). Against the teacher baseline, it remains positive throughout but lacks statistical resolution at this sample size (overall d=0.34d=0.34 [−0.10,0.82][-0.10,0.82] ). Subgroup cells contain 1515 – 2222 participants per arm and are uncorrected for multiplicity.
Figure 6: AI Cosmos shows the largest learning gains across both aptitude strata. Participants were split at the median pre-test score ( 46.746.7 ) into (a) lower- and (b) higher-prior-knowledge strata . Translucent points are individual pre/post scores; filled markers are group means ±\pm 95%95\% CI. The slope of each connecting line is the mean learning gain (annotated). Within each stratum, the arms were statistically equivalent at pre-test ( p=0.92p=0.92 and p=0.39p=0.39 ), yet AI Cosmos gained most in both ( +28.4+28.4 vs. +26.0+26.0 / +21.8+21.8 pp, lower; +14.5+14.5 vs. +9.6+9.6 / +10.6+10.6 pp, higher). As expected given their greater headroom, lower-prior-knowledge learners gained more across all arms (aptitude main effect F(1,105)=46.8F(1,105)=46.8 , p<10−4p<10^{-4} ); the group ×\times aptitude interaction was not significant ( F(2,105)=0.55F(2,105)=0.55 , p=0.58p=0.58 ). Because the arms were baseline-equivalent within each stratum, the consistent advantage of AI Cosmos indicates the overall effect is not a statistical artifact of its slightly lower aggregate starting point.
Figure 7: Results of pre- and post-test. Three arms ( n=17n=17 – 1919 ) shared a single base model; both AI Cosmos and the teacher baseline withheld complete code, whereas the vanilla assistant was unconstrained. (a) Pre-test scores (dots represent individuals; markers denote mean ±\pm 95%95\% CI). The arms were statistically equivalent at baseline ( F(2,51)=0.12F(2,51)=0.12 , p=0.89p=0.89 ). (b) Post-test scores . AI Cosmos outperformed the teacher baseline by a near-significant margin ( Δ=18.2\Delta=18.2 points, g=0.56g=0.56 ; Welch p=0.092p=0.092 ) and performed comparably to the unconstrained vanilla assistant ( g=0.21g=0.21 , n.s.). The LLM-as-Teacher baseline was the weakest arm overall. Given the sample size, the omnibus test was not significant ( F(2,51)=1.65F(2,51)=1.65 , p=0.20p=0.20 ).
Figure 8: Post-test submission outcomes by arm. (a) Rates of scored submissions ( >0>0 ; pale bars) versus criterion mastery (grade ≥\geq G4; solid bars). AI Cosmos and the vanilla assistant earned partial credit at nearly identical rates ( 82%82\% vs. 85%85\% ) but diverged sharply on true mastery ( 55%55\% vs. 38%38\% ), explaining their converging mean scores despite disparate learning outcomes. (b) Proportion of scored submissions achieving genuine mastery. Among credit-earning submissions, 68%68\% under AI Cosmos reached mastery, compared to 45%45\% for vanilla and 32%32\% for the LLM-as-Teacher baseline (Fisher’s exact: AI Cosmos vs. LLM-as-Teacher p=0.009p=0.009 ; vs. vanilla p=0.12p=0.12 ). Equivalently, AI Cosmos roughly halved the rate of hollow submissions—code that secures points without actually implementing the target algorithm.
Figure 9: Criterion mastery and failure modes. (a) Proportion of learners achieving criterion mastery (grade ≥\geq G4) on 00 , 11 , or 22 post-test problems. AI Cosmos learners mastered both problems (darkest band) at more than double the rate of controls ( 36.8%36.8\% vs. 17.6%17.6\% vanilla, 11.1%11.1\% LLM-as-Teacher), significantly outperforming the LLM-as-Teacher baseline in per-student mastery count (Mann–Whitney p=0.024p=0.024 ). (b) Post-test failure-mode rates. AI Cosmos achieved the highest flawless-implementation rate ( EOK , 52.6%52.6\% ; Fisher vs. teacher p=0.019p=0.019 ) and the lowest incomplete-template rate ( E02 ). Brute-force degeneration ( E06 , i.e., exploiting test cases without using the intended algorithm) appeared only under the unconstrained vanilla arm ( 5.9%5.9\% ), never under either code-withholding condition.
Figure 10: Differential impact of the execution workbench across learner aptitudes. (a) AI Cosmos advantage over the LLM-as-Teacher baseline (Hedges’ gg , 95%95\% CI) on post-test scores and criterion mastery, by track. The advantage is significant for mastery among novices in the basic-algorithm track (Mann–Whitney p=0.039p=0.039 ). Conversely, the effect is small and statistically unresolved for the stronger data-structure track, despite the latter covering harder algorithms (arm  × \,\times\, track interaction p=0.34p=0.34 ). (b)&(c) Pre- to post-test trajectories by arm within each track (faint lines represent individuals; markers denote mean ±\pm 95%95\% CI; annotated values indicate mean change). In the basic-algorithm track, the LLM-as-Teacher baseline actually regressed from its pre-test level ( −22-22 points) while AI Cosmos achieved the highest gains. In the data-structure track, all arms improved, and performance gaps narrowed.
Figure 11: Within-arm correlation between prior aptitude (within-track pre-test zz -score) and criterion mastery. Lines indicate OLS fits, and points represent vertically jittered individuals. (a) Vanilla and (b) LLM-as-Teacher baselines exhibit no predictive relationship ( r=0.08r=0.08 and r=−0.06r=-0.06 , respectively; both n.s.). When an assistant can freely supply code, recorded outcomes decouple from actual learner ability. (c) AI Cosmos shows a strong, stable correlation ( r=0.59r=0.59 , p=0.008p=0.008 ; robust to leave-one-out resampling, r∈[0.53,0.65]r\in[0.53,0.65] ) that significantly differs from the LLM-as-Teacher baseline (Fisher’s zz , p=0.040p=0.040 ). Requiring learners to construct and repair algorithms restores construct validity, making the score a faithful measure of the person.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

 Vanilla LLMLLM-as-TeacherAI CosmosTest ( pp )
 ( n=37n=37 )( n=37n=37 )( n=37n=37 ) 
Female (%)73.073.081.181.186.586.5χ2, 0.34\chi^{2},\ 0.34
Prior legal training (%)24.324.327.027.021.621.6χ2, 0.86\chi^{2},\ 0.86
Age (years)23.2±2.123.2\pm 2.123.8±3.023.8\pm 3.023.0±2.023.0\pm 2.0F, 0.41F,\ 0.41
AI literacy (/170)119.7±15.0119.7\pm 15.0119.8±13.8119.8\pm 13.8116.1±14.3116.1\pm 14.3F, 0.44F,\ 0.44
Self-efficacy, GSES15.1±4.015.1\pm 4.015.1±4.315.1\pm 4.313.6±3.713.6\pm 3.7F, 0.27F,\ 0.27
Appendix
Table 1: Participant statistics by arm, with tests of equivalence. Values are presented as mean ±\pm s.d..
 Vanilla LLMLLM-as-TeacherAI CosmosTest ( pp )
 ( n=17n=17 )( n=18n=18 )( n=19n=19 ) 
Female (%)11.811.10.0χ2\chi^{2} , 0.31
Age (years)18.5 ±\pm 0.618.7 ±\pm 0.719.3 ±\pm 1.9FF , 0.20
First-year (%)94.188.984.2χ2\chi^{2} , 0.64
Basic- algorithm track (%)47.138.947.4χ2\chi^{2} , 0.85
Data-structure track (%)52.961.152.6χ2\chi^{2} , 0.85
Appendix
Table 2: Participant characteristics by arm, with tests of equivalence. Values are mean ±\pm s.d.. Pre-test scores are within-track comparable. The two tracks were assessed on different target algorithms. Pre-test problems below criterion reports the share of pre-test submissions an expert graded below G4 (not yet a substantially complete, correct implementation of the target algorithm).
GradeAnchorDescription
G1Target algorithm not formedImplementation direction is inconsistent with the target algorithm, or no valid algorithmic intent can be identified. A brute-force or non-target solution is graded here even when it earns partial credit.
G2Intent present, key errorsThe target algorithm is attempted, but with defects in complexity, boundary handling, memory safety, or a critical invariant.
G3Main flow present, details unstableThe target algorithm’s main framework exists but still contains key implementation errors.
G4Framework essentially completeOnly minor, non-fundamental defects remain. (Criterion-mastery threshold.)
G5Correct and completeThe target algorithm is implemented correctly and passes; corresponds to error code EOK .
Appendix
Table 3: The five-level mastery scale used by the expert re-graders. Each grade is anchored to how completely and correctly a submission realizes the problem’s target algorithm. The criterion-mastery threshold used in the main text is grade ≥\geq G4.
CodeNameDefinitionSeverity
E00Not submitted / incompleteNo valid submission, or no assessable result was produced.medium
E01Compilation / syntax errorCode does not compile, or the submitted file is not executable source.high
E02Incomplete skeletonThe basic structure of the target algorithm was never formed.high
E03Critical maintenance errorA core invariant or key maintenance step is wrong, yielding incorrect results.high
E04State / boundary errorThe main flow is present, but a conditional branch or boundary case is mishandled.medium
E05Complexity / termination errorComplexity is inadequate or the termination strategy is unstable, failing some test cases.high
Appendix
Table 4: Error codebook. Every submission is tagged [code:name] ; the code records the specific failure mode underlying the grade. “Severity” reflects how far the failure is from a correct target-algorithm implementation. Codes E01 – E07 and E00 are mutually informative failure modes; N01 / N02 are annotations rather than errors; sub-variant suffixes (e.g., -S0 / -S1 ) distinguish source-availability and intent sub-cases and are merged into their base code for the aggregate analysis.
More in this topic

AI in Education

AI in Education studies how artificial intelligence can support teaching, learning, assessment, and educational decision-making.

See all 264 papers

Explore similar work

CardsList

  1. LLM-as-Code: Agentic Programming for Agent Harness

    Jun 14, 2026Junjia Qi, Zichuan Fu, Jingtong Gao +4AI Agent ReliabilityLLM Agent Orchestration

    City University of Hong Kong · Tencent Jarvis Lab

  2. CASCADE: Case-Based Continual Adaptation for Large Language Models During Deployment

    May 5, 2026Siyuan Guo, Yali Du, Hechang Chen +2Continual Learning for LLM AgentsContinual Learning for LLMs

    aSchool of Artificial Intelligence, Jilin University. · bEngineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University. · cInternational Center of Future Science, Jilin University. +3

  3. Schema: Discovering Unknown Environments via Agentic Program Induction

    Sep 30, 2026Guanning Zeng, Jiani Wang, Wenjie Ma +8LLM Agent HarnessesLLM Agent Planning

    Carnegie Mellon University · Impossible Research · UC Berkeley