Career-support assistants must distinguish user claims from evidence of current ability and withhold judgments when evidence is insufficient. DrawOffer separates language-model observation extraction from deterministic inference over source-linked records to maintain uncertain ability estimates, experience assessments, and revisable issues. We evaluate 102 fictional personas with 580 scored checkpoints, a historical full-context and Mem0 comparison, and three auxiliary knowledge-tracing datasets. On 37 test personas, DrawOffer passes 169/210 checkpoints; checkpoint-defined issue precision and recall are 0.875 and 0.636. Across three development simulations, 87.0% of checkpoint verdicts agree, including persistent failures. A cache-only development ablation replacing answer-evidence requirements with confidence-only settlement reduces majority-vote passes from 306 to 293 of 370 and issue precision from 0.927 to 0.725. Equalizing positive source weights improves predictive metrics without changing total passes; removing decay gives mixed results. These findings support an exploratory systems contribution centered on evidence sufficiency. Developer-authored scenarios, unmatched baseline models, historical test exposure, and incomplete public artifacts limit generalization and independent reproduction.
Figures & tables
Metric
DrawOffer v3
Predecessor v2
Ability-band checks
.913
.957
Checkpoint-defined issue precision
.875
.818
Checkpoint-defined issue recall
.636
.273
Experience-assessment checks
.581
.613
Inactive-issue checks
.962
.975
Issue-state transitions
.125
.000
Table 1: Expanded scenario test results (37 personas, 210 checkpoints). Check rows report pass rates; precision/recall use the checkpoint definitions in Section 4.4. The predecessor is a business-system baseline.
Configuration
All passes
Test passes
Test AUC
DrawOffer
105/129
40/45
.8093
Full context
109/129
40/45
.7677
Mem0 adapter
107/129
38/45
.7433
Table 2: Historical comparison on 25 personas, including nine test personas. Different model configurations limit attribution.
Configuration
ASSISTments
EdNet
Junyi
Engine, no trained difficulty
.7010
.6521
.7397
Engine, skill difficulty
.7189
.6765
.7863
Engine, problem difficulty
.7676
.7577
.7903
BKT
.7173
.6730
.7868
PFA
.7213
.6783
.7903
IRT
.7623
.7626
.7768
Table 3: Local knowledge-tracing configurations; five-fold mean AUC.
Configuration
Passes /370
Delta
Exploratory 95% interval
Full
306
0
—
Equal positive reliability
306
0.00
[−1.08,1.09]
No decay
303
−.81
[−1.94,.28]
Confidence-only settlement
293
−3.51
[−6.05,−1.33]
Table 4: Development majority-vote ablation. Delta is ablation minus full pass rate, in percentage points. Intervals cluster by persona.
The Chinese University of Hong Kong Hong Kong, China · Macao Polytechnic University Macao, China · The Hong Kong Polytechnic University Hong Kong, China