We describe a modern deep learning system that automatically identifies informative contextual examples (\qu{contexts}) for first language vocabulary instruction for high school student. Our paper compares three modeling approaches: (i) an unsupervised similarity-based strategy using MPNet's uniformly contextualized embeddings, (ii) a supervised framework built on instruction-aware, fine-tuned Qwen3 embeddings with a nonlinear regression head and (iii) model (ii) plus handcrafted context features. We introduce a novel metric called the Retention Competency Curve to visualize trade-offs between the discarded proportion of good contexts and the \qu{good-to-bad} contexts ratio providing a compact, unified lens on model performance. Model (iii) delivers the most dramatic gains with performance of a good-to-bad ratio of 440 all while only throwing out 70% of the good contexts. In summary, we demonstrate that a modern embedding model on neural network architecture, when guided by human supervision, results in a low-cost large supply of near-perfect contexts for teaching vocabulary for a variety of target words.
Figures & tables
Fig. 1 : Qwen3 multi‑head causal‑attention mechanism [ 21 ] followed by mean‑pooling token vectors whose positions corresponding to a target word are arbitrary in this illustration.
Proximity metric threshold
P(Y<0)
P(Y∈[0,0.5))
P(Y≥1)
throwout rate
good-to-bad ratio
# accepted
0.205
0.0141
0.1371
0.4633
0.0000
32.7902
67807
0.215
0.0141
0.1371
0.4633
0.0000
32.7902
67807
⋮
0.835
0.0084
0.0847
0.5282
0.6832
62.9937
18844
0.845
0.0077
0.0815
0.5313
0.7138
69.1462
16920
0.855
0.0067
0.0786
0.5343
0.7449
79.3465
15000
TABLE I : Out of sample results for the unsupervised MPNET model (Section II-D1 ) for the [word unseen] regime.
y^0 threshold
P(Y<0)
P(Y∈[0,0.5))
P(Y≥1)
throwout rate
good-to-bad ratio
# accepted
-0.415
0.0141
0.1371
0.4633
0.0000
32.7902
67807
-0.405
0.0141
0.1371
0.4633
0.0000
32.7902
67807
⋮
0.845
0.0026
0.0272
0.6801
0.1936
258.4796
37248
0.855
0.0026
0.0261
0.6887
0.2090
267.1828
36082
0.865
0.0025
0.0246
0.6973
0.2232
280.4713
34993
TABLE II : Out of sample results for the supervised model (Section II-E ) for the [word unseen] regime.
y^0 threshold
P(Y<0)
P(Y∈[0,0.5))
P(Y≥1)
throwout rate
good-to-bad ratio
# accepted
-1.105
0.0141
0.1371
0.4633
0.0000
32.7902
67807
-1.095
0.0141
0.1371
0.4633
0.0000
32.7902
67807
⋮
0.845
0.0027
0.0309
0.6777
0.2103
250.5859
36606
0.855
0.0026
0.0298
0.6846
0.2249
259.0106
35565
0.865
0.0026
0.0285
0.6919
0.2408
265.0000
34472
TABLE III : Out of sample results for the supervised + handcrafted features model (Section II-F ) for the [word unseen] regime.
Fig. 2 : RCC plots for all three models with AUC metrics (see legend).
Model
RMSE
R2
Null model
0.40
0
RF (from our previous work)
0.36
0.18
Supervised
0.31
0.38
Supervised + Handcrafted Features
0.30
0.42
TABLE IV : Out of sample regression metrics comparison for supervised models herein and in our previous work for the [word unseen] regime.
Fig. 3 : An example image in which a student can learn the target word supercilious along with an associated caption such as: “Argyle held his crested head with such localized gravity that one might suspect the rest of the garden was simply revolving around him. As the common pheasants scurried for fallen seed, Argyle offered them nothing but the heavy-lidded appraisal of a bored monarch, his supercilious gaze suggesting that their very existence was a mere clerical error in mother nature’s ledger.”