Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System
Organizations: Independent Researcher
Abstract
Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but what that layer should compute is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, each posed on a ladder of rungs that differ only in how much of the computation is done for the model. Answering directly, none of eleven open-weight models gets more than 4% of sexagenary days or 1% of collisions right, and supplying the attributes of every pillar does not change the collisions. A complete list of computed relations is not enough either: it leaves Qwen3-32B at 1%, with relations listed for other pairs misattributed to the queried pair, whereas a query-filtered layer from follow-up runs, stating only the queried relations pillar by pillar, gives 92%. On chess positions, two predictions locked in advance hold: attributes do not help, and unqueried relations are again misattributed. Three proprietary reasoning models compute 68--95% of collisions unaided, while reasoning in smaller open models overrides supplied facts. A fact layer should compute the interactions and return the ones the question asks about.
Figures & tables
| Abbr. | Steps | What is asked | |
|---|---|---|---|
| StE | 10 | 1 | polarity, element of a stem |
| BrE | 12 | 1 | polarity, element of a branch |
| ERel | 45 | 2 | sheng / ke relation, acting side |
| TG | 100 | 3 | ten-god label vs. a day master |
| Hid | 12 | 1 | ordered hidden stems of a branch |
| NaY | 60 | 1 | NaYin of a pillar (free text) |
| No thinking | Thinking | |||||||||||||
| Macro | DP | TrTG | Macro | DP | TrTG | |||||||||
| Model | closed | table | tool | closed | table | closed | tool | closed | table | tool | closed | table | closed | tool |
| Majority class | 23 | 23 | 23 | 5 | 5 | 3 | 3 | 23 | 23 | 23 | 5 | 5 | 3 | 3 |
| Qwen3-0.6B | 16 | 23 | 55 | 0 | 2 | 0 | 3 | 15 | 26 | 56 | 2 | 2 | 0 | 5 |
| Qwen3-1.7B | 25 | 36 | 70 | 2 | 2 | 0 | 25 | 31 | 46 | 68 | 2 | 2 | 0 | 18 |
| Qwen3-4B | 31 | 43 | 93 | 3 | 0 | 0 | 98 | 52 | 76 | 94 | 2 | 9 | 0 | 67 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Think | StE | BrE | ERel | TG | Hid | NaY | StR | BrR | Sea | DP | Abs | TrTG | Macro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Majority | 10 | 17 | 11 | 10 | 8 | 3 | 80 | 54 | 20 | 5 | 49 | 3 | 23 | |
| Qwen3-0.6B | – | 80 | 50 | 11 | 10 | 0 | 2 | 0 | 0 | 20 | 0 | 18 | 0 | 16 |
| Qwen3-0.6B | ✓ | 50 | 33 | 24 | 6 | 0 | 2 | 11 | 8 | 27 | 2 | 18 | 0 | 15 |
| Qwen3-1.7B | – | 90 | 75 | 11 | 12 | 8 | 3 | 0 | 53 | 22 | 2 | 27 | 0 | 25 |
| Qwen3-1.7B | ✓ | 80 | 75 | 22 | 8 | 0 | 3 | 53 | 37 | 43 | 2 | 52 | 0 | 31 |
| Qwen3-4B | – | 100 | 75 | 27 | 21 | 8 | 7 | 56 | 0 | 27 | 3 | 51 | 0 | 31 |
| Model | Think | StE | BrE | ERel | TG | Hid | NaY | StR | BrR | Sea | DP | Abs | TrTG | Macro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Majority | 10 | 17 | 11 | 10 | 8 | 3 | 80 | 54 | 20 | 5 | 49 | 3 | 23 | |
| Qwen3-0.6B | – | 90 | 83 | 9 | 12 | 33 | 10 | 0 | 0 | 18 | 2 | 15 | 0 | 23 |
| Qwen3-0.6B | ✓ | 80 | 75 | 22 | 7 | 58 | 8 | 4 | 4 | 23 | 2 | 27 | 0 | 26 |
| Qwen3-1.7B | – | 100 | 92 | 13 | 16 | 58 | 27 | 67 | 1 | 25 | 2 | 35 | 0 | 36 |
| Qwen3-1.7B | ✓ | 100 | 83 | 58 | 14 | 67 | 22 | 80 | 36 | 53 | 2 | 32 | 0 | 46 |
| Qwen3-4B | – | 100 | 100 | 22 | 18 | 100 | 28 | 36 | 6 | 50 | 0 | 50 | 0 | 43 |
| Model | Think | StE | BrE | ERel | TG | Hid | NaY | StR | BrR | Sea | DP | Abs | TrTG | Macro |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Majority | 10 | 17 | 11 | 10 | 8 | 3 | 80 | 54 | 20 | 5 | 49 | 3 | 23 | |
| Qwen3-0.6B | – | 100 | 92 | 18 | 99 | 92 | 52 | 40 | 35 | 28 | 93 | 6 | 3 | 55 |
| Qwen3-0.6B | ✓ | 90 | 67 | 40 | 85 | 42 | 32 | 44 | 33 | 95 | 92 | 44 | 5 | 56 |
| Qwen3-1.7B | – | 100 | 92 | 36 | 100 | 100 | 100 | 20 | 96 | 43 | 95 | 28 | 25 | 70 |
| Qwen3-1.7B | ✓ | 100 | 100 | 71 | 100 | 75 | 100 | 20 | 65 | 50 | 83 | 28 | 18 | 68 |
| Qwen3-4B | – | 100 | 83 | 78 | 100 | 100 | 100 | 100 | 100 | 98 | 100 | 58 | 98 | 93 |
| Transit pillar given | Only the date given | |||||||||||
| Model | Think | closed | +attrs | +full | table | +rel. | closed | +pillar | +attrs | +full | table | +rel. |
| Qwen3-0.6B | – | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1 |
| Qwen3-1.7B | – | 0 | 0 | 0 | 0 | 42 | 0 | 0 | 0 | 0 | 0 | 32 |
| Qwen3-4B | – | 0 | 0 | 0 | 0 | 54 | 0 | 0 | 0 | 0 | 0 | 49 |
| Qwen3-8B | – | 0 | 0 | 1 | 0 | 81 | 1 | 1 | 0 | 0 | 0 | 78 |
| Qwen3-14B | – | 0 | 0 | 0 | 0 | 68 | 0 | 0 | 0 | 0 | 0 | 60 |
| Model | Th. | closed | table | +attrs | +full | natal | +rel. |
| Qwen3-0.6B | – | 0 | 0 | 0 | 0 | 0 | 0 |
| Qwen3-1.7B | – | 0 | 0 | 0 | 1 | 2 | 2 |
| Qwen3-4B | – | 0 | 0 | 0 | 21 | 26 | 71 |
| Qwen3-8B | – | 0 | 0 | 0 | 30 | 24 | 41 |
| Qwen3-14B | – | 0 | 0 | 0 | 32 | 45 | 42 |
| Qwen3-32B | – | 0 | 0 | 0 | 71 | 79 | 81 |
| 8k cap | 24k cap | ||||||
|---|---|---|---|---|---|---|---|
| Model | Cat. | Cond. | Acc. | Trunc. | Acc. | Trunc. | Med. tok. |
| Qwen3-32B | DP | closed | 2 | 0 | 2 | 0 | 288 |
| DP | table | 29 | 2 | 35 | 0 | 1,926 | |
| TrTG | closed | 0 | 0 | 0 | 0 | 695 | |
| TrTG | table | 3 | 0 | 5 | 0 | 1,102 | |
| Qwen3.6-35B-A3B | DP | closed | 2 | 55 | 6 | 1 | 8,227 |
| closed | table | tool | ||||
|---|---|---|---|---|---|---|
| Model | Th. | WD | DP | 1 | WD | WD |
| Qwen3-0.6B | – | 9 | 0 | 33 | 15 | 100 |
| ✓ | 11 | 2 | 35 | 13 | 100 | |
| Qwen3-1.7B | – | 20 | 2 | 35 | 11 | 100 |
| ✓ | 22 | 2 | 32 | 15 | 100 | |
| Qwen3-4B | – | 34 | 3 | 67 | 51 | 100 |
| Day pillar, closed-book | table | Weekday, closed-book | |||||||
| Model | Acc. | corr. anchor | corr. count | no anchor | wrong only | trunc. | corr. count | anchor prec. | formula |
| Qwen3-4B | 2 | 1 | 0 | 77 | 17 | 3 | 26 | 63 | 99 |
| Qwen3-8B | 1 | 0 | 0 | 71 | 25 | 3 | 51 | 75 | 97 |
| Qwen3-14B | 0 | 0 | 0 | 72 | 27 | 1 | 36 | 72 | 93 |
| Qwen3-32B | 2 | 0 | 0 | 82 | 16 | 0 | 39 | 77 | 99 |
| Qwen3.6-35B-A3B | 2 | 61 | 47 | 0 | 20 | 55 | 93 | 72 | 99 |
| P: closed | P: +attrs | D: closed | D: table | D: +attrs | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 8k | 24k | 8k | 24k | 8k | 24k | 8k | 24k | 8k | 24k | |
| Qwen3.6-35B-A3B | 74 (0) | 69 (0) | 64 (0) | 75 (0) | 1 (43) | 1 (1) | 28 (37) | 46 (0) | 72 (0) | 62 (0) |
| Qwen3.8-27B | 82 (13) | 96 (0) | 68 (17) | 94 (0) | 2 (79) | 89 (4) | 26 (58) | 100 (0) | 69 (20) | 95 (0) |
| Qwen3.8-Flash-Next | 91 (3) | 96 (0) | 79 (12) | 94 (0) | 9 (75) | 94 (0) | 62 (31) | 99 (0) | 88 (7) | 94 (0) |
| Transit pillar given | Only the date given | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | Th. | +full | natal | +rel. | +full | natal | +rel. | ||
| Qwen3-1.7B | – | 0 | 0 | 42 | 0.00 | 0 | 0 | 32 | 0.00 |
| Qwen3-4B | – | 0 | 0 | 54 | 0.00 | 0 | 1 | 49 | 0.03 |
| Qwen3-8B | – | 1 | 2 | 81 | 0.02 | 0 | 4 | 78 | 0.05 |
| Qwen3-14B | – | 0 | 5 | 68 | 0.07 | 0 | 6 | 60 | 0.10 |
| Qwen3-32B | – | 1 | 10 | 92 | 0.10 | 0 | 1 | 90 | 0.01 |
| Model | Err. | AUROC | Conf. err. | |
|---|---|---|---|---|
| Qwen3-0.6B | 91 | 1.15 | 0.62 | 17 |
| Qwen3-1.7B | 82 | 1.08 | 0.63 | 26 |
| Qwen3-4B | 78 | 0.87 | 0.76 | 34 |
| Qwen3-8B | 76 | 0.93 | 0.70 | 24 |
| Qwen3-14B | 63 | 0.66 | 0.79 | 32 |
| Qwen3-32B | 62 | 0.92 | 0.82 | 24 |
| Locked decision rule | Runs | |
|---|---|---|
| H1 | closed look-ups (StE, BrE, Hid) | 2/20 |
| H1 | BrR recall precision | 3/20 |
| H2 | rule 1: pooled look-up error cut , McNemar sig. | 10/18 |
| H2 | rule 2: DP and TrTG cut below look-up cut | 17/18 |
| H3 | tool in every category | 5/20 |
| H4 | closed DP error (no thinking) | 11/11 |
| Check | Cases | Result |
|---|---|---|
| B1 stem relations, all ordered pairs | 100 | pass |
| B2 branch relations, all ordered pairs | 144 | pass |
| B3 day pillar, every date 1900-01-01–2100-12-31 | 73,414 | pass |
| B4 ten god, all (day master, stem) pairs | 100 | pass |
| B5 collision items: gold relations, ten god, day pillar | 160 | pass |
| C1 coverage, every kind colour square (empty board) | 768 | pass |
| Residual | Context lines | Exact (%) | ||||
| Condition | depth | on | off | direct | thinking | |
| BaZi collisions | ||||||
| closed | 43.0 | 4.5 | 0.0 | 0.0 | 0.1 | 21.1 |
| table | 43.0 | 4.5 | 0.0 | 0.0 | 0.1 | 40.2 |
| +pillar | 41.0 | 3.0 | 1.0 | 0.0 | 0.7 | 38.4 |
| +attrs | 41.0 | 3.0 | 1.0 | 4.0 | 0.1 | 35.4 |
| Hugging Face repository | Revision | GPUs | Runs |
|---|---|---|---|
| Qwen/Qwen3-0.6B | c1899de289 | 1 | no-think, sampled, think |
| Qwen/Qwen3-1.7B | 70d244cc86 | 1 | no-think, sampled, think |
| Qwen/Qwen3-4B | 1cfa9a7208 | 1 | no-think, sampled, think |
| Qwen/Qwen3-8B | b968826d9c | 1 | no-think, sampled, think |
| Qwen/Qwen3-14B | 40c069824f | 1 | no-think, sampled, think |
| Qwen/Qwen3-32B | 9216db5781 | 1 | no-think, sampled, think |
Knowledge Augmentation for Language Models
Knowledge augmentation for language models adds external or newly learned information to improve factuality, reasoning, and domain performance.
See all 187 papers