cs.CLNov 10, 2026

Compute the Interactions, Not the Attributes: What a Fact Layer Must Supply in the Chinese Sexagenary System

Authors: Huixue Han, Jiaying Liu

Organizations: Independent Researcher

Abstract

Language-model systems in rule-governed domains often pair the model with a deterministic fact layer, but what that layer should compute is rarely measured. We measure it in the Chinese sexagenary system underlying BaZi chart analysis, where every fact has one table-checkable value: 682 items over twelve fact categories and 160 transit--natal collision items, each posed on a ladder of rungs that differ only in how much of the computation is done for the model. Answering directly, none of eleven open-weight models gets more than 4% of sexagenary days or 1% of collisions right, and supplying the attributes of every pillar does not change the collisions. A complete list of computed relations is not enough either: it leaves Qwen3-32B at 1%, with relations listed for other pairs misattributed to the queried pair, whereas a query-filtered layer from follow-up runs, stating only the queried relations pillar by pillar, gives 92%. On chess positions, two predictions locked in advance hold: attributes do not help, and unqueried relations are again misattributed. Three proprietary reasoning models compute 68--95% of collisions unaided, while reasoning in smaller open models overrides supplied facts. A fact layer should compute the interactions and return the ones the question asks about.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Knowing the Rules, Applying the Rules: Evaluating Language Models on Traditional Chinese Bazi

    Oct 5, 2026Jiulin Li, Ping HuangLLM EvaluationCultural Knowledge in Language Models