cs.CLOct 16, 2026

Chinese Decision: A Family of Language Models for Single-Pass Probabilistic Decision-Making

Authors: Qiang Yan

Organizations: Institute of Computing Technology, Chinese Academy of Sciences

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Many language-based applications need a probability distribution over a small set of decisions rather than an open-ended response. We present Chinese Decision, a family of Chinese-language decision models that adapts pretrained causal language models to this interface. Each question is compiled into a shared instruction-and-candidate prompt; a single forward pass reads the final-position logits of the candidate letters and normalizes them over the valid options, without autoregressive answer generation. The interface covers Boolean judgments, categorical choices, and ordinal scores. We report three completed members built on Qwen3.5 0.8B, 2B, and 4B backbones, each trained for one epoch on the same 13.048 million decisions and released with a validation-fitted temperature and a Q8_0 quantization. On the full 307,900-decision CJ-Bench, the 4B model reaches 85.97% overall accuracy, 75.69% general-domain accuracy, and 84.00% average accuracy across medicine, law, and finance with a single checkpoint, above every reported baseline in each domain, at 1.00% expected calibration error. Between the 2B and 4B members, which share data, objective, and minimum batch, parent-clustered paired bootstrap intervals show gains in all four domains, largest in the two low-resource domains (+8.5 and +7.1 points), whereas the 2B gain over 0.8B in law is not statistically resolved. Analyses of the stored predictions further show that a single global temperature lowers pooled calibration error while worsening some domains, that larger members rank their own errors better for selective prediction, and that merging and quantization each change 0.4-1.1% of top-1 predictions while leaving aggregate metrics nearly unchanged. All comparisons rest on single training runs and imported baselines.

Figures & tables

Figure 1: One interface, three sizes. (a) A real customer message and the three questions of the repository’s example request, answered by the released 0.8B, 2B, and 4B models (Q8_0, CPU). Every question is one forward pass whose probability mass falls only on the valid candidate letters; the three sizes agree on urgency and department but not on the frustration level. (b) Accuracy of one released checkpoint per size on the full CJ-Bench, against the general model plus three specialists of Chinese-Jev and the hosted Jev model, whose values are reported rather than reproduced.
Table 1: The released family at a glance. Accuracy and ECE are full CJ-Bench results of the merged BF16 release at its fitted temperature; latency is the batch-one median over 25 benchmark inputs on a shared server (A800 GPU, BF16; 16 CPU threads, Q8_0).
Figure 2: The shared decision path , identical for all three sizes. A request is compiled by hmm-zh-v1 , the Qwen3.5 backbone with merged LoRA runs once, and only the logits of the valid candidate letters are normalized. The vocabulary bars are a schematic; the three candidate probabilities are the 4B model’s actual output for this question. The same distribution yields a Boolean probability, a categorical key, or an expected ordinal level.
Figure 3: A candidate tournament for the repository’s 41-intent customer-service example, run through the released 4B model (Q8_0, CPU). Each group is one ordinary choice question of at most 16 candidates; the two most probable candidates of each group (teal) advance to a six-candidate final. Four forward passes in total select damaged_item with probability 0.78 among the finalists.
Table 5
Table 4: Main results on the full CJ-Bench. One released checkpoint per size covers all four domains.
Table 5: Aggregate results of the six released files. BF16 rows are the merged releases; each Q8_0 row is its direct conversion.
Table 6: Released models by question type. Counts give the total and hard-target denominators.
Figure 4: Paired size effects with 95% parent-clustered bootstrap intervals (2,000 replicates, stratified by domain, same draws for every model). Filled points: the interval excludes zero. Every 2B →\to 4B effect is resolved; the 0.8B →\to 2B effect in law is not. Appendix Table 14 lists the values.
Figure 5: Full-benchmark trajectories of all 14 evaluated checkpoints , plotted against the share of the shared 13,048,000-decision epoch consumed rather than optimizer steps. All points are unmerged adapters at T=1T=1 ; lines connect measured checkpoints from one run per size and are not replicates.
StepGroupTop-1 agree (%)W →\to RR →\to WNet
0.8B →\to 2BAll89.6413,67310,608+3,065
 General83.666,7595,114+1,645
 Medical93.136,1244,833+1,291
 Legal75.28428389+39
 Finance79.31362272+90
2B →\to 4BAll90.7414,3178,640+5,677
Table 7: Paired top-1 transitions between released models of adjacent sizes on the full benchmark.
Table 8: Source-level changes from the 2B to the 4B release : the six largest gains and every source whose 95% paired interval is entirely negative.
Table 9: Temperature-only ablation on fixed logits : each final checkpoint at T=1T=1 versus the same logits divided by its validation-fitted temperature.
Figure 6: Calibration and ranking. Left: general-domain reliability of the released models (bins with fewer than 200 decisions omitted). Middle: risk–coverage curves over all hard targets. Right: domain ECE of the final checkpoints at T=1T=1 (red marks) and after dividing the same logits by the fitted temperature (bars).
Table 10: Selective prediction with the released models, ranking hard decisions by maximum candidate probability.
Table 11: Q8_0 against its BF16 source , paired over all 307,900 decisions at the same temperature.
Figure 7: Inference cost under one protocol : the same 25 benchmark inputs, three warm repeats each, batch size one, on a shared server (A800 80GB PCIe; Xeon Gold 5318Y with 16 threads). Left: median latency against host memory for every measured backend. Right: professional-mean accuracy against median latency for GPU BF16 (circles) and CPU Q8_0 (diamonds). Appendix Table 15 lists all values.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Table 12: Domain-by-type hard-target accuracy (%) of the released models.
Table 13: Every full-benchmark checkpoint evaluation of the three training runs.
Table 14: Accuracy with 95% parent-clustered bootstrap intervals and paired differences between adjacent sizes (percentage points); the values behind Figure 4 .
Table 15: Batch-one latency and host memory of the released models under one protocol; the values behind Figure 7 .
More in this topic

Language Modeling

Language modeling develops systems that learn patterns in text and other sequences to understand, predict, and generate language.

See all 519 papers

Explore similar work

CardsList

  1. Chinese-Jev: Bringing System One Model to Chinese-Language Tasks

    Sep 29, 2026Zexiao Wang, Zihao Zhang, Xudong Wang +5LLM Decision-MakingEfficient Language Model Inference

    Fudan University · National University of Singapore · University of Chinese Academy of Sciences

  2. Releasing the Inner Jev: A Simple Recipe for Open Decision Models

    Oct 12, 2026Jonathan Geuter, Andrew Cheng, David Alvarez-MelisLLM Decision-MakingLLM Fine-Tuning

    Harvard University · Kempner Institute

  3. Large Decision Models: Repurposing Large Language Models as Single-Pass Decision Makers

    Oct 11, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2LLM Decision-MakingEfficient Language Model Inference

    University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +2