WenSight-Decision-1 for Direct Decisions under E-commerce Rules

Mingwen Liu; Jun Sun; Sichen Wang

cs.CLOct 19, 2026

WenSight-Decision-1 for Direct Decisions under E-commerce Rules

Authors: Mingwen Liu, Jun Sun, Sichen Wang

Organizations: WenSight AI Lab · TooChuang

Abstract

E-commerce decisions require applying natural-language rules to the available case evidence. We introduce WenSight-Decision-1, a shared model that scores supplied options directly from rules and case information for six operational tasks, with examples spanning six languages. We fine-tune Qwen3.5-9B with reference options as supervision, using the same option scores for training and inference. On 1,440 synthetic operational cases reserved for final evaluation, accuracy rises from 61.60% for Qwen3.5-9B to 92.92% for WenSight-Decision-1 under identical inputs and scoring. Joint fine-tuning improves all six tasks, bringing heterogeneous operational judgments under a common decision interface.

Figures & tables

Panel of Figure 1 — Figure 1: A refund-review example and the direct option-scoring path.

Task	$N$	Accuracy (%)
Support routing	235	68.94 $\rightarrow$	87.23
Product matching	241	84.23 $\rightarrow$	96.68
Listing review	241	57.68 $\rightarrow$	95.02
Priority classification	241	49.79 $\rightarrow$	96.68
Next-step selection	241	54.77 $\rightarrow$	94.19
Action review	241	54.36 $\rightarrow$	87.55

Table 1: Final evaluation by task (A) and source (B). Accuracy pairs are Qwen3.5-9B

\rightarrow

WenSight-Decision-1 (%). Panel B includes 200 auxiliary ESCI cases in addition to the 1,440 operational cases.

Panel of Figure 2 — Figure 2: Paired behavioral tests. (a) Two edits of the refund case in Figure 1 ; all other facts remain fixed. Node labels give the expected answers. (b) Percentages and counts of pairs with both predictions correct, measured on separate development sets under the checkpoint-selection scoring configuration. Semantic predictions change in 7 of 719 paraphrase pairs (0.97%).

LLM Decision-Making

LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.

See all 242 papers

Releasing the Inner Jev: A Simple Recipe for Open Decision Models
Oct 12, 2026Jonathan Geuter, Andrew Cheng, David Alvarez-MelisLLM Decision-Making LLM Fine-Tuning
Harvard University · Kempner Institute
Decision Model Factory: Building Specialized Sovereign Decision Models
Oct 26, 2026Yuma IchikawaLLM Decision-Making Efficient Language Model Inference
Fujitsu Limited · RIKEN Center for AIP
StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams
Sep 29, 2026Jhen-Ke Lin, Chung Chun WangLLM Evaluation LLM Inference
National Yang Ming Chiao Tung University

Oct 12, 2026J/K move · Enter open · S save

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

Jonathan Geuter, Andrew Cheng, David Alvarez-Melis

Harvard University · Kempner Institute

LLM Decision-Making LLM Fine-Tuning Discrete Choice Modeling

Open paper AskPDF arXiv

LLM-based decision-making systems, also called System One models, have recently gained popularity, most notably through Jev (Almeida, 2026). Given a typed input state, a question, and a set of options, they return a probability for each option in a single forward pass, without text generation. We show that existing open instruction-tuned language models can achieve comparable performance without any additional training, by reading out answers from letters (A, B, C, etc.) in the model's output logits at the last token position. Furthermore, we show how a simple supervised fine-tuning recipe can improve their performance further. We evaluate four open instruction-tuned language models: Gemma 4 31B, Gemma 4 12B, Qwen3.6-27B, and Qwen3.5-4B. We fine-tune each using LoRA and a decision head initialized from a slice of the language model head. Furthermore, we introduce OpenDecisions, a novel 2,352-question validation dataset designed to reflect the task composition of JevBench (JevBench main- tainers, 2026) and which computes a Capability score based on accuracy and calibration metrics. Without fine-tuning, Gemma 4 31B scores 79.6 on OpenDecisions, compared to Jev's 80.1. After undergoing our fine-tuning recipe, it scores 84.4 on OpenDecisions and 81.5 on JevBench v1.7, outperforming Jev by 4.3 and 4.4 Capability points respectively and setting a new state-of-the-art on OpenDecisions. On a suite of public decision datasets, our models achieve accuracy comparable to Jev, with lower calibration error. Our results show that open language models already possess System One capabilities, and with simple fine-tuning they outperform Jev on a wide range of decision tasks. We release our models, serving code, and OpenDecisions.

Abstract

LLM Decision-Making

Explore similar work

Releasing the Inner Jev: A Simple Recipe for Open Decision Models

Decision Model Factory: Building Specialized Sovereign Decision Models

StreamDecisionBench: Evaluating Decisions in Force on Evolving Language Streams