cs.CLOct 10, 2026
SaveIs a 9B Decision Foundation Model Needed for Symptom-to-Disease Classification? Fine-Tuning Clef-Flash Against TF-IDF Baselines
Organizations: Department of Computer Science and Engineering, United International University
Abstract
Disease classification from patient-reported symptom descriptions is usually approached with lightweight discriminative models or with autoregressive large language models (LLMs), which understand language broadly but generate text token by token and require output parsing. We study decision foundation models, which score a closed set of candidate labels in a single forward pass without generating text. We adapt Clef-Flash, a 9-billion parameter decision model built on a Qwen3.5 backbone, to the 24-class Symptom2Disease benchmark using 4-bit quantized low-rank adaptation (QLoRA) on a stratified split of 960 training and 240 test cases. The fine-tuned model reaches 95.42% accuracy (95% CI 91.98-97.42%) and a macro F1 of 0.9551, up from 65.00% for the unadapted weights (62.92% through the remote API). However, TF-IDF linear SVM and logistic regression baselines trained on the same split reach 98.75% (95% CI 96.39-99.57%) with sub-millisecond CPU inference, so on this benchmark the fine-tuned model offers no accuracy advantage over a well-tuned linear classifier. A Leave-One-Symptom-Out clause-ablation analysis of four illustrative narratives shows predictions depending on distinguishing clauses: in a psoriasis example, removing the clause describing silvery scales lowers the predicted probability from above 99.99% to 0.12% and shifts the prediction to impetigo, while removing other clauses changes it by less than 0.001 percentage points. Decision architectures deliver structured, single-pass predictions from large language backbones, but short, lexically distinctive benchmarks such as Symptom2Disease cannot show an advantage over simple baselines; validation on longer, real clinical text and clinician review are needed.
Figures & tables
Figure 1: Overview of the decision pipeline and clause-level attribution. (a) A first-person symptom narrative is used as raw text input and segmented into symptom clauses for attribution. (b) Clef-Flash: a 4-bit frozen Qwen3.5-9B backbone with trainable LoRA adapters on the attention projections, and the trainable JointSchemaHead, which routes per-token evidence to option representations and scores all candidate conditions in one non-autoregressive forward pass, without generating tokens. (c) Predicted distribution for this narrative (log scale): Psoriasis receives a probability above . (d) Leave-One-Symptom-Out ablation: removing , “dry silvery scales peeling off” , lowers the Psoriasis probability by 99.88 percentage points and flips the prediction to Impetigo (99.59%), whereas removing any other clause changes it by less than 0.001 points. All values are measured model outputs for this one illustrative, author-written narrative.
| Model | Architecture | Accuracy [95% CI] | Macro F1 | Output Tok. | Latency |
| Panel A: Same split (this work) | |||||
| TF-IDF + Linear SVM | Uni/bigram TF-IDF, linear SVM | 98.75% [96.39–99.57] | 0.9876 | 0 | 0.7 ms (CPU) |
| TF-IDF + Logistic Regression | Uni/bigram TF-IDF, logistic regression | 98.75% [96.39–99.57] | 0.9876 | 0 | 0.8 ms (CPU) |
| TF-IDF + Random Forest | Uni/bigram TF-IDF, 500 trees | 96.67% [93.56–98.30] | 0.9665 | 0 | 45 ms (CPU) |
| Clef (27B), zero-shot API ( Cloudflare, 2026b ) | Qwen3.8-27B + JointSchemaHead | 66.25% [60.05–71.93] | 0.6186 | 0 | 0.92 s (API) |
| Clef-Flash (9B), zero-shot API ( Cloudflare, 2026a ) | Qwen3.5-9B + JointSchemaHead | 62.92% [56.65–68.78] | 0.5985 | 0 | 0.80 s (API) |
Table 1: Comparative Performance on Symptom2Disease. Panel A: all models use the same 960/240 split; classical baselines were tuned by 5-fold cross-validation on the training split and evaluated once on the test split (95% Wilson CIs in brackets). Classical latencies are single-record CPU inference (AMD Ryzen 7 250); zero-shot API latencies are mean per-request times to the remote Cloudflare Workers AI endpoint, including network round trips. Zero-shot rows use bare option names without disease criteria, matching the fine-tuning setup. The fine-tuned model’s local latency was not measured. Panel B: results as reported by the original authors on their own splits and hardware; they are not directly comparable to Panel A. “–” means not reported. † The paper cites Symptom2Disease but does not state explicitly that the reported result uses it. ‡ Averaging method not stated.
Figure 2: Training loss and test-set accuracy of Clef-Flash across 5 epochs. (A) Cross-entropy training loss progression across epochs, showing rapid and monotonic convergence from 0.900 down to 0.008. (B) Test-set accuracy from the zero-shot baseline (65.00%) to 95.42% across 5 training epochs. The test split was also used for checkpoint selection ( Section 3.3 ).
Figure 3: Normalized confusion matrix on the 240 held-out test cases. 15 of the 24 conditions achieved 100% recall (10 out of 10 cases correct), with 11 classes reaching 100% precision and recall. Six of the 11 misclassifications involve dengue, either as the true or the predicted class.
Figure 4: Class-by-class F1-score comparison between zero-shot Clef-Flash (remote API, bare option names) and the fine-tuned model. The largest gains occur for conditions with low zero-shot recognition, including Bronchial Asthma ( ), Drug Reaction ( ), Fungal Infection ( ), Malaria ( ), and Chicken Pox ( ).
| Disease Class | Zero-Shot | Prec. | Rec. | F1 | Gain ( ) | Disease Class | Zero-Shot | Prec. | Rec. | F1 | Gain ( ) |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Acne | 0.9474 | 1.0000 | 1.0000 | 1.0000 | +0.0526 | Malaria | 0.1538 | 1.0000 | 1.0000 | 1.0000 | +0.8462 |
| Arthritis | 1.0000 | 1.0000 | 1.0000 | 1.0000 | +0.0000 | Migraine | 0.5000 | 1.0000 | 1.0000 | 1.0000 | +0.5000 |
| Bronchial Asthma | 0.0000 | 1.0000 | 1.0000 | 1.0000 | +1.0000 | Pneumonia | 0.6061 | 1.0000 | 1.0000 | 1.0000 | +0.3939 |
| Cervical spondylosis | 0.9474 | 1.0000 | 0.9000 | 0.9474 | +0.0000 | Psoriasis | 0.5625 | 1.0000 | 0.9000 | 0.9474 | +0.3849 |
| Chicken pox | 0.1538 | 0.8333 | 1.0000 | 0.9091 | +0.7552 | Typhoid | 0.9000 | 0.9000 | 0.9000 | 0.9000 | +0.0000 |
| Common Cold | 0.6364 | 1.0000 | 1.0000 | 1.0000 | +0.3636 | Varicose Veins | 1.0000 | 1.0000 | 1.0000 | 1.0000 | +0.0000 |
Table 2: Classwise Diagnostic Performance on Held-Out Test Set (240 Cases). Zero-Shot is the F1 of Clef-Flash through the remote API with bare option names; Gain is the fine-tuned F1 minus the zero-shot F1.
Figure 5: Leave-One-Symptom-Out (LOSO) clause ablation on four illustrative, author-written narratives. Horizontal bars show the drop in predicted probability ( ) when individual symptom clauses are removed from the input narrative. Red bars mark clauses whose removal flips the prediction (Psoriasis to Impetigo upon removal of “dry silvery scales peeling off”, and Arthritis to Cervical Spondylosis upon removal of “painful swelling in my joints”). In the two multi-symptom narratives (Bronchial Asthma and Malaria), removing any single clause changes the probability by at most 0.02 percentage points.
| Target Disease | Ablated Symptom Expression | Clinical Category | Base | Post | Drop ( ) | Post-Ablation Diagnosis |
|---|---|---|---|---|---|---|
| Panel A: Single-Anchor Presentations (Decision Flips) | ||||||
| Psoriasis | dry silvery scales peeling off | HALLMARK DEFINING | 100.00% | 0.12% | 99.88% | Impetigo (99.59%) |
| Psoriasis | skin is cracked | SECONDARY COMPLICATION | 100.00% | 100.00% | 0.00% | Psoriasis (Stable) |
| Psoriasis | itchy red rash | NON-DEFINING / GENERAL | 100.00% | 100.00% | 0.00% | Psoriasis (Stable) |
| Psoriasis | bleeds occasionally | NON-DEFINING / GENERAL | 100.00% | 100.00% | -0.00% | Psoriasis (Stable) |
| Arthritis | painful swelling in my joints | HALLMARK DEFINING | 98.69% | 8.18% | 90.52% | Cervical spondylosis (39.01%) |
Table 3: Symptom Clause Ablation and Post-Ablation Predictions (four illustrative, author-written narratives; clause categories assigned by the author). Panel A: in the psoriasis and arthritis narratives, removing the clause labeled as the hallmark lowers the target probability by at least 90 percentage points and changes the predicted class. Panel B: in the bronchial asthma and malaria narratives, which contain several corroborating clauses, removing any single clause changes the probability by at most 0.02 points.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Disease Class | Clinical Domain | Train ( ) | Test ( ) | Total ( ) | Criteria Summary (adapted from NIH MedlinePlus) |
|---|---|---|---|---|---|
| Acne | Dermatological | 40 | 10 | 50 | Outbreaks of pimples, blackheads, whiteheads, cysts, and oily, inflamed skin on face, chest, or back. |
| Arthritis | Musculoskeletal | 40 | 10 | 50 | Joint inflammation, stiffness, swelling, reduced range of motion, and joint pain typically worse in the morning. |
| Bronchial Asthma | Respiratory | 40 | 10 | 50 | Wheezing, shortness of breath, chest tightness, coughing spells, exacerbated by exercise, cold air, or allergens. |
| Cervical spondylosis | Musculoskeletal | 40 | 10 | 50 | Neck pain, neck stiffness, muscle spasms, and radiating numbness, tingling, or weakness into shoulders, arms, and hands. |
| Chicken pox | Infectious / Febrile | 40 | 10 | 50 | Highly itchy blister rash progressing from spots to fluid-filled vesicles that scab over, with fever and body aches. |
| Common Cold | Infectious / Upper Resp. | 40 | 10 | 50 | Runny nose, nasal congestion, sneezing, mild sore throat, and mild cough with low-grade or absent fever. |
Table A1: Dataset breakdown and criteria summaries across the 24 conditions in Symptom2Disease. Each condition comprises 50 cases partitioned into 40 training cases (80%) and 10 held-out test cases (20%). Criteria summaries are paraphrased from NIH MedlinePlus.
| Configuration Category | Hyperparameter Parameter | Assigned Value |
| Base Architecture | Foundation Backbone | Qwen3.5-9B |
| Transformer Layers | 32 | |
| Hidden Dimension ( ) | 4096 | |
| Classification Head | JointSchemaHead (width 1024; 2 routing + 4 decoder layers) | |
| Quantization (QLoRA) | Base Weight Precision | 4-bit NormalFloat (NF4) |
| Double Quantization | Enabled (Nested 8-bit scaling) |
Table A2: Hyperparameter and training configuration for Clef-Flash adaptation. Values are taken from the training configuration file and training log in the code release.
| Disease Class | Base Zero-Shot F1 | Precision | Recall | Fine-Tuned F1 | F1 Gain ( ) | Support ( ) |
|---|---|---|---|---|---|---|
| Acne | 0.9474 | 1.0000 | 1.0000 | 1.0000 | +0.0526 | 10 |
| Arthritis | 1.0000 | 1.0000 | 1.0000 | 1.0000 | +0.0000 | 10 |
| Bronchial Asthma | 0.0000 | 1.0000 | 1.0000 | 1.0000 | +1.0000 | 10 |
| Cervical spondylosis | 0.9474 | 1.0000 | 0.9000 | 0.9474 | +0.0000 | 10 |
| Chicken pox | 0.1538 | 0.8333 | 1.0000 | 0.9091 | +0.7552 | 10 |
| Common Cold | 0.6364 | 1.0000 | 1.0000 | 1.0000 | +0.3636 | 10 |
Table A3: Comprehensive class-wise evaluation scorecard on the 240 held-out test cases. The table reports the zero-shot F1 score of Clef-Flash through the remote API (bare option names), fine-tuned precision, recall, and F1 score, absolute F1 gain ( ), and test support ( per class). Fine-tuning yields a +0.3566 macro F1 gain over the zero-shot baseline (0.5985). Per-class results were not saved for the local zero-shot evaluation.
| Case | Ground Truth | Predicted | Possible Symptom Overlap |
|---|---|---|---|
| 1 | Cervical spondylosis | Dengue | Neck and body aches shared with febrile myalgia. |
| 2 | Dengue | Chicken pox | Rash accompanied by fever. |
| 3 | Dengue | Typhoid | Fever with systemic and abdominal symptoms. |
| 4 | Fungal infection | Dengue | Skin rash; otherwise no clear overlap. |
| 5 | Hypertension | diabetes | Non-specific symptoms such as fatigue, dizziness, or blurred vision. |
| 6 | Hypertension | drug reaction | No clear clinical overlap. |
Table A4: All 11 misclassified test cases on the Symptom2Disease benchmark, taken from the confusion matrix. Per-case narratives and confidences were not saved in the evaluation outputs and are therefore not reported. The overlap column is the author’s interpretation and has not been reviewed by clinicians.
| Target Disease | Ablated Symptom Expression | Clinical Category | Base | Post | Drop ( ) | Post-Ablation Diagnosis |
|---|---|---|---|---|---|---|
| Single-Anchor Hallmark Presentations | ||||||
| Psoriasis | dry silvery scales peeling off | HALLMARK DEFINING | 100.00% | 0.12% | 99.88% | Impetigo (99.59%) [Flip] |
| Psoriasis | skin is cracked | SECONDARY COMPLICATION | 100.00% | 100.00% | 0.00% | Psoriasis (Stable) |
| Psoriasis | itchy red rash | NON-DEFINING / GENERAL | 100.00% | 100.00% | 0.00% | Psoriasis (Stable) |
| Psoriasis | bleeds occasionally | NON-DEFINING / GENERAL | 100.00% | 100.00% | -0.00% | Psoriasis (Stable) |
| Arthritis | painful swelling in my joints | HALLMARK DEFINING | 98.69% | 8.18% | 90.52% | Cervical spondylosis (39.01%) [Flip] |
Table A5: Full Leave-One-Symptom-Out (LOSO) clause ablation results for the four illustrative, author-written narratives. Removing the clause labeled as the hallmark lowers the target probability by at least 90 percentage points and changes the predicted class in the psoriasis and arthritis narratives. In the multi-symptom narratives, removing any single clause changes the probability by at most 0.02 points.
More in this topic
Medical Diagnosis
Medical diagnosis uses clinical data, tests, and images to identify diseases, assess risk, and guide care.
See all 261 papers