Case-Grounded Evidence Verification: A Framework for Constructing Evidence-Sensitive Supervision

Rule-based reasoning, as in eligibility checks and contract reviews, requires language models to assess evidence against individual conditions and combine their judgments under explicit rules. Errors in evidence assessment can leave a decision unchanged, but misinterpreting or overlooking decision-critical evidence can reverse it. Identifying such evidence allows more capable models to focus on checking the corresponding condition judgments, supporting accurate and safe decisions. Recognizing the evidence's criticality requires understanding how evidence affects a condition judgment and how that judgment affects the decision. To achieve the goal, we propose a INTERvention-based imPACT learning framework (InterPact), which enables counterfactual verification of evidence criticality in rule-governed decisions. Specifically, its evidence intervention constructor generates training pairs for a propagation verifier by editing case facts with a frozen language model while holding rules and non-target conditions fixed. Human-reviewed labels record the resulting condition and decision changes, while complete state-to-decision mappings supervise consequences beyond the observed edit. During training, the verifier weights learned conditional decision predictions by evidence-based condition probabilities through a fixed composition operation, propagating decision-change supervision into the base model. At inference, the trained base model directly judges criticality from the original case and target evidence, without human or stronger-model supervision. On single-case evidence criticality verification over adapted rule-governed decision cases, InterPact achieves 68.28% accuracy, outperforming all six baselines. These results support learned decision sensitivity as a basis for prioritizing evidence checks.

Evidence-grounded generation produces summaries whose claims should be supported by supplied evidence, but claim-level verifiers provide noisy feedback and can reward models that simply say less. We study this problem in clinical Brief Hospital Course summarization, where outputs must remain grounded in patient-specific EHR evidence. We introduce VERI-DPO, a preference-mining framework that converts noisy claim verification into coverage-controlled summary-level preferences. For each evidence-window prompt, VERI-DPO samples multiple candidate summaries, decomposes them into claims, verifies each claim against patient evidence, and forms a preference pair only when the chosen summary has better aggregate verifier-estimated support while retaining comparable verifiable content. Standard Direct Preference Optimization then distills these pairs into a single-sample policy, avoiding inference-time reranking. On patient-disjoint MIMIC-III-Ext-VeriFact-BHC test data, VERI-DPO reduces Not Supported rates from 10.7% to 1.9% under the mining verifier and from 11.6% to 6.4% under a separately prompted GPT-4o judge. In 100 blinded pairwise assessments by two domain researchers, VERI-DPO is preferred over the base model 56 times versus 18 times for factual faithfulness. In a locked zero-shot MIMIC-IV-Ext-BHC transfer test with 1,000 patients and no model adaptation, VERI-DPO lowers Not Supported rates with nearly unchanged scored-claim counts. Multi-seed ablations show that verifier-guided pair construction drives the gains, while coverage and anti-degeneration controls prevent apparent factuality improvements from coming from shorter or less checkable outputs.

Vision-Language Models (VLMs) should answer from concrete image evidence rather than language priors, dataset shortcuts, or irrelevant visual context. Existing perception-aware post-training methods encourage image use through global perturbations or attention proxies, but they do not test whether a sampled answer causally depends on the local evidence that supports it. We propose Counterfactual Evidence Disentanglement (CED), a training-time evidence audit for VLM grounding. For each response, CED neutralizes an object-centric Evidence Region and compares the resulting support drop against matched non-evidence Regions. We combine this signal with answer correctness inside GRPO, rewarding correct answers that rely on the evidence path rather than shortcut or nuisance paths. CED uses weak object-level proposals, requires no question-specific evidence annotations, and adds no inference-time overhead. Across nine public benchmarks and four backbones, CED outperforms prior RL-based post-training methods, with targeted analyses verifying its object-centric signal.