Tree-of-Evidence: Efficient "System 2" Search for Faithful Multimodal Grounding

Micky C. Nnamdi; Benoit L. Marteau; Yishan Zhong; J. Ben Tamo; May D. Wang

cs.LGApr 9, 2026

Tree-of-Evidence: Efficient "System 2" Search for Faithful Multimodal Grounding

Authors: Micky C. Nnamdi, Benoit L. Marteau, Yishan Zhong, J. Ben Tamo, May D. Wang

Organizations: Georgia Institute of Technology

Abstract

Large Multimodal Models (LMMs) achieve state-of-the-art performance in high-stakes domains like healthcare, yet their reasoning remains opaque. Current interpretability methods, such as attention mechanisms or post-hoc saliency, often fail to faithfully represent the model's decision-making process, particularly when integrating heterogeneous modalities like time-series and text. We introduce Tree-of-Evidence (ToE), an inference-time search algorithm that frames interpretability as a discrete optimization problem. Rather than relying on soft attention weights, ToE employs lightweight Evidence Bottlenecks that score coarse groups or units of data (e.g., vital-sign windows, report sentences) and performs a beam search to identify the compact evidence set required to reproduce the model's prediction. We evaluate ToE across six tasks spanning three datasets and two domains: four clinical prediction tasks on MIMIC-IV, cross-center validation on eICU, and non-clinical fault detection on LEMMA-RCA. ToE produces auditable evidence traces while maintaining predictive performance, retaining over 0.98 of full-model AUROC with as few as five evidence units across all settings. Under sparse evidence budgets, ToE achieves higher decision agreement and lower probability fidelity error than other approaches. Qualitative analyses show that ToE adapts its search strategy: it often resolves straightforward cases using only vitals, while selectively incorporating text when physiological signals are ambiguous. ToE therefore provides a practical mechanism for auditing multimodal models by revealing which discrete evidence units support each prediction.

Figures & tables

Figure 1: Overview of the Tree-of-Evidence (ToE) Framework. Phase I : Modality-specific classifiers are trained independently, with BioClinicalBERT Alsentzer et al. (2019) encoding notes and contextual data (CXR/ECG) concatenated as fixed priors. Phase II : Lightweight MLP selectors learn to score evidence units using Straight-Through Estimator (STE) top-k masking with frozen encoders. Phase III : At inference, beam search iteratively constructs compact evidence set by optimizing the scoring function, balancing decision agreement, probability stability, and sparsity.

Method	Type	Hard Evidence?	Multimodal?	Faithfulness?	Auditable Trace?
Attention Weights	Intrinsic (Soft)	✗	~	✗	✗
Gradient Saliency	Post-hoc	✗	~	✗	✗
LIME / SHAP	Post-hoc Surrogate	✗	~	~	✗
Concept Bottleneck	Intrinsic (Concepts)	✓	~	~	~
Tree-of-Evidence (Ours)	Inference Search	✓	✓	✓	✓

Table 1: Comparison of Interpretability Frameworks. ToE is distinct in offering an auditable trace (search history) over multimodal hard evidence.

Figure 2: The Faithfulness-Sparsity Frontier. Performance across evidence budgets

k

on MIMIC-IV (E1: In-Hospital Mortality, 5 seeds). (a) Sufficiency: ToE (Red

\star

) matches the full model’s predictive power (

\text{AUROC}\approx 0.80

) with as few as

k{=}5

units. (b) Fidelity: ToE achieves the lowest Fidelity MAE at sparse budgets (

k\leq 5

), reducing error by

{>}50\%

compared to Top-

k

Ranking (Blue

\bullet

) and Saliency (Gold

\blacktriangle

) at sparse budgets, proving it captures the model’s actual confidence rather than just label correlations.

$k$	Method	AUROC	Fidelity MAE ( $\downarrow$ )	ECE ( $\downarrow$ )
1	LIME	0.564 $\pm$ 0.006	0.229 $\pm$ 0.022	0.406 $\pm$ 0.011
	SHAP	0.764 $\pm$ 0.009	0.123 $\pm$ 0.006	0.320 $\pm$ 0.010
	ToE	0.783 $\pm$ 0.013	0.096 $\pm$ 0.005	0.297 $\pm$ 0.019
5	LIME	0.695 $\pm$ 0.010	0.171 $\pm$ 0.016	0.332 $\pm$ 0.018
	SHAP	0.801 $\pm$ 0.014	0.039 $\pm$ 0.002	0.302 $\pm$ 0.025
	ToE	0.800 $\pm$ 0.017	0.040 $\pm$ 0.003	0.280 $\pm$ 0.023

Table 2: Comparison with LIME and SHAP (E1: Hospital Mortality, 5 seeds). ToE achieves the best fidelity–sufficiency tradeoff at every budget

k

. Full results across all

k

in Appendix B , Figure A3 .

Model	Type	Params	AUROC	AUPRC
Llama-3.2-1B	text	1.2B	$0.532\pm 0.009$	$0.135\pm 0.005$
Llama-3.1-8B	text	8.0B	$0.691\pm 0.008$	$0.206\pm 0.008$
Med42-v2-70B	text	70B	$0.745\pm 0.009$	$0.293\pm 0.014$
MedGemma-27B (V)	vision	27B	$0.630\pm 0.009$	$0.190\pm 0.008$
ToE $k{=}5$	multi	109M	0.800 $\pm$ 0.017	0.310 $\pm$ 0.067

Table 3: LLM/MLLM comparison (E1: Hospital Mortality). ToE with 109M parameters outperforms all models up to 70B. Full 10-model results in Appendix C ; Figure A4 .

	Dataset	Task	Full AUROC	ToE $k{=}5$	Fid. MAE
E1	MIMIC-IV	Hosp. mort.	0.806 $\pm$ 0.015	0.800 $\pm$ .017	0.040 $\pm$ 0.003
E2	MIMIC-IV	Long LOS	0.747 $\pm$ 0.041	0.740 $\pm$ .046	0.031 $\pm$ 0.002
E3	MIMIC-IV	ICU mort.	0.816 $\pm$ 0.009	0.808 $\pm$ .011	0.042 $\pm$ 0.004
E4	MIMIC-IV	Post-obs.	0.794 $\pm$ 0.021	0.784 $\pm$ .023	0.041 $\pm$ 0.001
	eICU	ICU mort.	0.822	0.808	0.124
	LEMMA	Fault det.	0.741	0.730	0.106

Table 4: Cross-task and cross-dataset evaluation. ToE retains

\geq

98% of full-model AUROC at

k{=}5

across all settings. MIMIC-IV results: mean

\pm

std over 5 seeds; eICU and LEMMA-RCA results are from a single seed.

Modality	AUROC	AUPRC	Fidelity MAE
TS Only	$0.7876\pm 0.0075$	$0.3912\pm 0.0151$	$0.0445\pm 0.0730$
Notes Only	$0.5590\pm 0.0077$	$0.1338\pm 0.0047$	$0.3432\pm 0.2915$
Both	$0.8001\pm 0.0165$	$0.3096\pm 0.0672$	$0.0403\pm 0.0027$

Table 5: Modality Ablation Results (

k=5

). Notes-Only fails to ground predictions, while the Multimodal (Both) approach maintains high predictive power and stability comparable to the strong TS-Only baseline.

Configuration	AUROC	AUPRC	Fidelity MAE	Comp.
Full ( $\lambda{=}1.0$ , $\mu{=}0.05$ )	$0.8001\pm 0.0165$	$0.3096\pm 0.0672$	$0.0403\pm 0.0027$	$0.1112\pm 0.0143$
No Stability ( $\lambda{=}0.0$ , $\mu{=}0.05$ )	$0.7738\pm 0.0338$	$0.3069\pm 0.0622$	$0.0800\pm 0.0052$	$0.1371\pm 0.0153$
No Sparsity ( $\lambda{=}1.0$ , $\mu{=}0.0$ )	$0.7915\pm 0.0385$	$0.3191\pm 0.0682$	$0.0408\pm 0.0015$	$0.1025\pm 0.0115$
Top- $k$ Ranking (no search)	$0.7735\pm 0.0339$	$0.3066\pm 0.0625$	$0.0806\pm 0.0054$	$0.1357\pm 0.0158$

Table 6: Search Objective Ablation (

k=5

, MIMIC-IV Mortality, 5 seeds). The full ToE objective achieves the lowest Fidelity MAE. Removing stability (

\lambda{=}0

) doubles the error. At fixed

k

, the sparsity cost

\mu

has no effect (identical to Full). Top-

k

Ranking without beam search doubles the MAE (

+100\%

Metric	eICU	MIMIC-IV
TP search exhaustion rate	0.3%	7.2%
FP search exhaustion rate	7.3%	30.2%
FP / TP exhaustion ratio	25.6 $\times$	4.2 $\times$

Table 7: Search exhaustion rates for true positive (TP) vs. false positive (FP) predictions. When the model is wrong, the search exhausts its budget 4–26

\times

more often.

Case A: Efficient Triage (Patient A)	Case B: Multimodal Synergy (Patient B)
Task: Mortality Prediction	Task: Mortality Prediction
Full Model Prob: $0.0005$ (Low Risk)	Full Model Prob: $0.861$ (High Risk)
Final Trace: $k=1$ (Vitals Only)	Final Trace: $k=10$ (8 Vitals + 2 Notes)
Search Step 1: Add Vitals W5 $\hookrightarrow$ Evidence: [Physiological Window 5] $\hookrightarrow$ Sufficiency: $\mathbf{0.998}$ (Threshold Met)	Search Steps 1–8: Add Vitals W23, W1, …, W11 $\hookrightarrow$ Evidence: [Multiple Physiological Windows] $\hookrightarrow$ Sufficiency: $0.840$ (Plateau)
Outcome: The search terminates immediately. The model determines that the vital signs alone are sufficient to justify the “Low Risk” prediction. No notes are processed.	Search Step 9: Add Note N2 $\hookrightarrow$ Evidence: “…Coalescent, bilateral, perihilar opacities reflect alveolar edema… suggest volume overload.” $\hookrightarrow$ Sufficiency: $0.833$ (Stable)
	Search Step 10: Add Note N3 $\hookrightarrow$ Evidence: [Additional Radiology Report] $\hookrightarrow$ Sufficiency: $0.833$ (No Change)

Table 8: Qualitative comparison of ToE traces. (Left) ToE efficiently solves clear-cut cases using only vitals. (Right) ToE dynamically integrates clinical notes to surface interpretable clinical context, extracting specific medical concepts (e.g., “Alveolar Edema”) to ground the prediction.

Patient ID	Outcome	Prediction		Evidence Size ( $k$ )		Jaccard	Key Insight
		ToE	LLM	ToE	LLM	Overlap
1	Survived	✓	✓	1	5	20.0%	ToE solved via single vital; LLM was cautious.
2	Survived	✓	✓	6	9	50.0%	Strong agreement on deterioration intervals.
3	Died	✓	✓	8	11	46.2%	Both flagged critical physiological decline.
4	Survived	✓	✓	8	9	30.8%	LLM prioritized stable periods differently.
5	Died	✗	✓	8	11	35.7%	Audit Win: ToE exposed model blindness to GCS.

Table 9: ToE vs Zero-Shot LLM. Comparison on a representative subset of 5 patients. While the LLM achieves perfect prediction accuracy by leveraging external medical knowledge, ToE remains faithful to the underlying model’s logic. ToE selects significantly sparser evidence (Avg 6.2 vs. 9.0 windows) while maintaining reasonable overlap with the LLM’s clinical reasoning.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

$k$	E1: Hospital Mort.	E2: Long LOS	E3: ICU Mort.	E4: Post-Obs Mort.
1	$0.0413\pm 0.0050$	$0.0447\pm 0.0107$	$0.0326\pm 0.0050$	$0.0441\pm 0.0037$
3	$0.0885\pm 0.0104$	$0.0939\pm 0.0094$	$0.0749\pm 0.0092$	$0.0908\pm 0.0051$
5	$0.1112\pm 0.0143$	$0.1199\pm 0.0104$	$0.0961\pm 0.0131$	$0.1139\pm 0.0067$
8	$0.1293\pm 0.0177$	$0.1431\pm 0.0117$	$0.1138\pm 0.0159$	$0.1334\pm 0.0089$
12	$0.1417\pm 0.0199$	$0.1582\pm 0.0125$	$0.1270\pm 0.0180$	$0.1463\pm 0.0108$
16	$0.1489\pm 0.0216$	$0.1644\pm 0.0123$	$0.1342\pm 0.0190$	$0.1541\pm 0.0121$

Appendix

Table A1: ToE Comprehensiveness (

\uparrow

) across evidence budgets and four MIMIC-IV tasks (mean

\pm

std, 5 seeds).

Figure A1: Calibration Analysis. Full Model (Blue) vs. ToE at

k{=}5

(Red). ToE preserves calibration (ECE 0.254 vs. 0.259) while using sparse evidence.

Figure A2: Evidence Size Distribution (

k{=}5

, MIMIC-IV Mortality). Left: By mortality outcome, both groups concentrate at the budget cap, with survivors showing slightly more early stopping. Right: By prediction correctness, correct predictions exhibit greater evidence efficiency (more mass at

k<5

), while incorrect predictions exhaust the full budget in 97% of cases, indicating the search struggles to find supporting evidence when the model is wrong.

Figure A3: Complete comparison of ToE, LIME, and SHAP across four MIMIC-IV tasks (E1–E4, 5 seeds,

k\in\{1,3,5,10,15,20,25\}

). Rows: AUROC, AUPRC, Fidelity MAE, ECE. Columns: E1 (In-Hospital Mortality), E2 (Long LOS), E3 (ICU Mortality), E4 (Post-Obs Mortality). Shaded regions denote

\pm 1

std. ToE consistently achieves the best ECE across all tasks and competitive AUROC at sparse budgets (

k\leq 5

), while SHAP converges at higher

k

with lower MAE.

Figure A4: LLM/MLLM, CBM, and ToE Comparison on E1 (In-Hospital Mortality, MIMIC-IV). (a) AUROC and (b) AUPRC for 7 text-only LLMs (1B–70B), 3 vision LLMs (12–27B), Hard CBM (24 clinical concepts), and ToE at

k{=}1,5,10

(109M parameters). Error bars denote

\pm 1

std (bootstrap for LLMs/CBM, 5 seeds for ToE). Dashed line indicates the full model. ToE

k{=}5

outperforms the best 70B LLM (Med42) by

+0.048

AUROC with

640{\times}

fewer parameters, and adding vision to LLMs degrades performance.

	Beam Search (ToE)		Top- $k$ Ranking
$k$	AUROC	Fidelity MAE	AUROC	Fidelity MAE
1	$0.7833\pm 0.0128$	$0.0958\pm 0.0053$	$0.7676\pm 0.0362$	$0.1073\pm 0.0071$
3	$0.7967\pm 0.0152$	$0.0481\pm 0.0023$	$0.7739\pm 0.0343$	$0.0798\pm 0.0054$
5	$0.8001\pm 0.0165$	$0.0403\pm 0.0027$	$0.7735\pm 0.0339$	$0.0806\pm 0.0054$
8	$0.8028\pm 0.0162$	$0.0367\pm 0.0023$	$0.7722\pm 0.0325$	$0.0840\pm 0.0056$
12	$0.8039\pm 0.0167$	$0.0333\pm 0.0022$	$0.7713\pm 0.0314$	$0.0859\pm 0.0060$

Appendix

Table A2: Search vs Ranking Comparison Across Evidence Budgets (MIMIC-IV Mortality, 5 seeds).

$\tau_{\mathrm{STE}}$	Suff. AUROC ( $k{=}6$ )	AUPRC ( $k{=}6$ )	Suff. AUROC ( $k{=}1$ )
0.1	0.792	0.316	0.741
0.5	0.797	0.321	0.740
1.0	0.799	0.325	0.753
2.0	0.799	0.319	0.745
5.0	0.790	0.313	0.748

Appendix

Table A3: STE temperature sensitivity (eICU). Performance varies

<

1% across a 50

\times

range.

$k$	Space	AUROC	Fid. MAE	Comp.
1	Probability	0.755	0.090	0.029
1	Logit	0.749	0.100	0.040
5	Probability	0.773	0.030	0.097
5	Logit	0.768	0.054	0.131
12	Probability	0.773	0.014	0.130
12	Logit	0.769	0.051	0.189

Appendix

Table A4: Probability-space vs. logit-space stability. Probability-space achieves consistently lower fidelity MAE.

Dataset	$k$	ToE AUROC	Exhaustive	Gap
MIMIC-IV	1	0.7550	0.7543	+0.0007
MIMIC-IV	3	0.7706	0.7697	+0.0009
LEMMA-RCA	all	0.7181	0.7252	$-0.0071$

Appendix

Table A5: Optimality gap: ToE vs. exhaustive search.

Hyperparameter	Range	Dataset(s)	Key Finding
Stability space	Prob vs. Logit	MIMIC + eICU	Prob-space 44% lower MAE
$\tau_{\text{suff}}$	.70, .80, .90, .95	eICU	Higher $\rightarrow$ better fidelity
$\lambda$ (stability)	0, 1.0	MIMIC	$\lambda{=}0$ doubles MAE

Appendix

Table A6: Hyperparameter sensitivity summary.

Multimodal Model Interpretability

Multimodal model interpretability studies how AI systems combine and use images, text, audio, video, and other signals, and how to explain their decisions.

See all 129 papers

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction
Jul 10, 2026Nikkie Hooman, Zhongjie Wu, Eric C. Larson +1Multimodal Robustness Clinical Prediction
Southern Methodist University Dallas, USA
MedTRACE: Tool-Augmented Multimodal Clinical Reasoning Agents for Evidence-Grounded Decision-Making
Sep 13, 2026Ji Lu, Lifei Liu, Haoran Yu +5Medical Diagnosis Clinical Decision-Making
Vanderbilt University, United States · Wichita State University, United States · University of Florida, United States +4
Conditional Evidence Reconstruction and Decomposition for Interpretable Multimodal Diagnosis
Apr 18, 2026Shaowen Wan, Yanjun Lv, Lu Zhang +5Missing-Modality Learning Medical Diagnosis
Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ 07102, USA · Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA · Department of Computer Science, Indiana University Indianapolis, Indianapolis, IN 46202, USA +1

Jul 10, 2026J/K move · Enter open · S save

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

Nikkie Hooman, Zhongjie Wu, Eric C. Larson, Mehak Gupta

Southern Methodist University Dallas, USA

Multimodal Robustness Clinical Prediction Multimodal Model Interpretability Longitudinal EHR Modeling Multimodal Model Routing

Open paper AskPDF arXiv

Electronic health record (EHR) data are inherently multimodal, and leveraging multiple modalities can improve predictive performance. However, most existing approaches rely on deep fusion, which obscures how individual modalities contribute to predictions and limits the interpretability of multimodal reasoning. We propose an explicit multimodal routing framework for clinical prediction that enables interpretable, robust, and auditable reasoning across three EHR modalities: structured longitudinal variables (L), clinical notes (N), and chest X-rays (I). Our model constructs discrete unimodal, directional bimodal, and trimodal routes to capture both individual modality signals and asymmetric cross-modal interactions. To audit multimodal reasoning and assess robustness, we introduce inference-time route masking, which simulates missing modalities and reweights the remaining routes without retraining. We analyze changes in performance and routing weights under these scenarios to understand model decision-making. We evaluate our framework on multi-label phenotype prediction (K = 25) and binary ICU mortality prediction using trimodal patient stays from MIMIC-IV, revealing systematic differences in modality reliance across clinical condition groups. Overall, our framework offers a transparent, auditable, and practical approach to multimodal clinical prediction, providing interpretability, robustness, and insights into how different data sources drive model decisions.

Abstract

Multimodal Model Interpretability

Explore similar work

Multimodal Routing for Interpretable, Robust, and Auditable Clinical Prediction

MedTRACE: Tool-Augmented Multimodal Clinical Reasoning Agents for Evidence-Grounded Decision-Making

Conditional Evidence Reconstruction and Decomposition for Interpretable Multimodal Diagnosis