From Sampled Evidence to Radiology Reports: Objectives, Constraints, and Realization
Organizations: University of Pittsburgh · University of Texas Rio Grande Valley · The University of Manchester
Abstract
Sampling multiple reports creates an inference problem: how should sampled evidence be converted into a final decision, and does a better structured decision yield a better report? We study these questions through a framework that separates the decision signal, scoring objective, and feasible decision set, followed by text realization. On MIMIC-CXR and IU-Xray, controlled comparisons across 13 generator configurations per benchmark under grouped development/test evaluation reveal three findings. First, changing the objective under fixed evidence and rule capacity changes predictions and their held-out metric trade-offs. Second, relaxing candidate constraints has objective-dependent effects under shared signals and frozen scores. Diagnostics connect the IU macro-oriented assertion increase to near-zero label thresholds and show that limited candidate coverage coexists with selection headroom. Third, a controlled realization study with SFT-Qwen3-VL-8B shows that decision-level gains can be attenuated, amplified, or reversed. Change decomposition reveals that realization can remove incorrect assertions while losing correct findings, so target fidelity and reference accuracy need not improve together. Finding gains also do not consistently improve reference-based report quality. These results motivate separate evaluation of decision rules, candidate constraints, and realized reports.
Figures & tables
| MIMIC-CXR | IU-Xray | |||||||
|---|---|---|---|---|---|---|---|---|
| Contrast | Ma-14 | Mi-14 | Ma-5 | Mi-5 | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
| Ma–Maj [-1pt] (0/0/0) | +0.0644 [-1pt] (13/0/0) | -0.0033 [-1pt] (7/0/6) | +0.0590 [-1pt] (13/0/0) | +0.0238 [-1pt] (11/0/2) | +0.0553 [-1pt] (13/0/0) | -0.1423 [-1pt] (1/0/12) | +0.0767 [-1pt] (12/1/0) | +0.1234 [-1pt] (12/1/0) |
| Mi–Maj [-1pt] (0/0/0) | +0.0471 [-1pt] (12/0/1) | +0.0245 [-1pt] (11/0/2) | +0.0563 [-1pt] (12/0/1) | +0.0376 [-1pt] (12/0/1) | +0.0125 [-1pt] (6/4/3) | -0.0008 [-1pt] (4/1/8) | +0.0055 [-1pt] (5/6/2) | +0.0160 [-1pt] (6/6/1) |
| Ma–Mi [-1pt] (0/0/0) | +0.0172 [-1pt] (12/1/0) | -0.0278 [-1pt] (1/1/11) | +0.0028 [-1pt] (7/1/5) | -0.0138 [-1pt] (4/1/8) | +0.0428 [-1pt] (13/0/0) | -0.1415 [-1pt] (0/0/13) | +0.0712 [-1pt] (12/1/0) | +0.1074 [-1pt] (12/1/0) |
| Data | Contrast | Ma-14 | Mi-14 | Ma-5 | Mi-5 | RG | RL | GREEN | RaTE | B4 |
|---|---|---|---|---|---|---|---|---|---|---|
| MIMIC | RG–CX [-1pt] (0/0/0) | -0.0223 [-1pt] (0/0/13) | -0.0282 [-1pt] (1/0/12) | -0.0289 [-1pt] (0/0/13) | -0.0332 [-1pt] (1/0/12) | +0.0428 [-1pt] (13/0/0) | +0.0227 [-1pt] (12/0/1) | +0.0069 [-1pt] (11/0/2) | +0.0139 [-1pt] (12/0/1) | +0.0079 [-1pt] (10/0/3) |
| RL–CX [-1pt] (0/0/0) | -0.0243 [-1pt] (0/0/13) | -0.0355 [-1pt] (1/0/12) | -0.0333 [-1pt] (0/0/13) | -0.0436 [-1pt] (0/0/13) | +0.0183 [-1pt] (13/0/0) | +0.0369 [-1pt] (12/0/1) | +0.0035 [-1pt] (9/0/4) | +0.0086 [-1pt] (11/0/2) | +0.0146 [-1pt] (13/0/0) | |
| RG–RL [-1pt] (0/0/0) | +0.0020 [-1pt] (7/0/6) | +0.0073 [-1pt] (10/0/3) | +0.0044 [-1pt] (10/0/3) | +0.0104 [-1pt] (12/0/1) | +0.0246 [-1pt] (13/0/0) | -0.0142 [-1pt] (0/0/13) | +0.0033 [-1pt] (10/0/3) | +0.0053 [-1pt] (13/0/0) | -0.0067 [-1pt] (0/0/13) | |
| IU | RG–CX [-1pt] (0/0/0) | +0.0012 [-1pt] (6/3/4) | +0.0108 [-1pt] (5/3/5) | -0.0015 [-1pt] (2/9/2) | +0.0021 [-1pt] (2/9/2) | +0.0715 [-1pt] (13/0/0) | +0.0188 [-1pt] (13/0/0) | +0.0085 [-1pt] (8/0/5) | +0.0337 [-1pt] (13/0/0) | +0.0153 [-1pt] (12/0/1) |
| RL–CX [-1pt] (0/0/0) | +0.0028 [-1pt] (5/5/3) | +0.0052 [-1pt] (5/5/3) | -0.0008 [-1pt] (1/9/3) | -0.0019 [-1pt] (2/9/2) | +0.0364 [-1pt] (11/0/2) | +0.0316 [-1pt] (13/0/0) | +0.0030 [-1pt] (8/0/5) | +0.0158 [-1pt] (12/0/1) | +0.0187 [-1pt] (13/0/0) | |
| RG–RL [-1pt] (0/0/0) | -0.0017 [-1pt] (4/4/5) | +0.0056 [-1pt] (5/4/4) | -0.0007 [-1pt] (2/9/2) | +0.0040 [-1pt] (3/9/1) | +0.0351 [-1pt] (13/0/0) | -0.0129 [-1pt] (1/0/12) | +0.0055 [-1pt] (7/0/6) | +0.0179 [-1pt] (12/0/1) | -0.0034 [-1pt] (5/0/8) |
| MIMIC-CXR | IU-Xray | |||||||
|---|---|---|---|---|---|---|---|---|
| Effect | Ma-14 | Mi-14 | Ma-5 | Mi-5 | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
| [-1pt] (0/0/0) | +0.0069 [-1pt] (10/0/3) | -0.0322 [-1pt] (0/0/13) | -0.0068 [-1pt] (1/0/12) | -0.0086 [-1pt] (2/0/11) | -0.0010 [-1pt] (9/0/4) | -0.2208 [-1pt] (0/0/13) | -0.0038 [-1pt] (11/0/2) | -0.1023 [-1pt] (2/0/11) |
| [-1pt] (0/0/0) | +0.0181 [-1pt] (11/0/2) | -0.0269 [-1pt] (0/0/13) | +0.0138 [-1pt] (10/0/3) | +0.0141 [-1pt] (10/0/3) | +0.0439 [-1pt] (11/0/2) | -0.2206 [-1pt] (1/0/12) | +0.0526 [-1pt] (10/0/3) | -0.0411 [-1pt] (5/0/8) |
| [-1pt] (0/0/0) | +0.0196 [-1pt] (13/0/0) | +0.0002 [-1pt] (6/0/7) | +0.0206 [-1pt] (13/0/0) | +0.0008 [-1pt] (8/0/5) | -0.0085 [-1pt] (0/11/2) | +0.0001 [-1pt] (4/7/2) | -0.0205 [-1pt] (0/11/2) | -0.0009 [-1pt] (2/9/2) |
| [-1pt] (0/0/0) | +0.0289 [-1pt] (13/0/0) | +0.0075 [-1pt] (9/0/4) | +0.0328 [-1pt] (13/0/0) | +0.0135 [-1pt] (13/0/0) | +0.0254 [-1pt] (11/0/2) | +0.0035 [-1pt] (10/0/3) | +0.0628 [-1pt] (11/0/2) | +0.2085 [-1pt] (12/0/1) |
| [-1pt] (0/0/0) | -0.0127 [-1pt] (0/0/13) | -0.0323 [-1pt] (0/0/13) | -0.0274 [-1pt] (0/0/13) | -0.0093 [-1pt] (0/0/13) | +0.0076 [-1pt] (10/0/3) | -0.2209 [-1pt] (0/0/13) | +0.0167 [-1pt] (12/0/1) | -0.1015 [-1pt] (2/0/11) |
| Ma-14 | Mi-14 | Ma-5 | Mi-5 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Obj. | Contrast | ||||||||||||
| MIMIC-CXR | |||||||||||||
| Macro | P–C | +.0215 | -.0149 | +.0010 | -.0162 | -.0037 | +.0012 | -.0127 | +.0025 | -.0025 | -.0133 | +.0004 | +.0013 |
| F–C | +.0307 | -.0153 | +.0062 | -.0154 | +.0006 | +.0115 | +.0056 | -.0084 | -.0030 | +.0062 | -.0039 | -.0013 | |
| Micro | P–C | +.0220 | -.0079 | -.0052 | -.0010 | -.0023 | +.0007 | +.0160 | -.0018 | -.0048 | -.0024 | +.0002 | +.0012 |
| F–C | +.0243 | +.0034 | -.0066 | -.0022 | -.0011 | +.0022 | +.0215 | +.0041 | -.0081 | +.0020 | +.0013 | -.0021 | |
| MIMIC-CXR | IU-Xray | |||||||||||
| Method | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG |
| Greedy | .3531 | .5807 | .4712 | .5729 | .4866 | .3146 | .1020 | .5521 | .0467 | .0727 | .5731 | .3122 |
| MBR-1 | .3725 | .5903 | .5111 | .5775 | .4977 | .3382 | .0544 | .5825 | .0000 | .0000 | .5993 | .3960 |
| MBR-2 | .3742 | .5833 | .5145 | .5762 | .4942 | .3347 | .0629 | .5843 | .0000 | .0000 | .5692 | .3585 |
| CCS-Emb | .3881 | .6092 | .5178 | .6059 | .4952 | .3313 | .0775 | .5842 | .0000 | .0000 | .6022 | .3740 |
| CCS-Dev | .3789 | .5898 | .5145 | .5824 | .4970 | .3310 | .1022 | .5787 | .0707 | .1565 | .5553 | .2997 |
Appendix figures & tables59 assets
Supplementary material from the paper’s appendix.
Appendix
| Generator | Rule | Ma-14 | Mi-14 | Ma-5 | Mi-5 | P | R | |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-VL-8B | Maj | 17 | .3637 | .6136 | .4877 | .5946 | .5282 | .3335 |
| Ma | 3 | .4212 | .5940 | .5529 | .6073 | .3311 | .5893 | |
| Mi | 10 | .4036 | .6436 | .5527 | .6474 | .4308 | .4262 | |
| Q25-1 | Maj | 17 | .2885 | .5441 | .4163 | .5251 | .3941 | .2610 |
| Ma | 8 | .3651 | .6023 | .5148 | .5998 | .3943 | .4002 | |
| Mi | 9 | .3614 | .6018 | .5077 | .5988 | .4208 | .3849 |
| Generator | Rule | Ma-14 | Mi-14 | Ma-5 | Mi-5 | P | R | |
|---|---|---|---|---|---|---|---|---|
| Q35 | Maj | 17 | .2106 | .6235 | .1822 | .1833 | .4147 | .2491 |
| Ma | 1 | .2479 | .5491 | .3080 | .3841 | .2578 | .4113 | |
| Mi | 9 | .2102 | .6054 | .1811 | .2222 | .3142 | .2853 | |
| IV35 | Maj | 17 | .2072 | .6322 | .2339 | .3421 | .2719 | .2668 |
| Ma | 2 | .2332 | .5522 | .2879 | .3692 | .1839 | .4372 | |
| Mi | 18 | .2025 | .6319 | .2151 | .3087 | .3106 | .2611 |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +0.0176 [-0.0024, +0.0364] | -0.0495 [-0.0611, -0.0380] | +0.0002 [-0.0352, +0.0301] | -0.0401 [-0.0582, -0.0208] |
| Q25-1 | +0.0037 [-0.0022, +0.0113] | +0.0004 [-0.0037, +0.0048] | +0.0072 [-0.0053, +0.0259] | +0.0010 [-0.0059, +0.0082] |
| Q35 | +0.0611 [+0.0360, +0.0892] | -0.0076 [-0.0253, +0.0106] | +0.0822 [+0.0409, +0.1269] | +0.0870 [+0.0459, +0.1331] |
| IV35 | +0.0257 [-0.0008, +0.0509] | -0.0422 [-0.0552, -0.0293] | +0.0066 [-0.0167, +0.0313] | +0.0035 [-0.0201, +0.0293] |
| ESC-RL | +0.0120 [-0.0072, +0.0311] | -0.0362 [-0.0458, -0.0264] | -0.0121 [-0.0411, +0.0151] | -0.0377 [-0.0518, -0.0242] |
| MPO | +0.0150 [-0.0126, +0.0403] | -0.0632 [-0.0760, -0.0507] | -0.0373 [-0.0806, +0.0004] | -0.0674 [-0.0870, -0.0481] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +0.0575 [+0.0333, +0.0803] | -0.0196 [-0.0364, -0.0036] | +0.0652 [+0.0238, +0.1057] | +0.0128 [-0.0163, +0.0449] |
| Q25-1 | +0.0766 [+0.0558, +0.0986] | +0.0581 [+0.0431, +0.0734] | +0.0986 [+0.0655, +0.1395] | +0.0748 [+0.0465, +0.1092] |
| Q35 | +0.0393 [+0.0183, +0.0610] | -0.0141 [-0.0284, +0.0015] | +0.0381 [+0.0141, +0.0655] | +0.0438 [+0.0135, +0.0795] |
| IV35 | +0.0345 [+0.0070, +0.0624] | -0.0454 [-0.0602, -0.0313] | +0.0239 [-0.0024, +0.0527] | +0.0157 [-0.0105, +0.0449] |
| ESC-RL | +0.0761 [+0.0499, +0.1020] | +0.0016 [-0.0145, +0.0183] | +0.0650 [+0.0268, +0.0997] | +0.0178 [-0.0092, +0.0494] |
| MPO | +0.0547 [+0.0303, +0.0781] | -0.0329 [-0.0482, -0.0170] | +0.0127 [-0.0274, +0.0505] | -0.0279 [-0.0528, +0.0014] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +0.0376 [-0.0048, +0.0829] | -0.0563 [-0.0798, -0.0318] | +0.1269 [+0.0543, +0.2034] | +0.1618 [+0.0710, +0.2584] |
| IV35 | +0.0307 [-0.0124, +0.0770] | -0.0797 [-0.1077, -0.0515] | +0.0728 [-0.0135, +0.1632] | +0.0605 [-0.0307, +0.1541] |
| Q25-3 | +0.0815 [+0.0455, +0.1144] | -0.1812 [-0.2218, -0.1405] | +0.1375 [+0.0976, +0.1750] | +0.2195 [+0.1486, +0.2843] |
| Q25-1 | +0.0359 [+0.0187, +0.0565] | -0.2039 [-0.2356, -0.1722] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| ESC-RL | +0.0769 [+0.0474, +0.1080] | -0.1753 [-0.2099, -0.1408] | +0.1154 [+0.0748, +0.1570] | +0.1628 [+0.0812, +0.2420] |
| MPO | +0.0775 [+0.0415, +0.1138] | -0.1806 [-0.2195, -0.1418] | +0.1663 [+0.0953, +0.2376] | +0.1753 [+0.1040, +0.2410] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +0.0373 [-0.0209, +0.1073] | -0.0744 [-0.1027, -0.0453] | +0.1258 [+0.0258, +0.2337] | +0.2007 [+0.0939, +0.3109] |
| IV35 | +0.0260 [-0.0161, +0.0710] | -0.0800 [-0.1076, -0.0531] | +0.0540 [-0.0292, +0.1414] | +0.0271 [-0.0570, +0.1136] |
| Q25-3 | +0.1157 [+0.0888, +0.1411] | -0.1785 [-0.2205, -0.1368] | +0.1452 [+0.1076, +0.1805] | +0.2393 [+0.1794, +0.2965] |
| Q25-1 | +0.0359 [+0.0187, +0.0565] | -0.2046 [-0.2365, -0.1730] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| ESC-RL | +0.1178 [+0.0883, +0.1456] | -0.1894 [-0.2304, -0.1489] | +0.1535 [+0.1150, +0.1913] | +0.2654 [+0.2089, +0.3202] |
| MPO | +0.1060 [+0.0728, +0.1387] | -0.1780 [-0.2188, -0.1375] | +0.1744 [+0.1045, +0.2439] | +0.1957 [+0.1371, +0.2540] |
| Generator | Maj | Shared Ma | Shared Mi | Label Ma | NC | Const/AP | Deg. |
|---|---|---|---|---|---|---|---|
| Q25-3 | 0.3637 | 0.4212 | 0.4036 | 0.4206 | 0.4206 | 0.2172 | 0 |
| Q25-1 | 0.2885 | 0.3651 | 0.3614 | 0.4002 | 0.4002 | 0.2172 | 0 |
| Q35 | 0.3547 | 0.3940 | 0.3329 | 0.4310 | 0.4310 | 0.2172 | 0 |
| IV35 | 0.3770 | 0.4115 | 0.3858 | 0.4269 | 0.4269 | 0.2172 | 0 |
| ESC-RL | 0.3682 | 0.4442 | 0.4322 | 0.4612 | 0.4612 | 0.2172 | 0 |
| MPO | 0.3601 | 0.4148 | 0.3998 | 0.4444 | 0.4444 | 0.2172 | 0 |
| Generator | Maj | Shared Ma | Shared Mi | Label Ma | NC | Const/AP | Deg. |
|---|---|---|---|---|---|---|---|
| Q35 | 0.2106 | 0.2479 | 0.2102 | 0.2692 | 0.2454 | 0.1210 | 1 |
| IV35 | 0.2072 | 0.2332 | 0.2025 | 0.2693 | 0.2525 | 0.1210 | 1 |
| Q25-3 | 0.0556 | 0.1713 | 0.0899 | 0.1884 | 0.1884 | 0.1210 | 0 |
| Q25-1 | 0.0550 | 0.0909 | 0.0550 | 0.0909 | 0.0884 | 0.1210 | 5 |
| ESC-RL | 0.0556 | 0.1735 | 0.0965 | 0.1665 | 0.1665 | 0.1210 | 0 |
| MPO | 0.0552 | 0.1612 | 0.0837 | 0.1516 | 0.1516 | 0.1210 | 1 |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | -0.0212 [-0.0359, -0.0069] | -0.0374 [-0.0471, -0.0277] | -0.0183 [-0.0412, +0.0083] | -0.0420 [-0.0594, -0.0259] |
| Q25-1 | -0.0198 [-0.0316, -0.0077] | -0.0390 [-0.0505, -0.0273] | -0.0187 [-0.0364, +0.0011] | -0.0385 [-0.0574, -0.0195] |
| Q35 | -0.0104 [-0.0290, +0.0085] | +0.0260 [+0.0135, +0.0397] | -0.0224 [-0.0510, +0.0044] | +0.0011 [-0.0190, +0.0202] |
| IV35 | -0.0285 [-0.0459, -0.0126] | -0.0286 [-0.0432, -0.0149] | -0.0386 [-0.0638, -0.0130] | -0.0324 [-0.0504, -0.0155] |
| ESC-RL | -0.0253 [-0.0382, -0.0137] | -0.0377 [-0.0469, -0.0284] | -0.0341 [-0.0542, -0.0147] | -0.0431 [-0.0592, -0.0276] |
| MPO | -0.0320 [-0.0445, -0.0211] | -0.0385 [-0.0483, -0.0288] | -0.0507 [-0.0702, -0.0329] | -0.0457 [-0.0608, -0.0314] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | -0.0194 [-0.0382, -0.0016] | -0.0446 [-0.0568, -0.0327] | -0.0159 [-0.0450, +0.0110] | -0.0491 [-0.0726, -0.0290] |
| Q25-1 | -0.0260 [-0.0357, -0.0157] | -0.0456 [-0.0564, -0.0348] | -0.0251 [-0.0442, -0.0042] | -0.0450 [-0.0645, -0.0262] |
| Q35 | -0.0017 [-0.0208, +0.0167] | +0.0278 [+0.0148, +0.0415] | -0.0115 [-0.0417, +0.0150] | -0.0051 [-0.0244, +0.0131] |
| IV35 | -0.0346 [-0.0509, -0.0192] | -0.0395 [-0.0536, -0.0265] | -0.0568 [-0.0854, -0.0291] | -0.0518 [-0.0715, -0.0336] |
| ESC-RL | -0.0315 [-0.0468, -0.0167] | -0.0460 [-0.0584, -0.0337] | -0.0426 [-0.0704, -0.0165] | -0.0566 [-0.0782, -0.0364] |
| MPO | -0.0213 [-0.0358, -0.0069] | -0.0434 [-0.0539, -0.0331] | -0.0442 [-0.0639, -0.0254] | -0.0507 [-0.0683, -0.0344] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | -0.0018 [-0.0188, +0.0151] | +0.0072 [-0.0035, +0.0178] | -0.0024 [-0.0280, +0.0250] | +0.0071 [-0.0115, +0.0267] |
| Q25-1 | +0.0062 [-0.0069, +0.0198] | +0.0066 [-0.0030, +0.0164] | +0.0064 [-0.0156, +0.0286] | +0.0065 [-0.0100, +0.0233] |
| Q35 | -0.0087 [-0.0208, +0.0027] | -0.0018 [-0.0093, +0.0058] | -0.0108 [-0.0320, +0.0085] | +0.0062 [-0.0092, +0.0217] |
| IV35 | +0.0061 [-0.0058, +0.0183] | +0.0109 [-0.0014, +0.0234] | +0.0182 [-0.0066, +0.0438] | +0.0195 [+0.0024, +0.0371] |
| ESC-RL | +0.0062 [-0.0072, +0.0194] | +0.0083 [-0.0018, +0.0184] | +0.0085 [-0.0126, +0.0292] | +0.0135 [-0.0036, +0.0309] |
| MPO | -0.0108 [-0.0276, +0.0046] | +0.0050 [-0.0038, +0.0135] | -0.0065 [-0.0287, +0.0150] | +0.0050 [-0.0106, +0.0207] |
| Generator | RG | RL | GREEN | RaTE | B4 |
|---|---|---|---|---|---|
| Q25-3 | +.0257 | -.0144 | +.0005 | +.0028 | -.0064 |
| Q25-1 | +.0298 | -.0197 | +.0025 | +.0034 | -.0061 |
| Q35 | +.0103 | -.0009 | -.0023 | +.0034 | -.0000 |
| IV35 | +.0092 | -.0007 | +.0017 | +.0016 | -.0014 |
| ESC-RL | +.0252 | -.0197 | +.0030 | +.0046 | -.0103 |
| MPO | +.0255 | -.0150 | -.0023 | +.0076 | -.0064 |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +0.0128 [-0.0181, +0.0441] | -0.0131 [-0.0399, +0.0135] | +0.0049 [-0.0598, +0.0755] | -0.0308 [-0.1000, +0.0377] |
| IV35 | +0.0072 [-0.0362, +0.0447] | +0.1695 [+0.1272, +0.2106] | -0.0239 [-0.0860, +0.0331] | +0.0589 [-0.0077, +0.1221] |
| Q25-3 | -0.0095 [-0.0219, -0.0002] | -0.0038 [-0.0127, +0.0050] | -0.0056 [-0.0179, +0.0000] | -0.0134 [-0.0438, +0.0000] |
| Q25-1 | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| ESC-RL | -0.0029 [-0.0123, +0.0044] | -0.0089 [-0.0182, -0.0004] | +0.0051 [+0.0000, +0.0164] | +0.0128 [+0.0000, +0.0428] |
| MPO | +0.0081 [+0.0002, +0.0231] | +0.0131 [+0.0050, +0.0224] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +0.0370 [-0.0301, +0.0879] | -0.0522 [-0.0820, -0.0217] | -0.0037 [-0.0747, +0.0803] | -0.0434 [-0.1083, +0.0235] |
| IV35 | +0.0094 [-0.0292, +0.0489] | +0.1165 [+0.0750, +0.1578] | -0.0061 [-0.0577, +0.0440] | +0.0194 [-0.0474, +0.0857] |
| Q25-3 (3 epoch) | -0.0081 [-0.0208, +0.0019] | -0.0060 [-0.0159, +0.0036] | -0.0056 [-0.0179, +0.0000] | -0.0134 [-0.0438, +0.0000] |
| Q25-1 | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| ESC-RL | -0.0041 [-0.0129, +0.0006] | +0.0004 [-0.0054, +0.0054] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| MPO | +0.0006 [+0.0001, +0.0012] | +0.0120 [+0.0044, +0.0210] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | -0.0242 [-0.0731, +0.0351] | +0.0391 [+0.0125, +0.0667] | +0.0086 [-0.0541, +0.0724] | +0.0126 [-0.0378, +0.0654] |
| IV35 | -0.0022 [-0.0593, +0.0496] | +0.0530 [+0.0199, +0.0864] | -0.0178 [-0.0873, +0.0439] | +0.0396 [-0.0164, +0.0989] |
| Q25-3 | -0.0013 [-0.0050, +0.0008] | +0.0021 [-0.0072, +0.0113] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| Q25-1 | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| ESC-RL | +0.0012 [-0.0012, +0.0055] | -0.0093 [-0.0189, -0.0002] | +0.0051 [+0.0000, +0.0164] | +0.0128 [+0.0000, +0.0428] |
| MPO | +0.0075 [-0.0002, +0.0226] | +0.0011 [-0.0012, +0.0044] | +0.0000 [+0.0000, +0.0000] | +0.0000 [+0.0000, +0.0000] |
| Generator | RG | RL | GREEN | RaTE | B4 |
|---|---|---|---|---|---|
| Q35 | +.0094 | -.0058 | +.0052 | +.0054 | -.0003 |
| IV35 | +.0181 | -.0005 | -.0020 | -.0033 | -.0005 |
| Q25-3 | +.0662 | -.0116 | +.0176 | +.0357 | -.0039 |
| Q25-1 | +.0105 | -.0055 | -.0214 | +.0058 | -.0151 |
| ESC-RL | +.0500 | -.0206 | +.0014 | +.0305 | -.0148 |
| MPO | +.0693 | +.0006 | -.0267 | +.0307 | +.0082 |
| Utility | Ma-14 | Mi-14 | Ma-5 | Mi-5 | RG | RL | GREEN | RaTE | B4 |
|---|---|---|---|---|---|---|---|---|---|
| chexbert14 | 0.3814 | 0.5928 | 0.4955 | 0.5854 | 0.2942 | 0.3133 | 0.4988 | 0.6197 | 0.0864 |
| chexbert5 | 0.3710 | 0.5747 | 0.4937 | 0.5827 | 0.2927 | 0.3156 | 0.4964 | 0.6186 | 0.0873 |
| rougeL | 0.3571 | 0.5573 | 0.4622 | 0.5417 | 0.3124 | 0.3502 | 0.5023 | 0.6282 | 0.1011 |
| bleu4 | 0.3617 | 0.5601 | 0.4659 | 0.5437 | 0.3109 | 0.3471 | 0.5031 | 0.6284 | 0.1044 |
| embed | 0.3735 | 0.5716 | 0.4846 | 0.5648 | 0.3095 | 0.3323 | 0.5059 | 0.6293 | 0.0979 |
| bertscore | 0.3627 | 0.5640 | 0.4665 | 0.5458 | 0.3152 | 0.3429 | 0.5041 | 0.6291 | 0.0988 |
| Utility | Ma-14 | Mi-14 | Ma-5 | Mi-5 | RG | RL | GREEN | RaTE | B4 |
|---|---|---|---|---|---|---|---|---|---|
| chexbert14 | 0.0732 | 0.5468 | 0.0348 | 0.0366 | 0.3117 | 0.3099 | 0.6007 | 0.6220 | 0.0561 |
| chexbert5 | 0.0759 | 0.5317 | 0.0343 | 0.0331 | 0.3079 | 0.3089 | 0.5970 | 0.6197 | 0.0561 |
| rougeL | 0.0760 | 0.5519 | 0.0340 | 0.0347 | 0.3481 | 0.3415 | 0.6037 | 0.6378 | 0.0747 |
| bleu4 | 0.0748 | 0.5540 | 0.0340 | 0.0355 | 0.3424 | 0.3382 | 0.6020 | 0.6328 | 0.0788 |
| embed | 0.0743 | 0.5549 | 0.0309 | 0.0387 | 0.3547 | 0.3282 | 0.6163 | 0.6416 | 0.0706 |
| bertscore | 0.0756 | 0.5544 | 0.0341 | 0.0395 | 0.3553 | 0.3311 | 0.5921 | 0.6420 | 0.0658 |
| Dataset / contrast | Index | Text | Findings |
|---|---|---|---|
| MIMIC / CX vs RG | 0.9492 | 0.9492 | 0.3572 |
| MIMIC / CX vs RL | 0.9678 | 0.9678 | 0.3868 |
| MIMIC / RG vs RL | 0.8259 | 0.8259 | 0.3062 |
| IU / CX vs RG | 0.9784 | 0.9784 | 0.0844 |
| IU / CX vs RL | 0.9737 | 0.9737 | 0.0846 |
| IU / RG vs RL | 0.8837 | 0.8837 | 0.0631 |
| Dataset | Replacement label | Train prev. |
|---|---|---|
| MIMIC | Pleural Other | 0.0169 |
| MIMIC | Pneumothorax | 0.0334 |
| MIMIC | Consolidation | 0.0383 |
| MIMIC | Fracture | 0.0387 |
| MIMIC | Lung Lesion | 0.0395 |
| IU | Pneumothorax | 0.0043 |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +.0215 [+.0046, +.0384] | -.0162 [-.0217, -.0101] | -.0127 [-.0291, +.0043] | -.0133 [-.0200, -.0064] |
| Q25-1 | +.0060 [-.0126, +.0253] | -.0502 [-.0585, -.0419] | -.0079 [-.0278, +.0116] | -.0152 [-.0229, -.0078] |
| Q35 | -.0025 [-.0184, +.0130] | -.1237 [-.1328, -.1150] | -.0016 [-.0114, +.0068] | +.0029 [-.0004, +.0062] |
| IV35 | +.0188 [+.0004, +.0392] | -.0410 [-.0492, -.0331] | +.0058 [-.0052, +.0160] | +.0068 [+.0011, +.0124] |
| ESC-RL | +.0131 [+.0002, +.0263] | -.0123 [-.0165, -.0078] | -.0084 [-.0234, +.0068] | -.0095 [-.0150, -.0040] |
| MPO | +.0097 [-.0028, +.0228] | -.0171 [-.0216, -.0125] | -.0114 [-.0246, +.0021] | -.0125 [-.0185, -.0067] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +.0307 [+.0120, +.0504] | -.0154 [-.0255, -.0051] | +.0056 [-.0144, +.0257] | +.0062 [-.0068, +.0193] |
| Q25-1 | +.0301 [+.0087, +.0519] | -.0407 [-.0554, -.0269] | +.0254 [+.0005, +.0512] | +.0211 [+.0027, +.0406] |
| Q35 | +.0186 [+.0004, +.0370] | -.1037 [-.1172, -.0902] | +.0266 [+.0123, +.0420] | +.0408 [+.0239, +.0602] |
| IV35 | +.0257 [+.0056, +.0490] | -.0350 [-.0474, -.0227] | +.0301 [+.0147, +.0473] | +.0413 [+.0261, +.0587] |
| ESC-RL | +.0181 [+.0032, +.0340] | -.0109 [-.0198, -.0017] | +.0109 [-.0090, +.0309] | +.0102 [-.0029, +.0240] |
| MPO | +.0209 [+.0044, +.0377] | -.0153 [-.0248, -.0058] | +.0053 [-.0124, +.0243] | +.0046 [-.0090, +.0180] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +.0220 [+.0064, +.0396] | -.0010 [-.0031, +.0013] | +.0160 [-.0083, +.0429] | -.0024 [-.0065, +.0017] |
| Q25-1 | +.0317 [+.0102, +.0520] | -.0002 [-.0027, +.0023] | +.0222 [-.0068, +.0476] | -.0017 [-.0058, +.0019] |
| Q35 | +.0137 [-.0079, +.0331] | +.0014 [-.0017, +.0039] | +.0067 [-.0098, +.0199] | +.0066 [+.0024, +.0102] |
| IV35 | +.0383 [+.0155, +.0589] | +.0045 [+.0018, +.0074] | +.0060 [-.0155, +.0234] | +.0070 [+.0019, +.0118] |
| ESC-RL | +.0156 [-.0029, +.0327] | -.0014 [-.0036, +.0007] | +.0111 [-.0142, +.0361] | -.0018 [-.0054, +.0016] |
| MPO | +.0240 [+.0095, +.0389] | .0000 [-.0018, +.0019] | +.0283 [+.0085, +.0475] | +.0006 [-.0024, +.0035] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +.0243 [+.0065, +.0439] | -.0022 [-.0079, +.0034] | +.0215 [-.0069, +.0504] | +.0020 [-.0075, +.0112] |
| Q25-1 | +.0586 [+.0335, +.0818] | +.0121 [+.0045, +.0199] | +.0404 [+.0086, +.0685] | +.0153 [+.0028, +.0284] |
| Q35 | +.0371 [+.0114, +.0598] | +.0487 [+.0372, +.0603] | +.0388 [+.0191, +.0586] | +.0461 [+.0297, +.0646] |
| IV35 | +.0422 [+.0188, +.0646] | +.0146 [+.0068, +.0223] | +.0263 [+.0020, +.0482] | +.0330 [+.0184, +.0477] |
| ESC-RL | +.0222 [+.0019, +.0414] | -.0003 [-.0054, +.0050] | +.0152 [-.0114, +.0415] | +.0010 [-.0077, +.0093] |
| MPO | +.0319 [+.0146, +.0485] | +.0035 [-.0029, +.0102] | +.0390 [+.0155, +.0621] | +.0116 [+.0023, +.0216] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | -.0005 [-.0279, +.0241] | -.0152 [-.0209, -.0091] | -.0287 [-.0634, +.0033] | -.0109 [-.0185, -.0029] |
| Q25-1 | -.0258 [-.0573, +.0072] | -.0500 [-.0592, -.0409] | -.0301 [-.0533, -.0039] | -.0135 [-.0206, -.0063] |
| Q35 | -.0161 [-.0379, +.0078] | -.1250 [-.1339, -.1165] | -.0082 [-.0209, +.0075] | -.0036 [-.0073, +.0007] |
| IV35 | -.0195 [-.0433, +.0073] | -.0456 [-.0543, -.0369] | -.0002 [-.0197, +.0220] | -.0002 [-.0069, +.0066] |
| ESC-RL | -.0026 [-.0211, +.0180] | -.0109 [-.0153, -.0062] | -.0195 [-.0466, +.0097] | -.0077 [-.0136, -.0010] |
| MPO | -.0143 [-.0338, +.0050] | -.0171 [-.0216, -.0126] | -.0397 [-.0608, -.0192] | -.0131 [-.0189, -.0073] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q25-3 | +.0065 [-.0223, +.0331] | -.0131 [-.0231, -.0030] | -.0159 [-.0519, +.0182] | +.0043 [-.0080, +.0177] |
| Q25-1 | -.0285 [-.0650, +.0100] | -.0528 [-.0679, -.0387] | -.0149 [-.0403, +.0141] | +.0058 [-.0088, +.0218] |
| Q35 | -.0185 [-.0430, +.0093] | -.1524 [-.1634, -.1417] | -.0122 [-.0287, +.0057] | -.0054 [-.0167, +.0062] |
| IV35 | -.0164 [-.0426, +.0130] | -.0495 [-.0617, -.0377] | +.0038 [-.0186, +.0293] | +.0083 [-.0038, +.0212] |
| ESC-RL | -.0041 [-.0244, +.0188] | -.0106 [-.0192, -.0018] | -.0044 [-.0309, +.0258] | +.0092 [-.0035, +.0227] |
| MPO | -.0110 [-.0340, +.0132] | -.0188 [-.0279, -.0099] | -.0336 [-.0586, -.0083] | -.0070 [-.0205, +.0062] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +.0007 [-.0312, +.0332] | -.1956 [-.2169, -.1750] | +.0045 [-.0604, +.0651] | -.2368 [-.2921, -.1815] |
| IV35 | -.0408 [-.0810, +.0004] | -.1776 [-.1942, -.1612] | -.0598 [-.1202, +.0029] | -.2525 [-.3035, -.1984] |
| Q25-3 | -.0106 [-.0369, +.0097] | -.1975 [-.2243, -.1705] | +.0055 [+.0018, +.0101] | -.1414 [-.2178, -.0669] |
| Q25-1 | +.0158 [+.0057, +.0276] | -.0933 [-.1158, -.0716] | +.0123 [+.0018, +.0294] | +.0198 [+.0066, +.0364] |
| ESC-RL | -.0015 [-.0304, +.0222] | -.1469 [-.1683, -.1252] | +.0083 [+.0018, +.0175] | -.1078 [-.1674, -.0470] |
| MPO | -.0249 [-.0533, +.0058] | -.2826 [-.3138, -.2516] | -.0608 [-.1275, +.0059] | -.1913 [-.2647, -.1139] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | -.0357 [-.0795, +.0117] | -.3287 [-.3676, -.2918] | -.0310 [-.1101, +.0451] | -.2194 [-.2887, -.1504] |
| IV35 | -.0484 [-.0942, +.0007] | -.3221 [-.3537, -.2911] | -.0558 [-.1221, +.0167] | -.2471 [-.3013, -.1887] |
| Q25-3 | +.0198 [-.0158, +.0558] | -.1659 [-.2052, -.1266] | +.0495 [-.0024, +.1010] | -.0870 [-.1742, -.0038] |
| Q25-1 | +.0931 [+.0652, +.1195] | -.0061 [-.0423, +.0287] | +.1191 [+.0850, +.1515] | +.1173 [+.0924, +.1431] |
| ESC-RL | +.0515 [+.0176, +.0841] | -.1020 [-.1404, -.0642] | +.0632 [+.0152, +.1112] | -.0419 [-.1157, +.0312] |
| MPO | +.0009 [-.0355, +.0405] | -.2802 [-.3236, -.2370] | -.0428 [-.1230, +.0475] | -.1362 [-.2215, -.0487] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | -.0635 [-.1071, .0000] | -.0021 [-.0082, +.0019] | -.1333 [-.2000, .0000] | -.0095 [-.0363, +.0060] |
| IV35 | -.0476 [-.0714, .0000] | .0000 [-.0034, +.0028] | -.1333 [-.2000, .0000] | -.0045 [-.0279, +.0105] |
| Q25-3 | .0000 [.0000, .0000] | +.0013 [.0000, +.0033] | .0000 [.0000, .0000] | +.0014 [.0000, +.0051] |
| Q25-1 | .0000 [.0000, .0000] | .0000 [.0000, .0000] | .0000 [.0000, .0000] | .0000 [.0000, .0000] |
| ESC-RL | .0000 [.0000, .0000] | +.0006 [.0000, +.0021] | .0000 [.0000, .0000] | .0000 [.0000, .0000] |
| MPO | .0000 [.0000, .0000] | +.0007 [.0000, +.0021] | .0000 [.0000, .0000] | +.0012 [.0000, +.0045] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | -.0788 [-.1350, +.0113] | +.0042 [-.0136, +.0216] | -.1549 [-.2564, +.0192] | +.0343 [-.0548, +.1214] |
| IV35 | -.0748 [-.1161, -.0078] | -.0033 [-.0179, +.0121] | -.1781 [-.2805, -.0057] | -.0314 [-.1010, +.0331] |
| Q25-3 | +.0179 [-.0156, +.0467] | +.0024 [-.0153, +.0201] | +.0698 [+.0203, +.1206] | +.1789 [+.0861, +.2733] |
| Q25-1 | +.0705 [+.0523, +.0872] | +.0116 [-.0118, +.0356] | +.1520 [+.1084, +.1914] | +.3464 [+.2584, +.4246] |
| ESC-RL | +.0137 [-.0045, +.0336] | -.0003 [-.0178, +.0172] | +.0257 [-.0193, +.0758] | +.0927 [+.0162, +.1741] |
| MPO | +.0173 [-.0167, +.0509] | -.0026 [-.0218, +.0169] | +.0778 [+.0172, +.1388] | +.2226 [+.1126, +.3282] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +.0642 [-.0304, +.1348] | -.1935 [-.2167, -.1698] | +.1378 [-.0595, +.2558] | -.2273 [-.2907, -.1571] |
| IV35 | +.0068 [-.0456, +.0458] | -.1775 [-.1937, -.1614] | +.0736 [-.0961, +.1855] | -.2480 [-.3010, -.1907] |
| Q25-3 | -.0106 [-.0369, +.0097] | -.1988 [-.2258, -.1719] | +.0055 [+.0018, +.0101] | -.1428 [-.2195, -.0684] |
| Q25-1 | +.0158 [+.0057, +.0276] | -.0933 [-.1158, -.0716] | +.0123 [+.0018, +.0294] | +.0198 [+.0066, +.0364] |
| ESC-RL | -.0015 [-.0304, +.0222] | -.1476 [-.1690, -.1257] | +.0083 [+.0018, +.0175] | -.1078 [-.1674, -.0470] |
| MPO | -.0249 [-.0533, +.0058] | -.2833 [-.3145, -.2521] | -.0608 [-.1275, +.0059] | -.1926 [-.2664, -.1150] |
| Generator | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|
| Q35 | +.0431 [-.0648, +.1237] | -.3329 [-.3720, -.2943] | +.1239 [-.0850, +.2545] | -.2537 [-.3556, -.1518] |
| IV35 | +.0263 [-.0396, +.0773] | -.3189 [-.3540, -.2843] | +.1223 [-.0680, +.2460] | -.2156 [-.2991, -.1271] |
| Q25-3 | +.0019 [-.0360, +.0401] | -.1683 [-.2090, -.1270] | -.0204 [-.0891, +.0426] | -.2659 [-.3959, -.1389] |
| Q25-1 | +.0226 [-.0019, +.0479] | -.0177 [-.0558, +.0188] | -.0329 [-.0674, +.0040] | -.2290 [-.3040, -.1488] |
| ESC-RL | +.0378 [+.0030, +.0706] | -.1017 [-.1417, -.0622] | +.0374 [-.0112, +.0841] | -.1346 [-.2326, -.0382] |
| MPO | -.0164 [-.0579, +.0270] | -.2776 [-.3205, -.2350] | -.1206 [-.2157, -.0115] | -.3589 [-.4876, -.2293] |
| Generator | Ma near-zero | Ma assertions | Ma added | Ma add prec. | Mi near-zero | Mi assertions | Mi added | Mi add prec. |
|---|---|---|---|---|---|---|---|---|
| Q25-3 | 0 | 2.6851 | 794 | 0.2607 | 0 | 2.1522 | 218 | 0.3028 |
| Q25-1 | 0 | 3.0677 | 1476 | 0.2188 | 0 | 2.1620 | 376 | 0.4122 |
| Q35 | 0 | 3.9679 | 2535 | 0.1014 | 0 | 2.2849 | 477 | 0.4088 |
| IV35 | 0 | 2.9958 | 1173 | 0.1995 | 0 | 2.2004 | 376 | 0.4043 |
| ESC-RL | 0 | 2.6110 | 685 | 0.2686 | 0 | 2.1613 | 226 | 0.3319 |
| MPO | 0 | 2.6578 | 772 | 0.2552 | 0 | 2.1508 | 236 | 0.3644 |
| Generator | Ma near-zero | Ma assertions | Ma added | Ma add prec. | Mi near-zero | Mi assertions | Mi added | Mi add prec. |
|---|---|---|---|---|---|---|---|---|
| Q35 | 1 | 5.6121 | 1874 | 0.0454 | 0 | 1.1996 | 59 | 0.3559 |
| IV35 | 1 | 4.9798 | 1590 | 0.0314 | 0 | 1.1525 | 29 | 0.3448 |
| Q25-3 | 4 | 9.7601 | 3760 | 0.0662 | 0 | 1.2534 | 94 | 0.3191 |
| Q25-1 | 6 | 10.5448 | 4177 | 0.0864 | 0 | 1.3117 | 157 | 0.3376 |
| ESC-RL | 2 | 8.3879 | 3166 | 0.0853 | 0 | 1.2466 | 90 | 0.3000 |
| MPO | 2 | 10.1323 | 3927 | 0.0507 | 0 | 1.2735 | 119 | 0.3109 |
| Label | Train prev. | Ma threshold | Ma pos. rate | Ma add prec. | Mi threshold | Mi pos. rate | Mi add prec. |
|---|---|---|---|---|---|---|---|
| No Finding | 0.2375 | 0.3540 | 0.4118 | 0.3614 | 0.3172 | 0.4375 | 0.2077 |
| Enlarged Cardiomediastinum | 0.0931 | 0.0802 | 0.2312 | 0.1115 | 0.3172 | 0.0045 | 0.8824 |
| Cardiomegaly | 0.2576 | 0.3132 | 0.2978 | 0.4626 | 0.3172 | 0.2948 | 0.4140 |
| Lung Lesion | 0.0395 | 0.1100 | 0.0488 | 0.1896 | 0.3172 | 0.0063 | 0.2857 |
| Lung Opacity | 0.3045 | 0.2864 | 0.4273 | 0.3912 | 0.3172 | 0.3990 | 0.3393 |
| Edema | 0.0955 | 0.2771 | 0.1332 | 0.5294 | 0.3172 | 0.1161 | 0.5587 |
| Label | Train prev. | Ma threshold | Ma pos. rate | Ma add prec. | Mi threshold | Mi pos. rate | Mi add prec. |
|---|---|---|---|---|---|---|---|
| No Finding | 0.6270 | 0.3676 | 0.8360 | 0.6433 | 0.2710 | 0.9283 | 0.2550 |
| Enlarged Cardiomediastinum | 0.0268 | 0.0044 | 0.8420 | 0.0195 | 0.2710 | 0.0000 | – |
| Cardiomegaly | 0.0875 | 0.1132 | 0.4250 | 0.1722 | 0.2710 | 0.1261 | 0.4353 |
| Lung Lesion | 0.0446 | 0.0107 | 0.9719 | 0.0402 | 0.2710 | 0.0000 | – |
| Lung Opacity | 0.1258 | 0.1537 | 0.4310 | 0.1754 | 0.2710 | 0.1578 | 0.2811 |
| Edema | 0.0126 | 0.0037 | 0.9922 | 0.0107 | 0.2710 | 0.0000 | – |
| Generator | Unique | Zero error (%) | Oracle errors | Ma errors | Mi errors | Ma missed (%) | Mi missed (%) |
|---|---|---|---|---|---|---|---|
| Q25-3 | 8.4525 | 67.5978 | 0.4043 | 1.3436 | 1.2325 | 56.8436 | 53.7011 |
| Q25-1 | 8.6711 | 62.9888 | 0.4839 | 1.3561 | 1.2626 | 52.6536 | 51.0475 |
| Q35 | 4.7968 | 38.0587 | 1.1746 | 1.6948 | 1.5656 | 30.2374 | 24.8603 |
| IV35 | 6.0810 | 50.2095 | 0.8080 | 1.4560 | 1.3226 | 38.1983 | 33.1704 |
| ESC-RL | 8.9246 | 69.4134 | 0.3757 | 1.3547 | 1.2318 | 56.2151 | 54.0503 |
| MPO | 9.3149 | 67.2486 | 0.4351 | 1.3715 | 1.2584 | 53.8408 | 51.0475 |
| Generator | Cond. | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|---|
| Q25-3 | Oracle | 0.6822 | 0.8797 | 0.8344 | 0.9016 |
| Q25-3 | Macro C | 0.4722 | 0.6465 | 0.6158 | 0.6602 |
| Q25-3 | Micro C | 0.4382 | 0.6683 | 0.5992 | 0.6752 |
| Q25-1 | Oracle | 0.5996 | 0.8520 | 0.7825 | 0.8675 |
| Q25-1 | Macro C | 0.4607 | 0.6318 | 0.5938 | 0.6443 |
| Q25-1 | Micro C | 0.4073 | 0.6516 | 0.5808 | 0.6620 |
| Generator | Unique | Zero error (%) | Oracle errors | Ma errors | Mi errors | Ma missed (%) | Mi missed (%) |
|---|---|---|---|---|---|---|---|
| Q35 | 2.0673 | 67.0404 | 0.6009 | 0.9709 | 0.8072 | 16.1435 | 13.4529 |
| IV35 | 3.2601 | 71.3004 | 0.4753 | 0.9664 | 0.7691 | 23.0942 | 18.1614 |
| Q25-3 | 3.9776 | 73.9910 | 0.3498 | 1.5179 | 0.8274 | 61.6592 | 33.6323 |
| Q25-1 | 2.9641 | 68.6099 | 0.5022 | 1.7892 | 0.8274 | 73.0942 | 27.3543 |
| ESC-RL | 4.1143 | 76.9058 | 0.3251 | 1.5785 | 0.8117 | 66.1435 | 33.1839 |
| MPO | 3.5000 | 72.1973 | 0.3969 | 1.2578 | 0.8072 | 48.2063 | 29.3722 |
| Generator | Cond. | Ma-14 | Mi-14 | Ma-5 | Mi-5 |
|---|---|---|---|---|---|
| Q35 | Oracle | 0.3715 | 0.7248 | 0.5009 | 0.5793 |
| Q35 | Macro C | 0.2530 | 0.6110 | 0.2581 | 0.3779 |
| Q35 | Micro C | 0.2499 | 0.6386 | 0.3146 | 0.3151 |
| IV35 | Oracle | 0.4754 | 0.7850 | 0.5556 | 0.7200 |
| IV35 | Macro C | 0.2656 | 0.6114 | 0.3044 | 0.4071 |
| IV35 | Micro C | 0.2479 | 0.6567 | 0.3641 | 0.4217 |
| Objective | Target | Route | Ret. | Ins. | Ham. | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|---|
| macro | C | R | .8946 | .0360 | .0211 | .4375 | .2722 | .6010 | .2886 | .0799 |
| macro | C | G | .8919 | .0309 | .0208 | .4890 | .3200 | .6453 | .3575 | .1161 |
| macro | P | R | .8702 | .0315 | .0262 | .4274 | .2687 | .5992 | .2864 | .0777 |
| macro | P | G | .8645 | .0271 | .0264 | .4817 | .3147 | .6434 | .3542 | .1167 |
| macro | F | R | .8286 | .0284 | .0375 | .4243 | .2647 | .5973 | .2825 | .0723 |
| macro | F | G | .7958 | .0221 | .0426 | .4813 | .3156 | .6442 | .3529 | .1184 |
| Objective | Target | Route | Ret. | Ins. | Ham. | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|---|
| macro | C | R | .9634 | .0019 | .0034 | .4867 | .2617 | .5680 | .2707 | .0494 |
| macro | C | G | .8626 | .0673 | .0175 | .4963 | .3154 | .5953 | .3329 | .0961 |
| macro | P | R | .4443 | .0031 | .1954 | .4367 | .2182 | .5460 | .2303 | .0277 |
| macro | P | G | .4589 | .1876 | .2269 | .2920 | .2195 | .5162 | .2706 | .0612 |
| macro | F | R | .3639 | .0038 | .4444 | .0774 | .1049 | .4286 | .2083 | .0198 |
| macro | F | G | .3469 | .1332 | .4925 | .1981 | .1975 | .4991 | .2402 | .0454 |
| Data | Obj. | T/R | ||||
|---|---|---|---|---|---|---|
| MIMIC | macro | C/R | 121 | 167 | 31 | 69 |
| MIMIC | macro | C/G | 117 | 173 | 26 | 59 |
| MIMIC | macro | P/R | 158 | 235 | 23 | 69 |
| MIMIC | macro | P/G | 149 | 265 | 21 | 57 |
| MIMIC | macro | F/R | 206 | 381 | 19 | 70 |
| MIMIC | macro | F/G | 215 | 488 | 20 | 47 |
| Method | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|
| Plurality | .3900 | .6214 | .5229 | .6209 | .4947 | .3227 | .6307 | .3518 | .1124 |
| Most-typical | .3900 | .6214 | .5229 | .6209 | .4947 | .3227 | .6307 | .3518 | .1124 |
| Consensus-CB | .4005 | .6331 | .5338 | .6361 | .4876 | .3158 | .6425 | .3282 | .1016 |
| CCS-Emb | .3881 | .6092 | .5178 | .6059 | .4952 | .3313 | .6465 | .3617 | .1203 |
| Classifier probe | .4400 | .6163 | .5625 | .6348 | .4783 | .3106 | .6385 | .3234 | .1063 |
| CLIPScore | .4037 | .6046 | .5295 | .6101 | .4812 | .3118 | .6266 | .3346 | .1131 |
| Configuration | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|
| Macro, C G | .4334 | .6260 | .5901 | .6556 | .4818 | .3428 | .6454 | .3613 | .1195 |
| Macro, P G | .4669 | .6198 | .5843 | .6444 | .4796 | .3419 | .6452 | .3586 | .1208 |
| Macro, F G | .4777 | .6338 | .6031 | .6622 | .4797 | .3417 | .6469 | .3585 | .1231 |
| Micro, C G | .4130 | .6415 | .5627 | .6705 | .4818 | .3436 | .6450 | .3622 | .1194 |
| Micro, P G | .4315 | .6402 | .5811 | .6684 | .4818 | .3424 | .6440 | .3619 | .1200 |
| Micro, F G | .4287 | .6408 | .5803 | .6721 | .4843 | .3431 | .6448 | .3615 | .1207 |
| Method | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|
| Plurality | .0586 | .5940 | .0082 | .0202 | .5678 | .3129 | .6373 | .3009 | .0523 |
| Most-typical | .0875 | .5966 | .0082 | .0202 | .6028 | .3810 | .6510 | .3397 | .0751 |
| Consensus-CB | .0663 | .5924 | .0000 | .0000 | .5668 | .3122 | .6370 | .3003 | .0511 |
| CCS-Emb | .0775 | .5842 | .0000 | .0000 | .6022 | .3740 | .6454 | .3383 | .0785 |
| Classifier probe | .1266 | .5156 | .1046 | .1795 | .5259 | .2977 | .6131 | .2930 | .0578 |
| CLIPScore | .1762 | .5635 | .0987 | .1653 | .5689 | .3257 | .6222 | .3262 | .0796 |
| Configuration | Ma-14 | Mi-14 | Ma-5 | Mi-5 | GREEN | RG | RaTE | RL | B4 |
|---|---|---|---|---|---|---|---|---|---|
| Macro, C G | .1565 | .5549 | .1663 | .3425 | .5822 | .3772 | .6436 | .3551 | .0947 |
| Macro, P G | .0747 | .1387 | .1050 | .1714 | .2947 | .2459 | .5138 | .2674 | .0581 |
| Macro, F G | .0854 | .1258 | .1416 | .2183 | .2197 | .2327 | .5092 | .2520 | .0514 |
| Micro, C G | .1275 | .6262 | .1510 | .3239 | .6033 | .4002 | .6667 | .3670 | .0934 |
| Micro, P G | .1275 | .6275 | .1510 | .3262 | .6072 | .4020 | .6679 | .3675 | .0938 |
| Micro, F G | .1402 | .6203 | .2131 | .3676 | .5983 | .3992 | .6643 | .3667 | .0948 |
Radiology Report Generation
Radiology report generation uses medical images and clinical context to produce structured, clinically accurate findings and impressions.
See all 159 papers