Visual counterfactual explanations (VCEs) should alter a classifier's prediction while remaining realistic and preserving decision-irrelevant content. Image-space methods provide high-quality counterfactuals but are computationally costly, whereas latent-space methods are more efficient but often less faithful. We identify two sources responsible for this gap. First, latent generation recovers a natural sample from a perturbed representation, whereas classifier guidance pushes it toward a target class, so their update directions may oppose. Second, VAE reconstruction can alter regions that should remain unchanged. We introduce CLEAR, a latent Flow Matching framework for VCEs. CLEAR uses Flow Matching's explicit instantaneous velocity as a reference for the generative direction to decompose classifier guidance into cooperative and conflicting components. It preserves the cooperative component and adaptively attenuates only the conflicting one according to conflict strength. We further analyze conflict frequency and strength throughout generation to reveal how classifier guidance interacts with the generative process. CLEAR also dynamically identifies editable regions, constrains latent modifications, and directly preserves factual content outside them to reduce VAE-induced changes. Across five bidirectional tasks on two datasets, CLEAR retains latent-space efficiency while achieving counterfactual quality comparable to or better than strong image-space baselines, with a 2.5x speedup over the strongest baseline. Project page: https://yan-zen9.github.io/CLEAR/
Figures & tables
Figure 1: Motivation for Cooperative Flow Guidance. Additive guidance may cause excessive editing, while hard projection can lead to insufficient editing. CLEAR adaptively regulates conflicting guidance to produce a natural and effective counterfactual.
Figure 2: Overview of CLEAR. Starting from a partially noised factual latent, CLEAR evolves the counterfactual along a guided latent flow. At each timestep, the flow model provides the generative velocity and a one-step image estimate, from which the target classifier produces both the classifier force and the current decision-relevant support. The image-space support is mapped to the latent grid to constrain where classifier-driven modifications are introduced, while Cooperative Flow Guidance determines how the classifier force modifies the generative direction. Repeating these coupled updates yields a counterfactual that concentrates changes within the evolving editable support while preserving decision-irrelevant factual content.
Figure 3: From sampler-dependent denoising to cooperative flow guidance. (a) In DDPMs, the predicted noise is converted into a state update through a chosen sampler, making the update direction sampler-dependent. (b) Flow Matching directly predicts the instantaneous velocity vt , providing an explicit generative direction for comparison with the classifier force. (c) CLEAR reconciles the classifier force ut=−gt according to its directional conflict with vt . In the non-conflicting case (i.e. ⟨gt,vt⟩≤0 ), illustrated here by an orthogonal example, ut is retained. Under mild or severe conflict, CLEAR decomposes ut into parallel and orthogonal components and increasingly attenuates only the opposing component as the conflict strength grows, yielding the reconciled force u^t and guided velocity v^t=vt+αtu^t .
Smile
Age
Method
FID ↓
sFID ↓
FVA ↑
FS ↑
MNAC ↓
CD ↓
COUT ↑
FR(%) ↑
FID ↓
sFID ↓
FVA ↑
FS ↑
MNAC ↓
CD ↓
COUT ↑
FR(%) ↑
DiVE [ 36 ]
107.0
–
35.7
–
7.41
–
–
–
107.5
–
32.3
–
6.76
–
–
–
STEEX [ 23 ]
21.9
–
97.6
–
5.27
–
–
–
26.8
–
96.0
–
5.63
–
–
–
DiME [ 24 ]
18.1
27.7
96.7
0.67
2.63
1.82
0.65
97.0
18.7
27.8
95.0
0.66
2.10
4.29
0.56
97.0
ACE ℓ1 [ 25 ]
3.21
20.2
100.0
0.89
1.56
2.61
0.55
95.0
5.31
21.7
99.6
0.81
1.53
5.40
0.40
95.0
ACE ℓ2 [ 25 ]
6.93
22.0
100.0
0.84
1.87
2.21
0.60
95.0
16.4
28.2
99.6
0.77
1.92
4.21
0.53
95.0
Table 1: Quantitative comparison on CelebA-HQ across the Smile and Age tasks. Methods marked with † operate in latent space. Best and second-best results are highlighted in bold and underline , respectively.
Figure 4: Inference efficiency and counterfactual quality on CelebA-HQ Smile . All methods are evaluated on 1,000 images at 256×256 resolution with a batch size of 3. Runtime and peak GPU memory measure counterfactual inference only. † denotes a latent-space method.
Figure 5: Qualitative comparison on CelebA-HQ using DenseNet-121. From left to right: the factual image and counterfactual images generated by MaskDiME and CLEAR.
Method
Sorrel – Zebra
Persian Cat – Egyptian Cat
Cougar – Cheetah
FID ↓
sFID ↓
S3↑
COUT ↑
FR(%) ↑
FID ↓
sFID ↓
S3↑
COUT ↑
FR(%) ↑
FID ↓
sFID ↓
S3↑
COUT ↑
FR(%) ↑
DiME [ 24 ]
222.9
243.2
0.19
−0.31
0.0
322.8
352.1
0.44
−0.05
0.0
268.2
292.0
0.11
−0.16
0.0
ACE ℓ1 [ 25 ]
84.5
122.7
0.92
−0.45
47.0
93.6
156.7
0.85
0.25
85.0
70.2
100.5
0.91
0.02
77.0
ACE ℓ2 [ 25 ]
67.7
98.4
0.90
−0.25
81.0
107.3
160.4
0.78
0.34
97.0
74.1
102.5
0.88
0.12
95.0
FastDiME [ 46 ]
96.5
103.5
0.22
−0.44
14.0
193.6
207.1
0.10
0.01
20.0
133.0
141.1
0.12
−0.11
18.0
DVCE [ 1 ]
33.1
43.9
0.62
−0.21
57.8
46.6
59.2
0.59
0.60
98.5
46.9
54.1
0.70
0.49
99.0
Table 2: Quantitative comparison on ImageNet across three semantic class pairs. Methods marked with † operate in latent space. Best and second-best results are highlighted in bold and underline , respectively.
Figure 6: Qualitative results on ImageNet using ResNet-50. Each pair shows a factual image and the corresponding counterfactual generated by CLEAR for the paired target class.
Figure 7: Directional conflict analysis during generation. (a) Conflict frequency across Flow Matching timesteps. (b) Conflict strength, measured by the relative magnitude ∥ut∥∥2/∥vt∥2 . (c) Percentage of the conflicting component retained after Cooperative Flow Guidance, with Hard Projection and Additive Guidance corresponding to 0% and 100% retention, respectively.
Figure 8: Qualitative results under different conflict regimes. Rows show examples with different combinations of conflict frequency and strength. CLEAR consistently produces more natural and effective counterfactuals.
Variant
FID ↓
MNAC ↓
COUT ↑
FR(%) ↑
MSE ↓
CLEAR
2.18
1.30
0.65
99.6
0.000
Cooperative guidance
Additive Guidance
4.47
2.16
0.71
99.7
0.000
Hard Projection
3.66
1.63
0.64
95.6
0.000
Adaptive Support Preservation
Fixed Mask
3.40
1.57
0.61
93.7
0.000
Table 3: Main ablation on CelebA-HQ Smile . Each variant changes only the specified component while keeping all other settings fixed. MSE is measured outside each variant’s final editable support and is zero by construction for variants using factual-pixel composition.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure S1 : Additional qualitative results under different conflict regimes. Results on CelebA-HQ Age and ImageNet further demonstrate the consistent behavior of CLEAR across different conflict settings.
Figure S2 : Visualization of localized counterfactual editing. The editable support concentrates changes in target-relevant regions while preserving the remaining factual content.
Figure S3 : Exact preservation with Adaptive Support Preservation. From left to right: factual image, CLEAR counterfactual, and pixel-wise difference map. Pixels outside the editable support remain unchanged.
Setting
FID ↓
MNAC ↓
COUT ↑
FR(%) ↑
Default
2.18
1.30
0.65
99.6
Conflict sensitivity
τ=0.25
2.54
1.33
0.59
95.2
τ=1.0
4.28
1.71
0.68
99.7
Starting-time sensitivity
ts=0.4
5.42
3.27
0.74
99.9
Appendix
Table S1: Hyperparameter sensitivity on CelebA-HQ Smile . The default configuration uses τ=0.5 , ts=0.6 , N=20 , s=8 , and ρ=5% .
Counterfactual explanations show how changing an input could alter a model’s prediction, supporting model understanding, auditing, and actionable recourse.
Visual Counterfactual Explanations (VCEs) aim to explain image classifiers by generating minimally edited and realistic versions of an input image that change the classifier's prediction. Existing VCE methods are inherently classifier-dependent and therefore susceptible to classifier biases and failure modes, such as sensitivity to shortcut features and calibration errors. In this paper, we propose a classifier-free approach for visual counterfactual generation based on Contrastive Analysis (CA). Given two datasets corresponding to different classes (e.g., healthy and patients), we disentangle the generative factors that are common across the two datasets from those that are salient to each dataset, and generate counterfactual images by swapping only the salient factors. By operating directly on data distributions rather than decision boundaries, our method provides model-agnostic VCEs that are less sensitive to classifier biases. Our approach leverages the high-quality synthesis and well-structured latent space of StyleGAN2. We use the feature space F, instead than the usual W-space, to improve detail preservation. Unlike conventional CA approaches, which typically assume salient factors in only one dataset, we introduce an adapted framework and loss functions for VCE that allow multiple salient factors in each dataset. We evaluate our method on three medical imaging datasets and demonstrate superior counterfactual generation quality compared to existing approaches.
Yunlong He, Pietro Gori
LTCI, Télécom Paris, Institut Polytechnique de Paris, France
Visual counterfactual explanations aim to change classifier decisions through realistic and localized edits while preserving decision-irrelevant content. Existing DDPM-based methods typically perform classifier-guided editing along a long reverse denoising trajectory. The changing noise levels make semantic editability and spatial control difficult to balance, and the editable state is noisy, whereas the target classifier is trained on clean images. As a result, these methods require either costly recursive denoising or low-quality one-step estimates to obtain classifier-facing clean images. We propose FiRe, a Fixed-noise Refinement framework for visual counterfactual explanations. Rather than following a reverse denoising trajectory, FiRe maps the input to a fixed noise level and iteratively refines the noisy state at that level. To provide clean images for classifier guidance, FiRe first adapts Pixel Mean Flow to visual counterfactual explanation, enabling direct clean-image prediction from noisy states. To make fixed-noise refinement produce minimal and localized counterfactual edits, FiRe introduces three FiRe-specific controls: a dynamic dual-mask strategy, adaptive guidance, and early stopping, which determine where edits accumulate, which changes become visible, and when refinement stops. Experiments on five tasks across three datasets show that, compared with the strongest recent baseline, FiRe achieves about 3× faster online inference and 8× fewer FLOPs while obtaining comparable or state-of-the-art counterfactual quality.
Yan Zeng, Changlu Guo, Oskar Kristoffersen +3
Department of Applied Mathematics and Computer Science, Technical University of Denmark
Modern neural networks achieve strong performance but remain difficult to interpret in high-dimensional visual domains. Counterfactual explanations (CFEs) provide a principled approach to interpreting black-box predictions by identifying minimal input changes that alter model outputs. However, existing CFE methods often rely on dataset-specific generative models and incur substantial computational cost, limiting their scalability to high-resolution data. We propose SCE-LITE-HQ, a scalable framework for counterfactual generation that leverages pretrained generative foundation models without task-specific retraining. The method operates in the latent space of the generator, incorporates smoothed gradients to improve optimization stability, and applies mask-based diversification to promote realistic and structurally diverse counterfactuals. We evaluate SCE-LITE-HQ on natural and medical datasets using a desiderata-driven evaluation protocol. Results show that SCE-LITE-HQ produces valid, realistic, and diverse counterfactuals competitive with or outperforming existing baselines, while avoiding the overhead of training dedicated generative models.
Ahmed Zeid, Sidney Bender
Machine Learning Group, Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany