Deep models often rely on spurious correlations that fail on underrepresented groups. Existing methods either correct human-specified bias attributes or infer unknown biases from the learned representation space. However, annotated attributes may differ from the shortcuts actually used by a model, while a single representation provides incomplete and potentially noisy evidence. Our key insight is that a bias obscured in one feature space may be clearer in another. In this work, we propose CHORD (Cross-view Harmonic Outlier Reversal for Debiasing), a multi-view framework that allows samples to "sing in chorus" across diverse representation spaces to achieve robust cross-view consensus. CHORD aggregates aggregates within-class partitions in each view, which is a proxy for specifying candidate biases, to debias the target representation and classifier without bias supervision. Experiments on seven benchmarks across vision, language, speech, and medical imaging show consistent improvements in worst-group accuracy. Code and checkpoints are available at https://github.com/davelee-uestc/chord_debiasing_public .
Figures & tables
Figure 1: Overview of CHORD. CHORD separates proposing biases from testing them for reliable bias discovery in the target model. First, each view proposes a candidate bias. The intuition is that a noisy candidate bias pattern in one representation may be denoised in another. Each candidate partitions samples into majority and outlier subsets. We then test the candidate in the target space by removing the feature difference between the two subsets. Removing a real bias improves within-class compactness by moving outliers closer to their class centroid. Outliers whose nearest-centroid assignments return to their labeled class are used as minority candidates. Finally, we aggregate this evidence across views to retrain the classifier head.
Labels
Waterbirds
CelebA
MultiNLI
CivilComments
Mean
Tr.
Val
mAcc
WGA
mAcc
WGA
mAcc
WGA
mAcc
WGA
WGA
ERM
✗
✗
97.30
72.60
95.60
47.20
82.09
68.12
92.34
57.06
61.25
JTT [ 31 ]
✗
✓
93.30
86.70
88.00
81.10
78.60
72.60
91.10
69.30
77.43
MT [ 4 ]
✗
✗
93.00
86.40
91.30
78.00
———— Not Applicable ————
CNC [ 54 ]
✗
✗
90.90
88.50
89.90
88.80
-
-
81.70
68.90
-
AFR [ 38 ]
✗
✓
94.20
90.40
91.30
82.00
81.40
73.40
89.80
68.70
78.63
Table 1: Mean accuracy (mAcc) and worst-group accuracy (WGA) on four widely used benchmarks. For CHORD, we report means over 10 classifier seeds with fixed ERM features, with the standard deviations shown for WGA.
CREMA-D
MEAD
Mean
Method
mAcc
WGA
mAcc
WGA
mAcc
WGA
ERM
77.08
59.63
60.51
48.35
68.80
53.99
NSF
76.99
60.63
61.67
50.77
69.33
55.70
CHORD
77.11
67.02
61.19
52.84
69.15
59.93
GDRO ∗ [ 39 ]
76.02
59.20
61.30
47.29
68.67
53.25
Table 2: Macro-average of mAcc and WGA on CREMA-D and MEAD. GDRO ∗ averages attribute-matched experts, requiring oracle model switching.
Bias Label
Chexpert
Train
Val
mAcc
WGA
Gain
ERM
✗
✗
89.78
31.21
-
NSF
✗
✗
81.94
70.21
+39.00
CHORD
✗
✗
81.62
71.43
+40.22
GDRO [ 39 ]
✓
✓
78.90
74.50
+43.29
Table 3: Results on CheXpert. Both mAcc and WGA are reported; group-supervised GDRO is included as a reference.
Method
MV
SW
MD
WGA
ERM
47.20
Single View
✓
84.27
MV-Hard
✓
✓
86.78
CHORD
✓
✓
✓
89.06
Table 4: Component ablation on CelebA. MV adds multiple reversal-sample sets, SW aggregates their support as soft sampling weights, and MD uses the target transformation t0 for the final classifier update. We report mean WGA over ten random seeds.
Target Only: 84.27
Conf.
WGA
Conf.
WGA
BT
86.32
DINO
87.39
BLIP
87.49
SimCLR
87.78
CLIP
86.84
SwAV
86.71
Table 5: Effect of individual pretrained views on CelebA. We report WGA after adding one view to Target Only in (a) or removing one from CHORD in (b). Every addition improves WGA, while most removals lower it. BT denotes Barlow Twins.
Method
Target View
Rep. Views
Geo. View
WGA
ERM
68.12
Target Only
✓
73.12
Target + Pre. Views
✓
✓
74.19
Target + Geo. View
✓
✓
73.36
CHORD
✓
✓
✓
75.25
Table 6: View-level ablation on MultiNLI.
ResNet-50
ViT-B/16
Dataset
Method
mAcc
WGA
mAcc
WGA
Waterbirds
ERM
97.30
72.60
92.89
78.97
CHORD
95.48
93.11
96.64
89.42
CelebA
ERM
95.60
47.20
96.21
39.72
CHORD
90.78
89.06
93.26
87.36
Table 7: Cross-architecture comparison on Waterbirds and CelebA. We report mean accuracy (mAcc) and worst-group accuracy (WGA) for ResNet-50 and ViT-B/16. CHORD improves WGA in all four dataset–architecture settings.
Method
Train attr.
mAcc
WGA by evaluation attribute
Sex
Race
Age-B
Age-Q
SexRace
Mean
ERM
–
77.08
62.00
61.54
62.50
52.63
59.46
59.63
CHORD
–
77.11
66.70
69.62
69.41
64.52
64.86
67.02
GDRO
Sex
75.92
59.41
62.32
61.32
55.10
59.38
59.51
Race
74.04
55.56
57.89
60.49
53.49
48.44
55.17
Age-B
77.08
63.37
63.08
64.15
55.56
56.67
60.57
Table 8: Multi-attribute evaluation on CREMA-D (%). Each row evaluates one model across five post-hoc group definitions. Train attr. specifies GDRO’s group annotation. Underlined entries are the attribute-matched GDRO results. CHORD results are averaged over 10 classifier seeds on fixed ERM features; GDRO uses one model seed per attribute. Age-B/Q denote binary/quartile age groups; SexRace denotes sex × race. CHORD achieves highest WGA under all five definitions without attribute-specific training.
ERM
CHORD
Target
Eval. Attribute
mAcc
WGA
mAcc
WGA
Blond Hair
Gender
95.60
47.20
90.78
89.06±0.50
Gender
Blond Hair
98.64
87.22
97.73
96.03±0.51
Table 9: Attribute-role exchange on CelebA to assess transfer across prediction tasks. Blond Hair and Gender exchange their roles as the prediction target and evaluation attribute, using the standard ResNet-50 CHORD configuration without task-specific retuning. We report means over 10 random seeds, with WGA standard deviations. CHORD improves WGA under both assignments.