Whose Polarization Is It? Auditing Fairness in Network Polarization Measures

Stanisław Stępień; Michalina Janik; Akrati Saxena; Radosław Michalski

cs.SIOct 15, 2026

Whose Polarization Is It? Auditing Fairness in Network Polarization Measures

Authors: Stanisław Stępień, Michalina Janik, Akrati Saxena, Radosław Michalski

Organizations: Wroclaw University of Science and Technology, Wrocław, Poland · Leiden University, Leiden, The Netherlands

Abstract

A polarization score is supposed to describe a community, not its surroundings. Yet when a minority community is embedded in a larger network, its measured polarization can rise or fall with the size and structure of the majority around it. We ask whether polarization measures are therefore fair to minorities and develop an audit built around two criteria: measurement invariance (MI), which asks whether otherwise equivalent majorities and minorities receive equivalent scores, and structural exogeneity (SE), which asks whether a minority's score changes when its surrounding majority changes. Resolution sensitivity (RS) is treated as a prerequisite for meaningful fairness assessment. We audit seven polarization measure families spanning the three network-operationalizable dimensions of polarization, introducing Network-Weighted Distance (NWD) to fill an unserved extremeness-separation combination. Across controlled synthetic networks, mechanism-isolating diagnostic topologies, and real political and communication networks, no measure passes all applicable checks. Minorities are systematically scored as more polarized than equivalent majorities. We trace this dependence to two separate channels: majority structure can alter the opinions that opinion-reading measures consume, and, on a different path, opinion-orthogonal structural change can affect the arithmetic of structural measures. Reading opinions is thus double-edged, letting contamination in when structure moves opinions but shielding the score when structure changes and opinions do not. We close with practical guidance for reporting polarization scores without inheriting these biases.

Figures & tables

Figure 1: Coverage of the seven audited measures across the three Bramson dimensions: extremeness (D1), clustering (D2), and separation (D3). Each measure sits in the region of the dimensions it captures. Modularity appears here as a single measure (Q) but is audited in two operationalisations,

Q_{\text{opinion}}

and

Q_{\text{struct}}

Figure 2: The two fairness criteria and resolution-sensitivity prerequisite on Track A, per measure, ordered so opinion-reading measures sit above the two opinion-free ones (

Q_{\text{struct}}

, AEI). (A) Measurement invariance: Cohen’s

d

(majority

-

minority); red bars are negative (the minority scored higher), and bars past

\pm 0.3

(dashed) are Benjamini-Yekutieli-significant violations. (B) Structural exogeneity: semi-partial

R^{2}

of the minority score on majority parameters under Friedkin-Johnsen opinions, violations past

0.02

(dashed); under fixed bimodal opinions, the control condition, all measures are exogenous (

R^{2}\leq 0.01

). (C) Resolution sensitivity: detection rate by size ratio

r

, one cell per measure, darker is higher and bold marks detection

\geq 0.8

; the dashed box is the flagged region (

r<0.5

). AEI is degenerate on the single-community minority subgraphs, so its MI and SE entries are not substantively evaluable.

Measure	$R^{2}_{\text{maj}}$ (FJ)	Dominant predictor	SE (FJ)
$Q_{\text{opinion}}$	$0.00$	-	✓
$Q_{\text{struct}}$	$0.00$	-	✓
E-R	$0.070$	$p_{\text{out}}$	$\times$
$A$	$0.047$	$p_{\text{out}}$	$\times$
LE	$0.126$	$p_{\text{out}}$	$\times$
GED	$0.087$	$\log n_{\text{maj}}$	$\times$

Table 1: Structural Exogeneity, opinion-dynamics (Friedkin-Johnsen) channel: semi-partial

R^{2}_{\text{maj}}

of each measure’s minority-subgraph score on the majority parameters, the dominant majority predictor, and the violation verdict (Benjamini-Yekutieli-corrected). Under fixed (bimodal) opinions, the control condition, every measure is exogenous (

R^{2}\leq 0.01

, not shown). The Adaptive E-I Index is degenerate on single-community minority subgraphs.

Figure 3: Track B diagnostic audit: which measures fail which diagnostic constructions. Each cell records whether a measure follows the theoretically expected direction on the corresponding topology; red marks a mismatch, amber a calibration or baseline artifact, gray a case undefined for that measure, and hatching a case not evaluated. The columns are grouped as T1/T2 Structure vs. Opinion , T3/T4 Diversity vs. Conflict , T5/T6 Resolution vs. Null , and T7 Broker Dilution . NWD is evaluated only on T7, the purpose-built broker-dilution diagnostic for its walk-based mechanism; the remaining diagnostics were not assigned prespecified directional predictions for NWD. Full constructions, scores, and expected directions are in the appendix (Table 4 ).

Figure 4: T8 (Asymmetric Echo Chambers): mean majority

-

minority score difference across 100 replicate pairs. Asterisks mark Benjamini-Yekutieli-significant violations.

Measure	MI	SE ${}_{\text{FJ}}$	T8 size	RS	TB
$Q_{\text{opinion}}$	✓	✓	✓	$\times$	1
$Q_{\text{struct}}$	$\times$	✓	$\times$	$\times$	0
E-R	$\times$	$\times$	$\times$	✓	1
$A$	✓	$\times$	$\times$	$\times$	0
LE	$\times$	$\times$	$\times$	$\times$	3
GED	$\times$	$\times$	$\times$	$\times$	1

Table 2: Cross-measure audit overview, one column per check. MI (Measurement Invariance) and SE

{}_{\text{FJ}}

(Structural Exogeneity via the Friedkin-Johnsen channel) are the fairness checks; RS (Resolution Sensitivity) is a validity prerequisite, and T8 is a size-scaling diagnostic; TB is the Track B mismatch count. ✓ = pass,

\times

= violation, - = not meaningfully evaluable. † The Adaptive E-I Index is degenerate on the single-community minority subgraphs, so its MI and SE

{}_{\text{FJ}}

entries are definitional rather than substantive.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Diagnostic topologies (Track B). Rows are the three inverse pairs and two standalone diagnostics; within each inverse pair the two topologies probe a measure from opposite directions. Node color encodes opinion on a fixed scale (blue $-1$ , cream $0$ , dark red $+1$ ); layout and node size are illustrative. Construction and expected behaviour are given in Table 3 .

Topology	Construction	Isolates	Predicted behaviour	Criterion
Pair I - Structure without opinion vs. opinion without structure (probes Q, GED)
T1 Separated Consensus	Two dense communities, all $x=+0.5$	structure, no opinion	Q reports high (structural false positive); E-R and LE correctly $\approx 0$	SE
T2 Opinion-Separated Mixed	Dense E-R ( $p=0.7$ ), bimodal $\pm 1$ , edges opinion-blind	opinion, no structure	Q $\approx 0$ (miss); E-R max; LE high; GED should be nonzero	coverage
Pair II - Diversity vs. conflict (probes LE, GED)
T3 Diverse Community	One clique, $x\sim\mathcal{U}(-1,1)$	diversity, no separation	LE high (false positive from density $\times$ range); GED suppresses via its mesolevel term	coverage
T4 Disconnected Clusters	20 disconnected cliques, alternating $\pm 1$	distributional separation, no cross-edges	LE exactly $0$ (miss); E-R max	coverage

Appendix

Table 3: The eight diagnostic topologies (Track B), grouped into three inverse pairs and two standalone diagnostics. Each pair probes one measure from opposite directions; the final column records the role of each construction in the audit. Full generation parameters are given in the appendix.

Topology	Measure	Score	Expected	Direction check	Role
T1	$Q_{\text{opinion}}$	$0.0000$	ZERO	ok
T1	$Q_{\text{struct}}$	$0.4954$	HIGH	ok	MI
T1	E-R	$0.0000$	ZERO	ok
T1	$A$	-	N/A	N/A
T1	LE	$0.0000$	ZERO	ok
T1	GED	$0.0000$	ZERO	ok

Appendix

Table 4: Track B diagnostic results, reproduced from the pipeline output. Cell shading: green = score matched its theoretically expected direction; red = mismatch; amber = calibration finding (MOD/NONZERO) or undefined (N/A). Scores are on the pipeline’s normalized scale.

Topology	Measure	Score	Expected	Direction check	Role
T5	$Q_{\text{opinion}}$	$0.0303$	ZERO	MISMATCH	RS
T5	$Q_{\text{struct}}$	$0.0837$	ZERO	ok	RS
T5	E-R	$0.0241$	HIGH	ok
T5	$A$	$0.6238$	HIGH	ok
T5	LE	$0.0093$	HIGH	MISMATCH
T5	GED	$1444.8210$	MOD	MOD (baseline artifact)	RS

Appendix

Table 5: Track B diagnostic results (continued).

Group Fairness

Group fairness measures and reduces disparities in how algorithms predict, rank, allocate resources, and make decisions for different groups.

See all 140 papers

Linking Scalar-Intensity Language to Structural Polarization with Validated Signed-Network Measures
May 12, 2026Zhijin Guo, Li Zhang, Tyler Bonnet +2Social Media Analysis
University of Oxford · University College London · Imperial College London
SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization
Apr 8, 2026Usman Naseem, Robert Geislinger, Juan Ren +31Social Media Analysis Multilingual Text Classification
Macquarie University · University of Hamburg · Zayed University +23
PACIFIER: Pacing Opinion Depolarization via a Unified Graph Learning Framework
Feb 23, 2026Mingkai Liao, Zijun Zhang, Mingyang ZhouSocial Network Analysis Social Media Analysis
Shenzhen University, Shenzhen, China · Hubei University of Police, Wuhan, China

May 12, 2026J/K move · Enter open · S save

Linking Scalar-Intensity Language to Structural Polarization with Validated Signed-Network Measures

Zhijin Guo, Li Zhang, Tyler Bonnet, Janet B. Pierrehumbert+1

University of Oxford · University College London · Imperial College London

Social Media Analysis

Open paper AskPDF arXiv

Polarization in online communities is often studied through either language or interaction structure, but the two views are rarely connected within a unified framework. Prior work has linked them by constructing interaction graphs from human judgements of agreement and disagreement, leaving a gap between language as observed text and structure as an engineered representation of that text. We address this gap with a language-grounded signed-network pipeline that derives signed relations directly from conversational exchanges and links window-level language patterns to structural polarization over time. Before examining this relationship, we compare spectral and frustration-based polarization measures on synthetic benchmarks and real interaction networks. We find that frustration-based measures normalized by the graph's cycle-space capacity provide a more suitable basis for comparing polarization across networks of different sizes and densities. We therefore carry forward two complementary frustration-based measures: a weighted form that incorporates stance-model confidence and a count form based on edge signs. The weighted measure provides the better-behaved structural estimate and aligns more closely with polarization measured from human-labelled interactions, while the count measure reveals a stronger relationship with language. Across monthly Reddit Brexit discussions, greater prevalence of scalar-intensity language is associated with greater structural polarization, with a similar rank-level pattern in the human-labelled network. We find little evidence that language in one month predicts polarization in the next, whereas contemporaneous scalar-intensity prevalence provides useful information about polarization within the same month.

Abstract

Group Fairness

Explore similar work

Linking Scalar-Intensity Language to Structural Polarization with Validated Signed-Network Measures

SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization

PACIFIER: Pacing Opinion Depolarization via a Unified Graph Learning Framework