Scalable partial information decomposition for symptom networks via supervised embeddings

Cillian Hourican; Eric Dignum; Rick Quax; Debraj Roy

cs.LGAug 14, 2026

Scalable partial information decomposition for symptom networks via supervised embeddings

Authors: Cillian Hourican, Eric Dignum, Rick Quax, Debraj Roy

Organizations: Computational Science Lab, Institute for Informatics, University of Amsterdam

Abstract

Pairwise relationships among mental-health symptoms are routinely summarised asscalar edge weights, which cannot express whether two symptoms carry overlapping information about a third or information that appears only in combination. Partial information decomposition (PID) addresses this gap but is computationally intractable beyond a few sources. We introduce embedding-based PID (ePID), a scalable pipeline that compresses all nonfocal symptoms into a low-cardinality discrete embedding and computes a tractable two-source PID, yielding source-unique, remainder-unique, redundant, and synergistic components for each ordered source-target pair. We benchmarked 13 candidate embeddings on synthetic Bayesian networks calibrated to PHQ-9 and on 83 real-world datasets across five PID measures. A supervised Agglomerative Conditional Information Bottleneck (ACIB) embedding recovered the reference decomposition most accurately of the 13 embeddings tested, and did so for every PID measure yielding non-negative atoms once four or more symptoms were compressed (synergy recovery r = 0.92). The two instruments then diverged sharply. In PHQ-9 networks (UK Biobank,N = 154,291; Xinxiang student sample, N = 24,292) the surrounding symptom context carried most pairwise dependence through redundant and remainder-unique channels; synergy contributed 6 to 9%, and no directed edge was synergy-dominated in either cohort. In the 28-item Interpersonal Reactivity Index, 45% of source pairs were. The identical pipeline, applied without parameter changes, therefore returned opposite profiles for the two instruments, each consistent with how that instrument was constructed. By separating overlapping from interaction-dependent information, ePID provides a scalable, model-agnostic complement to standard symptom-network methodology to distinguish redundant and synergistic contributions to observed correlations.

Figures & tables

Figure 1: Methods comparison on the toy structural causal model Note. The toy system is a five-variable mixture structural causal model (

X_{1},X_{2},W\sim\mathrm{Uniform}\{0,\ldots,4\}

independent;

X_{4}=X_{5}=W

;

X_{3}=h(X_{1},X_{2})

with probability

0.5

else

X_{3}=W

, where

h

is the non-monotonic stylised mapping defined in Appendix J ). Top row: (a) True graph:

X_{1},X_{2},X_{4}\to X_{3}

with

X_{4}=X_{5}

marked. (b) PC network: only the

X_{1}\to X_{3}

edge survives (

\rho_{ij\mid\text{rest}}=+0.38

);

X_{4}

X_{5}

edges are undefined under naive PC because the deterministic equality

X_{4}=X_{5}

makes the conditioning covariance singular. (c) CMI network:

X_{1}\to X_{3}

and the non-monotonic

X_{2}\to X_{3}

edge (0.84 and 0.19 bits respectively);

X_{4}

X_{5}

contributions vanish because conditioning on the remaining system already includes the duplicate

X_{5}=X_{4}

, giving each zero conditional entropy given the rest and hence

I(X_{4};X_{3}\mid\text{rest})=0

; the edges disappear from vanishing conditional mutual information, not from the conditioning rule itself, and the same attenuation arises under near-redundancy when the sources are highly but not perfectly correlated. Bottom row: (d) the ePID pipeline. Six triplet-level two-source PIDs (one per source pair targeting

X_{3}

, shown at lower opacity to signal pair-dependent attribution) collapse via ACIB-supervised compression of the remainder

\{X_{2},X_{4},X_{5}\}

into a four-atom ePID for focal source

X_{1}

. All atom values are computed exactly under

I_{\min}

(Williams & Beer 2010); the corresponding BROJA decomposition is shown in Appendix Figure S1 . PC

=

partial correlation; CMI

=

conditional mutual information; PID

=

partial information decomposition; ePID

=

embedding-based PID; ACIB

=

Agglomerative Conditional Information Bottleneck.

Panel of Figure 2 — Figure 2: Overview of the ePID evaluation pipeline Note. An end-to-end view of how raw symptom data become a PID-decomposed network: a scalable embedding-based path used for the main analyses runs in parallel with a small-scale ground-truth path used to validate it. Raw data are preprocessed (imputation, discretisation) and then processed along these two paths. Path A (top, blue): for each source–target pair, the remainder symptoms are compressed into a discrete embedding using one of 13 methods, and a two-source PID is computed on (source, embedding, target). Path B (bottom, orange): for benchmark validation, the full $N$ -source PID is computed via the fast Möbius transform (FMT) and amalgamated to two-source atoms for comparison with Path A.

Figure 3: ePID on the toy SCM: redundancy lattice, amalgamated atoms, truth-versus-ePID recovery, and the embedding shortcut Note. Same SCM as Figure 1 ; visualises the four-source PID lattice that the main pipeline amalgamates over. (a) Full

N=4

antichain lattice for

\{X_{1},X_{2},X_{4},X_{5}\}\to X_{3}

under

I_{\min}

: 166 atoms in total, of which 9 carry non-zero information (sum

=

1.489 bits). Node area scales with atom value; colour encodes the type-1 amalgamation class. (b) The nine non-zero atoms grouped by their type-1 amalgamation class (synergy

\Sigma=0.839

, remainder-unique

\Sigma=0.242

, redundancy

\Sigma=0.408

bits), each labelled with its individual value. (c) Amalgamated four-atom view for focal source

X_{1}

: ground truth versus ePID (ACIB embedding,

\lvert E\rvert=5

); the arrow marks the amalgamation step that collapses the lattice into four atoms. ACIB recovers

\mathrm{Red}

exactly,

\mathrm{Syn}

85\%

, and

\mathrm{Unq}_{E}

73\%

, for

87\%

of total joint information. The lower row makes the ePID shortcut explicit: rather than enumerating the full lattice, the remaining sources

\{X_{2},X_{4},X_{5}\}

are compressed into a single embedding

E

and a two-source PID is computed on

(X_{1},E;X_{3})

, recovering the amalgamated atoms directly (synergy

0.711

, remainder-unique

0.176

, redundancy

0.408

bits). Grey nodes carry zero information. ACIB

=

Agglomerative Conditional Information Bottleneck.

Figure 4: Mean absolute error by embedding method and PID atom Note. Each panel shows mean absolute error (in bits) between the amalgamated ground-truth

N

-source PID and the embedding-based two-source PID approximation, for one of the four PID atoms: (a) source-unique

\mathrm{Unq}_{X_{i}}

, (b) remainder-unique

\mathrm{Unq}_{E}

, (c) redundancy

\mathrm{Red}

, (d) synergy

\mathrm{Syn}

. Methods are coloured by family: manifold / dim-reduction (blue: ACIB, SIR, PLS, MCA+

k

-means, SVD+

k

-means, NMF+

k

-means), information-theoretic feature selection (pink: CMIM, JMI, ReliefF, Greedy CMI), and clustering (green: Agglomerative Hamming, Spectral Hamming,

k

-modes). Errors are pooled across 50 synthetic Bayesian networks calibrated to PHQ-9, nine targets, and three source counts (

N\in\{3,4,5\}

); within each panel, methods are sorted by mean error. Within the manifold family, ACIB (the recommended default) achieves the lowest error across all four atoms. The clustering family is heterogeneous:

k

-modes performs comparably to manifold methods on

\mathrm{Red}

and

\mathrm{Unq}_{X_{i}}

, while Hamming-based clustering collapses

\mathrm{Syn}

toward zero (see also Figure S4 ). ACIB

=

Agglomerative Conditional Information Bottleneck; JMI

=

Joint Mutual Information; CMIM

=

Conditional Mutual Information Maximisation.

Figure 5: Entropy-normalised PID atom shares across source–target pairs Note. Each cell is the atom fraction $I_{\mathrm{atom}}(X\to Y)/H(Y)$ for a directed pair among the eight PHQ-9 symptoms (suicidality excluded); rows are sources, columns are targets, and diagonals are blanked. Top row (a): Xinxiang student sample; bottom row (b): UK Biobank. Columns show the three non-trivial PID channels under ACIB (redundancy, synergy, and remainder-unique) with shared colour scales within each atom to make cross-cohort comparisons visible. The source-unique atom is zero or near-zero across all edges under MMI with a target-optimised embedding (see text above) and is omitted; the conditional-mutual-information share at the edge level is visualised in Fig. 6 .

Panel of Figure 6 — Figure 6: Replication of ePID networks (top-3 incoming edges per target) in UK Biobank and the Xinxiang student sample Note. Nodes are eight PHQ-9 symptoms (suicidality excluded). Edges were first screened using permutation-tested conditional mutual information (CMI) with Benjamini–Hochberg FDR control ( $q=.05$ ) and then pruned for visualisation by retaining, for each target symptom $Y$ , the three incoming edges $X\rightarrow Y$ with the largest entropy-normalised conditional mutual information $\mathrm{nCMI}_{X\rightarrow Y}=I(X;Y\mid\mathbf{Z})/H(Y)$ . For each retained directed edge, we compute an embedding-based two-source partial information decomposition (PID) on $(X,E_{X\rightarrow Y};Y)$ using an ACIB embedding of the remaining symptoms. Edge thickness is proportional to total mutual information $I(X,E_{X\rightarrow Y};Y)$ . Coloured segments indicate the fraction of this total carried uniquely by the embedded remainder (blue), redundantly by $X$ and the remainder (orange), or synergistically (green); a terminal black segment indicates direction. Edges were selected for display using CMI; decompositions for all 56 directed pairs are reported in Appendix H . Because nCMI conditions on the full remaining symptom set, it captures the source’s unique-plus-synergistic information and excludes redundancy; since the source-unique atom is near zero throughout these networks, the top-three-per-target display, if anything, emphasises synergistic edges rather than hiding them. The reported atom fractions (Fig. 5 ) are in any case computed from all 56 directed decompositions and all informative edges, not only the displayed subset.

Figure 7: ePID symptom networks in non-depressed versus depressed UK Biobank respondents Note. Directed embedding-PID networks for the eight retained PHQ-9 symptoms (suicidality excluded), estimated separately by depression status. (a) non-depressed (PHQ-9 total

<10

); (b) depressed (PHQ-9 total

\geq 10

). The groups were equal-

N

matched at

n=8{,}879

per arm and each decomposition was averaged over five random subsamples of the larger group. Both panels display the same 24-edge skeleton (the top three incoming edges per target by entropy-normalised CMI, pooled across groups). For each directed edge, coloured segments partition the pair–target total mutual information into unique information carried by the source plus embedded remainder context (blue), redundancy (orange), and synergy (green); a terminal black segment indicates direction, and edge thickness is proportional to total mutual information on a scale shared across both panels. Relative to non-depressed respondents, the depressed network shifts from redundancy toward unique and synergistic information (mean shares: redundancy

0.34\rightarrow 0.15

, unique

0.47\rightarrow 0.59

, synergy

0.19\rightarrow 0.26

); the clearest visual signal is the contraction of the orange (redundancy) segments. Because the absolute synergy share is sample-size dependent (full-cohort PHQ-9 synergy

\approx 0.08

), the figure communicates the matched-

N

contrast between groups rather than an absolute synergy level. All 56 significant directed edges are shown in Fig. S16 .

Configuration	$n$	Synergy (mean)	Redundancy (mean)	Synergy-dominant pairs
IRI within-subscale ( $n_{\mathrm{sources}}=5$ , pooled)	140	15.5%	29.2%	4%
IRI global 28-item	133	23.1%	18.0%	71%
IRI cross-subscale triplet (raw 2-source PID)	1,903	21.8%	22.6%	45%

Table 1: Scale-dependent synergy-dominance gradient across IRI configurations Note. Synergy fraction increases from within-subscale IRI through cross-subscale triplet IRI to the global 28-item IRI; redundancy decreases in mirror image. Atom fractions are means over informative directed pairs whose ACIB embedding respected its 5% conditional-mutual-information loss tolerance (where applicable); synergy-dominant pairs are those for which the synergy share exceeds the redundancy share.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Domain	Items	$N$	Source	Citation
CSWS Items	Self-worth	35	680	OSF	( Briganti et al., 2019 )
CSWS Subscales	Self-worth	7	680	OSF	( Briganti et al., 2019 )
SHARE W7	Depression	17	77,000	SHARE	( Börsch-Supan, 2022b )
ELSA W6	Depression	17	10,000	UK Data	( Banks et al., 2023 )
HRS W13	Sleep/pain/health	18	21,000	HRS	( Health and Retirement Study, 2024 )
easySHARE 2004 (W1)	Summary health	8	30,000	SHARE	( Börsch-Supan, 2022a ; Gruber et al., 2014 )

Appendix

Table S1 : Benchmark datasets Note. The 83 datasets used in the multi-dataset embedding benchmark. Items indicates the number of substantive variables selected before preprocessing; N is the raw sample size before row-level missingness filtering. easySHARE waves (6a–6g) are drawn from the same longitudinal study but are treated as separate networks because each wave has an independent cross-section of respondents.

Method	Family	Package/Library	Key Hyperparameters
ACIB	Info-theoretic	Custom (npeet)	$K_{\max}=12$ , $K_{\min}=2$ , $\alpha=0.5$ ,
			loss tol. $=5\%$ , max_iter $=40$
Greedy CMI	Info-theoretic	Custom (npeet)	max_vars $=4$ , loss tol. $=5\%$
JMI	MI-based selection	scikit-feature	$k=3$
CMIM	MI-based selection	scikit-feature	$k=3$
ReliefF	MI-based selection	skrebate	$k=3$ , $n\_\text{neighbours}=100$

Appendix

Table S2 : Summary of the 13 embedding methods, grouped by family. All methods share a global random seed of 42. “

K

” denotes the number of clusters or output cardinality; “

k

” denotes the number of selected features; “

B

” denotes the number of quantile bins per dimension.

Figure S1 : Methods comparison under BROJA Note. Companion to main-text Figure 1 , identical structural model and pipeline, with the bottom-row two-source PIDs computed under the BROJA redundancy measure ( Bertschinger et al., 2014 ) instead of

I_{\min}

( Williams and Beer, 2010 ) . The top row (panels a – c : true graph, PC, CMI) is measure-independent. On the homogeneous source pair

(X_{1},X_{2})

, both source variables live in the

h

-channel of

X_{3}

and the two measures agree to several decimals. On heterogeneous source pairs

(X_{a},X_{b})

where one source lives in the

h

-channel and the other in the

W

-channel of

X_{3}

I_{\min}

allocates information across redundancy and synergy while BROJA assigns the same information to pure uniques: a known divergence between the two measures that motivates BROJA’s operationalist definition. BROJA

=

Bertschinger–Rauh–Olbrich–Jost–Ay; PC

=

partial correlation; CMI

=

conditional mutual information.

Panel of Figure 12 — Figure S2 : Mutual information distribution comparison between real data and synthetic Bayesian network ensemble Note. (a) Histogram overlay showing density distributions of MI values. (b) Cumulative distribution functions (CDFs) with Kolmogorov–Smirnov (KS) statistic quantifying the distributional match. The ensemble of 50 synthetic BNs closely matches the real-data distribution, validating the calibration procedure.

Panel of Figure 13 — Figure S3 : Detailed validation of Bayesian network ensemble calibration Note. (a) Quantile–quantile (Q–Q) plot comparing MI quantiles between real data and synthetic BNs. Points close to the diagonal indicate good distributional match. (b) Mean MI per network across the ensemble, showing consistent calibration with low variability (std $=$ 0.0382). The red dashed line indicates the ensemble average, with shaded region showing $\pm 1$ standard deviation.

Figure S4 : Ranking of 13 embedding methods by ground-truth synergy correlation Note. Pearson correlation

r

between embedding-approximated and ground-truth synergy across the synthetic Bayesian network benchmark (50 networks

\times

9 targets

\times

3 source counts

N\in\{3,4,5\}

n=5{,}377

source–target configurations; type-1 amalgamation throughout). Bars are coloured by method family: manifold / dim-reduction (blue), feature selection (green), clustering (orange). ACIB ranks first by this criterion; manifold-family methods occupy the top six positions. ACIB

=

Agglomerative Conditional Information Bottleneck; JMI

=

Joint Mutual Information; CMIM

=

Conditional Mutual Information Maximisation; SIR

=

Sliced Inverse Regression; MCA

=

Multiple Correspondence Analysis; PLS

=

Partial Least Squares; SVD

=

Singular Value Decomposition; NMF

=

Non-negative Matrix Factorisation.

Measure	MAE (bits)	RE (%)	Win rate (%)	95% CI
$I_{\wedge}$	0.057	19.4	84.4	[78.4, 91.1]
$I_{\min}$	0.105	44.7	77.9	[71.1, 85.0]
$I_{\mathrm{mmi}}$	0.106	45.2	75.3	[68.6, 82.9]
$I_{\mathrm{rr}}$	0.146	70.9	64.9	[56.2, 73.3]
$I_{\pm}$	0.247	115.1	2.6	[0.0, 4.4]

Appendix

Table S3 : ACIB embedding fidelity by PID measure at

N=5

sources Note. Mean absolute error (MAE, bits) is the per-atom absolute error averaged across the four PID atoms (source-unique, remainder-unique, redundancy, synergy) and across datasets. Mean relative error (RE) is the same quantity divided by the magnitude of the corresponding ground-truth atom. Win rate is the proportion of per-dataset contests (across 77–83 datasets) in which ACIB achieved the lowest error among the 13 candidate embeddings, with bootstrap 95% confidence intervals. For

I_{\mathrm{rr}}

, 12 of the 13 embeddings were available (greedy CMI lacked an

I_{\mathrm{rr}}

ground truth).

Measure	$\chi^{2}$	$p$	$\chi^{2}$	$p$	$\chi^{2}$	$p$
	$N=3$		$N=4$		$N=5$
$I_{\min}$	12.0	0.45	131.4	$9\times 10^{-23}$	145.9	$1\times 10^{-25}$
$I_{\mathrm{mmi}}$	12.0	0.45	128.6	$3\times 10^{-22}$	131.1	$1\times 10^{-22}$
$I_{\mathrm{rr}}$	12.0	0.45	134.9	$2\times 10^{-23}$	37.6	$9\times 10^{-5}$
$I_{\pm}$	12.0	0.45	252.7	$8\times 10^{-48}$	237.7	$1\times 10^{-44}$
$I_{\wedge}$	12.0	0.45	181.8	$4\times 10^{-33}$	238.3	$8\times 10^{-45}$

Appendix

Table S4 : Friedman test results by PID measure and number of sources Note.

\chi^{2}

statistic and

p

-value testing whether method rankings differ across datasets. At

N=3

, no significant differences were detected for any measure.

Measure	$N=4$	$N=5$
$I_{\min}$	15	22
$I_{\mathrm{mmi}}$	14	19
$I_{\mathrm{rr}}$	15	6
$I_{\pm}$	37	36
$I_{\wedge}$	23	35

Appendix

Table S5 : Number of significantly different method pairs (Nemenyi post-hoc) Note. Out of 78 possible pairwise comparisons between 13 methods. Zero pairs at

N=3

confirms that all methods perform equivalently when compressing only two sources.

Figure S5 : Embedding approximation error across PID measures and number of sources (all 13 methods) Note. Top row: mean absolute error (bits); bottom row: mean relative error (%). Each column corresponds to one PID measure, ordered by increasing difficulty. ACIB (bold orange) is the lowest-error method at

N\geq 4

for all measures except

I_{\pm}

. The dashed line at RE

=100

% marks the threshold where approximation error equals the total signal.

I_{\mathrm{rr}}

N=5

is based on 23 datasets; all others on 77–83 datasets.

Figure S6 : Mean relative error by embedding method and PID atom Note. Per-atom error normalised by the magnitude of the corresponding ground-truth atom, complementing the absolute-error view in Figure 4 of the main text. Panels, methods, and family colour coding are identical to the main-text figure. Relative-error normalisation makes small absolute errors visible when ground-truth atoms are small in magnitude; the ranking of methods within each panel is preserved against the absolute-error view. ACIB

=

Agglomerative Conditional Information Bottleneck.

Figure S7 : Atom-level error profiles by PID measure (ACIB,

N=5

) Note. For non-

I_{\pm}

measures, error is concentrated in the remainder-dependent atoms (

\mathrm{Unq}_{E}

\mathrm{Syn}

). For

I_{\pm}

, all atoms show high error due to disruption of signed cancellation structure.

Panel of Figure 21 — Figure S8 : Approximated versus ground-truth synergy for one representative method per family tier Note. Per-pair scatter of embedding-approximated synergy (y-axis) against ground-truth synergy (x-axis), in bits, across 5,377 ( $G,X_{t},N$ ) configurations from the synthetic Bayesian network benchmark. Each panel shows one representative method per family. (a) ACIB, manifold/dim-reduction family, Pearson $r=0.98$ . (b) JMI, feature-selection family, $r=0.41$ . (c) $k$ -modes, clustering family, $r=0.90$ . Dashed diagonal is the identity line. The visible density clustering near ground-truth synergy $\approx 0.10$ reflects multiple synthetic networks producing similar synergy values. Methods within each family vary in fidelity; main-text Figure S4 ranks all 13 methods. The corresponding scaling of synergy approximation error with the number of embedded sources is reported in Section F.2 . ACIB $=$ Agglomerative Conditional Information Bottleneck; JMI $=$ Joint Mutual Information.

Figure S9 : Synergy approximation error as a function of the number of embedded sources (

N=3,4,5

) Note. ACIB error increases sub-linearly from

0.0008

bits at

N=3

0.0056

bits at

N=5

Panel of Figure 23 — Figure S10 : Target-centred PID radial graphs for two representative targets in the Xinxiang student sample Note. Each edge is segmented into four PID atoms (teal: redundancy; blue: unique source; purple: unique embedding; orange: synergy); edge thickness scales with total mutual information.

Figure S11 : Comparison of bivariate dependency measures across 80 clinical and psychometric datasets Note. (a) Number of significant pairs detected by Pearson vs. Spearman correlation per dataset; near-perfect agreement indicates negligible non-linear monotonic structure. (b) Number of significant pairs detected by Pearson correlation vs. mutual information; MI generally detects fewer pairs, consistent with discretisation-induced information loss. (c) Distribution of the percentage of MI-significant pairs not detected by either correlation measure; the vast majority of datasets show fewer than 2.5% non-linear-only dependencies.

Panel of Figure 25 — Figure S12 : Xinxiang student sample PHQ-9 networks under three association measures Note. Three measures that progressively relax modelling assumptions of linearity and monotonicity, each leading to a similar network structure: (a) Pearson partial correlation, (b) Spearman partial correlation, (c) conditional mutual information.

Panel of Figure 26 — Figure S13 : Two ePID network views in the Xinxiang student sample Note. Directed edges represent ordered source $\rightarrow$ target relations. (a) Full ePID view: each edge is partitioned into remainder-unique (blue), redundancy (orange), and synergy (green) atoms from the two-source PID of $(X_{i},E_{i\to k};X_{k})$ ; the terminal black segment indicates direction. (b) No-remainder-unique view: the remainder-unique atom is removed and the remaining redundancy and synergy atoms are rescaled, highlighting dependence that involves the focal source beyond remainder-only predictability. Edge thickness encodes target-normalised total information $I(X_{i},E_{i\to k};X_{k})/H(X_{k})$ .

Panel of Figure 27 — Figure S14 : Agreement between partial Pearson and partial Spearman correlation networks in the Xinxiang student sample Note. Partial Pearson correlations (PPC) and Spearman partial correlations (SPC) were computed for each symptom pair while conditioning on all remaining PHQ-9 symptoms (suicidality excluded). (a) Edge-wise comparison of absolute partial correlations ( $|r|$ ) shows near-identical weighting across pairs (Pearson $r=0.997$ ; Spearman $\rho=0.995$ ). (b) Edge-wise differences (SPC $-$ PPC) are small across the matrix, indicating that rank-based partial correlation and linear partial correlation yield highly similar conditional-association structures in this setting.

Figure S15 : Cross-dataset comparison of information-theoretic dependence strength and decomposition stability (UK Biobank vs the Xinxiang student sample) Note. For each directed source

\to

target symptom pair (eight PHQ-9 symptoms; suicidality excluded), we compare total mutual information (TMI) and the synergy proportion (synergy/TMI) derived from the embedding-based PID framework. Edge-wise differences (UK Biobank

-

Xinxiang) show systematically larger dependence strengths in UK Biobank, whereas synergy-proportion differences are concentrated near zero. Synergy proportions show substantial cross-dataset agreement (Pearson

r=0.663

; Spearman

\rho=0.777

), indicating that the composition of dependence is more stable across datasets than the overall dependence magnitude.

Source	Target
Energy	Motor
Motor	Energy

Appendix

Table S6 : Directed CMI edges that were statistically significant in UK Biobank but not in the Xinxiang student sample (2 edges). UK Biobank contained all 56 directed symptom pairs; the Xinxiang student sample contained the complementary set of 54 edges.

Pruning rule	$\|E_{\mathrm{Xinxiang}}\|$	$\|E_{\mathrm{Biobank}}\|$	$\|E_{\cap}\|$	$\|E_{\cup}\|$	Jaccard
All significant (FDR)	54	56	54	56	0.96
CMI $\geq 0.05$ bits	10	6	6	10	0.60
Top 10 by $\mathrm{nCMI}$	10	10	6	14	0.43
Top 25% by $\mathrm{nCMI}$	14	14	9	19	0.47
Top 3 per target by $\mathrm{nCMI}$	24	24	17	31	0.55

Appendix

Table S7 : Overlap of statistically significant CMI edges across cohorts under alternative pruning rules. Rank-based rules use entropy-normalized CMI,

\mathrm{nCMI}=I(X;Y\mid\mathbf{Z})/H(Y)

. The Jaccard index is

|E_{\cap}|/|E_{\cup}|

for the directed edge sets.

Figure S16 : ePID networks in non-depressed versus depressed UK Biobank respondents (all significant edges) Note. Companion to main-text Fig. 7 , showing all 56 significant directed edges rather than the top-three-per-target subset. (a) non-depressed (PHQ-9 total

<10

); (b) depressed (PHQ-9 total

\geq 10

); equal-

N

matched at

n=8{,}879

per arm, five-seed average, eight PHQ-9 symptoms (suicidality excluded). Segment colours (unique = blue, redundancy = orange, synergy = green) and the shared total-mutual-information thickness scale are as in Fig. 7 . Over all 56 edges the redundancy-to-unique/synergy shift is larger than for the pruned display (mean shares: redundancy

0.26\rightarrow 0.11

, unique

0.60\rightarrow 0.66

, synergy

0.14\rightarrow 0.23

). As in the main figure, the absolute synergy share is

N

-dependent and the panels should be read as a matched-

N

contrast, not an absolute level.

Figure S17 : IRI cross-subscale synergy and redundancy patterns Note. Each panel is computed over informative triplets (total mutual information

I(X_{i},X_{j};X_{k})\geq 0.01

bits) in which the source pair and target item belong to three distinct IRI subscales. (a) Synergy-dominance rate (Synergy

>

Redundancy) for triplets targeting a Perspective-Taking item, by cross-subscale source pair; FS

+

PD tops the ranking at 69.4%, consistent with the main-text claim that mature cognitive empathy is informed synergistically by the affective extremes. (b) Mean redundancy fraction per source pair, pooled across all target subscales; EC

+

FS tops the ranking at 25.7%, and all three pairs containing Fantasy rank above all three pairs without it. (c) Synergy-dominance rate per target subscale, pooled across all cross-subscale source pairs; PT ranks highest at 48.9%, ahead of PD (46.8%), EC (45.6%) and FS (41.0%), consistent with developmental accounts in which cognitive empathy emerges later than its affective counterpart and comes to integrate it ( Decety and Holvoet, 2021 ) . PT

=

Perspective Taking; FS

=

Fantasy Scale; EC

=

Empathic Concern; PD

=

Personal Distress.

Panel of Figure 33 — Figure S18 : The same three-variable system as seen by three methods Note. PC (blue, dotted) only recovers the linear $X_{1}$ – $X_{3}$ edge; CMI (red, dashed) additionally detects the conditional $X_{1}$ – $X_{2}$ dependence; PID (green, solid) further splits the joint information (2.16 bits total) into unique, redundant, and synergistic atoms, making explicit the synergy that PC misses and CMI detects only indirectly. This example is constructed for methodological demonstration and does not represent an empirically identified mechanism.

loss tolerance	$K_{\max}=6$	$K_{\max}=10$	$K_{\max}=14$
$2\%$	0.111	0.059	0.048
$5\%$	0.111	0.059	0.048
$10\%$	0.111	0.059	0.048

Appendix

Table S8 : Mean total absolute error of the recovered atoms against ground truth (

N=5

I_{\mathrm{mmi}}

\alpha=0.5

). Rows (loss tolerance) are identical; columns (

K_{\max}

) carry all the variation.

Figure S21 : Full-context versus

n_{\mathrm{sources}}=5

synergy fraction. The 28-item IRI network saturates under full context (synergy collapses); the 9-item PHQ-9 does not.

Explore similar work

Apr 20, 2026

Modeling Higher-Order Brain Interactions via a Multi-View Information Bottleneck Framework for fMRI-based Psychiatric Diagnosis

Resting-state functional magnetic resonance imaging (fMRI) has emerged as a cornerstone for psychiatric diagnosis, yet most approaches rely on pairwise brain cortical or sub-cortical connectivities that overlooks higher-order interactions (HOIs) central to complex brain dynamics. While hypergraph methods encode HOIs through predefined hyperedges, their construction typically relies on heuristic similarity metrics and does not explicitly characterize whether interactions are synergy- or redundancy-dominated. In this paper, we introduce

O

-information, a signed measure that characterizes the informational nature of HOIs, and integrate third- and fourth-order

O

-information into a unified multi-view information bottleneck framework for fMRI-based psychiatric diagnosis. To enable scalable

O

-information estimation, we further develop two independent acceleration strategies: a Gaussian analytical approximation and a randomized matrix-based Rényi entropy estimator, achieving over a 30-fold computational speedup compared with conventional estimators. Our tri-view architecture systematically fuses pairwise, triadic, and tetradic brain interactions, capturing comprehensive brain connectivity while explicitly penalizing redundancy. Extensive evaluation across four benchmark datasets (REST-meta-MDD, ABIDE, UCLA, ADNI) demonstrates consistent improvements, outperforming 11 baseline methods including state-of-the-art graph neural network (GNN) and hypergraph based approaches. Moreover, our method reveals interpretable region-level synergy-redundancy patterns which are not explicitly characterized by conventional hypergraph formulations.

Kunyu Zhang, Qiang Li, Vince D. Calhoun +1

Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, 1081 HV, The Netherlands · Tri-Institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS), Georgia State University, Georgia Institute of Technology, and Emory University, Atlanta, GA, 30303, USA · Department of Physics and Technology, UiT - The Arctic University of Norway, Tromsø, 9019, Norway

Functional Connectivity Multi-View Learning Information Bottleneck Neuroimaging Brain Network Analysis

Ask PDF

May 5, 2026

Partial Effective Information Decomposition for Synergistic Causality

Causality is a central topic in scientific inquiry, yet for complex systems, the identification and analysis of synergistic causation remain a challenging and fundamental problem. In the context of causal relations among multivariate variables, a decomposition framework grounded in interventionist causation is still lacking. To address this gap, this paper proposes Partial Effective Information Decomposition (PEID), a framework that decomposes the influence of multiple source variables on a target variable under maximum-entropy interventions into unique and synergistic information, thereby providing a unified and computable characterization of synergistic causal relations. Theoretically, in the three-variable case, the proposed framework is compatible with the major axioms of Partial Information Decomposition (PID). Empirically, under maximum-entropy interventions, correlations among input variables are removed, causing redundancy to vanish and thereby enabling PEID to compute synergistic relations. Furthermore, based on this framework, it is possible to define causal graphs containing hyperedges as well as downward causation, thus offering a unified toolkit for analyzing cross-scale and multivariate causal mechanisms in complex systems. Finally, applying the framework to a machine-learning-based air quality forecasting task on KnowAir-V2, we demonstrate that PEID can extract interpretable inter-station causal structures from a learned dynamical model. These results suggest that PEID provides a general interventionist information-theoretic tool for analyzing multivariate and synergistic causal mechanisms in complex systems.

Mingzhe Yang, Shuo Wang, Jiang Zhang

School of Systems Science Beijing Normal University

Causal Discovery Causal Intervention

Ask PDF

Jun 17, 2026

The Representational Limit of Scalar Interactions: An Interventional Decomposition

Signed pairwise interaction scores fundamentally conflate uniqueness (U), redundancy (R), and synergy (S). We prove this on a minimal 3-way XOR structural causal model: faithful indices such as Shapley-Taylor return zero per pair, whereas projective indices such as Shapley Interaction spread the third-order effect into pair scalars that conflate the three mechanisms. We introduce Stochastic Hi-Fi, a post-hoc, retraining-free predictability decomposition that estimates per-feature U/R/S profiles by interventional masked inference. The estimator provides exact interventional semantics, finite-sample Monte Carlo bounds, strict variance reduction from coupled diamond sampling, and uniform finite-vocabulary convergence. Across tabular SCMs, Stochastic Hi-Fi recovers structure missed by scalar baselines (up to 411x larger interaction-magnitude recovery ratios). It also separates redundant and synergistic heads in the GPT-2 IOI circuit. On NIH ChestX-ray14, Stochastic Hi-Fi matches GradCAM on Pointing Game and improves substantially on Deletion AUC.

Potito Aghilar, Sabino Roccotelli, Stanislao Fidanza +3

Polytechnic University of Bari · University of Bari Aldo Moro

Feature Interaction Modeling Representation Disentanglement Shapley Value Attribution Causal Interpretability

Ask PDF