Forecast-Necessary Causal Discovery for Nonlinear Political Panel Data: Feedback, Functional Form, and the Dynamics of Democratization
Authors: Michael Coppedge, Dmitry Zaytsev, Valentina Kuskova
Organizations: Department of Political Science, University of Notre Dame, Notre Dame, IN, USA · Kellogg Institute, University of Notre Dame, Notre Dame, IN, USA · Lucy Family Institute for Data & Society, University of Notre Dame, Notre Dame, IN, USA
A non-significant coefficient in a dynamic panel model need not imply the absence of a relationship. It may instead reflect heterogeneous effects averaged toward zero, reciprocal dynamics overlooked by a recursive specification, or relationships masked by the omission of correlated covariates. Standard linear estimators cannot distinguish among these possibilities. We develop an inferential workflow for political panel data that resolves this ambiguity by combining flexible autoregressive estimation, forecast-necessity testing, functional characterization, and same-data linear benchmarking. The workflow first identifies relationships required for out-of-sample prediction, then characterizes their functional form across political contexts, and finally, distinguishes differences arising from estimator flexibility from those due to model specification. Applied to the causal sequence model of democratization on the V-Dem panel of 113 countries, the workflow reproduces the model's central finding - the protective belt of civil society, the rule of law, and institutionalized parties - while recovering reciprocal relationships from democracy to its institutional supports that a linear model cannot detect. Most importantly, three weak published direct effects, of which two are null, and one is marginally significant, receive three different diagnoses: one dissolves under the full specification, one reflects heterogeneous effects averaged toward zero, and one was masked by the reduced variable set. The workflow corrects the published record in both directions, removing one relationship and recovering two. More broadly, the workflow provides a framework for evaluating dynamic political theories under a model class capable of representing nonlinear and reciprocal mechanisms while preserving relationship-level interpretation and explicit inferential standards.
Figures & tables
Figure 1: The workflow applied to the causal sequence model. The questions at left state what each stage determines, independent of the application; the quantities at right report each stage’s output for the present one. Each stage’s output for the present application appears at right: the exogeneity screen admits five structural variables as causes only; flexible estimation yields a causal score matrix over 396 candidate relationships; forecast-necessity testing retains 160; the automatic threshold retains 84; and functional characterization assigns each retained relationship a sign, a functional form, and a stability measure across democratic contexts. A linear benchmark, estimated on identical data and variables, enters at the final stage: the diagnosis of each relationship follows from the comparison between the two model classes.
Baseline ( Coppedge et al., 2022 )
This article
Data
V-Dem panel, 113 countries, 1900–2001
Same, standardized
Outcome
Electoral Democracy Index
Same
Variables
10 of 23 candidates
All 23 consistently observed
Estimator
Linear recursive path model
Neural additive autoregression
Functional form
Constant coefficients
Flexible per relationship
Reciprocal effects
Lagged cross-paths; few significant
Estimated jointly; tested for necessity
Table 1: Design of the baseline model and of the present re-estimation. The data, outcome, and conceptual organization of the variables are held constant; the estimator, variable set, lag depth, and retention criterion change.
Dominant-necessary relationships (automatic threshold, 5.48×10−4 ): the estimated graph
84
Positively / negatively signed net effects
47 / 37
Relationships with a stable sign across democratic contexts
84 of 84
Relationships nonlinear in at least one democratic regime
73 of 84
Table 2: The estimated system at a glance. All quantities are from the seed-42 reference run; ensemble ranges appear in I , and the complete forecast comparison in H .
Figure 2: Signed net effects and functional forms across the estimated system. Cell color reports the direction and magnitude of each source variable’s net effect on an outcome, aggregated from the regime-specific response curves: blue denotes positive effects, red negative, and grey cells are uncharacterized. Cell text reports the consensus functional form across democratic contexts (LIN linear, THR threshold, SAT saturating, SIG sign-changing, OTH other); where forms differ across regimes, the modal form is shown, with the per-regime record in Table 3 and in G . Two system-level patterns are visible at a glance: the uniformly positive row for polyarchy, summarizing the reciprocal dynamics of Section 5.2 , and the uniformly negative row for agricultural income, summarizing the indirect suppression established in Section 5.3 .
Figure 3: Three weak direct effects, three diagnoses. Each panel shows the estimated response of polyarchy to one variable, computed separately within low, middle, and high democratic contexts. The direct effect of agricultural income (left) is flat throughout: no direct relationship survives once intermediate channels enter the model, and the baseline’s small negative estimate dissolves. The effect of GDP per capita (center) reverses direction across the range of development within every context, the pattern that a constant coefficient averages toward zero. The effect of literacy (right) is positive throughout and was masked in the baseline by the reduced variable set: once the specification expands, even a linear model detects it. In the published model these three direct effects appear as marginally significant, null, and null; the response curves show one effect dissolving and two emerging, a correction running in both directions.
Figure 4: The dominant-necessary causal network (84 edges, seed 42). Nodes are arranged in the causal-sequence layout of the baseline model’s path diagram: structural and distal causes at the left, then intermediate and proximate/episodic variables, the protective belt, and the outcome (polyarchy) at the right. Structural variables appear as sources only; edge width is proportional to the NAVAR causal score; blue arrows denote positive ICE-derived effects, red negative, and grey mixed or sign-changing. Edge inclusion is read directly from the GMM-filtered adjacency matrix ( w_active_gmm.csv ).
Relationship
Causal score (sign)
Retention
Baseline coefficient
Functional form
Diagnosis
Protective belt → polyarchy
Civil society (CSO)
0.073 (+)
5/5
0.0515***
linear → sat.
Replicated
Rule of law
0.039 (+)
5/5
0.0483***
linear → sat.
Replicated
Party institutionalization
0.033 (+)
5/5
0.097 (n.s.)
linear → sat.
Strengthened
Modernization variables → polyarchy
Literacy rate
0.040 (+)
4/5
0.0002 (n.s.)
linear → sat.
Masked by specification
Table 3: Relationship-level comparison: flexible workflow vs. linear baseline. Causal scores are unsigned contribution magnitudes from the dominant-necessary graph; signs and functional forms are derived from the functional characterization. Baseline coefficients from Coppedge et al. (2022) , Table 8.1 and Figure 8.2. The two columns are not on a common scale; the comparison is between verdicts, not magnitudes.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Variable
Code
Source
Category
Electoral democracy (polyarchy)
v2x_polyarchy
V-Dem v9
Outcome
CSO participatory environment
v2csprtcpt
V-Dem v9
Protective belt
Rule of law / state capacity
v2clrspct
V-Dem v9
Protective belt
Institutionalized parties
v2xps_party
V-Dem v9
Protective belt
Majoritarian electoral system
elecsys
V-Dem v9
Intermediate
Anti-system movement
v2csantimv
V-Dem v9
Proximate
Appendix
Table 4: The 23 variables of the final model. Categories follow Coppedge et al. (2022) .
Figure 5: The structural exogeneity screen. Bars show the intraclass correlation coefficient of each model variable, the fraction of its variance lying between countries rather than within them. The five variables above the 0.88 threshold (red) have essentially no within-country movement over the observation period and are designated structurally exogenous: they enter the analysis as candidate causes but are never modeled as outcomes. The threshold falls within a wide empty interval, between 0.856 (party institutionalization) and 0.987 (Protestant population share), so the classification does not depend on its exact placement. Slow-moving institutional stocks (party institutionalization, rule of law, literacy) sit high among the dynamic variables; episodic variables (campaigns, growth) sit lowest.
Figure 6: Estimated causal score matrix. Each cell reports how actively a source variable (rows) moves the prediction of an outcome variable (columns), before necessity testing. Shaded columns mark the five structurally exogenous variables, which enter as candidate causes only. The concentration of scores in a small share of cells reflects the sparsity that the ℓ1 penalty induces; the columns for polyarchy and its institutional supports carry the strongest signal. The complete matrix is reported numerically in Table 5 .
Figure 7: Forecast-necessity testing across the 396 candidate relationships. Left: one-sided Diebold–Mariano p -values for the exact ablation of each source (rows) from each target equation (columns); green cells are significant at α=0.05 , blank cells are excluded self-effects and masked structural targets, and p -values above 0.2 share the terminal color. Right: each candidate’s −log10 DM p -value against its causal score magnitude, with the dashed line at α=0.05 and p -values floored at 10−10 for display; blue points are the 160 forecast-necessary relationships. Necessity is not a monotone function of score: candidates with some of the largest scores fall below the line, and candidates with small scores clear it.
Figure 8: The automatic dominance criterion. Left: a two-component mixture fitted to the logarithm of mean loss differentials across the 160 forecast-necessary relationships separates a near-zero cluster from a substantive one; the dashed line marks their intersection, 5.48×10−4 , which serves as the retention threshold. Right: the same 160 relationships ranked by loss differential, with 84 above the threshold. No researcher-specified parameter enters the decision.
Figure 9: ICE response curves for the nine edges in Table 1 of the main text, computed from the pipeline’s ICE output. Each panel overlays the low (solid blue), mid (dashed grey), and high (dotted red) democratic regimes, defined as terciles of the lag-1 standardized polyarchy level of the prediction window. Horizontal axis: source value on the normalized 81-point grid; vertical axis: mean change in the predicted target. Curves are plotted as estimated, with no smoothing beyond the model itself.
Table 7: Linear edge verdicts. Block test that all eight lag coefficients of the source are zero in the target equation; lag-sum is the sum of the eight coefficients.
Edge
Retention
Score mean (sd)
Score range
Civil society (CSO) → Polyarchy
5/5
0.072 (0.001)
[0.070, 0.073]
Rule of law → Polyarchy
5/5
0.041 (0.001)
[0.039, 0.042]
Party institutionalization → Polyarchy
5/5
0.031 (0.001)
[0.030, 0.033]
Literacy rate → Polyarchy
4/5
0.039 (0.001)
[0.038, 0.041]
Log GDP per capita → Polyarchy
4/5
0.034 (0.002)
[0.033, 0.037]
Nonviolent campaign → Polyarchy
3/5
0.031 (0.001)
[0.030, 0.031]
Appendix
Table 8: Seed-ensemble robustness of headline edges (5 seeds). Score statistics are computed over seeds in which the edge is retained.
Nonlinear machine-learning models are increasingly used to discover causal relationships in time-series data, yet the interpretation of their outputs remains poorly understood. In particular, causal scores produced by regularized neural autoregressive models are often treated as analogues of regression coefficients, leading to misleading claims of statistical significance. In this paper, we argue that causal relevance in nonlinear time-series models should be evaluated through forecast necessity rather than coefficient magnitude, and we present a practical evaluation procedure for doing so. We present an interpretable evaluation framework based on systematic edge ablation and forecast comparison, which tests whether a candidate causal relationship is required for accurate prediction. Using Neural Additive Vector Autoregression as a case study model, we apply this framework to a real-world case study of democratic development, modeled as a multivariate time series of panel data - democracy indicators across 139 countries. We show that relationships with similar causal scores can differ dramatically in their predictive necessity due to redundancy, temporal persistence, and regime-specific effects. Our results demonstrate how forecast-necessity testing supports more reliable causal reasoning in applied AI systems and provides practical guidance for interpreting nonlinear time-series models in high-stakes domains.
Valentina Kuskova, Dmitry Zaytsev, Michael Coppedge
Lucy Family Institute for Data & Society · University of Notre Dame · Notre Dame, IN, USA +1
Causal discovery in time series is increasingly performed using nonlinear machine-learning models, yet the resulting causal relationships are almost always summarized by scalar edge scores. We argue that this practice obscures the true object learned by nonlinear autoregressive models: a state-dependent function whose effect varies across regimes, magnitudes, and contexts. We formalize function-valued causal influence for additive, contribution-decomposable architectures and show that scalar causal scores constitute a severe information bottleneck, conflating between-state variation with within-state residual noise. Using Neural Additive Vector Autoregression as a representative architecture, we introduce a practical framework based on Individual Conditional Expectation for estimating causal response functions directly from trained models. Through controlled synthetic experiments, we demonstrate that edges with indistinguishable scalar scores can exhibit qualitatively different functional behaviors, including monotonic, thresholded, saturating, and sign-changing effects. An applied case study on democratic development further shows that function-valued analysis reveals regime-specific and asymmetric causal structure systematically missed by score-centric approaches.
Valentina V. Kuskova, Dmitry Zaytsev, Michael Coppedge
Lucy Family Institute for Data & Society, University of Notre Dame, Notre Dame, Indiana, USA. · Department of Political Science, University of Notre Dame, Notre Dame, Indiana, USA.
Panel data, in which multiple units are repeatedly observed over time, arise throughout science and engineering. Quantifying predictive uncertainty in such settings is challenging because conformal prediction, while distribution-free and model-agnostic, classically relies on exchangeability assumptions that fail under temporal dependence and unit heterogeneity. We propose a simple online conformal framework for non-exchangeable panel data. The method exploits a key feature of online panel prediction: when a forecast is required for one unit, contemporaneous outcomes from related units may already be observed and can serve as a calibration panel. At each round, prediction sets are formed using currently observed calibration units together with two adaptive quantities: history-based similarity weights that emphasize calibration units resembling the target, and an adaptive miscoverage level that is updated whenever target feedback is revealed. This two-state design yields a stepwise coverage bound and a long-run coverage guarantee. Empirically, across synthetic and real panel data sets, the method improves coverage on the worst-covered target units through adaptive interval-width allocation rather than uniform inflation. The two states are complementary: similarity weights protect coverage when target feedback is sparse, while the adaptive level further improves coverage as feedback accumulates.
Daohong Tu, Kay Giesecke
Department of Management Science and Engineering Stanford University