Organizations: Ph.D candidate, Department of Politics and Public Administration, The University of Hong Kong. · Boya Postdoctoral Fellow, School of International Studies, Peking University. · Associate Professor, Department of Politics and Public Administration, The University of Hong Kong. · Assistant Professor, Department of Political Science, Columbia University.
The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.
Figures & tables
Dataset
N
Years †
Region
Variables ‡
Production Details
Countries
(D/C/I/P)
National Leaders
[ baturo2016cursus ]
–
1960–2010
Global
✓/✓/–/✓
2 RAs, 2009–14
[ baturo2022what ]
–
1950–2017
Global
–/✓/–/–
1 RA
[ baturo2023incumbent ]
132
1918–2019
Global
–/–/–/✓
Built on 11 datasets
[ bomprezzi2025wedded ]
177
1989–2018
Global
✓/–/–/✓
27 RAs, 1.6M entities
Table 1: Available Datasets on Political Elites Since 2007 ∗
Figure 1: Two coding strategies for elite biographies. Left: when a Wikipedia page exists, we code directly from the curated page with a single LLM pass. Right: when Wikipedia is missing or incomplete, we search across web sources and iteratively synthesize a synthetic report, then code from that report. The lower panel illustrates the structured output as an ordered biography (career, education, and affiliations) anchored on a timeline. This contrast highlights why extraction from open-web sources requires adaptive synthesis rather than one-shot retrieval.
Figure 2: Experiment 1 Results: LLM coding performance relative to the human baseline (China sample, N=197). Points indicate coefficient estimates with 95% confidence intervals. The human-coded baseline ( Human_wiki ) is normalized to zero. Positive values indicate that LLMs outperform human coders on the corresponding metric.
Figure 3: Agentic synthesis versus Wikipedia baseline (pooled U.S. and OECD samples, N=398 ). Points indicate coefficient estimates from Equation 2 with 95% confidence intervals. The Wikipedia baseline (LLM_wiki) is normalized to zero. Positive values indicate that agentic synthesis outperforms Wikipedia-based extraction.
Figure 4: Agentic synthesis effects by sample. Points report F1 estimates with 95% confidence intervals. Baselines are normalized to sample-specific Wikipedia means (U.S.: 0.82; OECD: 0.73).
Condition
Upstream evidence construction
Downstream coder
Representation
Typical context length
LLM_wiki
Human (Wikipedia)
LLM
Wiki narrative
∼ 8k
LLM_raw
Agent (fixed trajectory)
LLM
Raw concatenation
∼ 300k
LLM_refined
Agent (fixed trajectory)
LLM
Refined passages
∼ 30k
Table 4: Representations used in the diagnostic comparison
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Dimension
Classification ( fcls )
Extraction (Synthesis → Coding)
Pipeline
Single-step mapping
Two-stage: synthesis fsyn (retrieval fret + condensation g ) then coding fcode ; end-to-end fext
Output space
Y (finite, closed)
Intermediate evidence Dk⊆D , refined corpus x∈X , and trajectory τ^∈T (open, effectively unbounded)
Large Reasoning Models (LRMs) embedded in agentic frameworks have transformed information retrieval from static, long context question answering into open-ended exploration. Yet real world use requires models to discover and synthesize "long-tail" facts from dispersed sources, a capability that remains under-evaluated. We introduce PolitNuggets, a multilingual benchmark for agentic information synthesis via constructing political biographies for 400 global elites, covering over 10000 political facts. We standardize evaluation with an optimized multi agent system and propose FactNet, an evidence conditional protocol that scores discovery, fine-grained accuracy, and efficiency. Across models and settings, we find that current systems often struggle with fine-grained details, and vary substantially in efficiency. Finally, using benchmark diagnostics, we relate agent performance to underlying model capabilities, highlighting the importance of short-context extraction, multilingual robustness, and reliable tool use.
Whether political elites organise into rent-seeking coalitions that capture public resources or civic networks that sustain governance is a central question in comparative politics. Yet observing these complex, informal, and adversarial ties at scale has historically required intensive manual coding, while automated text-as-data methods have largely been limited to simple co-occurrence. Recent large language model (LLM) approaches offer a path forward but often rely on proprietary APIs, lack cross-lingual capability, and struggle with scalable entity resolution. We present a modular, fully open-weight pipeline for multilingual joint entity-relation extraction that builds signed, temporal knowledge graphs from massive unstructured news corpora. It combines span-based named-entity recognition (NER) with a three-stage linking cascade mapping mentions to language-independent Wikidata identifiers; a high-throughput, ontology-constrained mixture-of-experts model then uses guided decoding to extract directed, signed relationships grounded in a domain ontology. A full-coverage spot-check against a 3491-relation gold standard shows high textual correctness (68.2% strict to 93.7% lenient). Two large-scale case studies validate the pipeline against the public record. In Austria, it reconstructs a political party's complete lifecycle, dating internal fractures and tracking personnel into successor factions and court convictions. In a Polish corpus, it uncovers the overlapping economic and governance networks of state-enterprise patronage, alongside the structurally balanced, signed conflict network of the polarized Civic Platform (Platforma Obywatelska, PO)--Law and Justice (Prawo i Sprawiedliwość, PiS) duopoly. By bridging raw multilingual text and structured relational data, our framework provides a robust, replicable foundation for cross-national empirical computational social science.
Predictive political question answering (QA), such as predicting how a political actor will vote, goes beyond factual lookup. External political resources offer rich historical evidence, but rarely contain the answer itself. Existing LLM augmentation methods, including actor-profile-based simulation and knowledge graph evidence injection, improve political reasoning but largely treat external resources as knowledge-based evidence, leaving prediction-relevant signals under-modeled. We identify two complementary signals for predictive political QA: actor stances that capture issue-specific preferences, and high-order structure signals that capture indirect dependencies among political actors. We propose PSL, a dual-view framework that converts semi-structured political records into inference-oriented evidence for LLMs. PSL extracts stance signals from question-relevant actor records in a semantic view, and learns structure-aware actor representations from an actor interaction graph in a vector view. Across three real-world datasets and multiple LLMs, PSL consistently outperforms baselines, with ablations confirming the complementary gains of stance and structure signals.
Yinan Liu, Zihan Zhou, Zichun Jin +3
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China