Organizations: Ph.D candidate, Department of Politics and Public Administration, The University of Hong Kong. · Boya Postdoctoral Fellow, School of International Studies, Peking University. · Associate Professor, Department of Politics and Public Administration, The University of Hong Kong. · Assistant Professor, Department of Political Science, Columbia University.
The production of large-scale political datasets typically demands extracting structured facts from vast piles of unstructured documents or web sources, a task that traditionally relies on expensive human experts and remains prohibitively difficult to automate at scale. In this paper, we leverage Large Language Models (LLMs) to automate the extraction of multi-dimensional elite biographies, addressing a long-standing bottleneck in political science research. We propose a two-stage ``Synthesis-Coding'' framework for complex extraction task: an upstream synthesis stage that uses recursive agentic LLMs to search, filter, and curate biography from heterogeneous web sources, followed by a downstream coding stage that maps curated biography into structured dataframes. We validate this framework through three primary results. First, we demonstrate that, when given curated contexts, LLM coders match or outperform human experts in extraction accuracy. Second, we show that in web environments, the agentic system synthesizes more information from web resources than human collective intelligence (Wikipedia). Finally, we diagnosed that directly coding from long and multi-language corpora introduces bias that the synthesis stage can alleviate by curating evidence into signal-dense representations. By comprehensive evaluation, We provide a generalizable, scalable framework for building transparent and expansible large scale database in political science.
Figures & tables
Dataset
N
Years †
Region
Variables ‡
Production Details
Countries
(D/C/I/P)
National Leaders
[ baturo2016cursus ]
–
1960–2010
Global
✓/✓/–/✓
2 RAs, 2009–14
[ baturo2022what ]
–
1950–2017
Global
–/✓/–/–
1 RA
[ baturo2023incumbent ]
132
1918–2019
Global
–/–/–/✓
Built on 11 datasets
[ bomprezzi2025wedded ]
177
1989–2018
Global
✓/–/–/✓
27 RAs, 1.6M entities
Table 1: Available Datasets on Political Elites Since 2007 ∗
Figure 1: Two coding strategies for elite biographies. Left: when a Wikipedia page exists, we code directly from the curated page with a single LLM pass. Right: when Wikipedia is missing or incomplete, we search across web sources and iteratively synthesize a synthetic report, then code from that report. The lower panel illustrates the structured output as an ordered biography (career, education, and affiliations) anchored on a timeline. This contrast highlights why extraction from open-web sources requires adaptive synthesis rather than one-shot retrieval.
Figure 2: Experiment 1 Results: LLM coding performance relative to the human baseline (China sample, N=197). Points indicate coefficient estimates with 95% confidence intervals. The human-coded baseline ( Human_wiki ) is normalized to zero. Positive values indicate that LLMs outperform human coders on the corresponding metric.
Figure 3: Agentic synthesis versus Wikipedia baseline (pooled U.S. and OECD samples, N=398 ). Points indicate coefficient estimates from Equation 2 with 95% confidence intervals. The Wikipedia baseline (LLM_wiki) is normalized to zero. Positive values indicate that agentic synthesis outperforms Wikipedia-based extraction.
Figure 4: Agentic synthesis effects by sample. Points report F1 estimates with 95% confidence intervals. Baselines are normalized to sample-specific Wikipedia means (U.S.: 0.82; OECD: 0.73).
Condition
Upstream evidence construction
Downstream coder
Representation
Typical context length
LLM_wiki
Human (Wikipedia)
LLM
Wiki narrative
∼ 8k
LLM_raw
Agent (fixed trajectory)
LLM
Raw concatenation
∼ 300k
LLM_refined
Agent (fixed trajectory)
LLM
Refined passages
∼ 30k
Table 4: Representations used in the diagnostic comparison
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Dimension
Classification ( fcls )
Extraction (Synthesis → Coding)
Pipeline
Single-step mapping
Two-stage: synthesis fsyn (retrieval fret + condensation g ) then coding fcode ; end-to-end fext
Output space
Y (finite, closed)
Intermediate evidence Dk⊆D , refined corpus x∈X , and trajectory τ^∈T (open, effectively unbounded)