Framing by Wording, Framing by Selection: A Large-Scale Two-Dimensional Audit of French News Headlines, 2022-2025

Amr Sobhy

cs.CLAug 12, 2026

Framing by Wording, Framing by Selection: A Large-Scale Two-Dimensional Audit of French News Headlines, 2022-2025

Authors: Amr Sobhy

Organizations: Le French News Lab

Abstract

News headlines frame public issues both by what they select and by how they word it, yet computational framing work typically collapses these operations into a single score. We introduce a two-dimensional framework that separates salience framing, measured through four wording devices (loaded vocabulary, blame attribution, threat framing, rhetorical question), from selection framing, measured through outlet-level story-form and high-charge distributions. We build a 10,000-headline French supervision set using three LLM annotators with majority-vote resolution and human arbitration, validate the labels against two annotator-independent blind human studies, and apply the strongest classifier to 902,111 deduplicated headlines from 25 French outlets (2022-2025). Three main findings emerge. First, salience and selection divergence are positively correlated yet leave nearly half of outlet-level variance unexplained, populating interpretively distinct off-diagonal cells in a four-cell outlet typology. Second, default classification thresholds systematically inflate corpus-level salience estimates; a precision-floor recalibration protocol corrects this distortion. Third, group-mention analysis reveals sharply unequal salience contexts: headlines mentioning Jews, the Far-right, and Muslims carry the highest detected salience rates, which broad event-context composition does not fully explain (residuals are descriptive, not same-event causal estimates; per-group lexicon precision is reported alongside). To our knowledge, this is the largest framing-focused French headline audit to date; we release the supervision set, lexicons, and analysis code.

Figures & tables

Measure	Operational definition
Loaded vocabulary	Evaluative or emotionally charged wording whose neutral paraphrase would preserve the core event claim.
Blame attribution	Wording that assigns causal responsibility for a problem or negative outcome to an actor or group.
Threat framing	Wording that casts an actor, event, or situation as danger, invasion, crisis, or security risk.
Rhetorical question	Headline posed as a question. Operationalised by interrogative form, not by whether the question is judged rhetorical.
Us-vs-them	Explicit ingroup/outgroup contrast through naming, pronouns, identity markers, or oppositional structure.
Story form	10-class event/action category: POLICY, PARLIAMENT, CONFLICT, CRIME, SCANDAL, ELITE, SOCIAL, LAW, ELECTIONS, OTHER.

Table 1: Measurement schema. Salience devices are wording-level and non-mutually exclusive; story form and high-charge are selection-level.

Head	MinP	Thr.	P	R	F1
loaded	.70	.67	.733	.698	.715
blame	.70	.83	.719	.650	.683
threat	.60	.84	.723	.712	.718
rhet. q.	.85	.89	.957	.809	.877
us-vs-them ‡	.60	.89	.600	.625	.612
charge	.70	.64	.779	.785	.782

Table 2: Final v6 thresholds (validation set only) for the mixed-encoder ensemble; see Table 3 and Models. ‡ Us-vs-them is a supplementary indicator (below the

0.60

precision floor; Appendix Table 22 ), not a primary prevalence estimate.

Model	Loaded	Blame	Threat	Rhet.Q	UvT ‡	Charge	Story (macro-F1)
TF-IDF + Logistic Regression	.518	.468	.429	.491	.381	.572	.543
CamemBERT-base	.770	.657	.642	.900	.617	.769	.723
XLM-RoBERTa-large †	.758	.672	.749	.841	.633	.801	.731

Table 3: Held-out test-set F1 by model (final v6 split, seed 42, production thresholds); per-model means and three-seed robustness are in Models and Appendix Table 23 . † Production model; CamemBERT substituted for rhetorical question only (see Models). ‡ Us-vs-them is a supplementary indicator (below the

0.60

precision floor; Appendix Table 22 ), not a primary prevalence estimate.

Device	H–H $\kappa$	Maj–LLM $\kappa$	F1	Prec.	Rec.
Loaded vocabulary	.542	.608	.768	.644	.951
Blame attribution	.601	.718	.675	.579	.809
Threat framing	.613	.668	.564	.420	.857
Rhetorical question	.894	.869	.862	.820	.909
Us-vs-them †	.510	.596	.471	.369	.649
Any salience ‡	.680	.463	.780	.646	.984

Table 4: Independent blind re-annotation of the 499-headline validation sample (

N=499

) by two unaffiliated annotators. Maj–LLM = majority-vote human labels vs. LLM consensus; conflict rows excluded, effective

N

per head 383–488.

\kappa

interpretation bands follow Landis and Koch 1977 . † Us-vs-them is a supplementary indicator (below the

0.60

precision floor; see Appendix Table 22 ), not a primary prevalence estimate. ‡ Derived composite. See Appendix Table 17 for the corroborating

N=350

study.

Figure 1: Selection vs. salience divergence across French news outlets. Note that Ouest-France sits high in (b) because it is far from the baseline overall while its headlines deploy framing devices unusually rarely , not because they deploy them aggressively. (a) Each outlet’s distance from the corpus baseline decomposed into selection divergence ( $D_{\mathrm{sel}}^{\mathrm{JS}}$ , what stories are emphasised) and salience divergence ( $D_{\mathrm{sal}}^{\mathrm{JS}}$ , which framing devices are deployed in the headline). Both quantities are Jensen–Shannon divergences against the cross-outlet mixture distribution. Marker colour encodes the four-class typology defined in § 4.1 . (b) Same outlets re-expressed as a salience share ( $D_{\mathrm{sal}}^{\mathrm{JS}}/(D_{\mathrm{sal}}^{\mathrm{JS}}+D_{\mathrm{sel}}^{\mathrm{JS}})$ ) against summed divergence. The dashed line at $0.5$ separates outlets whose share falls on the selection side from those on the salience side. Vertical position in (b) is summed distance from the baseline, not salience intensity; outlets close to the bottom edge are close to the corpus baseline on both dimensions. Both divergences share the same bounded scale, so the share is well defined, but their expected magnitudes differ with support size (5 vs. 10 bins); both coordinates in (b) are therefore read ordinally — as which axis dominates within an outlet, and as rank distance from the baseline — not as an equivalence of magnitudes.

Cell	$n$	Sal.JS	Sel.JS	Sal.%
Double-distinctive	9	0.027	0.081	.484
Salience-dominant	3	0.017	0.018	.358
Selection-dominant	3	0.008	0.046	.510
Low-distinctiveness	10	0.004	0.014	.376

Table 5: Four-cell typology. Sal.JS/Sel.JS = salience/selection Jensen–Shannon divergence from the corpus baseline (§ 3 ); cells assigned by median splits; Sal.% = mean any-salience prevalence (selection-dominant outlets show the highest Sal.% yet the lowest Sal.JS; see Results). Outlet assignments in Fig. 1 and Appendix Table 11 .

Group	N	Sal%	Exp%	Res.	Top form
Jews	4,646	78.7	49.6	+29.1	CRIME
Far-right	3,981	77.7	50.0	+27.7	CONFLICT
Muslims	4,720	66.4	46.6	+19.8	CRIME
Migrants	9,606	58.4	40.7	+17.7	CRIME
Police	14,313	57.6	46.5	+11.1	CRIME
Unions	6,877	53.8	50.3	+3.6	CONFLICT

Table 6: Group-mention salience contexts (top 8 of 12 groups). Sal% = observed any-salience; Exp% = expected rate under year

\times

story-form reweighting (focal group excluded from cell baselines to avoid circular self-inclusion); Res. = observed

-

expected (pp). Residuals are descriptive, not same-event causal estimates. Full 12-group table in Appendix Table 13 .

Component	Count	Share
Politics	3,339	33.4%
Economy	3,335	33.4%
Society	3,326	33.3%
Any salience	4,093	40.9%
Loaded vocabulary	2,391	23.9%
Blame attribution	1,435	14.4%

Table 7: Final v6 label distribution in the 10,000-headline development set after majority-vote merging and story-form arbitration.

Group	Terms	Prec.	Rec.	F1
Migrants	26	1.000	.674	.805
Police	29	.873	.674	.761
RN	14	.831	.736	.780
LFI	9	.691	1.000	.818
Workers	14	.860	.491	.625
Farmers	40	.877	.934	.904

Table 8: Held-out validation of group-mention lexicons.

(a) Reliability summary ( $N=10{,}000$ )
Field	3-way agree.	Mean $\kappa$
Loaded vocabulary	81.1%	.661
Blame attribution	85.4%	.639
Threat framing	90.0%	.623
Rhetorical question	94.8%	.770
Us-vs-them	88.3%	.408

Table 9: (a) Compact reliability summary for the final v6 annotation stack, computed on the 10,000 rows shared by all three composite annotators after UVT harmonization. (b) Homogeneity check on the 499-headline validation sample; all cells are pairwise between individuals and therefore comparable. LLM–LLM = mean pairwise

\kappa

among the three annotators; LLM–hum. = each annotator against each human coder (6 pairs); H–H = the two coders. Shared blind spots would show as LLM–LLM exceeding both LLM–hum. and H–H, which holds for loaded vocabulary and threat framing only. On unrounded values, the LLM–LLM minus LLM–hum. gap is

+.177

and

+.139

for those two, against

+.067

for us-vs-them,

+.012

for blame attribution and

+.002

for rhetorical question.

Device (context)	Example and rationale
Loaded vocabulary (Jews)	“Au Dîner du Crif, François Bayrou dénonce «la bête délirante et meurtrière» de l’antisémitisme” (“At the Crif dinner, François Bayrou denounces the ‘deranged and murderous beast’ of antisemitism”) — Le Figaro , id 3789940 , test split: the noun phrase bête délirante et meurtrière is an evaluative monstrosity metaphor with no neutral paraphrase of the same event. The loaded lexicon is voiced by a quoted speaker rather than by the outlet, illustrating the attribution limit of the headline unit noted in § 3 .
Blame attribution (Police)	“Mort de Blessing Matthew : les gendarmes mis en cause” (“Death of Blessing Matthew: the gendarmes implicated”) — Mediapart , id 3561734 , validation split: causal responsibility for a death is assigned to a named institutional actor in the outlet’s own voice, with no attributing source.
Threat framing (Migrants)	“«Impression de submersion» migratoire : Yaël Braun-Pivet «gênée» par les propos de François Bayrou” (“Migratory ‘sense of submersion’: Yaël Braun-Pivet ‘uncomfortable’ with François Bayrou’s remarks”) — Le Parisien , id 1454100 , train split: the submersion/overload lexicon reframes a demographic process as a security emergency, consistent with the threat-device definition in Table 1 ; as above, the phrase is quoted rather than asserted.
Rhetorical question	“Dépenses de santé : faut-il rembourser les soins en fonction des revenus ?” (“Health spending: should care be reimbursed according to income?”) — Le Point , id 436456 , validation split: an interrogative headline that advances an evaluative proposition rather than seeking information.
Story form vs. topic (immigration)	“Immigration : vers plus de contrôle des mariages des personnes étrangères en situation irrégulière” (“Immigration: towards greater scrutiny of marriages of irregular foreign nationals”; La Croix , id 1376080 ) is POLICY; “Naufrage de migrants dans la Manche : quatre personnes mises en examen et incarcérées” (“Migrant shipwreck in the Channel: four charged and jailed”; JDD , id 2310601 ) is CRIME. Both are train-split rows. Same subject domain, different journalistic action.

Table 10: One example per primary salience device from the v6 supervision set, each carrying the stated device label by unanimous agreement of all three LLM annotators; the released headline identifier and split are given so every row can be located in the distributed data. The final row illustrates the story-form/topic distinction.

Outlet	N	Sal.JS	Sel.JS	High-Chg%	Typology
Fdesouche	16,124	0.038	0.147	82.1	Double-distinctive
Slate.fr	3,907	0.051	0.071	23.1	Double-distinctive
Causeur	3,056	0.033	0.099	30.4	Double-distinctive
Blast †	646	0.025	0.095	57.9	Double-distinctive
Ouest-France	90,423	0.021	0.082	4.6	Double-distinctive
Mediapart	7,092	0.016	0.075	48.7	Double-distinctive

Table 11: Full four-cell outlet typology with per-outlet headline counts. Median split thresholds: Sal.JS=0.0110, Sel.JS=0.0249. Displayed JS divergence values are rounded to three decimal places; cell assignments use full-precision values. L’Humanité’s full-precision Sal.JS (0.0110) equals the median; strict

>

convention places it in the low-salience half. † Blast (

N{=}646

): case profile, wide CIs (Table 12 ).

Domain	Case	N	Any-Sal% 95% CI	High-Chg% 95% CI
Outlet	Fdesouche	16,124	72.6 [71.9, 73.3]	82.1 [81.5, 82.6]
Outlet	Blast †	646	67.3 [63.6, 70.8]	57.9 [54.1, 61.6]
Outlet	Causeur	3,056	59.1 [57.4, 60.9]	30.4 [28.8, 32.1]
Outlet	Le Figaro	108,307	33.2 [32.9, 33.5]	31.2 [30.9, 31.5]
Group	Jews	4,646	78.7 [77.5, 79.9]	79.8 [78.6, 80.9]
Group	Far-right	3,981	77.7 [76.3, 78.9]	65.8 [64.3, 67.2]

Table 12: Representative 95% Wilson intervals for outlet-level and group-level headline rates, recomputed from the canonical paper artifacts. These are descriptive intervals over observed predicted headline rates only; they do not propagate classifier or lexicon uncertainty. † Blast:

N{=}646

headlines; estimates carry wide bootstrap CIs (displayed) and should be read as a case profile rather than a population estimate.

Group	N	Corpus%	Any-Sal%	Y $\times$ Story exp.%	Resid. pp	Dominant context
Jews	4,646	0.52	78.7	49.6	+29.1	CRIME 32.5%
Far-right	3,981	0.44	77.7	50.0	+27.7	CONFLICT 44.3%
Muslims	4,720	0.52	66.4	46.6	+19.8	CRIME 23.4%
Migrants	9,606	1.06	58.4	40.7	+17.7	CRIME 42.4%
Police	14,313	1.59	57.6	46.5	+11.1	CRIME 56.7%
Unions	6,877	0.76	53.8	50.3	+3.6	CONFLICT 56.8%

Table 13: Group-level salience contexts across 902,111 headlines, with coarse event-context normalization. Y

\times

Story exp.% is the expected any-salience rate after exact reweighting to each group’s year

\times

story-form composition, subtracting focal-group rows from each cell baseline. Resid. pp is observed minus expected; it is descriptive, not a same-event causal estimate.

Table 15

Head	Type	Headline	Error source
Loaded	FP	Nationalisons la dette ! ( Nationalize the debt! )	Surface-form political imperative over-triggers on the charged verb; annotators classified this as advocacy rather than framing rhetoric.
Loaded	FN	Pour Macron (et pour que rien ne change), bienvenue dans la commune la plus riche de France ( For Macron (and so that nothing changes), welcome to the richest municipality in France )	Sarcastic loaded expression ( pour que rien ne change ) requires discourse-level reading beyond surface form; ironic register under-represented in training.
Blame	FP	Nîmes : l’homme armé qui s’est suicidé au tribunal recherchait «des magistrats mafieux» ( Nîmes: the armed man who killed himself at the courthouse was looking for “mafia judges” )	Reported speech containing explicit blame language is attributed to a third party; model does not distinguish endorsed from reported blame frames.
Blame	FN	Chez TotalEnergies, détruire la planète continue de rapporter gros ( At TotalEnergies, destroying the planet is still highly profitable )	Implicit blame through an agentive nominalized construction; model under-weights non-explicit accusation patterns relative to overt subject-verb-object blame frames.
Threat	FP	Que risquent les Français qui partent faire le djihad ? ( What do French people who leave to wage jihad risk? )	Co-occurrence of risquent and djihad triggers the threat surface pattern; the actual frame is a juridical inquiry about legal consequences, not a threat directed at an audience.
Threat	FN	Explosion de l’immigration en France : les demandes d’asile ont bondi de plus de 30 % en 2022, les régularisations de clandestins en hausse de 8 % ( Explosion in immigration in France: asylum applications jumped by more than 30% in 2022, regularizations of undocumented migrants up 8% )	Threat metaphor ( explosion ) combined with quantitative surge framing; the statistical register suppresses the model’s threat signal relative to the metaphor.

Table 16: Qualitative error analysis: ten representative false positives (FP) and false negatives (FN) from the 1,501-headline test set, two per salience head, drawn from highest-confidence misclassifications. Headlines are shown verbatim (truncated where necessary); error-source annotations are manual.

Device	H–H $\kappa$	Interp.	A1–LLM $\kappa$	A2–LLM $\kappa$	Maj–LLM $\kappa$	Interp.	F1	Prec.	Rec.
Loaded vocabulary	.699	substantial	.638	.523	.682	substantial	.766	.629	.980
Blame attribution	.772	substantial	.568	.559	.648	substantial	.653	.510	.907
Threat framing	.588	moderate	.664	.409	.685	substantial	.746	.623	.930
Rhetorical question	.974	almost perfect	.895	.923	.920	almost perfect	.917	.930	.904
Us-vs-them	.564	moderate	.717	.588	.845	almost perfect	.769	.833	.714
Any salience	.636	substantial	.735	.511	.715	substantial	.909	.864	.959

Table 17: Corroborating independent blind study on a separate sample: two unaffiliated annotators (

N{=}350

Field	Mistral–human $\kappa$	Ensemble–human $\kappa$	$\Delta$
Loaded vocabulary	.553	.684	$-$ .131
Blame attribution	.351	.379	$-$ .028
Threat framing	.517	.614	$-$ .097
Rhetorical question	.324	.400	$-$ .076
Us-vs-them	.298	.546	$-$ .248
Any salience	.435	.559	$-$ .124

Table 18: French-specific Mistral comparison on the 499-headline human validation sample. Mistral = mistral-small3.2 ; ensemble = released v6 consensus labels.

\Delta

is Mistral–human

\kappa

minus ensemble–human

\kappa

; negative values indicate lower agreement than the released ensemble.

Table 19

Group	2022	2023	2024	2025
Jews	74.6% ( $\times$ 2.35)	75.3% ( $\times$ 2.26)	77.2% ( $\times$ 2.21)	84.1% ( $\times$ 2.21)
Far-right	74.3% ( $\times$ 2.34)	74.7% ( $\times$ 2.24)	79.6% ( $\times$ 2.27)	81.4% ( $\times$ 2.14)
Muslims	63.8% ( $\times$ 2.01)	66.7% ( $\times$ 2.00)	65.5% ( $\times$ 1.87)	67.9% ( $\times$ 1.78)
Police	58.4% ( $\times$ 1.84)	52.9% ( $\times$ 1.59)	58.0% ( $\times$ 1.66)	63.1% ( $\times$ 1.66)
Migrants	53.2% ( $\times$ 1.68)	59.4% ( $\times$ 1.78)	58.7% ( $\times$ 1.68)	62.3% ( $\times$ 1.64)

Table 21: Year-by-year any-salience rates for the five highest-salience groups in the full-corpus group audit. Parenthetical values are lifts relative to the corpus-wide any-salience baseline for the same calendar year, which rises from 31.7% (2022) to 38.1% (2025). The same three groups, Jews, the Far-right, and Muslims, rank highest in every year, indicating that the main RQ3 pattern is not reducible to a single late-period news cycle.

Group	UvT lift	UvT rate	Interpretation
Far-right	$\times$ 10.60	56.0%	elevated directional signal only; not a primary prevalence estimate
Muslims	$\times$ 6.83	36.1%	elevated directional signal only; explicit mentions only
LFI	$\times$ 2.97	15.7%	weaker directional signal in party/mobilisation coverage

Table 22: Supplementary us-vs-them analysis. Because the us-vs-them head achieved only moderate human–human (

\kappa=.510

) and human–model (

\kappa=.596

) agreement, it is retained as a supplementary indicator rather than a primary prevalence metric. The values above are therefore best read as qualitative profile signals rather than exact point estimates.

Hyperparameter	CamemBERT-base	XLM-RoBERTa-large
Pretrained model	camembert-base	xlm-roberta-large
Max sequence length (tokens)	48	48
Per-device batch size	16	8
Gradient accumulation steps	2 (eff. batch = 32)	4 (eff. batch = 32)
Learning rate	$2\times 10^{-5}$	$1\times 10^{-5}$
Weight decay	0.01	0.01

Table 23: Training hyperparameters for CamemBERT-base and XLM-RoBERTa-large on the final v6 supervision set. Both models train on the same 6,999-headline training split; thresholds are selected on the 1,500-headline validation split only. Results in Table 3 are from the seed-42 run; three-seed evaluation (seeds 42, 123, 456) confirms seed-robust model selection. Full per-seed results: v6_model/colab/multiseed_variance_eval.ipynb .

Device	Definition (YES)	Canonical YES examples	Canonical NO examples
Loaded vocabulary	Strong emotional, exaggerated, metaphorical, or value-laden wording beyond neutral reporting	catastrophe, chaos, désastre, submersion, effondrement ; metaphorical or dramatic phrasing	Neutral descriptive wording: réforme, mesure, discussion
Blame attribution	An actor is clearly presented as causing harm, failure, or a negative outcome	Explicit causation: provoque, entraîne, aggrave ; responsibility: responsable de, à cause de ; accusation: accusé de, mis en cause	Simple criticism or opposition without causation: X critique Y , X s’oppose à Y
Threat framing	Wording casts an actor, event, or situation as dangerous, destabilizing, or harmful	menace, danger, risque, fait craindre ; escalation: au bord de, sous pression ; metaphorical threat: déferlante, explosion de, submerge	Neutral reporting of events without danger framing
Rhetorical question	A question that implies judgment, criticism, or suspicion	Implies evaluation: Faut-il craindre ? Macron a-t-il échoué ? Jusqu’où ira…	Neutral informational questions: Que dit la loi ? Que sait-on ?
Us-vs-them †	Explicit opposition between two distinct groups, both holding social, political, ethnic, national, class, or institutional identity	La France face aux migrants ; class opposition: les 1% vs le reste du monde ; institutional: gouvernement vs syndicats	Politician vs. politician (CONFLICT); commercial competition; one actor + vague target; crime report victim/perpetrator
Charge band (HIGH)	HIGH = strong emotional intensity, crisis, violence, confrontation, or scandal	Crisis, attack, scandal, strong confrontation	Routine, neutral reporting (LOW)

Table 24: Condensed annotation schema for the released supervision criteria. † Us-vs-them requires both groups to hold explicit social, political, ethnic, national, class, or institutional identity; borderline guidance covers politician-vs.-politician (classified as CONFLICT) and commercial competition as canonical exclusions. The full annotation prompt, user template, lexicons, and analysis scripts are available at https://github.com/lefrenchnewslab/framing-wording-selection .

Metric	Value	N
Cohen’s $\kappa$ (nominal)	.654	642
Cohen’s $\kappa$ (linear)	.666	642
Krippendorff’s $\alpha$	.653	642
% raw agreement	69.9%	642
By primary annotator confidence
HIGH	.729	318

Table 25: Post-hoc human–human inter-annotator agreement on the 642 three-way story-form conflict cases resolved by single-annotator arbitration. A second independent annotator applied the same codebook without access to the primary annotator’s decisions. The original gold labels were not modified. Agreement is higher for cases the primary annotator rated HIGH-confidence (

\kappa=.729

) than MEDIUM-confidence (

\kappa=.586

), confirming well-calibrated self-assessed uncertainty. Lower agreement on conflict trios involving ELITE (

\kappa=.276

–.417) reflects the residual-category nature of ELITE in the schema. Interpretation thresholds follow Landis and Koch 1977 .

Outlet	Default cell	Perturbed cell(s)	Triggering perturbation(s)
Valeurs actuelles	double-distinctive	selection-dominant	raise salience median by 10% or 20%
Le HuffPost	low-distinctiveness	selection-dominant	lower salience median by 10% or 20%; any-salience-only perturbation except raising selection median
Marianne	selection-dominant	double-distinctive	lower salience median by 10% or 20%; any selection-only perturbation
Libération	low-distinctiveness	selection-dominant	lower selection median by 20%
BFMTV	salience-dominant	double-distinctive; low-distinctiveness	lower selection median by 10% or 20%; raise salience median by 20%
L’Humanité	selection-dominant	double-distinctive	lower salience median by 10% or 20%

Table 26: Outlets whose four-cell assignments change under

\pm

20% perturbations of the salience and selection median split points. All remaining outlets keep the same cell assignment under every perturbation, indicating that instability concentrates in near-boundary cases, not distinctive outliers.

Outlet	N	Full Any-Sal%	RQ-excl Any-Sal%	$\Delta$ pp	$\Delta$ Sal.JS
Fdesouche	16,124	72.6	72.2	0.5	$-.026$
Slate.fr	3,907	49.6	31.9	17.7	$-.032$
Causeur	3,056	59.1	50.0	9.1	$+$ .002
Blast †	646	67.3	65.9	1.4	$-.018$
Ouest-France	90,423	7.2	5.8	1.4	$-.002$
Mediapart	7,092	58.8	56.8	2.0	$-.009$

Table 27: Rhetorical-question-excluded any-salience robustness. Recomputing any-salience as loaded vocabulary OR blame attribution OR threat framing (dropping the rhetorical-question head) reduces the corpus-level detected rate from 34.6% to 30.6% (4.0 pp). The Sal.JS/Sel.JS Pearson correlation drops from

r=.736

r=.561

(

r=.650

excluding Fdesouche). Four near-median outlets flip typology cells: Mediapart and Valeurs actuelles (double-distinctive

\rightarrow

selection-dominant) and Marianne and L’Humanité (selection-dominant

\rightarrow

double-distinctive); all other 21 outlets are stable. † Blast (

N{=}646

): case profile, wide CIs (Table 12 ).

Head	Threshold	Val. Prev.	Val. Prec.	Prod. Prev.	Est. Prec.	$\Delta$ pp
Loaded vocabulary	0.67	23.9%	73.3%	17.0%	64.3%	$-$ 9.0
Blame attribution	0.83	14.7%	71.9%	13.6%	70.1%	$-$ 1.8
Threat framing	0.84	8.8%	72.3%	8.5%	71.5%	$-$ 0.8
Rhetorical question	0.89	7.3%	95.7%	4.4%	92.9%	$-$ 2.8
Us-vs-them	0.89	6.4%	60.0%	5.3%	55.0%	$-$ 5.0
Any-salience	OR	41.1%	83.8%	33.7%	79.0%	$-$ 4.8

Table 28: Estimated precision drift under class-prior shift. Expected production precision is computed from the validation-set TPR and FPR at each head’s chosen threshold, applying the production corpus prevalence via Elkan 2001 . The any-salience OR row is an XLM-R baseline diagnostic computed before the final CamemBERT rhetorical-question substitution, so its production prevalence (33.7%) is slightly lower than the final ensemble corpus any-salience rate in the main text (34.6%).

Min. headlines	N cells	$\beta$	SE	$p$
5	1,197	0.8793	0.0637	$<$ .001
10	1,184	0.7582	0.0660	$<$ .001
20	1,161	0.8552	0.0689	$<$ .001
30	1,151	0.9218	0.0684	$<$ .001

Table 29: Temporal panel robustness. (a) Sensitivity to sparse-cell exclusion thresholds (two-way outlet

+

month fixed effects, classical SEs): the within-outlet coupling is stable across thresholds. (b) Cluster-robust standard errors (clustered by outlet,

G{=}25

, small-sample correction): positive under both specifications but not individually significant, consistent with treating the panel as confirmatory rather than a stand-alone test.

Political Discourse Analysis

Political discourse analysis studies how language expresses ideology, frames issues, persuades audiences, and shapes political debate.

See all 64 papers

Mining Frames: Association Rule Mining as an Unsupervised Framework for News Framing Detection
Oct 22, 2026Hager Hesham MustafaAssociation Rule Mining
Anmat Computational Analysis Lab
Uncovering Temporal Framing in the News
May 29, 2026Tarek Mahmoud, Veronika Solopova, Premtim Sahitaj +7Multilingual Text Classification
MBZUAI, UAE · Technische Universität Berlin, Germany · German Research Center for Artificial Intelligence (DFKI), Germany +1
FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification
Jun 8, 2026Amr SobhyText Classification Cross-Domain Generalization
Le French News Lab

Oct 22, 2026J/K move · Enter open · S save

Mining Frames: Association Rule Mining as an Unsupervised Framework for News Framing Detection

Hager Hesham Mustafa

Anmat Computational Analysis Lab

Association Rule Mining

Open paper AskPDF arXiv

Computational framing analysis is dominated by supervised classifiers that require large annotated corpora and predefined frame taxonomies, limiting their use for non-English languages, novel events, and low-resource settings. This paper presents Association Rule Mining (ARM) as an unsupervised, annotation-free framework for news framing detection. Each document or sentence is treated as a transaction of lexical items, and high-lift co-occurrence rules are read as candidate framing signals (Gamson and Modigliani's condensing symbols), confirmed in context by journalist-researchers. The primary validation is Framing Gaza: 25,512 English-language articles from 16 US, UK, French, and German outlets covering the first year of the Gaza war (October 2023 to October 2024), analyzed with Apriori (mlxtend), platform-weighted support, POS-filtered n-grams, and lift-based scoring. ARM inductively surfaces interpretable patterns without predefined categories: a cross-platform military frame, {israeli, strike} to {kill}, appears as a closed rule in 14 of 16 outlets, and a month-by-month analysis recovers event-driven framing pivots (e.g., the April 2024 World Central Kitchen strike) with no prior event labeling. A systematic headline-body divergence is identified, with headlines carrying concentrated, high-lift editorial framing and body text reproducing institutional, procedural language. A methodological validation on the Seven Days project (1,575 Arabic articles on the 2026 Iran war) shows that FP-Growth enables sentence-level analysis with far lower memory overhead. Findings were validated through editorial uptake: two published investigative stories (Daraj Media; Saheeh Masr). Preprocessing code is released openly; the full pipeline is documented but withheld to preserve operational deployability.

Abstract

Political Discourse Analysis

Explore similar work

Mining Frames: Association Rule Mining as an Unsupervised Framework for News Framing Detection

Uncovering Temporal Framing in the News

FrenchNews-7: Benchmarking Cross-Publisher French News Editorial Desk Classification