stat.MLOct 27, 2026
SaveInterpretable Feature Discovery from Unstructured Data Using LLMs and Statistical Choice Models
Organizations: École polytechnique, Palaiseau, France · Now at the Institute for Computational and Mathematical Engineering (ICME), Stanford University. · Operations Department, Kellogg School of Management, Northwestern University, Evanston, IL, USA
Abstract
Feature extraction from unstructured data typically relies on opaque embeddings unsuited to interpretable decision-making. Interpretable features, on the other hand, typically require costly, and often infeasible, hand-engineering. Large language models (LLMs) offer a natural bridge, as they can read raw data and describe it in interpretable terms. One could simply ask an LLM to extract features from a large sample of the dataset, but context limitations and diluted attention can lead to unsatisfactory results. We introduce a method that automatically discovers interpretable, decision-relevant features from unstructured data. Our key idea is to perform pairwise comparisons between items, asking an LLM to articulate, in interpretable terms, the features distinguishing each pair. We treat these explanations, or rationales, as data: we develop a statistical choice model that combines these rationales across the comparisons to identify relevant features and estimate per-item, per-feature scores. We first test our method on a synthetic hiring dataset of cover letters with known ground-truth features and show that it significantly outperforms direct LLM feature extraction in feature recovery. We then apply it to the ChangeMyView (Reddit) debate dataset, where it discovers several interpretable features associated with persuasion while reaching predictive accuracy comparable to established baselines. We also show how the same rationales can be used to audit LLM decisions, yielding a holistic score whose decomposition over interpretable features reveals what the LLM rewards.
Figures & tables
Figure 1: Overview of the pipeline described in Section 3 . Items are first compared in pairs by the LLM, cited features are then clustered, and a statistical model learns the relevant features from the LLM’s choices. Finally, statistical diagnostics identify the reliable scores. The orange running example follows one pair of cover letters (the setup of Section 4 ) through the pipeline.
| Method | Cost | Tracked | Stat.R. | Sem.R. | Attrib. | LK | ||
|---|---|---|---|---|---|---|---|---|
| Ours | 3.45 | 21 | 100.0% | 0.850 | 100.0% | 76.2% | 0.090 | |
| Benchmarks | Direct LLM | 0.26 | 21 | 81.2% | 0.732 | 81.2% | 61.9% | 0.109 |
| Direct LLM–Multi-score | 3.42 | 21 | 81.2% | 0.787 | 81.2% | 66.7% | 0.118 | |
| Local LLMs | 2.35 | 23 | 75.0% | 0.702 | 68.8% | 47.8% | 0.110 | |
| Local LLMs–Multi-score | 3.27 | 23 | 75.0% | 0.737 | 68.8% | 47.8% | 0.113 | |
| Ablations | Pairwise comparisons + LLM scoring | 3.67 | 32 | 93.8% | 0.763 | 87.5% | 40.6% | 0.093 |
Table 1: Per-method estimates on the single full-data run. Costs are Batch-API prices in USD; is the number of features the method reports. Tracked, Sem.R., Attrib. in %; Stat.R. and LK are correlations.
Figure 2: Discovered features span more effective dimensions than prior ones while retaining predictive accuracy. In both panels, each point is an average over random -subsets of the feature set, so that sets of different sizes are compared at matched size. Left: noise-corrected participation ratio of the feature-score correlation matrix (Spearman) over the held-out exchanges; dotted: . Right: held-out accuracy of the logistic regression on the held-out pairs; chance . D: discovered ( features); D r : reliable subset of D ( ); P: prior ( ); S: sample-based ( ); BT/PW: Bradley–Terry/pointwise scoring; #words: reply length. Bands: bootstrap percentile intervals over held-out exchanges (left) and exchanges + pairs (right). Shaded: , where P and D r are not defined, and , where only S is defined. Details in Appendix D.3.1 .
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Which features matter for each judge on each dataset. Radial axis is (with estimation noise removed) on each judge’s reliable features ( , ). Bars are 95% delta-method intervals from the sandwich covariance. Axes are cross-judge clusters formed by meaning ( GPT-5.4 (high reasoning) ), restricted to constructs at least three of the four judges discovered (see Figures 6 and 7 for the full plots). An open marker denotes a construct absent from that judge’s reliable basis (never surfaced, or removed by the identifiability screen), not a zero estimate.
Figure 5: Per-judge overall score distribution and its agreement (Spearman rank correlation) with human scores.
Figure 6: Denoised per-feature spread by judge on ELLIPSE. Axes are cross-judge constructs formed by meaning ( GPT-5.4 (high reasoning) ); annotations give how many of the four judges discovered each construct. The radial coordinate is the denoised spread of each judge’s feature mapped to that construct, on per-judge standardized reliable feature sets ( , ); bars are 95% delta-method intervals from the sandwich covariance. An open marker denotes a construct absent from that judge’s reliable basis (never surfaced, or removed by the identifiability screen), not a zero estimate.
Figure 7: Denoised per-feature spread by judge on ASAP 2.0. Same construction, quantities, and conventions as Figure 6 , on the ASAP 2.0 corpus.
Figure 8: Out-of-fold reliability diagrams for the winner gate (left; unit: comparison) and the citation gate (right; unit: cell) of the model on ELLIPSE, one row per judge. Units are grouped into ten equal-width bins of predicted probability ; points show the observed event rate against the mean prediction , with bootstrap intervals (resampling comparisons, resp. cells). Faint line: in-sample fit; dashed: perfect calibration ( ). The expected calibration error is , with . Bottom strips: per bin, log scale.
Figure 9: Out-of-fold Brier skill of the two gates, per judge and corpus, with in-sample values overlaid. With the predicted probability and the gated event, the Brier score is and the skill score against a reference forecast is , where ; is perfect, matches the reference, and negative values are worse than it. Two references: chance ( ) and the empirical event rate of the training fold ( ). Intervals as in Figure 8 .
Figure 10: As Figure 8 , on ASAP 2.0.
Figure 11: The job advertisement used as the fixed evaluation context. Colored tags mark which requirement grounds which dial, using the same visual vocabulary as the letter template (Figure 12 ): the qualifications ground the education and numbered dials, and the preferred qualifications and responsibilities ground the six role-skill dials. Go is a mandatory requirement realized identically in every letter, not a dial. The remaining dials of Table 3 (school and employer prestige, prose fluency, and the four niche dials) have no anchor in the ad and are described in Appendix C.1 .
Figure 12: Example cover-letter template. Roman text is fixed template prose; angle-bracketed slots are filled from the ground-truth dial vector. Skill coverage sets how many core units are capability sentences; paragraph count is constant. Fluency is imposed afterward by injecting surface errors.
| Dial | Type | Levels | Group |
| Experience (years) | ordered | 3 / 5 / 7 | primary |
| Education level | ordered | BS / MS / PhD | primary |
| School selectivity | ordered | low / mid / high | primary |
| Employer prestige | ordered | startup / mid / big tech | primary |
| Languages known | ordered | 2 / 5 / 9 | primary |
| Production launches | ordered | 2 / 4 / 6 | primary |
Table 3: The 18 engineered ground-truth dials of the cover-letter corpus.
Figure 13: Out-of-fold reliability of the citation gate (left, all cells) and direction gate (right, cited cells) of the feature-discovery model. Points: observed rate per bin of predicted probability, with bootstrap intervals over comparisons; faint line: in-sample; dashed: perfect calibration; bottom: cells per bin (log scale).
Figure 14: Out-of-fold Brier skill of the two gates against chance and against the training-fold empirical rates; intervals as in Figure 13 .
| feature | Cite rate | Kept | |||
| AI experience | 0.613 | 3.6 | 0.937 | ✓ | |
| Security assessment experience | 0.596 | 2.9 | 0.935 | ✓ | |
| Security automation experience | 0.562 | 2.4 | 0.917 | ✓ | |
| Agentic systems experience | 0.544 | 2.7 | 0.924 | ✓ | |
| Professional experience | 0.531 | 3.8 | 0.960 | ✓ | |
| Algorithmic strength | 0.524 | 2.7 | 0.917 | ✓ |
Table 4: Features surfaced by our comparative method in the point estimate of Table 1 . Of surfaced features, we keep the features that have and , leaving features. Fitted (95% CI ). CIs for and coefficients were computed using ( B.3 ).
| Feature | Description | Scoring guidelines |
|---|---|---|
| Academic degree | Captures the formal level of academic preparation or equivalent technical grounding the applicant presents. | 1 No clear qualifying degree or equivalent preparation is shown. 2 Shows a bachelor’s degree or equivalent practical preparation. 3 Shows a master’s degree in a relevant technical field. 4 Shows a doctoral degree in a relevant technical field. |
| Adversarial practice | Captures ongoing offensive-security practice through CTF participation and competitive results. | 1 No ongoing offensive-security practice is shown. 2 Mentions occasional CTF participation or light adversarial practice. 3 Shows regular or serious CTF participation as continuing skill development. 4 Shows strong competitive results, such as near-top or top placements in substantial CTF fields. |
| Agent boundary design | Captures experience designing agentic systems with explicit control boundaries between Python agents and Go services. | 1 No agentic tooling or agent architecture is described. 2 Mentions agents or agentic systems only at a high level. 3 Describes building agentic systems with Python agents and Go back-end support. 4 Describes explicit guardrails such as bounded actions, permissions, auditing, or enforced boundaries between agents and services. |
| Algorithmic depth | Captures depth of data-structure and algorithm reasoning as evidenced by concrete redesign or optimization examples. | 1 Does not provide evidence of algorithmic or data-structure skill. 2 Claims strong algorithms knowledge, but with little concrete illustration. 3 Provides a clear example of using better algorithms or data structures to improve a system. 4 Provides a strong example with technical specificity and visible impact, such as removing a bottleneck or making a previously hard problem tractable. |
| Applied AI deployment | Captures whether the applicant has shipped AI or LLM features in production and understands reliability or evaluation concerns. | 1 No meaningful AI or LLM experience is shown. 2 Mentions AI interest or exposure without a clear shipped production example. 3 Shows direct experience shipping AI or LLM-backed functionality in production. 4 Shows shipped AI or LLM systems plus concrete attention to evaluation, reliability, or failure modes. |
| Developer enablement mindset | Captures whether the applicant frames security tooling as a way to help developers do the right thing with less friction. | 1 Does not frame security work in terms of helping other engineers. 2 Mentions helping teams only in broad or generic language. 3 Clearly emphasizes making secure behavior easier or more natural for developers. 4 Explicitly focuses on reducing friction, building good defaults, or serving busy developer teams through usable tooling. |
Table 5: The rubric elicited by Direct LLM. Descriptions and the per-score guidelines are reproduced verbatim as the judge produced them; the open elicitation lets the judge choose its own scale, and it selected a – scale for every feature.
Figure 15: Paired comparisons over bootstrap draws (with replacement) of the -letter corpus; every arm sees the same draws, and differences are taken within replicate. Each row shows the mean paired difference with its percentile CI ( : excludes zero); the right column gives Ours better , the share of replicates our method wins ( , the difference of discovered features, is annotated below each panel and carries no share). The panels bracket discovery variance from opposite sides and are not comparable to one another : (a) re-runs discovery per replicate on both sides, (b) freezes discovery on both sides, pooled over fusion and corpus draws. Both panels score our method at a reduced budget (average degree against the designed ); it leads on several metrics regardless. Exact values: Table 7 .
| Method | Cost | Tracked | Stat.R. | Sem.R. | Attrib. | LK | |
|---|---|---|---|---|---|---|---|
| Ours | 3.45 | 21 | 100.0% | 0.850 | 100.0% | 76.2% | 0.090 |
| resampling matchings | [16.2, 22.0] | [87.5, 93.8] | [0.737, 0.805] | [81.2, 93.8] | [71.4, 87.5] | [0.087, 0.095] | |
| Direct LLM | 0.26 | 21 | 81.2% | 0.732 | 81.2% | 61.9% | 0.109 |
| resampling letters | [17.2, 25.0] | [68.8, 87.5] | [0.649, 0.825] | [62.5, 87.5] | [40.6, 69.6] | [0.109, 0.135] | |
| Direct LLM–Multi-score | 3.42 | 21 | 81.2% | 0.787 | 81.2% | 66.7% | 0.118 |
| resampling letters | [21.0, 21.0] | [81.2, 81.2] | [0.788, 0.807] | [81.2, 81.2] | [57.1, 66.7] | [0.119, 0.145] |
Table 6: Per-method point estimates with run-variance bands. Tracked, Sem.R., Attrib. in %; Stat.R. and LK are correlations; bands are in the same units as the column. Below each method’s point estimates, the gray row gives that method’s own run-variance band ( percentiles across replicates), obtained by resampling the unit named at the left of the row. These bands are not confidence intervals for the point estimates above them – the point estimates come from the single full-data run, the bands from re-runs on resampled data, so a point estimate may fall outside its band – and they must not be compared across methods , since the arms resample different units. Paired, comparable contrasts: Table 7 .
| Difference | 95% CI | Ours better | |
| (a) Ours Direct LLM (discovery re-run per replicate on both sides) | |||
| — | |||
| Tracked ∗ | |||
| Stat.R. | |||
| Sem.R. ∗ | |||
| Attrib. ∗ | |||
Table 7: Paired comparisons over bootstrap draws (with replacement) of the -letter corpus; every arm sees the same draws, and differences are taken within replicate. Each row reports the mean difference, its percentile CI ( ∗ : excludes zero), and Ours better : the share of replicates in which our method wins in the metric’s preferred direction ( is descriptive and carries no share). The panels bracket discovery variance from opposite sides and are not comparable to one another . Both panels score our method at a reduced budget (average degree against the designed ) ; it leads on several metrics regardless. Design rationale, the conditional-fusion alternative, and budget details: Appendix C.4.2 .
Figure 16: Dial-recovery heatmap yielded by the features discovered by our method. The rows are the discovered features and the columns are the dials.
Figure 17: Dial-recovery heatmap yielded by the features discovered by Direct LLM. The rows are the discovered features and the columns the dials. The scores leak more between one another and the dial recovery is less clean than in Figure 16 as some dials are not recovered.
Figure 18: Left: probability that a dial is recovered as a function of the acceptance threshold used to merge similar features across the elicitations of Local LLMs (dials are matched to the merged features by thresholded Hungarian assignment at the calibrated ). Right: number of features surviving the merge, against the number of pooled elicitations, one curve per acceptance threshold. At , recovering every dial demands a high threshold, which leaves an unusable number of near-duplicate features, hence the need for an LLM taxonomy.
| Framing | Accuracy |
|---|---|
| P1 (unnamed, scrubbed) | |
| P2 (named) | |
| Lift | pp |
Table 8: Memorization probe on CMV exchanges pinned to the data of Table 2 (judge: GPT-5.4 mini ; chance ). The lift is assessed by a paired McNemar test on the same exchanges ( , ).
Figure 19: The reliable discovered features are more predictive, feature for feature, than the full discovered set and than the sample-elicited set. Each panel shows, as a function of the number of features , the difference in held-out accuracy between D r (the reliable discovered features) and another feature set. At each , an -regularized logistic regression is fitted on up to random -subsets of each set (all subsets when there are fewer), and accuracy is averaged over subsets; PW scores are averaged over two scoring runs. Shaded: bootstrap interval at each ( resamples of the held-out pairs and of the drawn subsets). (a, b) Against the full discovered set D, with the same scoring: D r is ahead at every , by point on average over under both scorings ( CIs for PW, for BT). (c, d) Against the sample-elicited set S, scored pointwise only: D r + PW is ahead at every , by points on average ( ); D r + BT’s advantage grows with , reaching points at ( ).
| Arm | Dims | Accuracy | CI |
|---|---|---|---|
| Emb. (full) | 3,072 | 64.88% | |
| Tan | 3,205 | 63.75% | |
| D + PW | 29 | 62.63% | |
| D r + PW | 16 | 62.13% | |
| Emb. | 3,072 | 61.63% | |
| D r + BT | 16 | 61.50% |
Table 9: Held-out accuracy of each method. D r : the reliable subset of the discovered features D. Emb. denotes text-embedding-3-large embeddings. Arms marked (full) are trained on all training pairs; all others on . Accuracies for PW arms are obtained from the average of two scoring runs. Zero-shot asks the judge directly to predict the -winning reply, with the query issued in both presentation orders and the two predictions averaged. Chance ; bootstrap percentile intervals ( resamples of the held-out pairs, shared across arms).
Figure 20: Both dimensionality measures agree on the ordering of the feature sets. Noise-corrected effective dimensionality against feature set size , averaged over -subsets of each feature set. Left: participation ratio, as in Figure 2 . Right: effective rank [ Roy and Vetterli, 2007 ] , with the normalized singular values. Both are computed on the disattenuated correlation matrix. The order holds under both measures. Bands, shading and dotted line as in Figure 2 .
Figure 21: Effectiveness of the discovered argumentation techniques (D + PW). The figure is the complete version of Figure 3 and shows each discovered feature’s effectiveness , i.e. the fraction of the test pairs in which the reply that earned the scores higher on that feature. is chance. Brackets give Wald intervals and is the Benjamini–Hochberg false-discovery rate across the tests. The discovered features are gathered by category. Exact values of the BH and each feature’s description can be found in Table 10 .
| Feature | Description | Win rate (95% CI) | |
|---|---|---|---|
| Concrete illustration ∗ | The argument makes its case through specific, vivid, or worked-through examples rather than abstraction alone. | 0.604 | 0.001 |
| Qualified rebuttal ∗ | The challenger narrows the claim, marks exceptions, or emphasizes contextual dependence instead of arguing absolutely. | 0.596 | 0.001 |
| Structured rebuttal development ∗ | The challenger builds an organized, multi-step, or multi-pronged case rather than a single bare point. | 0.583 | 0.001 |
| Scope broadening | The exchange expands or contracts the discussion by moving between narrow cases and wider principles or issue areas. | 0.583 | 0.001 |
| Counterexample use ∗ | The challenger undermines the OP with exceptions, alternative cases, or edge-case scenarios that break the generalization. | 0.581 | 0.001 |
| Challenger evidentiary support ∗ | The challenger backs the rebuttal with external evidence, citations, factual corrections, or expert authority. | 0.575 | 0.001 |
Table 10: Which argumentation techniques separate winners from losers (D + PW). This table details the results shown in Figures 3 and 21 . It presents each discovered feature (its name and description) together with its effectiveness , i.e. the fraction of the test pairs in which the reply that earned the scores higher on that feature. is chance. Brackets give Wald intervals and is the Benjamini–Hochberg false-discovery rate across the tests; features in bold satisfy . of the features separate winners, all of them positively. Features marked ∗ form the reliable subset D r ( features, and ).
| Feature | Description |
|---|---|
| Core Claim Relevance | It measures the degree to which the reply focuses on the central claim under dispute rather than drifting to side issues. |
| Reason Engagement | It measures the degree to which the reply directly addresses the original poster’s stated reasons, premises, or assumptions. |
| View Understanding | It measures the degree to which the reply accurately represents the other person’s position before challenging it. |
| Logical Coherence | It measures the degree to which the reply’s conclusions follow clearly and consistently from its premises. |
| Evidence Strength | It measures the degree to which the reply supports its claims with substantial, pertinent evidence or examples. |
| Source Trustworthiness | It measures the degree to which the reply relies on credible, authoritative, and verifiable sources when offering support. |
Table 11: Prior-elicited features (P). The features named by GPT-5.4 (high reasoning) for assessing the persuasiveness of a reply, with no data shown (Prompt ), in the order produced.
| Feature | Description |
|---|---|
| Topical responsiveness | How directly a reply addresses the poster’s actual view rather than pivoting to meta-issues or adjacent topics. |
| Premise targeting | Whether a reply attacks the core assumption or inference supporting the view instead of focusing on side details. |
| Clarification demand | How much a reply advances the discussion by asking the poster to specify vague claims, terms, or examples. |
| Conceptual reframing | Whether a reply changes the frame of the debate by redefining the key concept or explaining that the dispute is really about something else. |
| Thought experiment | Whether a reply uses an invented scenario to test the implications or consistency of the poster’s view. |
| Analogy use | How much a reply explains its point by mapping the issue onto a different familiar domain. |
Table 12: Sample-elicited features (S). The features named by GPT-5.4 (high reasoning) after reading a sample of exchange pairs, without their outcomes.
More in this topic
Interpretable ML
Interpretable machine learning develops models and explanations that make predictions, evidence, and decision processes understandable to people.
See all 310 papers