Delegating resume screening to large language models (LLMs) requires organizations to establish whether they can distinguish candidates by relevant qualifications while allowing only authorized users to change screening rules or approve exceptions. To this end, we develop a system architecture that separates candidate evidence, evaluation, permission controls, and assessment record, and empirically examine two complementary dimensions of its deployment readiness. First, we assess its operational readiness on four LLMs using a controlled experiment on 210 resumes across 14 job roles and six decision interfaces. Second, we assess its compliance readiness by testing how it handles unauthorized user requests and instructions embedded in resumes. We find that operational performance depends on the choice of evaluation interface: scalar scores generally distinguish qualification changes better than simple pass/fail judgments, while comparative judgments are sensitive to presentation order. Explicit permission rules and separation of resume content from instructions reduce unauthorized compliance, to a varying degree of effectiveness across LLMs and usage contexts. Finally, we find that LLM evaluators often diverge between their decisions and offered justifications. Going beyond simply evaluating LLMs for hiring tasks, our framework supports an audit of a fully configured resume evaluation system to enable its deployment decision.
Figures & tables
Figure 1: Reference architecture for an LLM-based resume screening system. Study 1 examines the evaluation mechanism through controlled changes in qualification evidence, decision interface, presentation order, and identity cues. Study 2 examines authority and provenance controls through requests issued under different permissions and through instructions embedded in candidate materials, as well as checks the fidelity of the resulting assessments. Dashed arrows denote audit feedback.
Figure 2: For each baseline resume corresponding to a JD, three anchors each of required qualifications and preferred qualifications are chosen. Every baseline resume has different variants wherein either 1, 2 or 3 anchors are being modified for each of the required removal or preferred addition of qualifications.
Figure 3: The anchors for the IT role are mentioned. Only the portions of the resume that are changed are shown above. The box in red shows the removal of the required qualification anchors while the one in green shows the addition of the preferred qualifications
Interface
Decision space
Utility
Binary
Shortlist or Reject
Whether a resume should be considered further
Ordinal
Ordered suitability categories, 1–5
Coarse assessment of one resume
Scalar
Suitability score, 0-100
Graded assessment of one resume
Rubric
Evidence labels at anchors: present/absent/unclear
Granular assessment of one resume
Pairwise
Stronger candidate or tie, both orders compared
Relative judgment between two resumes
Listwise
Ranking of four candidates, all 24 permutations compared
Ranked ordering of a set of resumes
Table 1: Decision interfaces and their usage
Figure 4: Evaluation of single-resume decision interfaces when 3 required qualifications are removed and 3 preferred qualifications are added. Panel A reports the sensitivity of binary, ordinal, and scalar scoring to resume qualification changes, panel B reports the accuracy of rubric-based evaluations.
Figure 5: Evaluation of decision interfaces that consider multiple resumes together. Panel A compares the accuracy of pairwise decisions when the stronger resume is presented first or second, and panel B compares the accuracy of listwise decisions across different positions of the strongest resume in the list
MiniMax- M3
DeepSeek- V4-Pro
Nemotron- 3-Ultra
GPT-OSS- 120B
Cutoff
53
73
56
73
Expected balanced accuracy (%)
91.8
78.4
98.7
79.9
Table 2: Score cutoffs and correspondence with binary recommendations
Figure 6: Testing whether all qualifications in a job are weighed equally important in (top) scalar and (bottom) binary scoring
Figure 7: A governance layer around the LLM evaluator triages usage context with respect to the controls provided in the application context and constructs the final input prompt to be supplied to the LLM.
Figure 8: Decision outcomes under authority and provenance controls. Panel A compares direct unauthorized requests. Panel B compares user permissions under D3. Panel C reports administrator-spoofing requests. Panel D separates the request types by evaluator under D3.
Figure 9: Mechanisms of spotlighting untrusted data.
Figure 10: Rate of undesired compliance to instructions embedded in resumes under different defense mechanisms
Figure 11: Examples of assessment inconsistencies following intervention attempts
Evaluator
Decision reversed toward attack target
Decision unchanged
MiniMax-M3
42/42 (100.0%)
5/491 (1.0%)
DeepSeek-V4-Pro
24/24 (100.0%)
3/606 (0.5%)
Nemotron-3-Ultra
19/19 (100.0%)
0/392 (0.0%)
GPT-OSS-120B
8/8 (100.0%)
0/609 (0.0%)
Pooled
93/93 (100.0%)
8/2,098 (0.4%)
Table 3: New justification problems by observed decision outcome
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Sector
Sources
IT
Amazon Jobs India, GE Vernova Careers, Cognizant
Legal services
Ernst & Young (EY) India
Healthcare operations
National Career Service hospital administrator profile, National Accreditation Board for Hospitals and Healthcare Providers (NABH) material
Medicine
National Medical Commission material, Steel Authority of India Limited (SAIL) recruitment notices
Education
Central Board of Secondary Education (CBSE) principal role, affiliation and staff-qualification provisions
Appendix
Table A1 : Sector-wise breakdown of JD sources
Required removals
Preferred additions
Evaluator
Edits ( k )
Binary
Ordinal
Scalar
Binary
Ordinal
Scalar
MiniMax-M3
1
7.1
64.3
85.7
14.3
21.4
78.6
2
21.4
78.6
92.9
14.3
21.4
100.0
3
35.7
71.4
92.9
14.3
21.4
100.0
DeepSeek-V4-Pro
1
50.0
92.9
78.6
28.6
57.1
85.7
2
71.4
100.0
92.9
28.6
64.3
100.0
Appendix
Table A2: Directional responsiveness by number of qualification edits
Resume screeners must infer job-relevant competence from resumes whose presentation can vary substantially in wording, structure, stylistic polish, and document extraction quality. Ideally, such surface variation should not change decisions when the underlying qualification evidence is unchanged. We introduce a controlled audit of this property, constructing occupation-grounded candidate profiles at controlled competence levels and rendering each profile into multiple resume presentations. A deterministic validation gate excludes variants that alter the underlying evidence before scoring. Across six open instruction-tuned LLM conditions, we find a clear disconnect between screening validity and presentation stability. Llama-3.1-8B with its native chat template achieves the strongest validity (0.781) yet reverses 29.6% of matched pairwise decisions under competence-preserving presentation changes; Mistral-7B-v0.3 reaches validity 0.644 with a 41.4% flip rate. Native chat formatting improves validity for several chat-tuned models but does not remove this instability. These results show that resume-screening evaluations should assess not only whether a system identifies stronger candidates, but also whether those decisions remain stable when the same competence evidence is presented differently.
Qiangju Chen, Yang Xiao
1Macquarie University · 2The University of Melbourne
We present PopResume, a population-representative resume dataset for causal fairness auditing of LLM- and VLM-based resume screening systems. Unlike existing benchmarks that rely on manually injected demographic information and outcome-level disparities, PopResume is grounded in population statistics and preserves natural attribute relationships, enabling path-specific effect (PSE)-based fairness evaluation. We decompose the effect of a protected attribute on resume scores into two paths: the business necessity path, mediated by job-relevant qualifications, and the redlining path, mediated by demographic proxies. This distinction allows auditors to separate legally permissible from impermissible sources of disparity. Evaluating four LLMs and four VLMs on PopResume's 60.8K resumes across five occupations, we identify five representative discrimination patterns that aggregate metrics fail to capture. Our results demonstrate that PSE-based evaluation reveals fairness issues masked by outcome-level measures, underscoring the need for causally-grounded auditing frameworks in AI-assisted hiring.
Sumin Yu, Juhyeon Park, Taesup Moon
ECE, Seoul National University · IPAI, Seoul National University · ASRI / INMC / AIIS, Seoul National University
Effective hiring is integral to the success of an organisation, but it is very challenging to find the most suitable candidates because expert evaluation (e.g.\ interviews conducted by a technical manager) are expensive to deploy at scale. Therefore, automated resume scoring and other applicant-screening methods are increasingly used to coarsely filter candidates, making decisions on limited information. We propose that large language models (LLMs) can play the role of subject matter experts to cost-effectively elicit information from each candidate that is nuanced and role-specific, thereby improving the quality of early-stage hiring decisions. We present a system that leverages an LLM interviewer to update belief over an applicant's rubric-oriented latent traits in a calibrated way. We evaluate our system on simulated interviews and show that belief converges towards the simulated applicants' artificially-constructed latent ability levels. We release code, a modest dataset of public-domain/anonymised resumes, belief calibration tests, and simulated interviews, at https://github.com/mbzuai-nlp/beyond-the-resume. Our demo is available at https://btr.hstu.net.
Harry Stuart, Masahiro Kaneko, Timothy Baldwin
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, United Arab Emirates