Building Interpretable Feature Representations for Resume-Vacancy Matching by Distilling Production LLM Signals

Ilya Chekin; Vyacheslav Malyugin; Vladimir Chirkov; Mikhail Yurushkin

cs.CLOct 2, 2026

Building Interpretable Feature Representations for Resume-Vacancy Matching by Distilling Production LLM Signals

Authors: Ilya Chekin, Vyacheslav Malyugin, Vladimir Chirkov, Mikhail Yurushkin

Organizations: BroutonLab · Curately

Abstract

Matching candidates to vacancies is central to recruitment, and a recruiter needs to see why a candidate fits, not only a single opaque relevance score. We provide this evidence as named, interpretable matching dimensions recruiters can act on - eight in our current deployment. We propose a two-part approach. The first is an LLM-based labeler whose prompts and feature definitions were refined from recruiter feedback while it served as an earlier production matching stage. In the current architecture, it is used only for offline labeling and is not called on online requests. The second is a feature bi-encoder distilled from it: a LoRA-adapted embedding backbone with compact per-dimension heads that runs on CPU and serves all online requests. Both parts keep improving: prompts are revised as feedback arrives, and the bi-encoder is retrained on the updated labels. The model is trained on 168,772 labeled vacancy-resume pairs (17,921 vacancies and 180,030 resumes). Recruiters using the service can confirm or revise surfaced feature predictions. On 927 recruiter-recorded values from this selected production-feedback subset, the deployed student agrees with the recorded decisions in 888 cases (95.79%). This is operational, non-blinded agreement rather than an independent human evaluation.

Figures & tables

Panel of Figure 1 — Figure 1: Feature bi-encoder architecture. Vacancy and resume are encoded independently by a shared Qwen3-Embedding-0.6B + LoRA encoder. The four-component pair representation $\boldsymbol{\varphi}$ feeds eight task-specific MLP heads that predict ordinal or binary matching features — the interpretable per-feature verdicts shown to recruiters. A query-only certification applicability head (Eq. 2 , left) depends only on the vacancy embedding $E(j)$ and masks the certification dimension when it is not required by the vacancy.

Panel of Figure 2 — Figure 2: Full training pipeline. Production-refined LLM prompts annotate vacancy-resume pairs offline with hard ordinal labels. A cross-encoder trained on these labels provides soft per-class targets for knowledge distillation. The feature bi-encoder is trained with a hybrid loss (Eq. 3 ) combining KD, direct supervision, monotonicity, and margin regularization, in two stages (frozen encoder, then LoRA fine-tuning). The deployed student removes every online LLM call.

Document	n	p50	p75	p90	p95	p99
Vacancy	17,921	503	725	997	1,236	2,018
Resume	180,030	1,321	2,038	3,235	4,205	6,244

Table 1: Document length in tokens (training corpus).

Method	Acc.	Macro-F1
Majority baseline	0.606	0.268
Original encoder + MLP	0.760	0.669
Cross-encoder (offline)	0.934	0.897
Bi-encoder (production)	0.886	0.849

Table 2: Fidelity to LLM labels on the single held-out, vacancy-disjoint corpus split described in § 4.1 , averaged over eight dimensions; these are not human ground-truth metrics. The cross-encoder is the offline teacher and the bi-encoder is the production student. Bold: best fidelity.

(a) Pair representation		(b) Training objective
$[\mathbf{u};\mathbf{v}]$	0.837	Hard-label cross-entropy	0.825
$[\mathbf{u};\mathbf{v};\mathbf{u}\odot\mathbf{v}]$	0.842	+ cross-encoder soft targets (KD)	0.840
$[\mathbf{u};\mathbf{v};\|\mathbf{u}-\mathbf{v}\|]$	0.846	+ KD + ordinal consistency	0.845
$[\mathbf{u};\mathbf{v};\|\mathbf{u}-\mathbf{v}\|;\mathbf{u}\odot\mathbf{v}]$	0.849	+ KD + margin (without ordinal)	0.844
		Full objective	0.849

Table 3: Controlled macro-F1 ablations. The pairwise terms follow established sentence-pair practice; the full representation and full objective perform best.

Dimension	n	Exact	Agree	Wilson 95% CI	Maj.
Skill Context	120	120	100.0%	96.90–100.00%	72.5%
Skill Proficiency	120	120	100.0%	96.90–100.00%	61.7%
Hard Skill Experience	120	120	100.0%	96.90–100.00%	38.3%
Industry Experience	120	120	100.0%	96.90–100.00%	47.5%
Experience Recency	87	87	100.0%	95.77–100.00%	87.4%
Job Title Match	120	118	98.3%	94.13–99.54%	35.8%

Table 4: Agreement of the deployed student with recruiter-recorded production decisions. Recruiters saw the deployed system prediction before confirming or revising it; the analysis is non-blinded and uses only explicitly recorded feedback. Wilson intervals quantify uncertainty in the observed proportions; 100% is sample agreement, not perfect population accuracy.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Work	Student model	Supervision	Interpretable output	Serving
Li et al. (2025)	Role-aware MoE ranker	LLM-generated job descr.	Single ranking score	GPU
Yu et al. (2026)	LLM re-ranker (Qwen3)	Contrastive + LLM rerank	Single relevance score	GPU
Fu et al. (2025)	Distilled generative LLM	LLM (fit + explanation)	Free-text explanation	GPU
Jouanneau et al. (2026)	Late cross-attn. ranker	Generative LLM, calib. KD	Single skill-fit score	GPU
Ours	Bi-encoder (0.6B+LoRA)	Prod. LLM $\to$ cross-enc. KD	Eight named ordinal/binary dimensions	CPU

Appendix

Table 5: Qualitative comparison with the closest LLM-for-person–job-fit systems. Prior systems distil or prompt an LLM into a single score (or a free-text rationale); ours exposes eight independently supervised, named dimensions.

Skill group (evaluated jointly by combined-skill prompt)
Feature	Classes (0 $\to$ 1 $\to$ 2)	What is evaluated
Skill Context	no_overlap $\to$ research $\to$ production	Context in which the overlapping required skill was applied: no evidence, R&D / experimentation, or live production / business delivery.
Experience Recency	old $\to$ recent $\to$ active	How recently the candidate used the main required skill cluster ( $>$ 3 yr, 1–3 yr, or current / $<$ 1 yr).
Skill Proficiency	no_overlap $\to$ practical $\to$ professional	Depth of hands-on ownership: no credible evidence, real applied use with limited scope, or substantial ownership and repeated delivery.
Hard Skill Experience	mismatch $\to$ close_match $\to$ match	How well the demonstrated hands-on hard-skill experience (type and level) meets the vacancy requirement.
Fit group (separate per-dimension prompts)

Appendix

Table 6: The eight matching dimensions predicted by the feature bi-encoder. Class 2 always represents the strongest match. Certifications is treated as binary after applicability gating; the remaining seven dimensions use three-class ordinal modelling via cumulative logits.

Feature	Majority	MLP,	Full
	baseline	orig. enc.	model
Skill Context	0.489	0.788	0.883
Skill Proficiency	0.240	0.642	0.801
Hard Skill Experience	0.218	0.739	0.846
Industry Experience	0.203	0.682	0.853
Experience Recency	0.279	0.467	0.828

Appendix

Table 7: Macro-F1 per feature on the held-out, vacancy-disjoint validation split. Majority baseline : majority-class prediction. MLP, orig. enc. : MLP head on original (non-fine-tuned) encoder embeddings. Full model : fine-tuned encoder with knowledge distillation, monotonicity, and margin losses (trained with the full hybrid loss). Bold marks the best result per feature.

Job-Candidate Matching

See all 13 papers

Synapse: Evolving Job-Person Fit with Explainable Two-phase Retrieval and LLM-guided Genetic Resume Optimization
Apr 2, 2026Ansel Kaplan Erol, Seohee Yoon, Keenan Hom +1Job-Candidate Matching Recommender Systems
Georgia Institute of Technology, Atlanta, GA, USA
ConFit v3: Improving Resume-Job Matching with LLM-based Re-Ranking
May 10, 2026Xiao Yu, Ruize Xu, Chengyuan Xue +6Reinforcement Learning Learning to Rank
Columbia University · New York, NY, USA · John Hopkins University +6
Mira-Embeddings-V1: Domain-Adapted Semantic Reranking for Recruitment via LLM-Synthesized Data
Apr 20, 2026Zhaohua Liang, Zhilin Wang, Renjie Cao +1Learning to Rank Semantic Search
OpenJobs AI · San Francisico, CA, USA

Apr 2, 2026J/K move · Enter open · S save

Synapse: Evolving Job-Person Fit with Explainable Two-phase Retrieval and LLM-guided Genetic Resume Optimization

Ansel Kaplan Erol, Seohee Yoon, Keenan Hom, Xisheng Zhang

Georgia Institute of Technology, Atlanta, GA, USA

Job-Candidate Matching Recommender Systems Large Language Model-Based Recommendation Large Language Model-Guided Optimization Explainable Recommendation

Open paper AskPDF arXiv

Modern recruitment platforms operate under severe information imbalance: job seekers must search over massive, rapidly changing collections of postings, while employers are overwhelmed by high-volume, low-relevance applicant pools. Existing recruitment recommender systems typically rely on keyword matching or single-stage semantic retrieval, which struggle to capture fine-grained alignment between candidate experience and job requirements under real-world scale and cost constraints. We present Synapse, a multi-stage semantic recruitment system that separates high-recall candidate generation from high-precision semantic reranking, combining efficient dense retrieval using FAISS with an ensemble of contrastive learning and Large Language Model (LLM) reasoning. To improve transparency, Synapse incorporates a retrieval-augmented explanation layer that grounds recommendations in explicit evidence. Beyond retrieval, we introduce a novel evolutionary resume optimization framework that treats resume refinement as a black-box optimization problem. Using Differential Evolution with LLM-guided mutation operators, the system iteratively modifies candidate representations to improve alignment with screening objectives, without any labeled data. Evaluation shows that the proposed ensemble improves nDCG@10 by 22% over embedding-only retrieval baselines, while the evolutionary optimization loop consistently yields monotonic improvements in recommender scores, exceeding 60% relative gain across evaluated profiles. We plan to release code and data upon publication.

Abstract

Job-Candidate Matching

Explore similar work

Synapse: Evolving Job-Person Fit with Explainable Two-phase Retrieval and LLM-guided Genetic Resume Optimization

ConFit v3: Improving Resume-Job Matching with LLM-based Re-Ranking

Mira-Embeddings-V1: Domain-Adapted Semantic Reranking for Recruitment via LLM-Synthesized Data