This paper investigates the task of predicting job experience levels in recruitment texts, aiming to automatically identify the qualifications required for positions. Unlike traditional text classification, recruitment texts typically possess explicit internal structures, with different paragraphs playing disproportionate roles in conveying experience clues. To address this, we propose a structure-aware Section-Aware BERT approach that segments and encodes key paragraphs (titles, responsibilities, requirements) for integrated modeling, building upon rule-based systems and classical baselines TF-IDF. Simultaneously, we evaluate large language models under both few-shot and fine-tuning settings on the same dataset to compare the capability boundaries of different modeling paradigms. Experimental results demonstrate that explicitly leveraging text structure significantly improves experience level prediction performance, particularly in scenarios with ambiguous job titles. Further error analysis reveals systemic challenges in this task, including confusion between Entry and Senior levels and the blurred boundaries of Mid-level positions. This research provides an effective modeling approach and analytical framework for understanding structured recruitment texts.
Figures & tables
Level
Title Patterns
Description Patterns
Executive
director, vice president, vp, chief, ceo, cto, cfo, head of, president
reports to ceo/board, leads team of 10+, p&l responsibility
Entry
entry-level, junior, intern, trainee, apprentice
no experience required, 0–2 years, recent graduate, will train
Senior
senior, sr., lead, principal, staff, architect
5+ years experience, 10+ years, extensive experience, proven track record
Mid
associate (excl. senior associate)
2–5 years experience, some experience
Table 1: Regular expression patterns for rule-based classification.
Figure 1: Baseline BERT architecture. The concatenated title and description are encoded as a single sequence.
Figure 2: Section-Aware BERT architecture. Four sections (Title, Responsibilities, Requirements, Other) are encoded separately by a shared BERT encoder, then fused via multi-head attention.
Model
Accuracy
Macro F1
Weighted F1
Test Samples
Classical & Rule-Based Models
Rule-Based (Regex)
0.5035
0.4284
0.4160
12,108
TF-IDF + Logistic Regression
0.7341
0.6351
0.7202
12,108
Transformer-Based Models
Baseline BERT
0.7835
0.7200
0.7807
12,108
Section-Aware BERT
0.8256
0.7600
0.8200
12,108
Table 2: Overall performance comparison across classical, transformer, and LLM-based models. For GPT-4o-mini models, evaluation is conducted on a stratified subset of 398 test samples due to API constraints.
Model
Entry
Mid
Senior
Executive
Macro Avg
Classical & Rule-Based Models
Rule-Based (Regex)
0.18
0.13
0.65
0.75
0.43
TF-IDF + Logistic Regression
0.78
0.36
0.77
0.63
0.64
Transformer-Based Models
Baseline BERT
0.83
0.53
0.80
0.73
0.72
Section-Aware BERT
0.88
0.54
0.85
0.79
0.76
Table 3: Per-class F1 scores for each model on the test set.
Sentence A
Sentence B
“you will lead the team”
“the team will lead you”
“entry-level management training”
“management of entry-level staff”
“assist senior engineers”
“senior engineer assists”
Table 4: Word order examples that TF-IDF cannot distinguish. Each pair implies different seniority levels but produces identical bag-of-words vectors.
Term
Total
Errors
Error Rate
coordinator
324
125
38.6%
analyst
529
199
37.6%
specialist
642
226
35.2%
engineer
1,080
352
32.6%
associate
582
166
28.5%
Table 5: TF-IDF error rates for polysemous job title terms.
Figure 3: TF-IDF confusion matrix. Bidirectional Entry ↔ Senior confusion dominates due to shared vocabulary across domains.
Class
Baseline
Section-Aware
Δ
Entry
83.3%
87.1%
+3.8%
Mid
47.2%
50.6%
+3.4%
Senior
83.0%
86.9%
+3.9%
Executive
69.5%
77.6%
+8.1%
Overall
78.5%
82.6%
+4.1%
Table 6: Recall comparison between Baseline BERT and Section-Aware BERT.
Figure 4: Section attention weights in Section-Aware BERT. Title receives the highest attention (36.5%), while Responsibilities and Requirements together contribute 40.3%.
Requirements
Responsibilities
Years mentioned
35.5%
Leadership verbs
65.6%
Degree required
36.7%
Strategic scope
20.5%
Certification
28.6%
Entry-level verbs
69.1%
Table 7: Experience signals by section (Section-Aware Wins samples).
True Label
Leader.
Strat.
Years
EntryV.
Entry
60.0%
15.0%
32.4%
68.7%
Mid
59.0%
13.5%
33.3%
65.4%
Senior
60.6%
17.4%
35.4%
62.7%
Executive
81.2%
40.0%
55.0%
55.0%
Table 8: Signal prevalence (%) by true experience level.
Confusion Fixed
Count
Example Title
Senior → Entry
323
Customer Advocacy Mgr
Entry → Senior
296
Process Engineer
Mid → Senior
80
Orthopedics Distributor
Exec. → Senior
70
Division Controller
Table 9: Top confusion pairs fixed by Section-Aware BERT.