WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain

Matthias De Lange; Warre Veys; Federico Retyk; Daniel Deniz; Warren Jouanneau; Mike Zhang; Aleksander Bielinski; Emma Jouffroy

cs.CLMar 17, 2026

WorkRB: A Community-Driven Evaluation Framework for AI in the Work Domain

Authors: Matthias De Lange, Warre Veys, Federico Retyk, Daniel Deniz, Warren Jouanneau, Mike Zhang, Aleksander Bielinski, Emma Jouffroy,

Organizations: TechWolf Ghent, Belgium · Avature Barcelona, Spain · Malt Paris, France · University of Copenhagen Copenhagen, Denmark ·

Abstract

Today's evolving labor markets rely increasingly on recommender systems for hiring, talent management, and workforce analytics, with natural language processing (NLP) capabilities at the core. Yet, research in this area remains highly fragmented. Studies employ divergent ontologies (ESCO, O*NET, national taxonomies), heterogeneous task formulations, and diverse model families, making cross-study comparison and reproducibility exceedingly difficult. General-purpose benchmarks lack coverage of work-specific tasks, and the inherent sensitivity of employment data further limits open evaluation. We present WorkRB (Work Research Benchmark), the first open-source, community-driven benchmark tailored to work-domain AI. WorkRB organizes 13 diverse tasks from 7 task groups as unified recommendation and NLP tasks, including job/skill recommendation, candidate recommendation, similar item recommendation, and skill extraction and normalization. WorkRB enables both monolingual and cross-lingual evaluation settings through dynamic loading of multilingual ontologies. Developed within a multi-stakeholder ecosystem of academia, industry, and public institutions, WorkRB has a modular design for seamless contributions and enables integration of proprietary tasks without disclosing sensitive data. WorkRB is available under the Apache 2.0 license at https://github.com/techwolf-ai/WorkRB.

Figures & tables

Figure 1. Overview of the community-driven WorkRB evaluation framework. A high-level overview diagram of the WorkRB framework showing tasks, models, and multilingual evaluation.

Dataset	Labels	Size (en)	Lang.
Occupation-to-Skill Rec.
ESCO Occupation-to-Skill ( le Vrang et al., 2014 ; De Lange et al., 2025 )	multi	3,039 (en) $\times$ 13,939 (en)	28
Skill-to-Occupation Rec.
ESCO Skill-to-Occupation ( le Vrang et al., 2014 ; De Lange et al., 2025 )	multi	13,492 (en) $\times$ 3,039 (en)	28
Similar Item Rec.
Job Title Sim. ( Zbib et al., 2022 ; Deniz et al., 2024 )	multi	105 (en) $\times$ 2,619 (en)	11

Table 1. Overview of WorkRB tasks. Dataset sizes are shown for the English (en) number of queries

\times

targets. For non-English datasets, we separately report the maximum number of queries and targets for cross-lingual (x), Bulgarian (bg), and Swedish (sv). ESCO-based target spaces vary by language and version. Table listing the 13 ranking tasks in WorkRB organized by task group, with their label type, number of queries and targets, and number of supported languages.

Model	SRec	ORec	SIRec	CRec	JNorm	SNorm	SExtr	All
#Multilingual datasets	28	28	12	12	72	35	84	271
Random Ranking	0.4	0.6	0.7	8.2	0.3	0.1	0.1	1.5
BM25 ( Robertson and Zaragoza, 2009 )	2.9	6.1	11.0	27.5	2.5	47.4	5.5	14.7
Qwen3-Embed (0.6B) ( Zhang et al., 2025 )	6.0	14.7	26.3	50.4	12.5	65.5	16.9	27.5
JobBERT-v3 ( Decorte et al., 2025a )	9.3	20.5	30.6	43.4	25.6	62.5	16.6	29.8
#EN-only datasets	1	1	2	2	2	1	3	12

Table 2. MAP per task group. Multilingual models are evaluated on all languages with no language aggregation; EN models are evaluated on English monolingual subsets only. Best result for both scenarios in bold .

AI and Labor

AI and labor studies how artificial intelligence changes jobs, wages, skills, hiring, and the organization and quality of work.

See all 93 papers

EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI
Oct 12, 2026Ruijia Yang, Bing Xu, Zulong Chen +5LLM Evaluation AI and Labor
The Hong Kong University of Science and Technology (Guangzhou) · Alibaba Group · Guangzhou University +1
WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks
Sep 20, 2026Yining Hua, Levi LianBenchmark Design AI Agent Benchmarks
Harvard University · Agent Evaluation Science Inc. · Raycaster +1
Design and Report Benchmarks for Knowledge Work
May 22, 2026Yining Hua, Hongbin Na, Cyrus Ayubcha +1Benchmark Design AI Agent Evaluation
Harvard University · University of Technology Sydney · Stanford University +1

Oct 12, 2026J/K move · Enter open · S save

EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI

Ruijia Yang, Bing Xu, Zulong Chen, Zhipeng Xu+4

The Hong Kong University of Science and Technology (Guangzhou) · Alibaba Group · Guangzhou University+1

LLM Evaluation AI and Labor

Open paper AskPDF arXiv

Recruitment applications span diverse operations, from drafting job descriptions and interpreting queries to organizing profiles, assessing fit, and recommending candidates and jobs. Systematic LLM evaluation therefore requires task coverage that reflects this range of business needs. We introduce EHR-Bench, an open Chinese benchmark comprising twelve tasks across six capability families and five recruitment scenarios. The tasks cover generation, extraction, classification, retrieval, semantic similarity, and reciprocal ranking within one evaluation framework. An internal recruitment dataset guides task definitions, the occupation taxonomy and sampling targets, output requirements, and scoring protocols. Following these specifications, we construct 8,336 task instances from public materials and controlled augmentation, with source-aware expert review and documented provenance. We evaluate twelve LLMs through task-level, capability-level, and aggregate comparisons. Closely ranked models exhibit distinct task strengths, and performance varies across input sources, reference evidence, and candidate pools. Five shared models retain the same observed Overall ordering on a private operational reference, with variation in capability and task correspondence. EHR-Bench provides a reproducible foundation for systematic model assessment, task-specific model selection, and further research on LLMs for recruitment. The project repository is available at https://github.com/EhrBench/EHR-Bench.

Abstract

AI and Labor

Explore similar work

EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI

WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks

Design and Report Benchmarks for Knowledge Work