Paper ID: 2410.16325 • Published Oct 18, 2024
This Candidate is [MASK]. Letters of Reference and Job Market Outcomes using LLMs
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
I implement a prompt-based learning strategy to extract measures of sentiment
and other features from confidential reference letters. I show that the
contents of reference letters is clearly reflected in the performance of job
market candidates in the Economics academic job market. In contrast, applying
traditional ``bag-of-words'' approaches produces measures of sentiment that,
while positively correlated to my LLM-based measure, are not predictive of job
market outcomes. Using a random forest, I show that both letter quality and
length are predictive of success in the job market. Letters authored by
advisers appear to be as important as those written by other referees.