Psychology

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

2 new papers

A weekly snapshot of new work published in Psychology.

Inside this field

Focused directions

Focused topic

Psychology

Within Psychology

59 papersExplore

Focused topic

Psychometrics

Within Psychology

45 papersExplore

Focused topic

Item Response Theory

Within Psychology

36 papersExplore

128 papers

Latest in Psychology

Browse all topics →

Extending Item Response Theory for Efficient and Meaningful Multilingual Evaluation
Jun 14, 2026Gili Lior, Tzviel Frostig, Gabriel Stanovsky +1Multilingual Benchmark Item Response Theory
LLM Judges Have Dark Current: A Psychometric Datasheet for LLM-as-a-Judge Evaluation
Jun 14, 2026Hiroyasu Usami, Keisuke Hara, Ayato Tsuboi +1Llm-As-A-Judge Large Language Model Judges
A Two-Stage Statistical Framework for Evaluating Associative Interference in Large Language Models
Jun 12, 2026Achraf Cohen, Andrew KincaidLarge Language Model Bias Item Response Theory
Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior
Jun 10, 2026Rafal Kocielnik, Pengrui Han, Peiyang Song +5Personality Psychometrics
Frozen Multimodal Embeddings for AI-Assisted Interview Assessment of Personality and Cognitive Ability
Jun 10, 2026Kuo-En Hung, Hung-Yue Suen, Shih-Ching Yeh +1Interviews Modality-Specific Encoders
SSR: Can Simulated Patients Learn to Stigmatize Themselves? Modeling Self-Stigma through Internal Monologue
Jun 6, 2026Kunyao Lan, Bingrui Jin, Zichen Zhu +1Simulated Patients Empathy
The Masked Advantage: Uncovering Local-Language Access to Cultural Knowledge in LLMs
Jun 5, 2026Yang Zhang, Xiao Fei, Amr Mohamed +6Multilingual Large Language Models Cultural Knowledge
Anchored, Not Graded: Vision-Language Models Fail at Slant-from-Texture Perception
Jun 4, 2026Qian Zhang, Michal Golovanevsky, Fulvio Domini +1Residual Anchoring Failure Mechanism
The Unsampled Truth: Quantifying Prompt Artifacts in LM Psychometrics
Jun 2, 2026Nils Schwager, Christoph Hau, Simon Münker +1Psychometrics Psychology
PsychoPass: Geometric Profiling of Multi-Turn Adversarial LLM Conversations
Jun 2, 2026Muberra Ozmen, Subhabrata MajumdarLanguage Model Evasion Attacks Jailbreak Attacks
A Multiscale Network with Supervised Contrastive Learning for Real-Time Facial Emotion Recognition
May 31, 2026Rejoy Chakraborty, Archisman Adhikary, Chayan Halder +3Facial Expression Recognition Particular Emotion-Aware Response
GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing
May 30, 2026Ming Wang, Shuang Wu, Bixuan Wang +7Psychometrics Psychology
Item Response Scaling Laws: A Measurement Theory Approach for Efficient and Generalizable Neural Scaling Estimation
May 29, 2026Sang Truong, Yuheng Tu, Rylan Schaeffer +1Item Response Theory Scaling Laws
Auditing LLM Benchmarks with Item Response Theory
May 28, 2026Sander Land, Daniel M. BikelItem Response Theory Theoretical Foundations
Supervised Semantic Differential for Cross-Cultural Concept Analysis: A Case Study of Human Affect
May 27, 2026Jan Sikora, Paweł Lenartowicz, Hubert PlisieckiCross-Cultural Study Affective
Efficient Safety Benchmarking via Item Response Theory
May 26, 2026Fabio Spagliardi, Mírian Silva, Ayan Datta +3Safety Evaluation Item Response Theory
Psychological Constructs in Shared Semantic Space
May 26, 2026Hubert PlisieckiPsychology Psychometrics
Overview of the PsyDefDetect Shared Task at BioNLP 2026: Detecting Levels of Psychological Defense Mechanisms in Supportive Conversations
May 24, 2026Hongbin Na, Zimu Wang, Zhaoming Chen +8Emotional Support Psychology
CUNY at CLPsych 2026: A Pipeline Approach to Classification and Summarization of Mental Health Changes
May 22, 2026Amirmohammad Ziaei Bideh, Shameed Charlomar Job, Ava Yahyapour +1Mental Health Psychology
DreamerNLplus: Interpretable Modeling of Mental Health Dynamics from Social Media Timelines using Hybrid Rule-Based and RAG Methods
May 21, 2026Maryia Zhyrko, Daisy Monika Lal, Erik van Mulligen +1Mental Health Psychology
Estimating Item Difficulty with Large Language Models as Experts
May 18, 2026Diana Kolesnikova, Kirill Fedyanin, Abe D. Hofman +2Item Response Theory Mean Opinion Scores
Why Do Safety Guardrails Degrade Across Languages?
May 16, 2026Max Zhang, Ameen Patel, Sang T. Truong +1Multilingual Safety Large Language Model Safety
Can We Trust AI-Inferred User States. A Psychometric Framework for Validating the Reliability of Users States Classification by LLMs in Operational Environments
May 15, 2026Izabella Krzeminska, Michal Butkiewicz, Ewa KomkowskaLarge Language Model Reliability Psychometrics
MindGap: A Conversational AI Framework for Upstream Neuroplastic Intervention in Post-Traumatic Stress Disorder
May 14, 2026Eranga Bandara, Ross Gore, Asanga Gunaratna +16Psychology Synaptic Plasticity
A Formative Study of Brief Affective Text as a Complement to Wearable Sensing for Longitudinal Student Health Monitoring
May 14, 2026Tamunotonye Harry, Johanna Hidalgo, Matthew Price +7Remote Physiological Measurement Physiological Data
Revealing the Gap in Human and VLM Scene Perception through Counterfactual Semantic Saliency
May 13, 2026Ziqi Wen, Parsa Madinei, Miguel P. EcksteinSemantic Gap Human
Assessing and Mitigating Miscalibration in LLM-Based Social Science Measurement
May 12, 2026Jinyuan Wang, Ningyuan Deng, Yi YangLarge Language Model Uncertainty Computational Social Science
The Scaling Law of Evaluation Failure: Why Simple Averaging Collapses Under Data Sparsity and Item Difficulty Gaps, and How Item Response Theory Recovers Ground Truth Across Domains
May 11, 2026Jung Min KangArtificial Intelligence Benchmarks Item Response Theory
Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence
May 10, 2026Alex Bogdan, Adrian de Valois-FranklinPsychometrics Psychology
Empirical Bayes 1-bit matrix completion
May 10, 2026Takeru MatsudaTensor Completion Bayesian
Reinforcement Learning Measurement Model
May 10, 2026Wenqian Xu, Feng JiPsychometrics
Nürnberg NLP at PsyDefDetect: Multi-Axis Voter Ensembles for Psychological Defence Mechanism Classification
May 8, 2026Philipp Steigerwald, Eric Rudolph, Jens AlbrechtPsychology Natural Language Processing Tasks
From 0-Order Selection to 2-Order Judgment: Combinatorial Hardening Exposes Compositional Failures in Frontier LLMs
May 8, 2026Hanmeng Liu, Shichao Weng, Xiulai Liu +3Item Response Theory Table Reasoning Datasets
An Interpretable and Scalable Framework for Evaluating Large Language Models
May 7, 2026Xinhao Qu, Qiang Heng, Hao Zeng +1Large Language Model Evaluation Item Response Theory
Drawing Lines in Psychological Space: What K-means Clustering Reveals in Simulated and Real Psychometric Data
May 7, 2026Pedro Henrique Ramos Pinto, Maria Jullyanna Ferreira Marques, Luiz Carlos Serramo LopezK-Means Psychometrics
The Pinocchio Dimension: Phenomenality of Experience as the Primary Axis of LLM Psychometric Differences
May 6, 2026Hubert Plisiecki, Sabina Siudaj, Kacper Dudzic +4Psychometrics Experience
Measuring Psychological States Through Semantic Projection: A Theory-Driven Approach to Language-Based Assessment
May 6, 2026Maria Luongo, Davide Marocco, Nicola MilanoPsychology
Ask Before You Diagnose: Safe-Psych, a Sequential Evaluation Benchmark for LLMs in Psychiatry
May 6, 2026Oriana Presacan, Andreea Grama, Larisa Irimină +6Psychiatric Reasoning Diagnostic Benchmark
Interpretable Difficulty-Aware Knowledge Tracing in Tutor-Student Dialogues
May 1, 2026Shuyan Huang, Alexander Scarlatos, Jaewook Lee +1Per-Competency Bayesian Knowledge Tracing Tutors
Psychologically Potent, Computationally Invisible: LLMs Generate Social-Comparison-Eliciting Posts They Fail to Detect
May 1, 2026Hua Zhao, Jiapei Gu, Michelle Mingyue GuHuman-Written Text Machine-Generated Text Detection
Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory
Apr 30, 2026Longwei Cong, Sonja Hahn, Sebastian Gombert +3Grading Item Response Theory
Beyond the Mean: Within-Model Reliable Change Detection for LLM Evaluation
Apr 30, 2026Jon-Paul CacioliLarge Language Model Evaluation Change
Uncovering Latent Patterns in Social Media Usage and Mental Health: A Clustering-Based Approach Using Unsupervised Machine Learning
Apr 27, 2026Md All Shahria, Sanjeda Dewan Mithila, Touhid Alam +2Mental Health Image Clustering
PhySE: A Psychological Framework for Real-Time AR-LLM Social Engineering Attacks
Apr 25, 2026Tianlong Yu, Yang Yang, Ziyi Zhou +5Language Model Evasion Attacks Psychology
Verbal Confidence Saturation in 3-9B Open-Weight Instruction-Tuned LLMs: A Pre-Registered Psychometric Validity Screen
Apr 24, 2026Jon-Paul CacioliConfidence Calibration Verbalization
An LLM-Native Psychometric Instrument Reveals a Self-Report--Behavior Gap Across 25 Models
Apr 24, 2026Juan Manuel ContrerasPsychometrics Latent Structure
FairTree: Subgroup Fairness Auditing of Machine Learning Models with Bias-Variance Decomposition
Apr 21, 2026Rudolf DebelakFairness Audits Subgroup Information
Has Automated Essay Scoring Reached Sufficient Accuracy? Deriving Achievable QWK Ceilings from Classical Test Theory
Apr 21, 2026Masaki UtoAutomated Essay Scoring Item Response Theory
AlphaContext: An Evolutionary Tree-based Psychometric Context Generator for Creativity Assessment
Apr 20, 2026Yixuan Wang, Yue Huang, Hong Qian +7Creativity Computerized Adaptive Testing
Before You Interpret the Profile: Validity Scaling for LLM Metacognitive Self-Report
Apr 20, 2026Jon-Paul CacioliPsychometrics Self-Assessments
LLMs can persuade only psychologically susceptible humans on societal issues, via trust in AI and emotional appeals, amid logical fallacies
Apr 18, 2026Alexis Carrillo, Salvatore Citraro, Ali Aghazhadeh Ardebili +5Persuasion Opinion Dynamics
CogEvolution: A Human-like Generative Educational Agent to Simulate Student's Cognitive Evolution
Apr 16, 2026Wei Zhang, Yihang Cheng, Zhirong Ye +1Self-Evolving Agents Tutors
Measuring the metacognition of AI
Mar 31, 2026Richard Servajean, Philippe ServajeanMetacognition Large Language Model Uncertainty
From Stateless to Situated: Building a Psychological World for LLM-Based Agents
Mar 26, 2026Boning Zhao, Yutong Hu, Xinnuo LiEmotional Support Situation
BRIDGE: Predicting Human Task Completion Time From Model Performance
Feb 6, 2026Fengyuan Liu, Jay Gala, Nilaksh +3Artificial Intelligence Benchmarks Psychometrics
Psychometric Comparability of LLM-Based Digital Twins
Dec 22, 2025Yufei Zhang, Zhihao MaPsychometrics Digital Twins
Latency-Response Theory Model: Evaluating Large Language Models via Response Accuracy and Chain-of-Thought Length
Dec 7, 2025Zhiyu Xu, Jia Liu, Yixin Wang +1Item Response Theory Large Language Model Evaluation
Adaptive Testing for LLM Evaluation: A Psychometric Alternative to Static Benchmarks
Oct 26, 2025Peiyu Li, Xiuxiu Tang, Si Chen +4Item Response Theory Large Language Model Evaluation
Psychological Steering in LLMs: An Evaluation of Effectiveness and Trustworthiness
Oct 6, 2025Amin Banayeeanzade, Ala N. Tak, Fatemeh Bahrani +5Personality Psychology
Human Psychometric Questionnaires Mischaracterize LLM Behavior
Sep 12, 2025Woojung Song, Dongmin Choi, Yoonah Park +3Psychometrics Persona

Jun 14, 2026 · cs.CLJ/K move · Enter open · S save

Extending Item Response Theory for Efficient and Meaningful Multilingual Evaluation

Gili Lior, Tzviel Frostig, Gabriel Stanovsky, Matan Eyal

Google Research · The Hebrew University of Jerusalem · PhaseV Trials

Multilingual benchmarks are central to evaluating large language models (LLMs) across languages, but they suffer from three issues: exhaustive evaluation scales linearly with the number of languages, automatic translation introduces errors that are easily missed at scale, and some items conflate general and culture-specific knowledge. We address all three with a unified statistical framework, Multilingual-IRT, which extends Item Response Theory with per-language difficulty deviations, split discriminability separating content from language effects, and per-language ability residuals. Fitting Multilingual-IRT on 25 LLMs across 29 languages of MMLU-Pro-X, we show that its fitted parameters support three practical applications: predicting unobserved (item, LLM, language) instances with 11-16% lower binary cross-entropy than the strongest accuracy-based baseline, surfacing candidate translation errors distributed across all 28 non-English languages, whereas accuracy-based baselines concentrate detections in a few languages, and recovering culture-specific items that accuracy-based baselines miss.

Multilingual Benchmark Item Response Theory

Open paper Ask about it PDF