Empirical Study

Follow this field Ask about this field

Recent momentum

emerging

0 papers in the last 28 days · 0.0% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this field, kept on the site without email delivery.

Period ending 2026-09-21

15 new papers

A weekly snapshot of new work published in Empirical Study.

Period ending 2026-09-14

12 new papers

A weekly snapshot of new work published in Empirical Study.

Period ending 2026-09-07

11 new papers

A weekly snapshot of new work published in Empirical Study.

Inside this field

Focused directions

Focused topic

Empirical Study

Within Empirical Study

184 papersExplore

Focused topic

Systematic Study

Within Empirical Study

69 papersExplore

Focused topic

Multi-Model Study

Within Empirical Study

52 papersExplore

303 papers

Latest in Empirical Study

Browse all topics →

Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents
May 28, 2026Alejandra Zambrano, Sara Vera Marjanovic, Imene Kerboua +2Large Language Model Planning Web Agents
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
May 28, 2026Vahideh ZolfaghariDeception Multi-Model Study
Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Study
May 27, 2026Irune Zubiaga, Aitor Soroa, Rodrigo AgerriLarge Language Model Evaluation Llm-As-A-Judge
Do We Really Need Quantum Machine Learning?: A Multidimensional Empirical Study
May 27, 2026Sudip Vhaduri, Ryan Gammon, Sayanton DibboQuantum Machine Learning Quantum Kernels
Ground Truths in Suicide Research: The Current State of AI-Based Suicide Detection in Social Media
May 26, 2026Yaakov Ophir, Ofri Hefetz, Refael Tikochinski +6Suicide Risk Social Media
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Muon Optimizer Adversarial Training
A Systematic Study of Behavioral Cloning for Scientific Data Annotation
May 26, 2026Ishaan Singh Chandok, Core Francisco ParkBehavior Cloning Annotations
Memory Architectures for Multi-Turn Text-to-SQL: A Benchmark and Empirical Study
May 25, 2026Ravi Kumar Tummalapenta, Suman AddankiText-To-Sql Large Language Model Memory
Explaining Too Much? Understanding How Large Language Model Reasoning Traces Influence Performance and Metacognition
May 25, 2026Daniela Fernandes, Daniel Buschek, Lev Tankelevitch +2Reasoning Traces Large Language Model Reasoning
Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation
May 24, 2026Yangneng Chen, Jing LiLarge Vision Language Models Mitigation
An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
May 22, 2026Mohammed Kharma, Ahmed Sabbah, Mohammad Alkhanafseh +2Code Generation Large Language Model Safety
Human-AI Collaboration in Science at Scale: A Global Large-scale Randomized Field Experiment
May 22, 2026Binglu Wang, Weixin Liang, Jiahui Xue +4Scientific Discovery Multi-Model Study
Empirical Analysis and Detection of Hallucinations in LLM-Generated Bug Report Summaries
May 22, 2026Hinduja Nirujan, Shreyas Patil, Abdallah Ayoub +2Bug Hallucinations
From Raw Experience to Skill Consumption: A Systematic Study of Model-Generated Agent Skills
May 22, 2026Zisu Huang, Jingwen Xu, Yifan Yang +13Reusable Agent Skills Experience
Philosophical Dispositions as Behavioral Constraints for AI-Assisted Code Review: An Empirical Study
May 21, 2026Kaushal BansalCode Quality Artificial Intelligence Reviewers
LLM Code Smells: A Taxonomy and Detection Approach
May 21, 2026Zacharie Chenail-Larcher, Brahim Mahmoudi, Naouel Moha +2Code Quality LLM Inference Efficiency
More Context, Larger Models, or Moral Knowledge? A Systematic Study of Schwartz Value Detection in Political Texts
May 21, 2026Víctor Yeste, Paolo RossoSentence-Level Classification Moral Reasoning
Relational Linear Properties in Language Models: An Empirical Investigation
May 21, 2026Giovanni Valer, Luigi Gresele, Marco Bronzini +1Graph Language Models Empirical Study
Stdlib or Third-Party? Empirical Performance and Correctness of LLM-Assisted Zero-Dependency Python Libraries
May 20, 2026Peng Ding, Rick StevensStandards Correctness
Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing
May 20, 2026Georg Jank, Mattia Piccinini, Sebastian Wenk +3Autonomous Racing Steering
Quadratic Characterizations for Reachability Analysis of Neural Networks
May 19, 2026Elias Khalife, Mazen Farhood, Pierre-Loic GarocheNeural Network Verification Reachability
Towards Multi-Model LLM Schedulers: Empirical Insights into Offloading and Preemption
May 19, 2026Mert Yildiz, Pietro Spadaccino, Alexey Rolich +2Optimal Scheduling Offloading
KAN-MLP-Mixer: A comprehensive investigation of the usage of Kolmogorov-Arnold Networks (KANs) for improving IMU-based Human Activity Recognition
May 18, 2026Mengxi Liu, Sizhen Bian, Vitor Fortes +5Human Activity Recognition Kolmogorov-Arnold Networks
A Large-Scale Study on the Accuracy vs Cost Trade-offs of Training and Evaluation Settings in Fine-Grained Image Recognition
May 18, 2026Edwin Arkel Rios, Augusto Christian Surya, Oswin Gosal +5Fine-Grained Perception Counterfactual Learning
An Empirical Study of Privacy Leakage Chains via Prompt Injection in Black-Box Chatbot Environments
May 18, 2026Hongjang Yang, Hyunsik Na, Daeseon ChoiIndirect Prompt Injection Language Model Evasion Attacks
Factorized Latent Dynamics for Video JEPA: An Empirical Study of Auxiliary Objectives
May 16, 2026Santosh PremiVideo Pretraining Joint-Embedding Predictive Architectures
Empirical evaluation of Time Series Foundation Models for Day-ahead and Imbalance Electricity Price Forecasting in Belgium
May 16, 2026Chi Bui, Maria Margarida Mascarenhas, Arnaud Verstraeten +1Electricity Price Forecasting Time Series Foundation Models
Multi-Paradigm Agent Interaction in Practice:A Systematic Analysis of Generator-Evaluator, ReAct Loop,and Adversarial Evaluation in the buddyMe Framework
May 16, 2026Xiaohua Wang, Chao Han, Kai Yu +2Multi-Agent Systems Adversarial Evaluation
Transformer Scalability Crisis: The First Comprehensive Empirical Analysis of Performance Walls in Modern Language Models
May 14, 2026Mahdi Naser Moghadasi, Faezeh GhaderiTransformer Architectures Large Models
Can Visual Mamba Improve AI-Generated Image Detection? An In-Depth Investigation
May 14, 2026Mamadou Keita, Wassim Hamidouche, Hessen Bougueffa Eutamene +3Ai-Generated Image Detection Generative Adversarial Network
LLM-Based Robustness Testing of Microservice Applications: An Empirical Study
May 13, 2026Hrushitha Goud Tigulla, Marco VieiraMicroservices Empirical Study
What Does LLM Refinement Actually Improve? A Systematic Study on Document-Level Literary Translation
May 13, 2026Shaomu Tan, Dawei Zhu, Ke Tran +5Machine Translation Refinement
What properties of reasoning supervision are associated with improved downstream model quality?
May 13, 2026Mikołaj Langner, Dzmitry Pihulski, Jan Eliasz +5Table Reasoning Datasets Downstream Fine-Tuning
Code-Centric Detection of Vulnerability-Fixing Commits: A Unified Benchmark and Empirical Study
May 13, 2026Nils Loose, Joseph Bienhüls, Kristoffer Hempel +2Commits Code Quality
When Do LLMs Generate Realistic Social Networks? A Multi-Dimensional Study of Culture, Language, Scale, and Method
May 13, 2026Sai Hemanth Kilaru, Sriram Theerdh Manikyala, Raghav Upadhyay +3Large Language Model Bias Graph Homophily
Pretraining Strategies and Scaling for ECG Foundation Models: A Systematic Study
May 12, 2026M A Al-Masud, Nils StrodthoffElectroencephalography Foundation Models Electronic Health Record Foundation Models
Three Regimes of Context-Parametric Conflict: A Predictive Framework and Empirical Validation
May 12, 2026Pruthvinath Jeripity VenkataConflict Empirical Study
An Empirical Study of Automating Agent Evaluation
May 12, 2026Kang Zhou, Sangmin Woo, Haibo Ding +14Agentic Evaluations Coding Agents
Attractor-Vascular Coupling Theory: Formal Grounding and Empirical Validation for AAMI-Standard Cuffless Blood Pressure Estimation from Smartphone Photoplethysmography
May 11, 2026Timothy Oladunni, Farouk Ganiyu AdewumiCuffless Blood Pressure Estimation Remote Photoplethysmography
Threat Modelling using Domain-Adapted Language Models: Empirical Evaluation and Insights
May 11, 2026Saba Pourhanifeh, AbdulAziz AbdulGhaffar, Ashraf MatrawyThreat Models Large Language Model Adaptation
Thinking with Novel Views: A Systematic Analysis of Generative-Augmented Spatial Intelligence
May 11, 2026Yanbing Zhang, Bo Wang, Jianhui Liu +9Spatial Reasoning Novel View Synthesis
Towards Generalist Game Players: An Investigation of Foundation Models in the Game Multiverse
May 11, 2026Kuan Zhang, Dongchen Liu, Qiyue Zhao +12Multimodal Agents Artificial Intelligence Systems
Security Risks in Tool-Enabled AI Agents: A Systematic Analysis of Privileged Execution Environments
May 10, 2026Hardik GoelArtificial Intelligence Safety Artificial Intelligence Agents
AI-Automation Tooling in Computer Engineering Education: Mixed-Methods TAM/UTAUT Evidence for a General Acceptance Attitude
May 10, 2026Aung PyaeComputer Science Artificial Intelligence Literacy
LLM hallucinations in the wild: Large-scale evidence from non-existent citations
May 8, 2026Zhenyue Zhao, Yihe Wang, Toby Stuart +3Large Language Model Hallucination Citations
Prompt Engineering Strategies for LLM-based Qualitative Coding of Psychological Safety in Software Engineering Communities: A Controlled Empirical Study
May 8, 2026Moaath Alshaikh, Tasneem Alshaher, Ricardo Vieira +7Qualitative Research Software Engineering
Gradient-Based LoRA Rank Allocation Under GRPO: An Empirical Study
May 8, 2026Yash Ganpat SawantGradient-Free Rank Allocator Group Relative Policy Optimization
A Systematic Investigation of RL-Jailbreaking in LLMs
May 7, 2026Montaser Mohammedalamen, Kevin Roice, Reginald McLean +1Large Language Model Jailbreaks Jailbreak Evaluations
How Well Do LLMs Perform on the Simplest Long-Chain Reasoning Tasks: An Empirical Study on the Equivalence Class Problem
May 7, 2026Chun Zheng, Lianlong Wu, Bingqian Li +2Large Language Model Reasoning Large Reasoning Models
Examining Human-Like Behaviors in LLMs: A Multi-Dimensional Analysis of Model Behaviors, User Factors, and System Prompts
May 7, 2026Sunnie S. Y. Kim, Margit Bowler, Leon A GatysEmpirical Study Objectives
Empirical Evidence for Simply Connected Decision Regions in Image Classifiers
May 7, 2026Arjhun Swaminathan, Mete AkgünImage Classification Topological Deep Learning
Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions
May 7, 2026Chengjie Wang, Jingzheng Wu, Xiang Ling +2Code Quality Large Language Model Safety
Does Synthetic Data Help? Empirical Evidence from Deep Learning Time Series Forecasters
May 7, 2026Hugo Cazaux, Eyjólfur Ingi Ásgeirsson, Hlynur StefánssonSynthetic Time Series Synthetic Data
An Empirical Study of Proactive Coding Assistants in Real-World Software Development
May 7, 2026Lehui Li, Ruixuan Jia, Guo-Ye Yang +1Code Generation Software Engineering
How Far Are VLMs from Privacy Awareness in the Physical World? An Empirical Study
May 6, 2026Junran Wang, Xinjie Shen, Zehao Jin +1Real-World Environments AI Privacy Risks and Protection
The Predictive-Causal Gap: An Impossibility Theorem and Large-Scale Neural Evidence
May 6, 2026Kejun LiuCausal Representation Learning Reward-Prediction Error
Empirical Study of Pop and Jazz Mix Ratios for Genre-Adaptive Chord Generation
May 6, 2026Jinju LeeMix Best-Epoch Metrics
Are LLMs Ready for Conflict Monitoring? Empirical Evidence from West Africa
May 5, 2026Hoffmann Muki, Olukunle OwolabiConflict Humanitarian Reports
Exposing LLM Safety Gaps Through Mathematical Encoding:New Attacks and Systematic Analysis
May 5, 2026Haoyu Zhang, Mohammad Zandsalimy, Shanu SushmitaLarge Language Model Safety Large Language Model Performance
Quantitative Sobolev Approximation Bounds for Neural Operators with Empirical Validation on Burgers Equation
May 4, 2026Nicole HaoLogarithmic Sobolev Inequalities Fourier Neural Operators

May 28, 2026 · cs.CLJ/K move · Enter open · S save

Does The Way You Plan Matter? An Empirical Study of Planning Representations for LLM Web Agents

Alejandra Zambrano, Sara Vera Marjanovic, Imene Kerboua, Xing Han Lù+1

Concordia University · Mila - Quebec AI Institute · University of Copenhagen · Universite Claude Bernard Lyon · McGill University

Despite recent advances, LLM-based web agents still struggle with limited exploration, omission of critical steps, and sensitivity to task constraints. Prior work suggests that many of these failures stem from weaknesses in planning, yet the impact of alternative natural language plan representation remains unexplored. To address this, we introduce PlanAhead, a static planner-executor framework that evaluates the impact of plan representation in agent performance. We first automatically categorize WebArena tasks into 3 difficulty levels, enabling consistent difficulty grading without human annotation. Then we systematically evaluate 4 different plan representations on the tasks categorized as hard: sequential subgoals, narrative, pseudocode, and checklist; across different families of multimodal LLM powered agents (OpenAI, Alibaba, and Google). To account for stochastic variability, we introduce two novel evaluation metrics: Achievement Rate (AR) and Solved-Task Consistency (STC). Our results show that both, the plan formulation and the underlying LLM generating the plan, significantly influence web-agent robustness and task success.

Large Language Model Planning Web Agents Planning Methods Empirical Study

Open paper Ask about it PDF