LLM Agent Evaluation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

115 papers in the last four weeks, up 140% on the four weeks before. 1.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 793

All topics

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents
Aug 6, 2025Thassilo M. Schiepanski, Nicholas PiëlLLM Agent Evaluation Web Agents
Surfly BV
Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions
Jul 7, 2025Yuanzhe Hu, Yu Wang, Julian McAuleyLLM Agent Memory LLM Agent Evaluation
University of California, San Diego
SheetMind: Actions Set Accuracy, Agents Set the Failure Mode
Jun 14, 2025Lyuhao Chen, Xi Cheng, Yanming Kang +11Multi-Agent LLM Systems LLM Agent Evaluation
Carnegie Mellon University · Cornell University · University of California, Berkeley +2
AgentMisalignment: Measuring the Propensity for Misaligned Behaviour in LLM-Based Agents
Jun 4, 2025Akshat Naik, Emma Gouné, Patrick Quinn +4AI Alignment LLM Agent Evaluation
Independent Researcher · Department of Engineering University of Cambridge
Towards LLM Agents for Earth Observation
Apr 16, 2025Chia Hsiang Kao, Wenting Zhao, Cheryl Lam +10Code Generation LLM Agent Evaluation
Cornell University · Columbia University
How Well Can Modern LLMs Act as Agent Cores in Radiology Environments?
Dec 12, 2024Qiaoyu Zheng, Chaoyi Wu, Weike Zhao +5Healthcare LLM Agent Evaluation
Shanghai Jiao Tong University, Shanghai, China · Department of Radiology, Renmin Hospital of Wuhan University, Wuhan, China
LLM-BabyBench: Can Language Models Plan in Worlds They Can Simulate?
Date pendingIdriss Malek, Omar Choukrani, Daniil Orel +5LLM Planning LLM Agent Evaluation
DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reports
Date pendingRuizhe Li, Mingxuan Du, Benfeng Xu +3Deep Research Agents LLM Agent Evaluation
University of Science and Technology of China · Metastone Technology, Beijing, China
WorkflowPerturb: Calibrated Stress Tests for Evaluating Multi-Agent Workflow Metrics
Date pendingMadhav Kanda, Sharad Agarwal, Rodrigo Fonseca +2LLM Agent Evaluation AI Agent Benchmarks
University of Illinois Urbana-Champaign · Microsoft
GRACE-DS: a Guarded Reward-guided Agent Correction Environment in Data Science
Date pendingAleksandr Tsymbalov, Danis Zaripov, Artem Epifanov +1LLM Agent Evaluation AI Agent Benchmarks
AI Talent Hub, ITMO University, Saint Petersburg, Russia · HSE University
OpenFinGym: A Verifiable Multi-Task Gym Environment for Evaluating Quant Agents
Date pendingKaicheng Zhang, Wen Ge, Lei Jiang +5Quantitative Finance LLM Agent Evaluation
University of Edinburgh · University College London · Alan Turing Institute +1
Harbor Adapters and Harbor-Index: Infrastructure and a Curated Meta-Dataset for Large-Scale Agentic Evaluation
Date pendingLin Shi (Audrey), Haowei Lin (Audrey), Zixuan Zhu (Audrey) +123LLM Agent Evaluation AI Agent Evaluation
Your Agent Says Yes: Interpreting Adversarial Market Behavior Beyond Individual Transactions
Date pendingZelin Li, Yiyun Su, Matt White +3LLM Agent Evaluation AI Agent Security
The Ohio State University · Rutgers University · University of California, Berkeley +3

Aug 6, 2025, cs.AIJ/K move · Enter open · S save

Beyond Pixels: Exploring DOM Downsampling for LLM-Based Web Agents

Thassilo M. Schiepanski, Nicholas Piël

Surfly BV

LLM Agent Evaluation Web Agents

Open paper AskPDF arXiv

The advent of large language models (LLMs) has sparked an evolution of autonomous web browsing agents: given a web browsing task and serialised user interface (UI) state, an LLM is expected to suggest input actions that incrementally solve the given task. The central challenge lies in serialising UI state for LLMs. Web agents have increasingly relied on grounded graphical UI (GUI) snapshots - screenshots augmented with visual cues - favoured for their modest input token footprint. Document object model (DOM) snapshots, serialised as HTML, represent a compelling alternative that leverages previously demonstrated HTML interpretation capabilities of LLMs. Their excessive token footprint, however, has precluded reliable deployment with web agents to date. We propose D2Snap, an algorithm to downsample the DOM, premised on preserving actionability and actionability-discriminating features. We evaluate D2Snap-downsampled DOM snapshots using a snapshot-variant web agent (GPT-4o) on a dataset sampled from Online-Mind2Web. Whilst 42% of raw DOM snapshots exceed the model context window (128 x 10^3 tokens), all D2Snap-downsampled DOM snapshots of our reference configuration fit, at a mean context utilisation of 16.5%. Against the 67% success rate of a grounded GUI snapshot baseline, our configuration attains 73% (+5.8%pt; 95% CI -13.6 to +26.0%pt; McNemar, p = 0.47), excluding a deficit (one-sided 95%) beyond 11%pt. Image input moreover appears to add little to snapshot utility; grounding text alone attains 62% (-5.8%pt; McNemar, p = 0.37).

Momentum

Related topics

Latest papers 793