SpatialTrust: A Benchmark for Environmental Risk Recognition in Secure Authentication

Junbin Lu; Hsiang-Wei Huang; Saesha Wadhwa; Yu Ting Hsu; Jenq-Neng Hwang

cs.CRAug 30, 2026

SpatialTrust: A Benchmark for Environmental Risk Recognition in Secure Authentication

Authors: Junbin Lu, Hsiang-Wei Huang, Saesha Wadhwa, Yu Ting Hsu, Jenq-Neng Hwang

Organizations: University of Washington, United States

Abstract

Visual environmental risk recognition plays an important role in secure authentication, where a user's surroundings may reveal sensitive information or introduce potential security risks. However, existing evaluations of multimodal large language models (MLLMs) rarely examine whether models can reliably recognize, localize, and explain such risks in spatially grounded authentication scenarios. We present SpatialTrust, a question-answering benchmark for evaluating environmental risk recognition in secure authentication. SpatialTrust assesses five complementary abilities: sensitive factor detection, direct factor identification, indirect factor identification, direct factor explanation, and indirect factor explanation. We evaluate both proprietary and open-source MLLMs and find that current models show limited performance, especially in understanding and explaining indirect risks, indicating that spatial risk awareness remains a challenging capability for MLLMs. In addition, we introduce SpatialTrustGuard, a structured QA-and-audit pipeline that improves Qwen3-VL-30B-A3B-Instruct from 36.78% to 41.12% overall. Our findings highlight the need for dedicated benchmarks and structured inference methods to improve the trustworthiness of MLLMs in secure authentication.

Figures & tables

Figure 1: Example SpatialTrust item. Each image is paired with a visible-evidence-only system prompt and five structured questions.

Figure 2: Overview of the SpatialTrust benchmark construction and evaluation pipeline. The benchmark is built through video-frame sampling, image preprocessing, identity-level and situation-level image editing, QA generation, and human review; evaluation compares model predictions against ground truth using answer accuracy and grounded judging.

Figure 3: Examples of situation-level diversification in SpatialTrust. The editing stage produces both images without an added sensitive situation and images with varied authentication-related sensitive-factor configurations.

Statistic	Count
Images	577
Image resolution	$1440\times 960$
QA items	2,885
QA items per image	5
Answer-only QA items	577
Grounded QA items	2,308

Table 1: Dataset statistics of SpatialTrust.

Figure 4: Overview of SpatialTrustGuard. Compared with direct one-pass VLM answering, SpatialTrustGuard decomposes inference into candidate selection, conservative self-verification, and grounded answer generation with locked intermediate choices.

Proprietary Models
Model	Sensitive Factor Detection	Factor Identification		Factor Explanation		Overall
Model	Sensitive Factor Detection	Direct	Indirect	Direct	Indirect	Overall
GPT-5.4	68.80	26.08	5.83	24.44	4.53	25.94
Gemini-2.5-flash	66.38	40.81	12.36	38.91	10.60	33.81
Open-Source Models
GLM-4.1V-9B-Thinking	67.07	14.21	8.64	11.96	6.21	21.62

Table 2: Model performance on SpatialTrust with an IoU threshold of 0.5 for grounded evidence.

Variant	Sensitive Factor Detection	Factor Identification		Factor Explanation		Overall
Variant	Sensitive Factor Detection	Direct	Indirect	Direct	Indirect	Overall
Qwen3-VL-30B-A3B-Instruct	67.42	39.17	28.65	25.74	22.91	36.78
+ Staged QA Decomposition	66.38	38.13	31.57	31.72	27.01	38.96
+ Conservative Self-Verification	65.51	41.07	34.20	35.44	29.38	41.12

Table 3: Ablation study of SpatialTrustGuard with an IoU threshold of 0.5 for grounded evidence.

Multimodal Large Language Models

Multimodal large language models combine language models with image, video, audio, and other inputs for grounded understanding and generation.

See all 1,092 papers

LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation
Sep 27, 2026Jie Wang, Yanbo Sun, Zheng Yan +2LLM Evaluation Trust in AI
State Key Laboratory of Integrated Services Networks, School of Cyber Engineering, Xidian University · Hangzhou Institute of Technology, Xidian University · Department of Computer Science, Purdue University
ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
Sep 29, 2026Ruochen Zhang, Yao Huang, Yitong Sun +5Multimodal Reasoning LLM Safety
Beihang University Beijing, China · Tsinghua University Beijing, China · Institute of Artificial Intelligence Beihang University Beijing, China State Key Laboratory of AI Safety
TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors
Sep 4, 2026Thu-Hien Trinh-Thi, Hai-Yen Vong, Thanh-Ha Ung-Dung +1LLM Safety Evaluation LLM Safety Benchmarks
Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam

Sep 27, 2026J/K move · Enter open · S save

LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation

Jie Wang, Yanbo Sun, Zheng Yan, Jiahe Lan+1

State Key Laboratory of Integrated Services Networks, School of Cyber Engineering, Xidian University · Hangzhou Institute of Technology, Xidian University · Department of Computer Science, Purdue University

LLM Evaluation Trust in AI Prompt Engineering

Open paper AskPDF arXiv

Trust evaluation plays a critical role in cybersecurity by supporting risk mitigation and decision-making. A variety of trust evaluation methods have been proposed, with learning-based approaches offering high accuracy and automation. However, they often require substantial ground truth, suffer from low training efficiency, lack support for basic trust properties, and provide limited explainability. Large Language Models (LLMs) offer a compelling alternative due to their strong zero-/few-shot reasoning abilities and broad knowledge. To this end, we propose LLM4Trust, the first benchmark framework that systematically explores the capabilities of LLMs for trust evaluation. We first construct diverse trust graphs to model five basic trust properties and design corresponding property understanding tasks. We then assess the ability of eight representative LLMs to understand these properties under nine prompt methods. Based on this exploration, we identify the most effective LLM-prompt combinations and apply them to five real-world datasets for validating LLMs' trust evaluation capability. During this process, we propose two strategies to extract key information from large-scale trust graphs, addressing the context window limitations of LLMs. Extensive experiments show that LLMs can effectively understand basic trust properties and have great potential for real-world trust evaluation, particularly under limited supervision. However, they remain vulnerable to attacks targeting trust graphs and demonstration examples used in few-shot prompting, and incur high inference costs. Accordingly, we propose a defense mechanism and batch inference to improve the robustness and efficiency of LLM-based trust evaluation. The source code of LLM4Trust is available at https://github.com/Jieerbobo/LLM4Trust

Abstract

Multimodal Large Language Models

Explore similar work

LLM4Trust: Exploring the Capabilities of Large Language Models for Trust Evaluation

ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models

TIER: Threat Implicitness Benchmark for Evaluating LLM Safety Behaviors