Proactive Rejection and Grounded Execution: A Dual-Stage Intent Analysis Paradigm for Safe and Efficient AIoT Smart Homes
Authors: Xinxin Jin, Zhengwei Ni, Zhengguo Sheng, Victor C. M. Leung
Organizations: School of Information and Electronic Engineering (Sussex Artificial Intelligence Institute), Zhejiang Gongshang University, Hangzhou 310018, China. · Sussex Artificial Intelligent Institute, Zhejiang Gongshang University, Hangzhou 310018, China · Department of Engineering and Design, University of Sussex, BN1 9RH, Brighton, U.K. · Artificial Intelligence Research Institute, Shenzhen MSU-BIT University, Shenzhen 518172, China · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen 518060, China · Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada.
As Large Language Models (LLMs) transition from information providers to embodied agents in the Internet of Things (IoT), they face significant challenges regarding reliability and interaction efficiency. Direct execution of LLM-generated commands often leads to entity hallucinations (e.g., trying to control non-existent devices). Meanwhile, existing iterative frameworks (e.g., SAGE) suffer from the Interaction Frequency Dilemma, oscillating between reckless execution and excessive user questioning. To address these issues, we propose a Dual-Stage Intent-Aware (DS-IA) Framework. This framework separates high-level user intent understanding from low-level physical execution. Specifically, Stage 1 serves as a semantic firewall to filter out invalid instructions and resolve vague commands by checking the current state of the home. Stage 2 then employs a deterministic cascade verifier-a strict, step-by-step rule checker that verifies the room, device, and capability in sequence-to ensure the action is actually physically possible before execution. Extensive experiments on the HomeBench and SAGE benchmarks demonstrate that DS-IA achieves an Exact Match (EM) rate of 58.56% (outperforming baselines by over 28%) and improves the rejection rate of invalid instructions to 87.04%. Evaluations on the SAGE benchmark further reveal that DS-IA resolves the Interaction Frequency Dilemma by balancing proactive querying with state-based inference. Specifically, it boosts the Autonomous Success Rate (resolving tasks without unnecessary user intervention) from 42.86% to 71.43%, while maintaining high precision in identifying irreducible ambiguities that truly necessitate human clarification. These results underscore the framework's ability to minimize user disturbance through accurate environmental grounding.
Figures & tables
Fig. 1: Failure mechanisms in existing reactive frameworks. (a) Task Omission: The agent loses global context when trapped in multi-turn interactions. (b) Forced Hallucination: The agent forcibly maps an unexecutable command to an incorrect entity due to the lack of a proactive rejection mechanism.
Framework Type
Representative Methods
Reasoning Paradigm
Ambiguity Strategy
Physical Safety
Main Limitation
Slot Filling
BERT-based
Supervised / Template
None
High (Template-bound)
Poor generalization to unseen commands
Direct Generation
GPT-3.5/4
End-to-End
Model Probability
Low (Prone to Hallucination)
Severe entity hallucinations; no feedback
Reactive Agents
SAGE [ 3 ] , ReAct [ 16 ]
Iterative (Think-Act Loop)
Passive (On Error)
Medium
Interaction Frequency Dilemma ; Disturbing
Uncertainty Planning
Ren et al. [ 4 ]
Probabilistic Alignment
Confidence Threshold
Medium
Linguistic confidence = Physical truth
DS-IA (Ours)
This Work
Analyze-then-Act (Dual-Stage)
Active State-Awareness
Very High (Cascade Check)
Dependent on real-time snapshot freshness
TABLE I: Comparison of Smart Home Control Frameworks. DS-IA achieves the best balance between physical safety and interaction efficiency.
Fig. 2: Workflow of the Dual-Stage Intent-Aware (DS-IA) Framework. This image shows the modular architecture: Stage 1 proactively analyzes intent, and Stage 2 utilizes a cascade verifier shield to ensure physical feasibility.
Task Category
Instruction Example
Targeted Metric & Capability
VS (Valid Single)
"Turn on the lamp."
EM & F1: Evaluates normal instruction-following recall.
VM (Valid Multi)
"Turn on lamp, lock door."
IS (Invalid Single)
"Turn on the oven."
Rejection Rate: Evaluates safety and hallucination suppression.
IM (Invalid Multi)
"Turn on oven and heater."
MM (Mixed Multi)
"Turn on lamp and oven."
EM & F1: Evaluates filter ability.
TABLE II: Taxonomy of HomeBench Tasks and Metric Mapping. (Assuming St does NOT contain an Oven)
Method
VS (Valid Single)
IS (Invalid Single)
VM (Valid Multi)
IM (Invalid Multi)
MM (Mixed Multi)
Overall
EM
F1
EM
F1
EM
F1
EM
F1
EM
F1
EM
F1
Baseline
66.96%
66.89%
14.07%
14.08%
37.55%
69.18%
0.00%
5.82%
0.49%
33.23%
29.98%
35.72%
SAGE
48.63%
49.18%
29.84%
29.84%
9.09%
47.42%
0.00%
0.00%
1.77%
29.80%
1.77%
30.84%
DS-IA (Ours)
50.40%
50.37%
87.04%
87.16%
50.00%
76.98%
38.46%
73.08%
24.49%
77.42%
58.56%
74.90%
TABLE III: Performance Comparison on HomeBench (EM: Exact Match, F1: F1-Score). Bold indicates best performance.
Fig. 3: Visual comparison of handling a non-existent device instruction on HomeBench. SAGE hallucinates a target (Forced Grounding), while DS-IA correctly rejects the command (Early Rejection).
Method
Safety
Stage 1 (Intent Analysis)
Stage 2 (Code Generation)
IS EM (Rejection)
Calls
Tokens
Calls
Tokens
No IA Baseline
83.29%
0
0
1,000
2,982,070
DS-IA (Ours)
88.69%
1,000
3,262,848
819
2,554,231
TABLE IV: Ablation on Safety and Efficiency (1,000 Tasks). DS-IA boosts physical safety (Rejection Rate) while trading cheap input tokens to massively reduce expensive autoregressive code generation cycles.
Task Category
SAGE (Baseline)
DS-IA (Ours)
Global Judgment Metrics
Clarification Succ. Rate (Ambiguous)
75.00%
75.00%
Autonomous Succ. Rate (Clear)
42.86%
71.43%
Detailed Task Succ. Rate Breakdown
Intent Resolution
33.33%
70.83%
Device Resolution
46.15%
69.44%
TABLE V: Task Success Rate (Succ. Rate) Breakdown on SAGE Benchmark. Success is achieved by either safe autonomous execution or correct proactive clarification.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Fig. 4: The complete constraint-aware prompt templates utilized in the DS-IA framework. (a) The Stage 1 prompt acts as a semantic firewall to extract diagnostic reasoning. (b) The Stage 2 prompt generates the final execution sequence, integrating strict negative constraints ( e.g., utilizing error_input for unexecutable actions) to enforce physical grounding.
Smart homes are evolving toward complex state-dependent living environments, requiring Large Language Models (LLMs) to reason over user intent, preferences, and multi-device interactions. However, existing smart-home benchmarks often focus on static instruction-to-API mapping or limited simulations, failing to evaluate whether LLMs can reason, interact, and act reliably in realistic household scenarios. To address these limitations, we introduce SMH-Bench, a comprehensive benchmark for evaluating LLMs in smart-home environments. Built upon HomeEnv, an executable and verifiable smart-home simulator, SMH-Bench contains 1,100 high-quality tasks spanning 7 categories and 22 fine-grained subcategories. It further stratifies tasks across simple, medium and complex homes, ranging from small apartments to dense multi-room environments with 135 devices. Experiments show that although frontier LLMs achieve strong performance on explicit control and query tasks, they still exhibit significant weaknesses in automation task scheduling, ambiguity handling and personalized reasoning, especially as home complexity increases. We hope SMH-Bench will facilitate the development of more reliable, context-aware, and practically deployable smart-home agents.
Kuan Li, Shuo Zhang, Huacan Wang +12
Midea Group · Beijing University of Posts and Telecommunications · Donghua University +2
Large language model agents are moving beyond text-only interaction toward physical-world control, with smart homes as a representative domain. Real domestic interaction requires understanding ambiguous intents, operating in dynamic environments, and performing multi-turn reasoning. However, existing methods struggle to generate high-quality training data for smart home agents. We propose HomeFlow, a verifiable data flywheel for this domain. HomeFlow uses HomeEnv as a unified simulation environment and HomeMaker to procedurally generate diverse home settings. Subsequently, Blueprint compiles open-ended user intents into executable state-based success conditions, while MCTS-Flow synthesizes diverse, verifiable multi-turn trajectories through environment-guided tree search. We then optimize the agents via supervised fine-tuning and step-wise RLVE, which facilitates iterative improvement through authentic physical feedback. We further construct SmartHome-Bench to evaluate the agent across various smart home tasks. On this benchmark, HomeFlow-RL-4B and HomeFlow-RL-8B achieve task success rates of 84.60% and 87.03%. It is worth noting that HomeFlow-RL-8B even surpasses the leading GPT-5.5 by 1.23 percentage points.
Yi Gu, Huacan Wang, Shuo Zhang +10
1Midea Group · 2Beijing University of Posts and Telecommunications · 3Donghua University +1
Recent advancements in Large Language Models (LLMs) have empowered home assistants with natural language interaction capabilities. However, current assistants overlook the progressive omission that occurs in human dialogue as shared context accumulates, leading to more elliptical expressions for efficient communication. Thus, current assistants still struggle to interpret such elliptical expressions accurately, which limits their effectiveness in real-world applications. In practical smart home scenarios, assistants face two major challenges caused by elliptical commands: (1) referential ambiguity caused by different environmental expectations among multiple users; and (2) intention ambiguity resulting from user preferences that evolve over time or change with the environment. To address these challenges, we introduce PEC-Home, the first simulated home dataset specifically designed for interpreting progressively elliptical commands in smart homes. Extensive experiments on various LLMs, including GPT-4o, show that existing home assistants struggle to execute user-intended operations based solely on elliptical commands. Even when equipped with tools for storing and retrieving user dialogue history, execution accuracy remains below that achieved with complete commands.}.
Yingyu Shan, Zeming Liu, Silin Li +4
Beijing Institute of Technology · Beihang University · Baidu Inc.