Organizations: Department of Computer Science and Technology, Tsinghua University, China · University of Southern California, USA · Shanghai Jiao Tong University, China · State Key Laboratory of Internet Architecture, Tsinghua University, China
Mobile advertising dominates app monetization but introduces risks ranging from intrusive user experience to malware delivery. Existing detection methods rely either on static analysis, which misses runtime behaviors, or on heuristic UI exploration, which struggles with sparse and obfuscated ads. In this paper, we present MANA, the first agentic multimodal reasoning framework for mobile ad detection. MANA integrates static, visual, temporal, and experiential signals into a reasoning-guided navigation strategy that determines not only how to traverse interfaces but also where to focus, enabling efficient and robust exploration. We implement and evaluate MANA on commercial smartphones over 200 apps, achieving state-of-the-art accuracy and efficiency. Compared to baselines, it improves detection accuracy by 30.5%-56.3% and reduces exploration steps by 29.7%-63.3%. Case studies further demonstrate its ability to uncover obfuscated and malicious ads, underscoring its practicality for mobile ad auditing and its potential for broader runtime UI analysis (e.g., permission abuse). Code and dataset are available at https://github.com/MANA-2026/MANA.
Figures & tables
Figure 1. MANA as a multimodal reasoning agent for efficient mobile ad detection, fusing heterogeneous signals to jointly reason about “where and how to go”.
Figure 2. Examples of different ad types: (a) integrated within the app interface as a banner; (b) shown as an intrusive interstitial that blocks normal interaction; (c) implemented as a “More Apps” list without standard SDK patterns, requiring semantic reasoning to recognize as advertising.
Figure 3. Screenshots of two example apps. The middle panels show the view-hierarchy nodes corresponding to the blue-boxed components in the screenshots, while red boxes highlight ad-related elements.
Figure 4. The system overview of MANA.
Figure 5
Figure 7. Cross-rendering consistency scheme, which combines Accessibility Service with visual techniques to produce semantically described actionable components for hierarchy-based and canvas-rendered apps.
Figure 8. Examples of MANA reasoning on a single UI screen. (a) a view-metadata case where hierarchical metadata identifies the “Other apps” button as a promotional redirection; (b) A visual cues case ( e . g ., “FREE COINS”) in a rendering app reveal advertising intent.
Figure 9. Cross-app regularities captured by memory. (a) and (b) both exhibit recurring ad-triggering patterns via the “Hidden Menu → More Apps” option, (c) shows the corresponding memory entry abstracted into a reusable representation.
Figure 10. An illustration of the structured prompt in MANA. The system prompt defines the task and heuristics, while the integrated prompt supplies multi-faceted context for decision-making.
Figure 11. Representative success case of MANA.
Figure 11Figure 12Figure 13
Figure 20. Automatic detection and analysis of malicious advertising by MANA. (a) A custom in-app ad is detected. (b) The interaction triggers a redirection to a suspicious website. (c) The system captures the ensuing browser warning, and (d) submits the URL to VirusTotal, which confirms it as malicious.
Figure 21. Case study on permission abuse.
Figure 22. Representative failure case of MANA.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 23. An example of a hierarchy-based app.
Figure 24. An example of a hierarchy-based app with VLM-annotated UI element.
Online advertising studies how digital ads are targeted, ranked, priced, measured, and delivered while balancing budgets, business goals, and user interests.
The rapid advancement of modern technology has led to a significant increase in the use of smart devices, such as smartphones and tablets, resulting in the widespread adoption of mobile applications. Although applications are required to undergo malware screening before being published on official app stores, many malicious applications successfully evade detection by concealing sophisticated malware variants. These malicious behaviors are often activated only during runtime, making them difficult to identify through conventional static analysis. As a result, malware may remain undetected until after installation, potentially causing irreversible damage to users and their devices. This study presents a real-time Android malware detection framework that analyzes application behavior to accurately identify and classify complex malware. The proposed approach employs a hybrid dynamic analysis technique to distinguish malicious applications from benign ones. After preprocessing and filtering the collected dataset, the applications are classified using multiple machine learning algorithms. A comprehensive performance evaluation is conducted to compare the effectiveness of different classification techniques in terms of detection accuracy and execution time. Experimental results demonstrate that a hybrid model combining Random Forest and a Multilayer Perceptron achieves the best overall performance, attaining an accuracy of 97.5% with an execution time of 22.945 seconds. The proposed framework can enhance mobile device security by enabling timely detection of malicious applications and reducing the risk of cyberattacks.
Md Faisal Ahmed, Zarin Tasnim Biash, Abu Raihan Shakil +4
Department of Computer Science and Engineering Brac University Dhaka, Bangladesh
Mobile graphical user interface (GUI) agents enable AI models to autonomously operate smartphones on behalf of users. However, most existing systems focus primarily on optimizing task accuracy and rely on cloud-hosted models for inference, which introduces privacy concerns and network-dependent latency. As a result, fully on-device deployment of mobile GUI agents remains underexplored. We propose MobileExplorer, a new framework that accelerates on-device inference for vision-based mobile GUI agents via online exploration. The key idea is to exploit the long per-step reasoning time of vision-language models (VLMs) by performing lightweight, parallel exploration of UI elements. During model inference, the agent proactively probes semantically relevant UI elements and records these exploration traces as structured memory. To ensure reliable execution in live mobile environments, we design a two-level rollback mechanism that robustly restores the initial UI state when a fast but naive backtracking strategy fails. The collected exploration traces are then summarized into concise contextual hints and injected into the prompt to enhance the subsequent reasoning step. We evaluate MobileExplorer on multiple off-the-shelf devices using the AndroidWorld benchmark, as well as newly designed, more complex tasks and dynamic on-device environments. MobileExplorer reduces the average number of reasoning steps and end-to-end latency by 23%, while maintaining or improving task success rates by up to 5%. A video demonstration of MobileExplorer performance in the real world is available at https://youtu.be/thK7MJmdlvM .
Runxi Huang, Liyu Zhang, Shengzhong Liu +1
Hong Kong University of Science and Technology · Shanghai Jiao Tong University
The rapid development of GUI foundation models and mobile GUI agents has spurred numerous evaluation benchmarks, yet most rely on simulated environments or open-source applications, leaving real-world closed-source applications largely unevaluated. The core difficulty is that closed-source applications do not expose internal states, making traditional automatic verification inapplicable. To bridge this gap, we introduce AndroidDaily, a large-scale benchmark comprising 350 realistic daily-use tasks across 94 high-frequency Android applications spanning transportation, shopping, local services, entertainment, content creation, social media, and everyday utilities. To enable automatic and verifiable assessment in these opaque environments, we propose Guideline-grounded Reviewer for Automatic Diagnostic Evaluation (GRADE), a process-aware evaluator built on a three-tiered system of observable external guidelines: operational obligations, output quality, and negative constraints. GRADE tracks the agent's visual trajectory against these criteria and produces step-level diagnostic judgments, turning long-horizon, open-ended mobile interactions into verifiable evaluation without relying on hidden internal states. Experiments show that GRADE achieves 87.37% agreement with human evaluators. The strongest model reaches a 62.0% success rate on AndroidDaily, highlighting a substantial gap between current reasoning capabilities and practical execution in realistic mobile workflows.
Yifan Sui, Xin Huang, Hongbing Li +14
1Beijing University of Posts and Telecommunications · 2StepFun · 3Waseda University