How Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call Traces
Organizations: Shanghai Artificial Intelligence Laboratory · University of Electronic Science and Technology of China · The University of Sydney
Abstract
Large language model (LLM)-based agents interact with external tools through protocols such as the Model Context Protocol (MCP), producing tool-call traces that may expose behavioral signals about users. In this work, we systematically study the risk of user-profile inference from MCP tool-call traces, where observers infer user attributes from protocol-valid tool interactions without access to user conversations. We introduce PrivacyTrace, a benchmark containing 1,000 synthetic user profiles and 4,000 agent trajectories across four domains, generated with real MCP tools from 25 servers. Experiments with four mainstream LLM attackers show that tool-call traces enable a 26.25% attack success rate for profile inference, compared with 17.97% from prior-based guessing. Further analysis reveals that tool arguments and returned results provide the main trace-grounded evidence, while linking traces across sessions further amplifies profiling risk. These findings demonstrate that MCP tool-call traces can act as effective behavioral side channels for user profiling, motivating the development of privacy-aware mechanisms for agent-tool interactions. Code and data are available at https://github.com/AI45Lab/PrivacyTrace.
Figures & tables
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Stage | Model | Prompt / Interface | |
|---|---|---|---|
| Profile Construction | |||
| Seed audit | GLM-5.2 | build_seed_audit_prompt | |
| PII completion | DeepSeek-V4-Pro | build_completion_prompt | |
| Government-ID audit | GLM-5.2 | build_government_id_audit_prompt | |
| Trajectory Generation | |||
| Scenario planner | DeepSeek-V4-Flash | profile_conditioned_task_planner | |
| Attribute | Target | Deterministic rule / semantic fallback |
|---|---|---|
| age | Integer age in years | Exact integer or interval containing truth; no LLM fallback. |
| sex | Male or female | Canonical aliases; fallback accepts equivalent category, not demographic associations. |
| ethnicity | East Asian, SE Asian, South Asian, MENA, Black, White, Latino, Indigenous, Pacific Islander, Mixed | Broad-category aliases; fallback permits more specific ethnicity that entails the target. |
| citizenship | Country of citizenship | Country/demonym aliases; fallback requires citizenship, not residence. |
| current_location | Current city and country | City plus canonical country; fallback permits equivalent names and unambiguous omission, not a region alone. |
| birth_location | Birth city and country | Same location rule applied to birthplace. |
| Family | Fitted margins | Primary aggregate source | Construction |
|---|---|---|---|
| Population | Country; age sex; country age sex | UN DESA World Population Prospects 2024 ( United Nations, Department of Economic and Social Affairs, Population Division, 2024 ) | Restrict the 2025 medium-variant population to ages 18–80 and normalize over the 26-country frame. |
| Urban location | Country current-city size | UN DESA World Urbanization Prospects 2025, File 21 ( United Nations, Department of Economic and Social Affairs, Population Division, 2025 ) | Group 8,422 cities with at least 50,000 residents into seven size bands; sample cities within each band by population. |
| Migration | Country citizenship; country birth country; country sex migration status | UN DESA International Migrant Stock 2020 ( United Nations, Department of Economic and Social Affairs, Population Division, 2020 ) | Use destination–origin tables directly when their basis matches the field; otherwise use the documented birth-country or citizenship proxy. |
| Education | Country education | World Bank WDI educational-attainment indicators, sourced from UNESCO UIS ( World Bank, 2026b ) | Difference cumulative attainment rates into eight mutually exclusive education levels. |
| Ethnicity | Country ethnicity | UNdata Table 26, national censuses, and UN DESA migrant-stock aggregates ( United Nations Statistics Division, n.d.a ; United Nations, Department of Economic and Social Affairs, Population Division, 2020 ) | Map compatible census categories to the project taxonomy; use broad migrant-origin regions when no comparable census table exists. |
| Religion | Country religion | Pew Religious Composition by Country, 2020 ( Hackett et al., 2025 ) | Map country estimates to seven religious categories. |
| Attacker | ASR | Input | Output | Price in/out | Cost | Failed |
| (%) | (M) | (M) | ($/M) | ($) | views | |
| MiniMax-M3 | 25.29 | 190.29 | 5.51 | 0.30 / 1.20 | 63.70 | 12 |
| DeepSeek-V4-Flash | 26.25 | 195.05 | 5.42 | 0.22 / 0.66 | 46.49 | 11 |
| GPT-5.5 | 27.78 | 151.97 | 10.67 | 5.00 / 30.00 | 1,079.88 | 87 |
| Gemini-3-Flash | 28.32 | 220.88 | 14.90 | 0.50 / 3.00 | 155.13 | 19 |
| Slot category | Count | Share of all slots |
| Correct under both attacks | 13,163 | 19.36% |
| Correct only under one-shot | 4,689 | 6.90% |
| Correct only under two-stage | 4,398 | 6.47% |
| Wrong under both attacks | 45,750 | 67.28% |