Context-Aware Classification and Grading of Sensitive Information in Online Conversational Health Data
Organizations: School of Computing, Macquarie University, Australia
Abstract
Online medical consultations contain sensitive health information whose privacy implications depend not only on the entities mentioned but also on how those entities are described in context. Existing classification and grading approaches often map health-information entities directly to predefined sensitivity levels, potentially overlooking whether a condition is confirmed, suspected, negated, hypothetical, or merely planned for investigation. In this study, we formulate sensitive-information grading in online medical dialogues as a context-aware evaluation task. We develop a standard-informed operational framework that incorporates assertion status, experiencer, test-result status, and information granularity. We further design a naturalistic evaluation setting together with contrastive cases that minimally alter negation, uncertainty, experiencer, or granularity, and compare large language models under mention-only and full-context conditions. The study aims to quantify the contribution of contextual information to sensitivity grading and to characterize safety-critical over- and under-grading errors. Our framework provides a reproducible basis for evaluating whether LLMs can distinguish sensitive entity mentions from contextually established sensitive disclosures.
Figures & tables
| Health data classification | Definition | Scope |
| Personal attribute data | Information that can identify a specific individual, either on its own or when combined with other information. | Demographic information, personal identifiers, contact information, biometric information, and health-monitoring device identifiers. |
| Health status data | Information that reflects an individual’s health condition or is closely related to it. | Chief complaints, medical history, symptoms and signs, laboratory and examination data, family history, lifestyle information, and genomic or other biological data. |
| Medical application data | Information that reflects the provision and use of medical services, including outpatient care, hospitalization, and discharge. | Medical records, prescriptions, examination and laboratory reports, medication information, surgical and anesthesia records, nursing records, and referral or discharge records. |
| Medical payment data | Information related to expenses incurred for medical or insurance services. | Medical transactions, payment information, insurance status, and insurance-related records. |
| Health resource data | Information that reflects the capabilities and characteristics of health service personnel, health plans, and health systems. | Hospital basic information, healthcare resources, personnel information, and hospital operation data. |
| Public health data | Information related to public health matters affecting populations at the national or regional level. | Environmental health, disease surveillance, epidemic information, disease prevention, and population-level birth and mortality data. |
| Information type | Contextual condition / granularity | Reference grade |
| Special disease | Confirmed, specific individual | L5 |
| Special disease | Suspected | L2 |
| Special disease | Negated | L2 |
| Special disease | Hypothetical / planned | L2 |
| Special disease | Generic / no individual | N/A |
| Sensitive test result | Confirmatory positive | L5 |