Measuring Hallucination in Large Language Models for Cyber Threat Intelligence: An Exploratory Study
Organizations: Korea University, 145 Anam-ro, Seongbuk-gu, Seoul, 02841, Republic of Korea · KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea · Kwangwoon University, 20 Kwangwoon-ro, Nowon-gu, Seoul, 01897, Republic of Korea · Chungbuk National University, 1 Chungdae-ro, Seowon-gu, Cheongju, Chungcheongbuk-do, 28644, Republic of Korea
Abstract
Large language models (LLMs) are increasingly used in cyber threat intelligence (CTI) tasks, such as summarization and question answering. However, they may produce hallucinations, false or misleading content not grounded in the source text, which can mislead analysts and corrupt downstream security decisions. This paper presents an exploratory measurement study of hallucinations in LLM-based CTI. We introduce HalluVision, a reproducible framework that generates LLM outputs, extracts domain-specific entities using a fine-tuned named entity recognition model, and evaluates factual consistency using multiple similarity metrics. Using 4,940 real-world cybersecurity articles, we measure hallucinations in outputs from two commercial LLMs and one open-weight LLM. Measurements are validated against 100 human-annotated outputs and two established faithfulness detectors. Results show that semantic similarity aligns with human judgments, whereas entity-based metrics exhibit limited agreement. Differences between models are concentrated in question answering, while entity extraction quality limits what entity-level metrics can reveal.
Figures & tables
| Type | Summarization Prompt | Q&A Prompt |
| Role | NER expert in the CTI domain | NER expert in the CTI domain |
| Instructions | - Summarize blog content in 50–70 words. - Use only words and content from the original text. | - Answer five predefined entity-based questions. - Use short answers (ideally one entity). - Output “no answer” if info is missing. |
| Input | {original_text} | {original_text} |
| Output | Single summary paragraph | 5 question–answer pairs |
| Source | Content |
| Original Article | 300 restaurants and at least 50,000 payment cards compromised by two separate campaigns against MenuDrive, Harbortouch and InTouchPOS services. Magecart campaigns have been skimming payment-card credentials of unsuspecting customers using three online restaurant-ordering systems, affecting about 300 restaurants that use the services and compromising tens of thousands of cards so far, researchers have found. Two separate ongoing Magecart campaigns have injected e-skimmer scripts into the online ordering portals of restaurants using three separate platforms: MenuDrive , Harbortouch, and InTouchPOS, researchers from Recorded Future revealed in a blog post this week. (…) “This campaign likely began no later than Jan. 18, 2022, and as of this report, a portion of the restaurants remained infected,” they noted in the post. However, the malicious domain used for the campaign, which researchers identified as authorizen[.]net , has been blocked since May 26, they said. |
| Abstractive Summarization | Two Magecart campaigns have compromised over 50,000 payment cards from approximately 300 restaurants using MenuDrive , Harbortouch, and InTouchPOS services. Researchers from Recorded Future reported that e-skimmer scripts were injected into these platforms, exposing customers’ payment card data and personal information. The campaigns began in late 2021 and early 2022, with ongoing infections and malicious domains still active. Smaller restaurant platforms remain attractive targets for cybercriminals due to their vulnerabilities. |
| Question & Answering | Q1. What is the main Malware?: Magecart Q2. What is the main System?: MenuDrive Q3. What is the main Indicator?: authorizen[.]net Q4. What is the main Vulnerability?: no answer Q5. What is the main Organization?: Recorded Future |
| Entity Type | Description |
| Malware | The name of the primary malware discussed in the article. If no specific name is provided, it may refer to a general malware category. |
| System | Operating systems (e.g., Linux, Windows), software (e.g., Chrome, Slack), or hardware leveraged by threat actors in attacks. |
| Indicator | Patterns used to detect malicious activity, such as domain names, URLs, IP addresses, filenames, hashes, email addresses, and port numbers. |
| Vulnerability | Common Vulnerabilities and Exposures (CVE) identifiers (e.g., CVE-2024-9264) and exploit mentions (e.g., command injection vulnerabilities) explicitly stated in the article. |
| Organization | Entities directly affected by cyber threats discussed in the article. Organizations that merely observed or reported the threats are excluded. |
| Metric | Preprocessing Required | Applicable To | Interpretation of High Score | Interpretation of Low Score |
| Cosine Similarity | Sentence Embedding | AS | Semantically consistent with the source | Semantic deviation or abstraction |
| Answer Matching Similarity – Precision | NER | AS, Q&A | Relevant entities correctly included | Irrelevant or spurious entities included |
| Answer Matching Similarity – Recall | NER | AS, Q&A | Most reference entities are retrieved | Key entities are missing |
| Frequency-based Entity Similarity | NER | AS | Similar entity-type distribution | Topic drift or entity imbalance |
| # of Articles | |||
| Task | Source / Model | Threatpost | The Hacker News |
| Article Scraping | — | 4,940 | 500 |
| NER Fine-tuning Data | MITRE | 50 | |
| Abstractive Summarization | gpt-4o-mini | 4,940 | 500 |
| gpt-3.5-turbo | 4,940 | 500 | |
| Llama-3-8B-Instruct | 500 | — | |
| Task | Model | Metric | Malware | System | Indicator | Vulnerability | Organization |
| Abstractive Summarization | gpt-4o-mini | Precision | 0.7401 | 0.7187 | 0.4977 | 0.1775 | 0.7641 |
| Recall | 0.2241 | 0.1924 | 0.0315 | 0.0717 | 0.1786 | ||
| gpt-3.5-turbo | Precision | 0.6979 | 0.6800 | 0.4000 | 0.1923 | 0.7377 | |
| Recall | 0.2199 | 0.1885 | 0.0291 | 0.0714 | 0.1637 | ||
| Q&A | gpt-4o-mini | Precision | 0.6442 | 0.5789 | 0.0288 | 0.1338 | 0.7475 |
| Recall | 0.1263 | 0.1120 | 0.0146 | 0.0885 | 0.1452 |
| Model | Cosine Similarity | Answer Matching Similarity | |
| Precision | Recall | ||
| Llama-3-8B-Instruct | 0.8659 | 0.7126 | 0.1547 |
| gpt-3.5-turbo | 0.8847 | 0.6917 | 0.1766 |
| gpt-4o-mini | 0.8911 | 0.7219 | 0.1831 |
| Label | Llama-3-8B- Instruct | gpt-3.5-turbo | gpt-4o-mini |
| Malware | 0.3002 | 0.3489 | 0.4527 |
| System | 0.2617 | 0.2353 | 0.3642 |
| Indicator | 0.0142 | 0.0101 | 0.0101 |
| Vulnerability | 0.0183 | 0.0304 | 0.0503 |
| Organization | 0.6227 | 0.6491 | 0.7022 |
| Overall Precision | 0.3936 | 0.3073 | 0.5221 |
| Task | Model | Metric | Malware | System | Indicator | Vulnerability | Organization |
| Abstractive Summarization | gpt-4o-mini | Precision | 0.6549 | 0.6939 | 0.4595 | 0.2032 | 0.8158 |
| Recall | 0.1760 | 0.1537 | 0.0405 | 0.0777 | 0.2322 | ||
| gpt-3.5-turbo | Precision | 0.6851 | 0.7167 | 0.4571 | 0.2434 | 0.8058 | |
| Recall | 0.1812 | 0.1630 | 0.0381 | 0.0701 | 0.1974 | ||
| Q&A | gpt-4o-mini | Precision | 0.5214 | 0.5619 | 0.0430 | 0.1840 | 0.7723 |
| Recall | 0.1001 | 0.1048 | 0.0191 | 0.0813 | 0.1662 |
| Doc | CS | AMS | FES | Human Rating |
| #11 | 79.3 | 33.3 | 25.0 | H H H |
| #2078 | 82.1 | 33.3 | 78.6 | H P H |
| #3916 | 95.2 | 100.0 | 75.0 | F F F |
| Baselines | HalluVision | |||||
| Doc | FActScore | QAGS | Cos. sim. | AM prec. | Human Rating | Main reason given |
| #4042 | 1.000 | 0.546 | 0.923 | 0.000 | P P H | Unsupported elaboration |
| #3313 | 1.000 | 0.709 | 0.929 | 0.250 | F P P | Wrong entity type |
| #4087 | 1.000 | 0.660 | 0.940 | 1.000 | P P P | Unsupported elaboration |
| NER Errors | |||||||
| Label | Precision | Recall | Boundary | Label | Not Entity | Missed | |
| Malware | 56 | 0.626 | 0.976 | 9 | 5 | 7 | 5 |
| System | 50 | 0.733 | 0.929 | 6 | 3 | 4 | 15 |
| Indicator | 12 | 0.968 | 0.771 | 0 | 2 | 0 | 18 |
| Vulnerability | 3 † | 0.000 | 0.000 | 2 | 1 | 0 | 22 |
| Organization | 83 | 0.760 | 0.969 | 7 | 5 | 8 | 9 |
| Task | Model | Metric | Malware | System | Indicator | Organization |
| Correction factor (Summarization: of CyNER) | 0.611 | 0.681 | 0.746 | 0.736 | ||
| Correction factor (Q&A: of CyNER) | 0.976 | 0.929 | 0.771 | 0.969 | ||
| Abstractive Summarization | gpt-4o-mini | Precision | 0.740 1.000 | 0.719 1.000 | 0.498 0.667 | 0.764 1.000 |
| Recall | 0.224 0.367 | 0.192 0.283 | 0.032 0.042 | 0.179 0.243 | ||
| gpt-3.5-turbo | Precision | 0.698 1.000 | 0.680 0.999 | 0.400 0.536 | 0.738 1.000 | |
| Recall | 0.220 0.360 | 0.189 0.277 | 0.029 0.039 | 0.164 0.222 | ||
| Original Article | […] Wikileaks’ Web site has been the target of denial of service attacks , which began even before the group published the leaked diplomatic cables as part of a campaign it dubbed “Cablegate.” While the source of those attacks isn’t clear, a hacker using the handle th3j35t3r claimed responsibility for it, citing the need to retaliate for the leak. More recently, Wikileaks has enlisted the services of hundreds of mirror sites to host the cablegate documents in a coordinated effort to keep them from being driven from the Internet. […] |
| Summarized Text ( hallucinated ) | Paypal permanently restricted Wikileaks’ account due to policy violation, sparking online feud with hacker-activists. Anonymous planned DDoS attack on Paypal in response. Wikileaks faced ongoing denial of service attacks, with mirror sites hosting leaked documents. PayPal and Amazon denied services to Wikileaks, as U.S. Attorney General authorized actions against the organization. |
| Reason | The summary implies that the DDoS attacks were associated with or originated through the mirror sites. However, the original text clearly states that mirror sites were a defensive measure in response to DDoS attacks. |
| Original Article | Hallucinated Entity | Ground-Truth ( CyNER ) | LLM-generated Answer ( gpt-3.5-turbo ) | Reason |
| And while it’s in vogue to refer to REvil and DarkSide as “franchise models” which supply Ransomware-as-a-Service, it is important to remember that the franchisees are effectively freelance cybercriminals. | REvil | Malware | Organization | REvil is a MALWARE in this article, not an ORGANIZATION. |
| Now researchers from Blackberry have connected the dots, painting a picture of a malware that has evolved five times in twelve months. | Blackberry | Organization | System | Blackberry is correctly labeled as an Organization, not a System, as it acts as a sighting group. |
| Log4j , ransomware, cloud vulnerabilities, phishing: Cyber threats are manifold. They all pale, however, in comparison to the security black holes that walk around on two legs. | Log4j | Not Identified | Malware | Log4j is a Java-based library and should be classified as a System, not Malware. |
| APT29, aka Cozy Bear, has used Tor and a technique called domain fronting in order to secure backdoor access to targets for nearly two years running. | Tor | System | Indicator | Tor is a System as a browser, not an INDICATOR. |
| Task | Metric | (4o-mini 3.5-turbo), 95% confidence interval | |
| QA | AM-prec | 4,843 | +0.2228 [0.2151, 0.2307] |
| AS | AM-prec | 4,940 | +0.0310 [0.0210, 0.0400] |
| AS | Cossim | 4,939 | +0.0070 [0.0063, 0.0077] |
| AS | Freq-sim | 4,490 | +0.0157 [0.0100, 0.0212] |
| Work | Scope | CTI Entity Extraction | Hallucination Analysis | Cybersecurity-specific |
| Liao et al. [ 14 ] | IoC extraction from cybersecurity articles | ● | ○ | ● |
| TTPDrill [ 6 ] | CTI ontology extraction from unstructured text | ● | ○ | ● |
| LLM-TIKG [ 5 ] | LLM-based CTI knowledge graph construction from unstructured text | ● | ○ | ● |
| GapFinder [ 9 ] | Information-gap analysis in cybersecurity articles | ● | ○ | ● |
| Jiang et al. [ 8 ] | Information-gap analysis in cybersecurity databases | ● | ○ | ● |
| Vulcan [ 10 ] | CTI entity extraction from unstructured text | ● | ○ | ● |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Parameter | Value | Applies to |
| Model snapshot | gpt-3.5-turbo-0125 | gpt-3.5-turbo |
| gpt-4o-mini-2024-07-18 | gpt-4o-mini | |
| Meta-Llama-3-8B-Instruct | Llama-3-8B-Instruct | |
| Temperature | 0.0 | All models |
| Top- | 1.0 | All models |
| Seed | 42 | OpenAI models |
LLM Hallucination Detection
Large language model hallucination detection identifies unsupported, incorrect, or fabricated claims in model outputs and traces.
See all 217 papers