Tracing Pharmacological Knowledge In Large Language Models
Authors: Basil Hasan Khwaja, Dylan Chen, Guntas Toor, Anastasiya Kuznetsova
Organizations: School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA · Viterbi School of Engineering, University of Southern California, Los Angeles, CA, USA · Stephen J. R. Smith Faculty of Engineering and Applied Science, Queen’s University, Kingston, ON, Canada · Department of Molecular and Cellular Biology, Scripps Research, La Jolla, CA, USA
Large language models (LLMs) have shown strong empirical performance across pharmacology and drug discovery tasks, yet the internal mechanisms by which they encode pharmacological knowledge remain poorly understood. In this work, we investigate how drug-group semantics are represented and retrieved within Llama-based biomedical language models using causal and probing-based interpretability methods. We apply activation patching to localize where drug-group information is stored across model layers and token positions, and complement this analysis with linear probes trained on token-level and sum-pooled activations. Our results demonstrate that early layers play a key role in encoding drug-group knowledge, with the strongest causal effects arising from intermediate tokens within the drug-group span rather than the final drug-group token. Linear probing further reveals that pharmacological semantics are distributed across tokens and are already present in the embedding space, with token-level probes performing near chance while sum-pooled representations achieve maximal accuracy. Together, these findings suggest that drug-group semantics in LLMs are not localized to single tokens but instead arise from distributed representations. This study provides the first systematic mechanistic analysis of pharmacological knowledge in LLMs, offering insights into how biomedical semantics are encoded in large language models.
Figures & tables
Example prompt for adrenergic alpha-agonists
Paired example prompt for adrenergic alpha-antagonists
Question: Which compound belongs to the class of adrenergic alpha-agonists? A) mh-76 compound B) ketoprofen Answer:
Question: Which compound is known to act as adrenergic alpha-antagonists? A) ketoconazole B) quinidine Answer:
Example prompt for central nervous system depressants
Paired example prompt for central nervous system stimulants
Question: Which compound would be grouped into central nervous system depressants? A) tadalafil B) amitriptyline, perphenazine drug combination Answer:
Question: Which compound falls under central nervous system stimulants? A) viomycin B) 4-fluoroamphetamine Answer:
Table 1: Example paired prompts used to probe semantic representations
Model
Accuracy
BioGPT
0.600
OpenBioLLM-8B
0.920
BioMistral-7B
0.860
Llama-3.1-8B-Instruct
0.900
Gemma3-4B
0.860
Table 2: Accuracy of models evaluated on the drug class–name relationship dataset.
Figure 1: Logit difference distributions for clean (blue) and counterfactual (orange) prompts across Llama-based models.
Figure 2: Activation patching of the Llama-3.1-8B-Instruct on the random prompt
Figure 4: Activation patching of the Llama-3.1-8B-Instruct on the random prompt
Figure 6: Metrics of linear probes trained on activations extracted from Llama-based models. Blue and orange lines indicate train and test metrics for the model trained on individual token activations, while red line shows both train and test performance for the model trained on sum-pooled tokens.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
0.5200 ± 0.0116
0.5318 ± 0.0029
0.5895 ± 0.1949
0.5949 ± 0.1975
0.5556 ± 0.0000
0.5556 ± 0.0000
Appendix
Table 3: Linear probe performance on adrenergic group activations extracted before the first transformer layer of Llama-Instruct and trained on individual tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
Appendix
Table 4: Linear probe performance on adrenergic group activations extracted before the first transformer layer of Llama-Instruct and trained on sum-pooled tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
0.6291 ± 0.0090
0.6382 ± 0.0022
0.6962 ± 0.0984
0.7009 ± 0.1005
0.7333 ± 0.0000
0.7333 ± 0.0000
Appendix
Table 5: Linear probe performance on central nervous system group activations extracted before the first transformer layer of Llama-Instruct and trained on individual tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
Appendix
Table 6: Linear probe performance on central nervous system group activations extracted before the first transformer layer of Llama-Instruct and trained on sum-pooled tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
0.5521 ± 0.0093
0.5907 ± 0.0093
0.5388 ± 0.0582
0.5857 ± 0.0669
0.6111 ± 0.0162
0.6539 ± 0.0156
Appendix
Table 7: Linear probe performance on adrenergic group activations extracted before the first transformer layer of OpenBioLLM and trained on individual tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
Appendix
Table 8: Linear probe performance on adrenergic group activations extracted before the first transformer layer of OpenBioLLM and trained on sum-pooled tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
0.6371 ± 0.0071
0.6679 ± 0.0079
0.6254 ± 0.0035
0.6627 ± 0.0116
0.7416 ± 0.0083
0.7690 ± 0.0096
Appendix
Table 9: Linear probe performance on central nervous system group activations extracted before the first transformer layer of OpenBioLLM and trained on individual tokens
Test Accuracy
Train Accuracy
Test F1 score
Train F1 score
Test ROC-AUC
Train ROC-AUC
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
1.0000 ± 0.0000
Appendix
Table 10: Linear probe performance on central nervous system group activations extracted before the first transformer layer of OpenBioLLM and trained on sum-pooled tokens
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.