LLMs as Validators for Cross-Chain Bridges: Evaluating Source-Level and Transaction-Level Attack Detection on Solana
Organizations: Chittagong University of Engineering and Technology Bangladesh · East Delta University Bangladesh · University of Tennessee at Chattanooga USA · The University of Akron USA
Abstract
Cross-chain bridges have suffered substantial asset losses due to vulnerabilities in their on-chain programs. On Solana, missing validation checks can enable exploits even when the underlying blockchain operates correctly. The ability of large language models (LLMs) to diagnose these flaws from source code or transaction records, rather than merely raise alarms, remains insufficiently understood. We implement a native Rust lock-and-mint bridge on a local Solana validator with four exploitable variants: two model flaws from the Cashio and Wormhole incidents, while the other two represent vulnerability classes documented in Solana security audits. Paired vulnerable and secure programs, together with successful attacks, benign sessions, and blocked attempts, allow us to distinguish correct diagnoses from unsupported alarms. We evaluate six configurations of five open-weight models across 16,800 responses, compare source inputs with and without comments, and grade 7,200 answers on vulnerable inputs using two blinded LLM graders. The results reveal substantial differences across conditions: without comments, models diagnose 24% of vulnerable source inputs while frequently flagging the secure bridge. Transaction diagnosis is high for the missing-signer scenario but averages 6% across the other three scenarios. Original comments, some of which describe the flaw, increase source diagnosis to 60%. These findings highlight the importance of distinguishing correct vulnerability diagnoses from unsupported alarms in bridge security review.
Figures & tables
| Attack | Real-world basis | Function | Check removed | Exploit (attacker) | Secure bridge |
|---|---|---|---|---|---|
| A1 mint confusion | Cashio (2022) | lock | deposited mint equals the approved collateral mint | deposits 500,000 of a worthless self-made token and receives 500,000 real wrapped tokens | WrongMint |
| A2 sysvar spoof | Wormhole (2022) | release | instructions sysvar is the real sysvar address | passes a fake instruction list with a dummy Ed25519 check; wrapped tokens are minted with no relay signature | UnsupportedSysvar |
| B1 missing signer | Signer authorization (Solana audit class) | require_admin | admin account signed (key match kept) | calls set_admin with the admin’s key but no signature and makes itself admin | MissingSignature |
| B2 arbitrary CPI | Arbitrary CPI (Solana audit class) | redeem | token program is the real SPL Token program | passes its own program as the token program; redeeming one wrapped token drains the vault | WrongTokenProgram |
| Input | Answer | Flagged | Diagnosed |
|---|---|---|---|
| Vulnerable | yes, planted bug named | TP | TP |
| Vulnerable | yes, other cause | TP | FN |
| Vulnerable | no | FN | FN |
| Secure or benign | yes | FP | FP |
| Secure or benign | no | TN | TN |
| Source code | Trans. | Total | ||
| No comm. | Orig. comm. | record | ||
| Evaluation Size | ||||
| Responses | 4,800 | 4,800 | 7,200 | 16,800 |
| Graded responses | 2,400 | 2,400 | 2,400 | 7,200 |
| Diagnosis Grading | ||||
| Claude positive | 574 | 1,433 | 579 | – |
| Flagged | Diagnosed | Blocked | |||||
| Model | Recall | F1 | Acc. | Recall | F1 | Acc. | flagged |
| Source code | |||||||
| gpt-oss-120b | 0.79 | 0.65 | 0.58 | 0.55 | 0.50 | 0.46 | – |
| DeepSeek-V3.2 | 1.00 | 0.67 | 0.50 | 0.25 | 0.22 | 0.12 | – |
| Llama-3.3-70B | 1.00 | 0.67 | 0.50 | 0.00 | 0.00 | 0.00 | – |
| Nemotron-3-Super | 0.98 | 0.66 | 0.51 | 0.24 | 0.22 | 0.14 | – |
| Model | No comments | Original comments |
|---|---|---|
| gpt-oss-120b | 218 | 306 |
| DeepSeek-V3.2 | 99 | 202 |
| Llama-3.3-70B | 1 | 188 |
| Nemotron-3-Super | 95 | 214 |
| Gemma-4-31B | 63 | 269 |
| DeepSeek-V3.2 (thinking) | 98 | 254 |
Software Vulnerability Detection
Software vulnerability detection identifies insecure code, pinpoints weaknesses, and helps developers validate or prevent exploitable bugs.
See all 152 papers