Organizations: National Taiwan University, Taipei, Taiwan · National Yang Ming Chiao Tung University, Hsinchu, Taiwan · National Dong Hwa University, Hualien, Taiwan
Web applications rely heavily on hyperlinks to connect disparate information resources. However, the dynamic nature of the web leads to link rot, where targets become unavailable, and more insidiously, semantic drift, where a valid HTTP 200 connection exists, but the target content no longer aligns with the source context. Traditional verification tools, which primarily function as crash oracles by checking HTTP status codes, often fail to detect semantic inconsistencies, thereby compromising web integrity and user experience. While Large Language Models (LLMs) offer semantic understanding, they suffer from high latency, privacy concerns, and prohibitive costs for large-scale regression testing. In this paper, we propose SemLink, a novel automated test oracle for semantic hyperlink verification. SemLink leverages a Siamese Neural Network architecture powered by a pre-trained Sentence-BERT (SBERT) backbone to compute the semantic coherence between a hyperlink's source context (anchor text, surrounding DOM elements, and visual features) and its target page content. To train and evaluate our model, we introduce the Hyperlink-Webpage Positive Pairs (HWPPs) dataset, a rigorously constructed corpus of over 60,000 semantic pairs. Our evaluation demonstrates that SemLink achieves a Recall of 96.00%, comparable to state-of-the-art LLMs (GPT-5.2), while operating approximately 47.5 times faster and requiring significantly fewer computational resources. This work bridges the gap between traditional syntactic checkers and expensive generative AI, offering a robust and efficient solution for automated web quality assurance.
Figures & tables
Fig. 1: The automated data collection pipeline for the HWPPs dataset. The system crawls seed URLs, extracts hyperlink context ( Hinfo ) and target page content ( Pinfo ), filters for accessibility, and stores valid pairs.
Fig. 2: The SemLink Feature Extraction Pipeline. The system handles both text-based and image-based links, augmenting them with contextual “Side-Text” derived from the DOM tree structure.
Fig. 3: The SemLink Siamese Network Architecture. Two inputs are processed by identical SBERT backbones and embedding layer with shared weights. Then, semantic difference vector is passed through MLP to predict semantic similarity.
Model/Tool
Accuracy
Precision
Recall
F1-Score
Generative Oracles (LLMs)
Llama-3-8B
86.98%
80.70%
97.20%
88.18%
Llama-3-70B
95.37%
95.44%
95.30%
95.37%
GPT-3.5 Turbo
94.55%
99.37%
89.65%
94.27%
GPT-4o
98.40%
99.95%
96.85%
98.37%
GPT-5.2
99.10%
99.90%
98.30%
99.09%
TABLE I: Performance Comparison with Baselines on Test Set
Feature Configuration
Accuracy
Precision
Recall
F1-Score
Anchor Text Only
92.03%
94.45%
89.30%
91.80%
Anchor + Side-Text
92.65%
90.31%
95.56%
92.86%
Anchor + Image OCR
92.28%
94.11%
90.20%
92.11%
Full Context (SemLink)
92.70%
90.06%
96.00%
92.93%
TABLE II: Ablation Study: Impact of Feature Extraction Strategies
Method
F1-Score
No Image Text (Baseline)
84.07%
Attributes Only ( alt / title )
89.86%
OCR Only
86.46%
Attributes + OCR
90.60%
Attributes + OCR + LabelDroid
88.81%
TABLE III: Ablation: Image Text Extraction Methods (Image Links Only)
Model
Recall
Speed (links/sec)
Speedup
Llama-3-8B
97.20%
0.87
8.7x
Llama-3-70B
95.30%
0.10
1x
GPT-3.5 (API)
89.65%
1.27
12.7x
GPT-4o (API)
96.85%
1.17
11.7x
GPT-5.2 (API)
98.30%
0.65
6.5x
SemLink
96.00%
30.87
308x
TABLE IV: Efficiency Comparison: Throughput vs. Recall
Fig. 4: Efficiency vs. Performance. SemLink (Green Star) is the sole approach within the real-time threshold ( <0.1 s).
Fig. 5: A “Soft Link Rot” example. The server returns HTTP 200, but the content is an error message. Traditional tools mark this as Valid; SemLink correctly marks it as Irrelevant.
Fig. 6: An example of Semantic Drift. The anchor text promises specific news content, but the target page has drifted to a general landing page. SemLink identifies this as Irrelevant due to low semantic overlap.
Fig. 7: Visualizing the Side-Text Heuristic. SemLink extracts the parent header “Campus Life” to disambiguate the generic “Read More” anchor text, correctly validating the link.
Fig. 8: A False Negative case. The link redirects to a login portal. SemLink classifies this as “Irrelevant” due to lack of semantic overlap, though it is functional behavior.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Parameter
Search Space
Optimal
Triplet Margin ( α )
{0.5,1.0,2.0}
1.0
Triplet Weight ( λ1 )
{0,0.5,1.0}
0
BCE Weight ( λ2 )
{0.5,1.0,2.0}
1.0
Dense Layers
{1,2,3}
2
Appendix
TABLE V: Hyperparameter Search Space and Optimal Values