From Rules to Neural Graphs: Scalable Structured Prediction for Patent Prior Art Search

Nikolai Zenovkin; Sebastian Björkqvist

cs.IROct 1, 2026

From Rules to Neural Graphs: Scalable Structured Prediction for Patent Prior Art Search

Authors: Nikolai Zenovkin, Sebastian Björkqvist

Organizations: IPRally Technologies Oy, Helsinki, Finland

Abstract

Patent search requires processing documents routinely exceeding tens of thousands of tokens. Most neural retrieval approaches operate on truncated inputs, limiting their effectiveness. Graph-based retrieval addresses this by representing each patent as a structured invention graph, but constructing these graphs relies on brittle rule-based parsers. We present the neural parser, which adapts biaffine attention from dependency parsing to predict invention graphs directly from patent text. Our local biaffine attention restricts pairwise scoring to a sliding window, reducing complexity from $O(n^2)$ to $O(n \cdot w)$ . Since local and global scoring share the same weights, the model trains on short sequences and deploys on documents exceeding 40,000 tokens without retraining. Distilled from 1 million rule-parsed documents, it surpasses its teacher at 3 $\times$ lower inference cost: neural graphs improve citation recall by 0.5% on short queries and 1.1% on full documents in a downstream Graph Transformer retrieval system.

Figures & tables

Figure 1: End-to-end patent prior art search pipeline. Patent documents and user queries are parsed into invention graphs, encoded by a Graph Transformer [ 5 ] into dense vectors, and matched via nearest neighbor search. This work replaces the rule-based parser with a neural alternative (highlighted).

Figure 2: A patent claim (US20250197034A1) and the corresponding invention graph. Blue bullets denote feature nodes; purple bullets denote functional relations. Indentation reflects hierarchical (part-of) edges. Features referenced inside relations are shown in uppercase.

Panel of Figure 3 — Figure 3: Model architecture. The Feature Head predicts parent structure and edge labels (syntactic vs. semantic); the Relation Head predicts functional relationships.

Figure 4: Global vs. local biaffine score matrices. Global attention computes all $n^{2}$ scores; local attention computes only $n(w{+}2)$ scores within a sliding window plus two pseudo-node columns. Both modes share the same weight matrix $\mathbf{W}$ , enabling training with global attention on short sequences and inference with local attention on long sequences.

Parameter	Value
MLP dimensions (parent / label / relation)	256 / 128 / 256
Max subword tokens per word	3
Optimizer	AdamW (lr $=10^{-4}$ )
LR ratio (encoder vs. task layers)	1:10
Warmup	10% of total steps, then linear decay
Batch size (tokens per worker) $\times$ accumulation	1,200 $\times$ 16

Table 1: Training hyperparameters. Architecture and regularization details in Sections 3.2 – 3.3 .

Model	IoU	IoU@90	Precision	Recall	Coverage
Rule-based (teacher)	1.000	1.000	1.000	1.000	0.847
Neural	0.653	0.142	0.761	0.780	0.861

Table 2: Intrinsic graph quality on the test set (5,000 documents). The rule-based teacher achieves IoU=1.0 by definition since it generates the training labels.

Configuration	Parser	All	Chemical	Non-Chemical
Claims	Rule-based	0.4163	0.3790	0.4218
Claims	Neural	0.4183 ∗	0.3799	0.4240 ∗
Desc.	Rule-based	0.3336	0.3060	0.3377
Desc.	Neural	0.3373 ∗∗	0.3095	0.3414 ∗∗

Table 3: Search recall (Recall@3). Claims : queries re-parsed, corpus rule-based. Descriptions : both re-parsed on a corpus subset. Significance via McNemar’s paired test (one-sided):

{}^{*}p<0.05

{}^{**}p<0.01

Metric	Rule-based	Neural
Graph Depth	2.157	2.057
Word Coverage	0.847	0.862
Feature Coverage	0.514	0.521
Relation Coverage	0.369	0.376

Table 4: Graph structure comparison between rule-based and neural parsers.

Figure 5: Rule-based vs. neural parser output for a chemical patent claim (WO2025012182A1). The rule-based parser extracts only 3 features, while the neural parser identifies 13 features and 5 relations capturing element compositions. This domain—chemical compositions with numerical ranges—is particularly challenging for hand-crafted rules.

Configuration	IoU	P	R	Cov.
100K dataset	0.614	0.727	0.753	0.797
1M dataset	0.651	0.766	0.772	0.846
+ short-doc filter, loss tuning	0.653	0.761	0.780	0.861
+ root penalization	0.637	0.748	0.768	0.866

Table 5: Ablation of training configuration. All models use 100 training epochs; metrics reported at best checkpoint (max IoU). Coverage: fraction of document words assigned to graph features.

Window Size	Recall@3	Doc. Range
512	0.4658	—
1024	0.4676	$>$ 15K words
2048	0.4690	5K–15K words
Global	0.4714	$<$ 5K words

Table 6: Effect of attention window size on search recall (Recall@3), measured using all-claims queries. In production, an adaptive strategy selects the largest window fitting NVIDIA L4 memory (24 GB): global for

<

5K words, 2048 for 5K–15K, 1024 for

>

15K.

Explore similar work

Citation-Driven Multi-View Training for Patent Embeddings: QaECTER and Sophia-Bench
Apr 24, 2026Younes Djemmal, You Zuo, Kim Gerdes +1Patent Text Embeddings
ALMAnaCH · 1AI Lab, Questel, Paris, France · 3Inria, Paris, France +3
Heterogeneous Dependency Graph-Guided Attentionfor Patent Representation Learning
May 11, 2026Yongmin Yoo, Qiongkai Xu, Zhangkai Wu +1Patent Interleaved Graph Attention
Frontier AI Research Centre, Macquarie University
Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering
May 22, 2026Amirhossein Yousefiramandi, Ciaran CooneyText Embeddings Clustering
Clarivate, Intellectual Property Barcelona, Spain 08025

Apr 24, 2026, cs.IRJ/K move · Enter open · S save

Citation-Driven Multi-View Training for Patent Embeddings: QaECTER and Sophia-Bench

Younes Djemmal, You Zuo, Kim Gerdes, Kirian Guiller

ALMAnaCH · 1AI Lab, Questel, Paris, France · 3Inria, Paris, France+3

Patent Text Embeddings Copyright Defenses Multi-View

Open paper AskPDF arXiv

Patent retrieval underpins critical decisions in innovation, examination, and IP strategy, yet progress has been hampered by the absence of benchmarks that reflect the diversity of real world search scenarios. We address this gap with two contributions. First, we introduce Sophiabench, a large-scale patent retrieval benchmark comprising 10,000 queries and 75,000 corpus documents stratified across ten years, eight IPC technology sections, and twelve filing jurisdictions. Unlike prior benchmarks, Sophia-bench tests retrieval using 12 different query types-from structured patent fields to AI-generated summaries-and evaluates results against citation-based ground truth enhanced with a novel domain-relevance metric (InScope). Together, these enable systematic measurement of how well models perform across query types, technology domains, and jurisdictions. Second, we introduce QaECTER, a 344M-parameter embedding model trained on patent citation graphs and multi-view self-alignment. Despite its compact size, QaECTER establishes a new state of the art for patent retrieval. It outperforms the #1 model on the English retrieval text embedding benchmark (RTEB), a model 23x larger, as well as all existing patent specific models across every query type, IPC section, and jurisdiction on Sophia-bench, with gains of up to 7.2% average NDCG@10 over the next-best model. These results are confirmed on an independent external benchmark, where QaECTER surpasses all prior models without requiring task-specific instruction prompts. Both the benchmark and the model are designed for practical deployment in large-scale patent search systems.

Abstract

Explore similar work

Citation-Driven Multi-View Training for Patent Embeddings: QaECTER and Sophia-Bench

Heterogeneous Dependency Graph-Guided Attentionfor Patent Representation Learning

Benchmarking Patent Embeddings: A Multi-Task Evaluation of 22 Models Across Retrieval, Classification, and Clustering