Modeling Claim Dependency Structure for Patent Litigation Prediction with Graph Attention Networks
Authors: Takao Arai, Hiroyasu Inoue
Organizations: Graduate School of Information Science University of Hyogo · Graduate School of Information Science University of Hyogo Center for Computational Science, RIKEN
Patent litigation imposes substantial costs on firms and distorts R&D incentives, making early risk identification a practically important task. While prior work has applied BERT-based models to patent claim text, two fundamental limitations remain: flat sequence encoding loses the dependency structure between independent and dependent claims that legally determines patent scope, and feeding the entire claim set to a single encoder discards legally critical text. A six-model ablation on 1.34 million USPTO utility patents confirms that per-claim encoding, graph connectivity, attention, and Attentional Aggregation each provide independent, additive predictive value. We propose ClaimGAT, a Graph Attention Network that encodes each claim independently, constructs a directed claim dependency graph, processes it with GATConv layers, and aggregates independent claims via Attentional Aggregation to yield both a litigation risk score and claim-level gate weights that enable post-hoc structural analysis. ClaimGAT achieves an AUC-ROC of 0.818 and a lift of 4.89x at the top 10%, using only information observable at the time of patent grant. It reveals a tendency in high-risk patents for structural selection and content sensitivity to diverge, a pattern consistent with defensive claim drafting.
Figures & tables
Figure 1: Flat sequence encoding (left) versus the proposed ClaimGAT (right), on the same six-claim example. Flat encoding concatenates all claims, discarding C5 and C6 (hatched) beyond the 512-token limit and erasing inter-claim references. ClaimGAT encodes each claim independently and builds a directed dependency graph (filled = independent, h=0 ; open = dependent, h≥1 ); black arrows mark each dependent claim’s reference to its parent, which is also the GATConv aggregation direction, so C5 and C6 are retained. Blue arrows feed the aggregated independent claims into Attentional Aggregation, whose gate weights ( w1,w4 , illustrative) yield a risk score and claim-level attribution.
Figure 2: Architecture of ClaimGAT and the five baseline models. Each column represents one model in the six-stage ablation. ∗ BERT = bert-for-patents (BERT-Large, 1024-dim), frozen in graph-based models. IC pool = Independent-Claim Centric Pooling (mean pool over independent-claim nodes only). Att. Agg. = Attentional Aggregation ( Fey & Lenssen 2019a ) . BN = Batch Normalization; backward cit. = backward citation count; CPC = Cooperative Patent Classification section (9-dim multi-hot). † Proposed model.
Table 1: Test-set performance of all models. Bold indicates the best value in each column. Lift@ K % is the ratio of litigated patents in the top- K % of model rankings to that of a random draw. A horizontal rule separates graph-based models from BERT baselines.
Figure 3: AUC-ROC (left) and AUC-PR (right) for all six models on the test set. The dashed line represents the random baseline.
Pre-trained language models advance patent classification and retrieval by encoding claims as flat token sequences, but they overlook the dependency hierarchy among claims. Incorporating this hierarchy into self-attention poses two challenges. First, claim dependencies include relation types with different levels of reliability, so treating them uniformly may allow noisy technical relations to interfere with more reliable legal citations. Second, claim dependencies are defined at the claim level, whereas Transformer attention operates over tokens, making direct structural injection nontrivial. We propose the Patent Heterogeneous Attention-Guided Graph Encoder (PHAGE), which constructs a typed claim graph that distinguishes legal citations from technical relations. PHAGE projects this claim-level topology into token-level attention through a connectivity mask and learnable relation-aware biases, and fine-tunes the encoder using a dual-granularity contrastive objective that combines inter-patent taxonomy with intra-patent topology. At inference, the graph-specific attention components are removed, allowing representations to be generated through a standard encoder forward pass without CDG construction. Experiments on patent classification, retrieval, and clustering show that PHAGE consistently outperforms domain-adapted and citation-aware baselines, demonstrating the value of claim-level structural guidance for graph-free patent representation learning.
Autoregressive decoders emit flat token sequences and cannot enforce hierarchical constraints across output segments, a limitation that becomes acute in patent claim generation, where a claim set forms a dependency forest whose scope must narrow monotonically with depth. Topology and content are mutually dependent: a dependent claim's wording must reflect its parent's scope, yet the parent must be chosen before that wording exists, so neither post-hoc parsing nor grammar-constrained decoding suffices. We propose SPG (Structure-aware Patent Generation), which predicts topology inside the autoregressive pass. A pointer head selects each dependent claim's parent, and its gradients, together with a depth-adaptive scope regularizer, reshape the shared decoder's representations during training. A second stage then applies a violation-weighted preference objective over self-generated deficient candidates, supplying the negative signal that granted-patent corpora lack. On HUPD-DCG, SPG on Llama-3-8B-Instruct recovers 79.0% of gold parent links, a quantity its training reward never supervises, and raises antecedent consistency from 0.292 to 0.478 over a supervised baseline of equal scale, with expert evaluation corroborating these gains.
Yongmin Yoo, Zhangkai Wu, Longbing Cao
Frontier AI Research Centre, Macquarie University · School of Computing, FSE, Macquarie University
Automated patent claim validation demands low error tolerance. However, existing approaches face a rigidity-resource dilemma: lightweight encoders cannot track long-range legal dependencies, while exhaustive LLM verification incurs 4-5X higher overhead at million-claim scale. A naive confidence-based cascade cannot resolve this because binary validity scores fail to distinguish structurally distinct error types which require different reasoning depths. We propose a two-stage framework: Adaptive Cost-efficient Evaluation (ACE), which exploits the categorical structure of patent errors for uncertainty-aware routing. In the first stage, a fine-tuned encoder projects claims into a K+1 distribution over legal error types, whose predictive entropy serves as the routing signal. Claims exceeding an entropy threshold are escalated to the second stage, where an expert LLM executes a schema-constrained Chain-of-Patent-Thought (CoPT) protocol to map claim elements against 35 U.S.C. standards whose schema constraint reduces per-claim latency by 42% while producing legally grounded verdicts. We further present a 40,000-claim dataset ACE-40k with MPEP-grounded annotations, where ACE surpasses competitive baselines including a supervised 70B-parameter LLM while reducing costs by 78%. On real USPTO rejection data, the routing mechanism transfers without re-calibration, reducing inference time by 60% while maintaining competitive recall.
Yongmin Yoo, Qiongkai Xu, Longbing Cao
Frontier AI Research Centre, Macquarie University · School of Computing, FSE, Macquarie University