Automated patent claim validation demands low error tolerance. However, existing approaches face a rigidity-resource dilemma: lightweight encoders cannot track long-range legal dependencies, while exhaustive LLM verification incurs 4-5X higher overhead at million-claim scale. A naive confidence-based cascade cannot resolve this because binary validity scores fail to distinguish structurally distinct error types which require different reasoning depths. We propose a two-stage framework: Adaptive Cost-efficient Evaluation (ACE), which exploits the categorical structure of patent errors for uncertainty-aware routing. In the first stage, a fine-tuned encoder projects claims into a K+1 distribution over legal error types, whose predictive entropy serves as the routing signal. Claims exceeding an entropy threshold are escalated to the second stage, where an expert LLM executes a schema-constrained Chain-of-Patent-Thought (CoPT) protocol to map claim elements against 35 U.S.C. standards whose schema constraint reduces per-claim latency by 42% while producing legally grounded verdicts. We further present a 40,000-claim dataset ACE-40k with MPEP-grounded annotations, where ACE surpasses competitive baselines including a supervised 70B-parameter LLM while reducing costs by 78%. On real USPTO rejection data, the routing mechanism transfers without re-calibration, reducing inference time by 60% while maintaining competitive recall.
Figures & tables
Figure 1: High-stakes patent claim validation requires low error tolerance while demanding cost efficiency. Adaptive Cost-efficient Evaluation (ACE) leverages the throughput of encoder models and the inference accuracy of Chain-of-Patent-Thought (CoPT) reasoning.
Category
Description
Count
Pass
Published patent
20,000
Fail: Antecedent
Lack of antecedent basis
4,000
Fail: Dependency
Invalid cross-reference
4,000
Fail: Logical
Impossible causality or circularity
4,000
Fail: Ambiguity
Indefinite terminology
4,000
Fail: Syntax
Structural format violations
4,000
Table 1: Statistics of the constructed ACE dataset. We enforce a strictly balanced distribution of 1:1 between valid and invalid samples.
Figure 2: The ACE framework. An uncertainty-aware Gatekeeper routes patent claims based on predictive entropy: confident predictions ( U≤τ ) are resolved via the Fast Path, while ambiguous claims ( U>τ ) are escalated to an instruction-tuned Expert LLM that performs Chain-of-Patent-Thought (CoPT) reasoning for statutory verification.
Model / Method
Acc. (%)
F1 (%)
AUC (%)
Latency (s)
Est. Cost ($)/M
Sanity Check & General Baselines
Vanilla BERT
49.60
66.25
50.93
0.12
$100
Vanilla Longformer
75.52
73.11
47.26
0.42
$350
Frozen BERT + Logistic Regression
69.27
68.12
66.20
0.08
$67
TF-IDF + Logistic Regression
73.38
72.23
83.00
< 0.01
< $1
Domain-Specific Experts
Table 2: Main Performance Results. Domain-specific baselines use pre-trained representations without task-specific fine-tuning. ACE Hybrid processes 1M claims at 1,247,a785,733). Cost estimated at $3.00/hr for NVIDIA H100. AUC is reported only for single-model classifier.
Figure 3: ROC curves for claim validity discrimination. The Gatekeeper (AUC=0.9716) substantially outperforms baselines.
Model
Recall (%)
Time (s)
Llama-3-8B (zero-shot)
13.2
103.1
Llama-3-70B (zero-shot)
8.3
355.3
PatentBERT Standalone
56.4
7.2
Llama-3-70B + SFT + CoPT
65.2
6,820.9
ACE Hybrid (Ours)
66.2
2,674.8
Table 3: Results on ACE-Real112b (204 real USPTO §112(b) rejections). All samples are genuine Office Action rejections; recall is reported as the transfer metric. No parameters were re-tuned on this set.
Figure 4: Fine-grained Gatekeeper performance across error types. Antecedent Basis records the lowest recall, confirming the long-range tracking bottleneck.
Failure Pattern
Count
%
Cross-claim dependency / Nested antecedent
38
55.1
Claim-set semantic scope ambiguity
17
24.6
Examiner-specific subjective judgment
14
20.3
Table 4: Breakdown of 69 missed cases on ACE-Real112b.
Figure 5: Risk-Coverage Trade-off. Retained F1 and operational cost as a function of escalation rate.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Label
Claim Text
Error Type
Pass
A semiconductor device comprising a substrate, a gate structure formed over the substrate, and a spacer adjacent to gate structure.
None
Fail
A semiconductor device comprising the gate structure formed over the substrate, wherein the substrate is configured to receive the gate structure of claim 1.
Antecedent
Fail
The system of claim 1, wherein the processor of claim 3 further executes the module of claim 7.
Dependency
Fail
A battery including a charging unit that generates electrical energy from the discharged battery cell.
Logical
Appendix
Table 5: Representative samples from the ACE dataset. Invalid claims contain single targeted errors injected based on MPEP guidelines.
Parameter
Value
Rationale
Base Model
PatentBERT-base
Domain-Adapted Efficiency
Max Sequence Length
512
Standard BERT Limit
Batch Size
32
Stable Gradient Estimation
Learning Rate
2×10−5
Optimal for Fine-tuning
Epochs
5
Sufficient Convergence
Weight Decay
0.01
L2 Regularization
Appendix
Table 6: Hyperparameters for Gatekeeper Training
Parameter
Value
Rationale
Base Model
Meta-Llama-3-70B-Instruct
Instruction-Following Capability
Quantization
4-bit (NF4)
Memory Efficiency (QLoRA)
LoRA Rank ( r )
16
Parameter Efficiency
LoRA Alpha ( α )
32
Scaling Factor ( 2×r )
LoRA Dropout
0.05
Regularization
Learning Rate
2×10−4
Optimal for QLoRA Fine-tuning
Appendix
Table 7: Hyperparameters for Llama-3 Evaluator Training (QLoRA)
Error Type
Recall
F1-Score
Nature
Semantic Reasoning Tasks
Ambiguity
90.75%
0.8985
Semantic Interpretation
Logical
86.45%
0.8645
Inferential Reasoning
Structural Verification Tasks
Dependency
92.37%
0.9130
Claim Hierarchy
Antecedent
82.77%
0.8141
Long-Range Reference
Appendix
Table 8: Fine-grained Performance of the Gatekeeper (Test Set). The model struggles with structural errors (Antecedent, Syntax), highlighting the specific areas where the LLM Evaluator is most needed.
Role
Content
System
You are a Senior Patent Attorney.
Your task is to evaluate the validity of a patent claim based on specific error types.
If ANY error exists, output {"verdict": "Fail"} . If NO error, output {"verdict": "Pass"} .
User
Claim: { claim_text }
Assistant
(Model Generation Target)
Appendix
Table 9: The Standard SFT prompt template. This strictly follows the format used during the supervised fine-tuning stage to prevent distribution shift during inference.
Role
Content
System
You are a Senior Patent Attorney.
Your task is to evaluate the validity of a patent claim based on specific error types.
[Special Instruction]: To ensure strict adherence to 35 U.S.C. § 112, you must explicitly document your reasoning process using the Chain of Patent Thought (CoPT) protocol before concluding.
1. [Element Parsing] : Identify preamble, transition, and body elements.
Appendix
Table 10: The full prompt template used for the Stage 2 Specialist LLM (SFT + CoPT). The [Special Instruction] tag and 2-shot examples are designed to override the model’s training bias towards short responses, enforcing a detailed legal analysis path.
Component
Content
Input Claim (Excerpt)
1. (canceled) 2. The workpiece made according to the method of claim 8, further comprising an intermediate layer containing TiAlN provided between the TixSi1-xN layer and the substrate body of the workpiece. (...) 5. The workpiece made according to the method of claim 8, further comprising an intermediate layer containing CryAl1-yN provided between the TixSi1-xN layer and the substrate body of the workpiece (...) (...) 8. A method for coating a workpiece having a coating comprising at least one TixSi1-xN layer, wherein x ≤ 0.85 (...), the method comprising: using a sputtering process to produce the TixSi1-xN layer (...)
CoPT Output
{"reasoning": "Step 1: Parsed claim elements. Step 2: Antecedent Basis error: Claim 2 refers to claim 8, but claim 8 is not yet defined. Claim 5 refers to claim 8, but claim 8 is not yet defined. Step 3: No Definiteness errors. Step 4: Fail due to Antecedent Basis errors.", "verdict": "Fail"}
Ground Truth
Fail (§112(b) rejection in USPTO Office Action)
Appendix
Table 11: Example of CoPT protocol output on a real USPTO §112(b) case. The model identifies forward-reference dependencies (claims 2 and 5 referencing the later-defined claim 8) through structured step-by-step reasoning, providing actionable grounds for the rejection verdict.
Configuration
F1 (%)
Cost ($/M)
Llama-3-8B SFT (Standalone)
83.5
$667
ACE Hybrid w/ 8B Expert
89.3
$233
ACE Gatekeeper (Stage 1 Only)
91.58
$100
ACE Hybrid w/ 70B Expert
94.95
$1,247
Appendix
Table 12: Effect of expert model scale on ACE Hybrid performance (ACE-40k test set). The 8B expert degrades below the Gatekeeper alone, confirming that 70B-class reasoning is necessary for the escalated high-uncertainty subset.
Escal Ratio
Stage-2 Samples
Recall
Cost ($/M)
10%
20
60.8
$673
20%
40
65.7
$1,247
30%
61
72.1
$1,820
40%
81
78.9
$2,393
50%
101
86.3
$2,967
Appendix
Table 13: Threshold sensitivity on ACE-Real112b. The recall-cost trade-off is smooth and monotonic across escalation ratios.
The European Patent Office (EPO) reported record filings in 2025, and the 2026 EPO Guidelines hold applicants strictly responsible for LLM-assisted content under Article 83 and Rule 42, creating pressure to triage suspected AI-generated patent text. Two constraints make this hard. First, realistic prosecution settings often have only consumer GPUs with about 8 GB VRAM, not datacenter-class scoring stacks. Second, Article 84 of the European Patent Convention requires claims to be clear and concise, pushing human drafting onto the same low-perplexity, low-burstiness manifold that LLMs occupy. We benchmark three open-source zero-shot detectors on 500 granted EPO H04 telecom patents versus 500 LLM-generated counterparts using five prompting strategies, all under the consumer hardware envelope. At claim level, all detectors exceed 60 percent false-positive rate: Binoculars 78.3 percent, Fast-DetectGPT 61.3 percent, DetectGPT 80.5 percent. The failure persists under Qwen2.5-3B-Instruct regeneration, LoRA-adapted Pythia-2.8B scoring heads, cross-IPC replication on A61K, C07D, and F03D (mean FPR 84.6 percent), and H100 re-evaluation with published Falcon-7B and GPT-J-6B heads, arguing the issue is structural rather than substitute-model capacity. A seven-feature linguistic-complexity logistic regression reaches 74.0 percent accuracy at 28.1 percent FPR, a 13 percentage-point gain over a perplexity-only baseline at a comparable operating point, without using likelihood at inference and within the same hardware budget.
Anubhab Banerjee
Nokia Solutions and Networks GmbH & Co. KG, Munich, Germany.
Patent litigation imposes substantial costs on firms and distorts R&D incentives, making early risk identification a practically important task. While prior work has applied BERT-based models to patent claim text, two fundamental limitations remain: flat sequence encoding loses the dependency structure between independent and dependent claims that legally determines patent scope, and feeding the entire claim set to a single encoder discards legally critical text. A six-model ablation on 1.34 million USPTO utility patents confirms that per-claim encoding, graph connectivity, attention, and Attentional Aggregation each provide independent, additive predictive value. We propose ClaimGAT, a Graph Attention Network that encodes each claim independently, constructs a directed claim dependency graph, processes it with GATConv layers, and aggregates independent claims via Attentional Aggregation to yield both a litigation risk score and claim-level gate weights that enable post-hoc structural analysis. ClaimGAT achieves an AUC-ROC of 0.818 and a lift of 4.89x at the top 10%, using only information observable at the time of patent grant. It reveals a tendency in high-risk patents for structural selection and content sensitivity to diverge, a pattern consistent with defensive claim drafting.
Takao Arai, Hiroyasu Inoue
Graduate School of Information Science University of Hyogo · Graduate School of Information Science University of Hyogo Center for Computational Science, RIKEN
Patent claim drafting is a challenging legal drafting task that requires technical expertise, precise linguistic control, strict adherence to formal conventions, and the preservation of complex logical relationships among claim elements. While Chain-of-Thought (CoT) prompting has been widely used to improve the reasoning capabilities of large language models (LLMs), recent evidence suggests that its benefits may be limited, or even negative, in highly structured or pattern-sensitive tasks. Therefore, this paper investigates whether CoT prompting benefits patent claim generation. We propose a task-specific CoT method for patent claim generation and evaluate its effectiveness through both automatic metrics and human expert assessment. Our results show that reasoning-enhanced prompting can improve claim quality. Moreover, we demonstrate a counter-intuitive but important empirical finding: implicit CoT, where reasoning is kept internal rather than explicitly verbalized, consistently outperforms explicit CoT. Through systematic analysis, we show that explicit CoT can introduce an unnecessary information bottleneck for claim generation. Verbalized reasoning may compromise the quality of final outputs through three specific mechanisms: abstraction of critical details, disruption of internalized generation patterns, and cascading error propagation. Our findings provide new insights into legal tasks and CoT applications.
Lekang Jiang, Wenjun Sun, Stephan Goetz
University of Cambridge · National Science Library, Chinese Academy of Sciences; Department of Information Resources Management, School of Economics and Management, University of Chinese Academy of Sciences.