cs.IROct 24, 2026

Novel or Not? PatEx-Gym: Agentic Prior-Art Search for Novelty Assessment in Patent Examination

Authors: Qiyao Wang, Xinyi Chen, Hongbo Wang, Jiayuan Lu, Jianhang Li, Minghuan Tan, Yuan Lin, Min Yang

Organizations: Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Dalian University of Technology · Shenzhen University of Advanced Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Patent examination is a challenging real-world task that requires agents to discover sparse prior art, iteratively refine queries, inspect technical evidence, and produce novelty assessments documented in Office Actions (OAs) across heterogeneous scientific and technical domains. However, existing benchmarks largely focus on static and limited candidate sets, leaving the full search--reasoning--assessment process underexplored. We present PatEx-Gym, the first framework for developing and evaluating agents for patent examination, which integrates three components. First, PatEx-Bench provides 1K realistic examination tasks, two large-scale retrieval corpora containing over 1M patents and 52K scientific papers, and a harness runtime supporting iterative search, document visits, reasoning, and OA generation. Second, we introduce PASS, an annotation-free Prior-Art Agentic Search Trajectory Synthesis framework. Specifically, it exploits document visits as implicit relevance signals and evidence-acquisition anchors to model intra-turn query behavior, inter-turn search progression, and trajectory-level evidence and outcome quality, yielding PatEx-SynData. Third, we train PatEx-Agent at the 4B scale via supervised fine-tuning on the curated trajectories, establishing a lightweight baseline. Extensive experiments show that final decision accuracy alone does not fully reflect examination quality, while prior-art discovery remains difficult even for strong frontier models. Our PatEx-Agent achieves competitive performance against substantially larger general-purpose agents. These results show that training on PatEx-SynData enables a compact 4B model to serve as a competitive baseline for prior-art search and patent examination. We view patent examination as a demanding testbed for agentic search and deep research agents.

Figures & tables

Figure 1: Overview of the Harness Runtime in PatEx-Gym, supporting iterative reasoning, prior-art retrieval, document inspection, and OA generation. rtr_{t} , utu_{t} , and ztz_{t} denote the reasoning, tool call, and tool results at step tt , respectively.
BenchmarkTest #Evidence PoolReasonOAAgenticNPL
PAR4PC2,8968 candidates✓✗✗✗
IPBench Task 3–5314no prior art✓✗✗✗
FiNE-Patents1,8441 given document✓✗✗✗
PatRe48010 references✓✓✗✗
PatEx-Bench (Ours)1,0001M + 52K✓✓✓✓
Table 1: Comparison of patent examination benchmarks. Evidence Pool denotes the searchable candidates. Where PatEx-Bench comprises 1M patents and 52K papers. Reason denotes LLMs reasoning beyond retrieval alone; OA denotes Office Action generation; Agentic indicates whether provides a harness for iterative tool use and multi-step task execution, and NPL denotes support for non-patent literature.
Figure 2: Data distributions for PatEx-Bench and its retrieval corpora. International Patent Classification (IPC) including: A, human necessities; B, performing operations and transporting; C, chemistry and metallurgy; D, textiles and paper; E, fixed constructions; F, mechanical engineering, lighting, heating, weapons, and blasting; G, physics; and H, electricity.
Figure 3: Overview of Prior-Art Agentic Search Trajectory Synthesis (PASS) for PatEx-SynData. (A) Visit-guided, relevance-aware rollout segmentation divides trajectories into semantic turns using visits as anchors. rr denotes reasoning, SS denotes search, VV denotes visit, ZZ denotes the returned tool results, and WW denotes workspace tools, e.g. Write . (B) Hierarchical quality modeling evaluates multidimensional quality. (C) Tournament-based ranking with rejection sampling for high-quality trajectories.
Figure 4: Training data filtering and the distributions of token length and tool calls.
ModelScoreDec.Stat.Search-RVisit-RSelect-R
Codex Harness
Claude-Opus-5-max7.3487.056.948.735.632.4
Qwen3.8-Max-xhigh6.7685.654.841.430.129.1
GLM-5.3-max6.7186.754.540.429.428.4
Claude-Opus-4.7-max6.5586.655.641.628.726.2
DeepSeek-V4-Flash-0731-max6.1779.950.244.126.824.7
Table 2: Main results on PatEx-Bench. Score is on a 0–10 scale. Decision Accuracy (Dec.) and Statute F1 (Stat.) evaluate the final examination decision and statutory grounds, respectively. Searched (Search-R), Visited (Visit-R), and Selected (Select-R) report the recall (%) at the corresponding stages. Best results in each column are bolded ; second-best results are underlined .
Figure 5: NPL recall in PatEx-Bench. Each bar reports NPL recall at three stages, i.e. , Search-R, Visit-R, and Select-R.
Figure 6: Ablation of sampling strategies for prior-art search.
Figure 7: Retrieval recall under tool isolation (left) and semantic-retriever replacement (right). Where Qwen-Emb refers to Qwen3-Embedding-0.6B.
Figure 8: Case study of two AI-domain patent applications. The left example retains relevant prior art through search, inspection, and selection, while the right example retrieves the relevant reference but fails to use it in the final evidence set.
More in this topic

Novelty Search

Novelty search explores behaviors or ideas that differ from an archive, enabling discovery when rewards are sparse or misleading.

See all 13 papers

Explore similar work

CardsList
May 5, 2026

PatRe: A Full-Stage Office Action and Rebuttal Generation Benchmark for Patent Examination

Patent examination is a complex, multi-stage process requiring both technical expertise and legal reasoning, increasingly challenged by rising application volumes. Prior benchmarks predominantly view patent examination as discriminative classification or static extraction, failing to capture its inherently interactive and iterative nature, similar to the peer review and rebuttal process in academic publishing. In this paper, we introduce PatRe, the first benchmark that models the full patent examination lifecycle, including Office Action generation and applicant rebuttal. PatRe comprises 480 real-world cases and supports both oracle and retrieval-simulated evaluation settings. Our benchmark reframes patent examination as a dynamic, multi-turn process of justification and response. Extensive experiments across various LLMs reveal critical insights into model performance, including differences between proprietary and open-source models, as well as task asymmetries between examiner analysis and applicant-side rebuttal. These findings highlight both the potential and current limitations of LLMs in modeling complex, real-world legal reasoning and technical novelty judgment in patent examination. We release our code and dataset to facilitate future research on patent examination modeling.

Qiyao Wang, Xinyi Chen, Longze Chen +4

Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Dalian University of Technology +2

LLM EvaluationLegal Document AnalysisText Generation
AskPDF
Oct 26, 2026

PatentAgentBench: Benchmarking LLM Agents with Professional Patent Task Suites

Patent practice lies at the intersection of technical, legal, and linguistic work, making it challenging yet crucial to technological innovation. Recently, rapid advances in large language models (LLMs) and agentic systems have demonstrated the potential to complete realistic business workflows and even patent works. This raises a question: can current AI systems perform patent work reliably? Existing benchmarks cover a limited range of patent tasks and simplify evidence acquisition by providing reference materials or candidate documents. In this paper, we introduce PatentAgentBench, a benchmark comprising 510 instances across eight task suites: novelty search, freedom-to-operate (FTO) search, design FTO, invalidity search, claim charting, patent drafting, office-action response, and patent translation. Curated from real patent records and professional materials, the benchmark combines task-specific metrics with expert-reviewed annotations and rubrics. Retrieval tasks such novelty search, FTO, etc., require agents to search and compare patent collections without a supplied candidate pool. The drafting and OAR tasks assess the quality of agents' work given technical disclosure and office action notice. We evaluate seven LLMs across four agent harnesses on PatentAgentBench, finding that current systems still struggle to identify novelty, assess risk, and understand practitioner intent throughout patent workflows.

Yijie Hu, Wei Li, Junjie Zhou +6

AI Eval & Alignment, PatSnap Ltd. Co.

LLM Agent EvaluationAI Agent BenchmarksInformation Retrieval
AskPDF
May 4, 2026

Is It Novel and Why? Fine-Grained Patent Novelty Prediction Based on Passage Retrieval

Novelty assessment is a critical yet complex task in the examination process for patent acceptance, requiring examiners to determine whether an invention is disclosed in a prior art document. The process involves intricate matching between specific features of a patent claim and passages in the prior art. While prior work has approached novelty prediction primarily as a binary classification task at the claim level, we argue that this formulation is susceptible to spurious correlations and lacks the granularity required for practical application. In this work, we introduce FiNE-Patents (Fine-grained Novelty Examination of Patents), a novel dataset comprising 3,658 first patent claims annotated with fine-grained, feature-level prior art references extracted from European Search Opinion (ESOP) documents. We propose shifting the evaluation paradigm from simple binary classification to a joint retrieval and abstract reasoning task at the feature level, requiring models to identify specific passages from a prior art document that disclose individual claim features, and to identify which features of a claim make it novel. We implement and evaluate LLM-based workflows that decompose claims into features, analyze each feature against prior art, and finally derive a claim-level novelty prediction. Our experiments demonstrate that these workflows outperform embedding-based baselines on passage retrieval and novel feature identification. Furthermore, we show that unlike trained classifiers, LLMs are robust against spurious correlations present in the claim-level novelty classification task. We release the dataset and code to foster further research into transparent and granular patent analysis.

Valentin Knappich, Anna Hätty, Simon Razniewski +1

Bosch Center for AI Stuttgart, Germany · University of Augsburg Augsburg, Germany · ScaDS.AI & TU Dresden Dresden, Germany

Legal IR
AskPDF