cs.CLMay 18, 2025
SaveEnriching Patent Claim Generation with European Patent Dataset
Organizations: University of Cambridge
Abstract
Drafting patent claims is time-intensive, costly, and requires professional skill. Therefore, researchers have investigated large language models (LLMs) to assist inventors in writing claims. However, existing work has largely relied on datasets from the United States Patent and Trademark Office (USPTO). To enlarge research scope regarding various jurisdictions, drafting conventions, and legal standards, we introduce EPD, a European patent dataset. EPD presents rich textual data and structured metadata to support multiple patent-related tasks, including claim generation. This dataset enriches the field in three critical aspects: (1) Jurisdictional diversity: Patents from different offices vary in legal and drafting conventions. EPD fills a critical gap by providing a benchmark of European patents to enable more comprehensive evaluation. (2) Quality improvement: EPD offers high-quality granted patents with finalized and legally approved texts, whereas others consist of patent applications that are unexamined or provisional. Experiments show that LLMs fine-tuned on EPD significantly outperform those trained on previous datasets and even GPT-4o in claim quality and cross-domain generalization. (3) Real-world simulation: We propose a difficult subset of EPD to better reflect real-world challenges. Results reveal that all tested LLMs perform substantially worse on challenging samples, which highlights the need for future research.
Figures & tables
| Dataset | Size | Title | Abst | Claim | Des | Cit | PC | Years | Source | Version | Primary Purpose |
| WIPO-alpha | 75K | ✓ | ✓ | ✓ | ✓ | ✓ | 1998-2002 | WIPO | Appl | Classification | |
| USPTO-2M | 2M | ✓ | ✓ | ✓ | ✓ | 2006-2015 | USPTO | Gra | Classification | ||
| PatentMatch | 6.3M | ✓ | ✓ | <2020 | EPO | Appl & Gra | Retrieval | ||||
| BigPatent | 1.3M | ✓ | ✓ | ✓ | 1971-2018 | Gra | Summarization | ||||
| HUPD-DCG | 9.5K | ✓ | ✓ | ✓ | ✓ | ✓ | 2017 | USPTO | Appl | Claim generation | |
| Patent-CR | 22.6K | ✓ | 2024 | EPO | Appl & Gra | Claim revision |
Table 1: Comparison of related patent datasets. Abbreviations: Abst – Abstract, Dsc – Description, Cit — Citation, PC – IPC/CPC codes, Appl – Application, and Gra – Granted version.
| Application | Granted | |
| # Patents | ||
| # Patents (all) | 73,980 | |
| # Patents (incl. both version) | 32,988 | |
| # Documents | 32,988 | 73,980 |
| Average # Tokens | ||
| Title | 15.8 | 15.5 |
Table 2: Dataset statistics. The granted patents from EPO in this dataset do not include the abstract section.
| Statistics | HUPD-DCG | EPD |
| Basic Statistics | ||
| # Tokens | 926 495 | 929 381 |
| # Claims | 13.6 6.9 | 11.5 3.9 |
| # Independent Claims | 1.9 0.9 | 1.4 0.6 |
| # Dependent Claims | 11.7 6.5 | 10.1 3.8 |
| Per Claim Length | 81.1 52.5 | 84.5 31.6 |
Table 3: Comparison of claim statistics with standard deviations between HUPD-DCG and EPD. The methods to calculate these statistics are introduced in Appendix F . A smaller readability score indicates higher readability.
| EPD | HUPD-DCG | |||||||
| Model | BLEU | R-1 | R-L | BERTScore | BLEU | R-1 | R-L | BERTScore |
| Law-specific LLMs | ||||||||
| SaulLM-7B | 13.53 | 39.95 | 26.46 | 83.19 | 12.68* | 36.63* | 25.10* | 83.13* |
| Base LLMs | ||||||||
| Llama-3.1-8B (Zero-shot) | 28.52 | 60.76 | 39.58 | 86.32 | 34.30 | 59.79 | 40.91 | 87.64 |
| Llama-3.1-8B (Few-shot of HUPD) | 25.69 | 65.16 | 41.73 | 86.43 | 28.61 | 63.46 | 41.73 | 86.43 |
Table 4: Claim generation results on EPD (from EPO) and HUPD-DCG (from USPTO) datasets. The best scores for each metric are marked in bold . Scores of * are from Jiang et al. (2025c) .
Figure 1: LLM-as-a-judge evaluation results on EPD and HUPD-DCG.
Figure 2: LLM-as-a-judge performance difference between EPD and HUPD-DCG (EPD minus HUPD-DCG)
| Difficult Samples | Easy Samples | |||||||
| Model | BLEU | R-1 | R-L | BERTScore | BLEU | R-1 | R-L | BERTScore |
| Base LLMs | ||||||||
| Llama-3.1-8B (Zero-shot) | 24.41 | 58.54 | 35.90 | 85.46 | 37.14 | 65.21 | 47.04 | 88.04 |
| Llama-3.1-8B (one-shot of HUPD) | 21.77 | 56.75 | 37.31 | 85.54 | 34.18 | 64.13 | 50.77 | 88.22 |
| Llama-3.1-8B (one-shot of EPD) | 28.85 | 60.24 | 38.98 | 86.77 | 42.24 | 67.85 | 53.96 | 89.81 |
| GPT-4o (Zero-shot) | 17.99 | 57.72 | 36.42 | 85.20 | 28.43 | 65.19 | 47.72 | 87.48 |
Table 5: Claim generation results on difficult and easy examples of the EPD dataset. The best scores for each metric are marked in bold .
Figure 3: LLM-as-a-judge performance on difficult and easy subsets of EPD.
Figure 4: LLM-as-a-judge performance difference between difficult and easy subsets (difficult minus easy).
| Structure | Label | Description |
| Section | F | Mechanical engineering ; Lighting; Heating; Weapons; Blasting |
| Class | F02 | Combustion engines; hot-gas or combustion-product engine plants |
| Sub-class | F02D | Controlling combustion engines |
| Group | F02D 41 | Electrical control of supply of combustible mixture of its constituents |
| Sub-group | F02D 41/02 | Circuit arrangements for generating control signals |
Table 6: Example of International Patent Classification (IPC) scheme from Jiang and Goetz (2025)
| Section | Description | Single | Both |
| A | Human necessities | 11,152 | 6,223 |
| B | Performing operations; transporting | 9,754 | 7,003 |
| C | Chemistry; metallurgy | 3,308 | 2,249 |
| D | Textiles; paper | 393 | 326 |
| E | Fixed constructions | 1,683 | 753 |
| F | Mechanical engineering; lighting; heating; weapons; blasting | 3,260 | 2,973 |
Table 7: Number of documents in different IPC sections
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | HUPD | EPD | ||||||||||
| Section | Class | Subclass | Section | Class | Subclass | |||||||
| Acc. | F1 | Acc. | F1 | Acc. | F1 | Acc. | F1 | Acc. | F1 | Acc. | F1 | |
| Llama-3.1-8B | 56.37 | 36.03 | 32.64 | 17.11 | 25.48 | 12.11 | 51.59 | 41.02 | 31.21 | 17.59 | 21.64 | 10.37 |
| Llama-3.1-8B-FT (HUPD) | 81.69 | 74.23 | 72.84 | 50.98 | 63.54 | 36.27 | 76.13 | 71.42 | 61.74 | 42.96 | 47.34 | 27.36 |
| Llama-3.1-8B-FT (EPD) | 78.41 | 71.73 | 67.12 | 45.94 | 55.22 | 30.51 | 80.77 | 74.77 | 67.63 | 48.44 | 54.78 | 33.41 |
Table 8: Patent classification results on HUPD and EPD datasets (Accuracy and F1).
| Model | HUPD | EPD | ||||
| R-1 | R-2 | R-L | R-1 | R-2 | R-L | |
| Llama-3.1-8B | 53.49 | 31.65 | 39.06 | 56.14 | 36.08 | 42.96 |
| Llama-3.1-8B-FT (HUPD) | 60.88 | 44.43 | 50.56 | 69.66 | 56.28 | 62.08 |
| Llama-3.1-8B-FT (EPD) | 59.71 | 43.70 | 50.10 | 69.48 | 56.77 | 62.60 |
Table 9: Results of patent summarization as measured by ROUGE score.
| HUPD-DCG | EPD | |||||||
| Model | BLEU | R-1 | R-L | BERTScore | BLEU | R-1 | R-L | BERTScore |
| Fine-tuned LLMs with RLHF | ||||||||
| Llama-3.1-8B-FT (DPO) | 20.85 | 51.88 | 31.45 | 84.10 | 33.05 | 57.69 | 34.14 | 86.13 |
| Llama-3.1-8B-FT (Mixed-DPO) | 25.96 | 53.69 | 36.14 | 85.83 | 38.41 | 59.89 | 38.44 | 87.45 |
Table 10: Claim generation results on HUPD-DCG (from USPTO) and EPD (from EPO) datasets.
| Difficult Samples | Easy Samples | |||||||
| Model | BLEU | R-1 | R-L | BERTScore | BLEU | R-1 | R-L | BERTScore |
| Fine-tuned LLMs with RLHF | ||||||||
| Llama-3.1-8B-FT (DPO) | 31.51 | 56.99 | 32.43 | 85.83 | 36.10 | 59.03 | 37.61 | 86.73 |
| Llama-3.1-8B-FT (Mixed-DPO) | 35.74 | 58.39 | 35.72 | 86.81 | 43.71 | 62.64 | 43.77 | 88.74 |
Table 11: Claim generation results on difficult and easy examples of the EPD dataset.
| Gold claims 1. Collar life preserver for water sports that includes a compressed inflatable life preserver (1), a cartridge (2) that is filled with CO2, a solenoid valve (3) in connection with the cartridge (2), a discharge valve (4), a sensor (5) associated with a microprocessor (6) programmed with an algorithm and emission/reception means between the valve (3) and the microprocessor (6), where the cartridge (2) can be recharged after each use, the algorithm constantly receives information of the user’s body functions and by means of predetermined values knows when the user is unconscious, the microprocessor (6) sends a signal to the solenoid valve (3) of the CO2 cartridge (2) for its automatic filling and there is a cord to manually activate the CO2 cartridge; wherein the sensor (5) includes a cardiac sensor that detects an altered heart rate, characterized in that the sensor (5) further includesa pulse oximeter that detects a low level of oxygen saturation in the blood,a movement sensor and a sensor that captures muscle contractions. |
| Llama-3.1-8B Claim 1: A device for automatic inflation of a life preserver, comprising:a collar placed around the user’s neck;an inflatable life preserver (1) internally compressed, said inflatable life preserver being connected to a CO2 cartridge (2);a solenoid valve (3) connected to the CO2 cartridge (2) for filling the inflatable life preserver (1);a discharge valve (4) for emptying the inflatable life preserver (1) after use;a sensor (5) associated with a microprocessor (6) programmed with an algorithm to detect the user’s body functions and determine when the user is unconscious; emission/reception means between the valve (3) and the microprocessor (6) for sending a signal to the solenoid valve (3) for automatic filling of the inflatable life preserver (1) when the user is unconscious. Claim 2: The device of claim 1, wherein the sensor (5) includes a pulse oximeter, a cardiac sensor, a muscle contraction sensor, a movement sensor, and a depth sensor for detecting the user’s physical condition. Claim 3: The device of claim 1, wherein the sensor (5) is capable of sending external radio signals to be captured by life guards or emergency services when the user is unconscious. Claim 4: The device of claim 1, wherein the CO2 cartridge (2) can be replaced by any other means of gas generation, such as decompression of a solid by heat. Claim 5: The device of claim 1, further comprising a rechargeable battery for powering the electronic elements. Claim 6: The device of claim 1, wherein the inflatable life preserver (1) is designed to keep the user’s head out of the water at all times, ensuring breathing can be maintained if the user is unconscious. Claim 7: The device of claim 1, wherein the device is designed for use in aquatic sports, such as surfing, and is capable of detecting the user’s physical condition, including oxygen saturation, heart rate, and muscle contractions. Claim 8: The device of claim 1, wherein the device can be used as an alternative for other sports where there is no specific protection, such as underwater fishing or for children or persons with little mobility. Claim 9: The device of claim 1, wherein the device has a manual activation cord for the CO2 cartridge (2) in case of need. Claim 10: The device of claim 1, wherein the inflatable life preserver (1) has a discharge valve (4) for emptying the inflatable life preserver (1) after use. |
| Llama-3.1-8B-FT (EPO) 1. Device for the protection of a user in aquatic sports, comprising a collar (1) that is placed around the user’s neck, said collar (1) being inflatable by means of a cartridge (2) that is filled with CO2, said inflatable collar (1) comprising a solenoid valve (3) in connection with said cartridge (2) that allows its filling, said inflatable collar (1) also comprising a discharge valve (4) that permits emptying the inflatable collar (1) after its use, said inflatable collar (1) presenting a sensor (5) associated with a microprocessor (6) programmed with an algorithm that constantly receives information on the user’s body functions and is capable, through predetermined values, of knowing when the user is unconscious, said device having emission/reception means between said valve (3) and said microprocessor (6), said sensor (5) being able to send a signal that alters the threshold of the predetermined value and said microprocessor (6) to send a signal to the solenoid valve (3) of the cartridge (2) of CO2 for the automatic filling, in such a way that this device keeps the user’s head out of the water at the time in which the life preserver is used and, therefore, breathing can be maintained if the user is unconscious. 2. Device according to claim 1, characterised in that it comprises a cord to activate the cartridge of CO2 manually in case of need. 3. Device according to claim 1, characterised in that it comprises a rechargeable battery that keeps the electronic elements in operation. 4. Device according to claim 1, characterised in that said microprocessor (6), programmed with the predetermined algorithm, detects by means of said sensor (5) an altered signal of the information obtain and sends a signal for the opening of the solenoid valve (3) for the activation of the cartridge (2). 5. Device according to claim 1, characterised in that said sensor (5) includes a pulse oximeter that detects a low level of oxygen saturation in the blood, a cardiac sensor that detects an altered heart rate, a sensor that captures muscle contractions, a movement sensor, in addition to a depth sensor for athletes that practice water sports, in particular, surfing. 6. Device according to claim 1, characterised in that it comprises a sensor capable of sending external radio signals capable of being captured by life guards or emergency services when the unconsciousness of the user is detected. |
| Llama-3.1-8B-FT (USPTO) 1. A device for a life preserver, comprising: a collar composed internally by a compressed inflatable life preserver, the collar being placed around a user’s neck and being inflated by means of a cartridge filled with CO2; the inflatable life preserver having a solenoid valve in connection with the cartridge that allows its filling; the inflatable collar having a discharge valve that permits emptying the inflatable collar after its use; the inflatable collar having a sensor associated with a microprocessor programmed with an algorithm that constantly receives information on the user’s body functions and is capable, through predetermined values, of knowing when the user is unconscious; the device having emission/reception means between the valve and the microprocessor, with this sensor being able to send a signal that alters the threshold of the predetermined value and the microprocessor to send a signal to the solenoid valve of the cartridge of CO2 for the automatic filling, in such a way that this device keeps the user’s head out of the water at the time in which the life preserver is used and, therefore, breathing can be maintained if the user is unconscious. |
| Continue on next page |
| 2. The device of claim 1, wherein the device has a cord to activate the cartridge of CO2 manually in case of need. 3. The device of claim 1, wherein the device has a rechargeable battery that keeps the electronic elements in operation. 4. The device of claim 1, wherein the microprocessor, programmed with the predetermined algorithm, detects by means of the sensor an altered signal of the information obtain and sends a signal for the opening of the solenoid valve for the activation of the cartridge. 5. The device of claim 1, wherein the sensor includes a pulse oximeter that detects a low level of oxygen saturation in the blood, a cardiac sensor that detects an altered heart rate, a sensor that captures muscle contractions, a movement sensor, in addition to a depth sensor for athletes that practice water sports, in particular, surfing. 6. The device of claim 1, wherein the sensor is capable of sending external radio signals capable of being captured by life guards or emergency services when the unconsciousness of the user is detected. 7. The device of claim 1, wherein the CO2 cartridge can be replaced by any other means of gas generation, for example decompression of a solid by heat. |
| Llama-3.1-8B-FT (Mixed) 1. A device for a life preserver, comprising a collar (1) that is placed around the user’s neck, said collar (1) being inflatable by means of a cartridge (2) of compressed gas, said collar (1) comprising a solenoid valve (3) in connection with said cartridge (2) for the filling of said collar (1), said collar (1) comprising a discharge valve (4) for the emptying of said collar (1) after its use, said collar (1) comprising a sensor (5) associated with a microprocessor (6) programmed with an algorithm that constantly receives information on the user’s body functions and is capable of determining when the user is unconscious, said device comprising emission/reception means between said valve (3) and said microprocessor (6), said sensor (5) being able to send a signal that alters the threshold of the predetermined value and said microprocessor (6) to send a signal to said solenoid valve (3) of said cartridge (2) of compressed gas for the automatic filling, in such a way that this device keeps the user’s head out of the water at the time in which the life preserver is used and, therefore, breathing can be maintained if the user is unconscious. 2. The device according to claim 1, characterised in that it comprises a cord to activate said cartridge (2) of compressed gas manually in case of need. 3. The device according to claim 1 or 2, characterised in that it comprises a rechargeable battery that keeps the electronic elements in operation. 4. The device according to any of the previous claims, characterised in that said microprocessor (6), programmed with said predetermined algorithm, detects by means of said sensor (5) an altered signal of the information obtain and sends a signal for the opening of said solenoid valve (3) for the activation of said cartridge (2). 5. The device according to any of the previous claims, characterised in that said sensor (5) includes a pulse oximeter that detects a low level of oxygen saturation in the blood, a cardiac sensor that detects an altered heart rate, a sensor that captures muscle contractions, a movement sensor, in addition to a depth sensor for athletes that practice water sports, in particular, surfing. 6. The device according to any of the previous claims, characterised in that it comprises a sensor capable of sending external radio signals capable of being captured by life guards or emergency services when the unconsciousness of the user is detected. |
| GPT-4o 1. An inflatable life preserver device comprising: - a collar configured to be placed around a user’s neck; - an inflatable life preserver housed within the collar; - a CO2 cartridge connected to the inflatable life preserver for inflation; - a solenoid valve operatively connected to the CO2 cartridge to control the release of CO2; - a sensor configured to monitor the user’s body functions; - a microprocessor programmed with an algorithm to receive data from the sensor and determine when the user is unconscious; - wherein the microprocessor sends a signal to the solenoid valve to inflate the life preserver upon detecting unconsciousness, thereby maintaining the user’s head in a safe position above water. 2. The device of claim 1, wherein the sensor includes: - a pulse oximeter for detecting oxygen saturation levels in the blood; - a cardiac sensor for monitoring heart rate; - a sensor for detecting muscle contractions; - a movement sensor; - a depth sensor for aquatic sports applications. 3. The device of claim 1, further comprising: - a manual activation cord for the CO2 cartridge to allow manual inflation of the life preserver. 4. The device of claim 1, further comprising: - a rechargeable battery to power the electronic components of the device. 5. The device of claim 1, wherein the sensor is further configured to send external radio signals to alert lifeguards or emergency services upon detecting the user’s unconsciousness. 6. The device of claim 1, wherein the CO2 cartridge is replaceable with an alternative gas generation means, such as the decompression of a solid by heat. 7. The device of claim 1, wherein the collar is designed to be used in various sports, including underwater fishing and surfing, and is suitable for children or individuals with limited mobility. 8. The device of claim 1, wherein the microprocessor is configured to adjust the threshold values for detecting unconsciousness based on the sensor data. 9. The device of claim 1, wherein the inflatable life preserver includes a discharge valve for deflating the collar after use. 10. The device of claim 1, wherein the microprocessor and sensor are configured to continuously monitor and analyze the user’s physiological data to ensure timely activation of the life preserver. |
| Llama-3.1-8B-FT (Mixed-DPO) 1. A device (1, 2, 3, 4, 5, 6) for a user, comprising a collar (1) configured to be placed around a neck of the user, the collar (1) comprising a compressed inflatable life preserver (1) and a sensor (5) associated with a microprocessor (6) configured to be programmed with an algorithm, wherein the inflatable life preserver (1) is configured to be inflated by a cartridge (2) (2) comprising compressed CO2, wherein the inflatable life preserver (1) comprises a solenoid valve (3) in connection with the cartridge (2) for filling the inflatable life preserver (1) with CO2 from the cartridge (2), wherein the inflatable life preserver (1) comprises a discharge valve (4) for emptying the inflatable life preserver (1) after its use, wherein the sensor (5) is configured to constantly receive information on the user’s body functions, wherein the microprocessor (6) is configured to determine, based on the received information, when the user is unconscious, wherein the device comprises emission/reception means between the solenoid valve (3) and the microprocessor (6), wherein the sensor (5) is configured to send a signal to the microprocessor (6) and the microprocessor (6) is configured to send a signal to the solenoid valve (3) to open the solenoid valve (3) for the automatic filling of the inflatable life preserver (1) with CO2 from the cartridge (2) when the user is unconscious, wherein the sensor (5) comprises a pulse oximeter configured to detect a low level of oxygen saturation in the blood, |
| Continue on next page |
| a cardiac sensor configured to detect an altered heart rate, a sensor configured to capture muscle contractions, a movement sensor and a depth sensor, wherein the sensor (5) is configured to send external radio signals capable of being captured by life guards or emergency services when the unconsciousness of the user is detected, and wherein the device comprises a rechargeable battery configured to keep the electronic elements in operation,characterized in that the sensor (5) is configured to send a signal that alters a threshold of a predetermined value of the algorithm, and in that the microprocessor (6) is configured to send a signal to the solenoid valve (3) of the cartridge (2) of CO2 for the automatic filling of the inflatable life preserver (1) with CO2 from the cartridge (2) when the user is unconscious, wherein the microprocessor (6), programmed with the predetermined algorithm, detects by means of the sensor (5) an altered signal of the information obtained and sends a signal for the opening of the solenoid valve (3) for the activation of the cartridge (2). 2. The device (1, 2, 3, 4, 5, 6) according to claim 1, wherein the sensor (5) is configured to send the external radio signals when the user is unconscious. 3. The device (1, 2, 3, 4, 5, 6) according to claim 1, wherein the sensor (5) is configured to send the external radio signals when the user is unconscious and the sensor (5) is configured to send the external radio signals when the user is in a real and imminent danger. 4. The device (1, 2, 3, 4, 5, 6) according to claim 1, wherein the sensor (5) is configured to send the external radio signals when the user is unconscious and the sensor (5) is configured to send the external radio signals when the user is in a real and imminent danger, wherein the sensor (5) is configured to send the external radio signals when the user is in a predetermined time under water. 5. The device (1, 2, 3, 4, 5, 6) according to claim 1, wherein the sensor (5) is configured to send the external radio signals when the user is unconscious and the sensor (5) is configured to send the external radio signals when the user is in a real and imminent danger, wherein the sensor (5) is configured to send the external radio signals when the user is in a predetermined time under water, wherein the sensor (5) is configured to send the external radio signals when the user is in a predetermined depth under water. |
Table 12: Example model outputs of patent EP3676170B1.
| Instructions: |
| You will be given the draft claims and the referenced claims of the same patent. Your task is to rate the draft claims on eight metrics using the referenced claims as the gold standard, and each scored from 1 to 10. Please make sure you read and understand these instructions carefully. Please keep this document open while reviewing, and refer to it as needed. |
| Evaluation Criteria: |
| 1. Coverage: Does the candidate claim fully capture all essential technical features described in the referenced claim? |
| 2. Technical Precision: Does the candidate claim provide precise definitions and clear characterizations of technical features, compared to the referenced claim? |
| 3. Specificity: Does the candidate claim explicitly describe technical features in concrete detail rather than using overly abstract or generalized language, compared to the referenced claim? |
Table 13: LLM-as-a-judge prompt used for patent claim evaluation.
More in this topic
Text Generation
Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.
See all 422 papers