A Dynamic Self-Evolving Extraction System
Organizations: Megagon Labs
Abstract
The extraction of structured information from raw text is a fundamental component of many NLP applications, including document retrieval, ranking, and relevance estimation. High-quality extractions often require domain-specific accuracy, up-to-date understanding of specialized taxonomies, and the ability to incorporate emerging jargon and rare outliers. In many domains--such as medical, legal, and HR--the extraction model must also adapt to shifting terminology and benefit from explicit reasoning over structured knowledge. We propose DySECT, a Dynamic Self-Evolving Extraction and Curation Toolkit, which continually improves as it is used. The system incrementally populates a versatile, self-expanding knowledge base (KB) with triples extracted by the LLM. The KB further enriches itself through the integration of probabilistic knowledge and graph-based reasoning, gradually accumulating domain concepts and relationships. The enriched KB then feeds back into the LLM extractor via prompt tuning, sampling of relevant few-shot examples, or fine-tuning using KB-derived synthetic data. As a result, the system forms a symbiotic closed-loop cycle in which extraction continuously improves knowledge, and knowledge continuously improves extraction.
Figures & tables
| Model | Recall | Avg # Ext-Trip | ||
|---|---|---|---|---|
| Base | Iter-0 | GPT 4.1 | 22.80 | 15.34 |
| GPT 4.1 mini | 12.72 | 7.71 | ||
| Kimi K2.5 | 32.03 | 20.37 | ||
| LLaMA-3.3 70B | 23.23 | 13.30 | ||
| DySECT - Positive | Iter-1 | GPT 4.1 | 30.62 | 17.33 |
| GPT 4.1 mini | 20.22 | 13.59 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Document Information Extraction
Document Information Extraction turns text, tables, forms, and images into structured, searchable data.
See all 290 papers