cs.AIAug 22, 2026

DynaContext: Self-Improving Dynamic Contextualization of Optimized Prompts for Heterogeneous Parameter Extraction

Authors: Joe Yu, Shibin Thomas Stanley Paul, Sven Mayer

Organizations: Technical University of Munich, Munich, Germany · TU Dortmund University, Dortmund, Germany · Research Center Trustworthy Data Science and Security, Dortmund, Germany

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Automated prompt and skill optimization typically produces a single static instruction that is reused across inference instances until the next optimization cycle. However, this approach cannot adapt when the required context, constraints, and evidence vary from one instance to another. For instance, parameter extraction from electronic component descriptions breaks this assumption: resistors, capacitors, transistors, and connectors require different fields, unit constraints, and demonstrations, and each input provides a different evidence state. We introduce DynaContext, a framework that combines an offline-optimized extraction core, learned with GEPA or SkillOpt, with inference-time contextual adaptation and validation-gated self-improvement. DynaContext routes each item through internal, external, or fallback evidence paths and composes an item-specific prompt from the core, schema, evidence, unresolved fields, and validated demonstrations. Deterministic validation and an LLM judge gate every output, uncertain cases go to human review, and only human-verified corrections enter the demonstration memory. On a single-category benchmark, average accuracy increases from 86.6% for the base prompt to 96.9% for standalone SkillOpt and 98.6% for the best DynaContext configuration. Across 850 heterogeneous gold parameter facts, average field-level F1 increases from 51.8% for an unoptimized, demonstration-free control to 59.2% with dynamic demonstrations alone, 66.9% with the optimized core alone, and 71.0% with both. Holding the model fixed, the full configuration outperforms the deployed static-prompting pipeline by 17.3 F1 points on average.

Figures & tables

Figure 1: Route-aware DynaContext extraction workflow. Requests are preprocessed and routed by MPN availability through internal lookup, external API, and PDF retrieval, or description-based category fallback. Schema-guided extraction is followed by postprocessing into the final JSON output.
Figure 2: Validation, human-review, and self-improvement loop. Accepted corrections update validated memory and feed the offline GEPA judge-optimization cycle; rejected cases are not promoted.
BenchmarkRecordsEvaluation scope
Ceramic capacitor142One component family; 7 target parameters per record
Multi-component13313 level-3 categories; 850 gold facts over 131 scored records
Table 1: Benchmark summary for the single-component and heterogeneous multi-component evaluations.
Core, retrievalMean acc.[95% CI]Best run
Base Prompt, none86.64[82.6, 90.7]90.14
Base Prompt, best94.83[92.5, 97.1]98.79
GEPA, none95.56[95.0, 96.1]96.68
GEPA, best98.49[98.0, 99.0]99.50
SkillOpt, none96.93[96.4, 97.4]97.89
SkillOpt, best98.59[98.3, 98.9]99.20
Table 2: Single-component ceramic-capacitor accuracy (%). “None” is the core with no retrieved demonstrations; “best” is its strongest retrieval policy, Hybrid except for GEPA, where Vector wins. Mean and interval are over the eight deployments; “Best run” is the single strongest deployment for that condition.
ConditionMean F1[95% CI]Best run
Base prompt, none51.76[48.9, 54.7]55.81
Base prompt, k5k559.22[57.1, 61.4]63.33
SkillOpt, none66.89[61.7, 72.1]73.86
SkillOpt, k5k570.98[65.9, 76.1]77.71
Static-10 (reference)53.96[49.9, 58.0]61.66
Table 3: Two-factor ablation on the heterogeneous benchmark: F1 (%) over 850 gold facts, crossing the extraction core with dynamic demonstrations at fixed k=5k=5 . Intervals are 95% tt -intervals over the eight deployments; paired effects are reported in the text. Static-10 uses a different prompt from the four crossed conditions, so it is shown for reference and is not part of the ablation.
ConfigurationF1Cost ($)Time (s)
GPT-5.5, k5k575.712.955768.9
DeepSeek-V4-Pro, k3k377.930.498422.3
GPT-5.4 mini, k5k577.420.203174.5
Table 4: Cost and runtime of the three strongest configurations. The model varies across configurations, so this compares deployments rather than methods.
More in this topic

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 429 papers

Explore similar work

CardsList
Feb 18, 2026

Evolutionary Context Search for Automated Skill Acquisition

Large Language Models cannot reliably acquire new knowledge post-deployment -- even when relevant text resources exist, models fail to transform them into actionable knowledge without retraining. Retrieval-Augmented Generation attempts to bridge this gap by surfacing relevant documents at inference time, yet similarity-based retrieval often fails to identify context that actually improves task performance. We introduce Evolutionary Context Search (ECS), an evolutionary method that searches context combinations using accuracy on a small development set, requiring only inference calls without weight updates. ECS moves beyond semantic similarity to discover non-obvious context pairings that significantly boost performance. Our empirical results show that ECS improves BackendBench by 27% and ττ-bench airline by 7%. The evolved contexts are model-agnostic, as those evolved with Gemini-3-Flash transfer effectively to Claude Sonnet and DeepSeek. This suggests that ECS opens a path toward automated context discovery for skill acquisition -- an efficient alternative to manual prompt engineering or costly fine-tuning.

Qi Sun, Stefan Nielsen, Rio Yokota +1

Sakana AI, Japan · Institute of Science Tokyo, Japan

Retrieval-Augmented ICLLLM Agent Skill Learning
AskPDF
Sep 29, 2026

Prompt2Skill: Unsupervised Skill Optimization From Natural Language Instructions

Skills are external artifacts that Large Language Models (LLMs) consume at inference time to improve their performance on specialized domains by incorporating relevant procedural and domain knowledge. Expert-authored skills are expensive to produce, and the resulting artifacts are not optimized for the specific model that consumes them, whose failure modes can vary with version, scale and training. In addition, emerging tasks may fall outside the scope of existing skill libraries, creating a need to develop new skills before curated training data become available. Recent works have explored automated skill optimization through reflection, but they require a curated, in-distribution training set, which users might not always have. To address these limitations, we present Prompt2Skill, a framework that builds skills from natural-language task description alone. From the prompt, the system derives a task specification, discovers or synthesizes datasets, and refines the skill in a closed loop of reflective editing. Across four domains spanning question answering, reading comprehension, spreadsheet manipulation, and mathematical reasoning, Prompt2Skill consistently outperforms the direct prompting baseline, achieving an average improvement of 10.8 across open-source and frontier models.

Bo Ni, Li Li, Ryan A. Rossi +2

Vanderbilt University · University of Southern California · Adobe Systems

LLM Agent Skill LearningLanguage Model Self-Improvement
AskPDF
Sep 18, 2026

One Prompt Does Not Fit All: Self-Meta-Evolve for Personalized Information Extraction

Large language models (LLMs) are increasingly deployed for enterprise information extraction (IE), where the same document must be reorganized differently for each user. Existing prompt optimization methods, however, rely on a single prompt optimized against a global objective, which is misaligned with the inherent user heterogeneity of real workplaces. We formulate enterprise IE as per-user prompt adaptation under interaction feedback and propose Self-Meta-Evolve, a hierarchical framework that maintains a dedicated prompt for each user and continuously refines it through a dual-loop process: an inner loop that edits structured prompts based on persona-conditioned feedback, and an outer loop that evolves the meta-prompt itself by distilling successful editing patterns. To enable scalable training and evaluation, we release a persona-driven IE benchmark of 292 simulated enterprise users, paired with a reproducible persona-generation pipeline grounded in O*NET occupational taxonomies. On this benchmark, Self-Meta-Evolve achieves a 74.58% success rate, outperforming the strongest prompt-optimization baseline by 13.56 absolute points, and reaches 52.54% within only two iterations. A double-blind human study with twenty real professionals further confirms that prompts adapted by our framework win against static baselines in 71% of pairwise comparisons.

Hongliang Li, Lu Wang, Yong Xu +6

Microsoft · Beijing Jiaotong University

Automatic Prompt OptimizationLLM Information ExtractionLLM PersonalizationDocument Information ExtractionLLM Self-Refinement
AskPDF