cs.HCMar 25, 2026
SaveContext-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems
Organizations: Aarhus University, Denmark · Maastricht University, Netherlands · TU Wien, Austria
Abstract
Domain experts possess tacit knowledge that they cannot easily articulate through explicit specifications. When experts modify AI-generated artifacts by correcting terminology, restructuring arguments, and adjusting emphasis, these edits reveal domain understanding that remains latent in traditional prompt-based interactions. Current systems treat such modifications as endpoint corrections rather than as implicit specifications that could reshape subsequent reasoning. We propose context-mediated domain adaptation, a paradigm where user modifications to system-generated artifacts serve as implicit domain specification that reshapes LLM-powered multi-agent reasoning behavior. Through our system Seedentia, a web-based multi-agent framework for sense-making, we demonstrate bidirectional semantic links between generated artifacts and system reasoning. Our approach enables specification bootstrapping where vague initial prompts evolve into precise domain specifications through iterative human-AI collaboration, implicit knowledge transfer through reverse-engineered user edits, and in-context learning where agent behavior adapts based on observed correction patterns. We present results from an evaluation with domain experts who generated and modified research questions from academic papers. Our system extracted 46 domain knowledge entries from user modifications, demonstrating the feasibility of capturing implicit expertise through edit patterns, though the limited sample size constrains conclusions about systematic quality improvements.
Figures & tables
Figure 1. Context-Mediated Domain Adaptation transforms ephemeral user interactions into persistent domain knowledge. As user knowledge and LLM model knowledge deviate we analyze user interaction and edits in order to extract implicit domain knowledge. Through iterative refinement our approach expands the shared context substantially, capturing domain-specific terminology, conventions, and patterns. This accumulated knowledge persists in an LLM-agnostic format, enabling system improvements across sessions and participants while maintaining compatibility with different language models.
| Knowledge Category | Description & Characteristics | Example Knowledge Patterns |
| Domain Terminology Evolution | Systematic vocabulary preferences and specialized language usage patterns derived from user corrections to technical terms and domain-specific expressions | Replacing “chart” with “visualization,” preferring “participants” over “users,” adopting field-specific acronyms and technical terminology |
| Methodological Refinements | Improvements to research approaches, analytical frameworks, and domain-specific practices reflecting expert knowledge of proper methodologies | Specifying statistical analysis requirements, adding ethical considerations, refining experimental design elements, emphasizing reproducibility standards |
| Conceptual Depth Changes | Theoretical nuances, conceptual relationships, and domain-specific considerations that add scholarly depth beyond surface-level content | Adding theoretical frameworks, clarifying causal relationships, introducing domain-specific constraints, connecting concepts to established literature |
Table 1. Knowledge categories extracted from user edits. The system identifies three distinct types of domain knowledge from user modifications to generated artifacts. Domain terminology evolution captures vocabulary preferences through direct text corrections. Methodological refinements encode expert knowledge about research practices and analytical frameworks. Conceptual depth changes add theoretical nuance and scholarly connections that distinguish expert from novice discourse.
Figure 2. Interaction modalities. Implementation of interaction modes defined in our Context-Mediated Domain Adaptation framework. These interfaces demonstrate how user modifications are captured and transformed into domain knowledge through bidirectional semantic links, enabling the system to learn from expert corrections and improve subsequent artifact generation. A side-by-side comparison of two interface screenshots. The left panel (a) shows "Direct Manipulation Mode" with a research question interface displaying "Research Question 1" at the top, followed by three sections: "Research Question" with editable text about Chart-of-Thought prompting strategy, "Contribution Summary" with a paragraph of text, and "Edit" buttons in the top right of each section. A red border highlights the editable content area. The right panel (b) shows "Prompt-based Regeneration Mode" with an identical layout, but includes a highlighted yellow section indicating content that will be regenerated based on user prompts. Both interfaces show quality indicators and word counts (25 words) in the top right corner. The screenshots demonstrate different interaction approaches for modifying AI-generated research questions.
Figure 3. Prompt-based generation. Complete workflow for prompt-based artifact regeneration showing the input dialog for natural language instructions and the asynchronous generation process. The interface maintains application responsiveness during AI processing, demonstrating the fire-and-forget architecture that decouples user interactions from computational workloads. A side-by-side comparison showing the prompt-based generation workflow. The left panel (a) displays a modal dialog box titled "Prompt based regeneration" overlaying a darkened research questions interface. The dialog contains explanatory text about how the system will regenerate content, a text input field labeled "Regeneration Prompt" with example prompts, and two buttons at the bottom: "Cancel" (gray) and "Regenerate" (blue). The right panel (b) shows the same interface during regeneration, with Research Question 1 displaying a spinner icon and grayed-out text indicating it is being regenerated, while Research Question 2 below remains fully visible and interactive with normal contrast. Both panels show the "Research Questions" header with "1 of 2 modified" status in the top right.
Figure 4. Edit history visualization. The AIContentWrapper component provides integrated edit history functionality that powers context-mediated domain adaptation. A side-by-side comparison of edit history interfaces. The left panel (a) shows an AIContentWrapper component with a white background displaying the title "Collective Meaning-Making and Knowledge Building through AI" at the top, followed by three light blue boxes containing action items: "Generate Visualization" for Augmented Collective Meaning-Making Publication Network, "Generate Visualization" for Impact of AI on Collaborative Knowledge Construction, and "Create Visualization" to define a custom visualization. A "Key Insights" section appears below with explanatory text. In the top right corner is a "History" badge button with a clock icon. The right panel (b) shows a modal dialog titled "Edit History - Collective Meaning-Making and Knowledge Building through AI" overlaying a darkened version of the same interface. The dialog displays a chronological timeline with two entries labeled "Item edit - Interview" from 1 minute ago, each showing a diff view with green highlighted additions and red highlighted deletions in the text. The dialog includes tabs for "Interview" and "Title" at the top.
| Table | Purpose |
| agent_tasks | Central coordination for asynchronous AI processing with status tracking via status , input_data , and output_data |
| task_type | Defines available AI workflows (research question generation, knowledge extraction, context injection) using code and label |
| task_action | Individual processing steps within multi-agent workflows for debugging using attempts and error_message |
| action_type | Categorizes processing actions (extraction, analysis, generation) for workflow orchestration |
| api_logs | External API interactions for literature retrieval tracking search_terms and papers_found |
| project_agent_log | Real-time logging of agent processing steps using log_type and message for monitoring and debugging |
Table 2. Database tables supporting asynchronous multi-agent task processing. The system uses six interconnected tables to coordinate AI agent workflows, track execution status, and log processing details. The agent_tasks table serves as the central coordinator, while supporting tables define workflow types, decompose tasks into atomic actions, categorize processing steps, record external API calls, and maintain real-time execution logs for debugging and monitoring.
Figure 5. Agentic task processing graph. The backend workflow graph is centered on the planner router node, which conditionally dispatches tasks to specialized nodes for paper retrieval, context-based research question generation, and edit-driven knowledge extraction. Node outputs are merged back into a unified state and persisted via the agent tasks infrastructure, enabling asynchronous execution while maintaining traceable bidirectional links between user edits, extracted domain insights, and subsequent generations. Visualization of the backend workflow graph centered on the
planner router node, which conditionally dispatches tasks to specialized nodes for paper retrieval (fetch_paper_content), context-based research question generation (generate_evaluation_questions), and edit-driven knowledge extraction (extract_implicit_knowledge). Node outputs are merged back into a unified state and persisted via the agent_tasks infrastructure, enabling asynchronous execution while maintaining traceable bidirectional links between user edits, extracted domain insights, and subsequent generations.| Table | Purpose |
| evaluation_research_questions | Implements Bidirectional Domain-Adaptive Representation by storing initial/final states for knowledge extraction using initial_question , current_question , and edit distance metrics |
| implicit_domain_knowledge | Materializes Adaptive Context Object with categorized domain insights and provenance tracking through knowledge_category and source_question_ids |
| ai_entity_edits | Granular interaction tracking for behavioral analysis capturing edit_type , original_value , and user_prompt |
| ai_entity_metadata | Generation context preservation for bidirectional semantic links using generation_prompt and model_parameters |
| evaluation_sessions | Session management with comprehensive metrics including edit_distance_score and LLM monitoring via langfuse_trace_id |
| evaluation_participants | Participant management with domain_expertise profiling and evaluation_status tracking |
Table 3. Database tables for knowledge extraction and evaluation tracking. Eight tables implement the core knowledge representation infrastructure for context-mediated domain adaptation. The evaluation_research_questions table stores initial and final artifact states to enable knowledge extraction from user modifications. The implicit_domain_knowledge table materializes extracted domain expertise with category labels and provenance tracking. Supporting tables capture granular edit operations, preserve generation context for bidirectional semantic links, manage evaluation sessions with comprehensive metrics, profile participant expertise, store research papers for AI processing, and log detailed UI interactions for behavioral analysis.
Figure 6. Context-mediated domain adaptation workflow tracing. Langfuse tracing demonstrates how the bidirectional learning cycle operates: user modifications flow through the extract_implicit_knowledge node (shown processing three user interactions), with extracted knowledge subsequently injected into the generate_evaluation_questions node’s system prompt. Notice how the interface makes the complete knowledge transfer visible, from hierarchical execution flow (center) to detailed prompts and performance metrics (right), enabling validation of our CMDA framework’s core claim that user edits systematically enhance AI reasoning. A screenshot of the Langfuse tracing interface showing multi-agent workflow execution. The interface has three main sections. The left panel displays a hierarchical trace list with timestamps, showing alternating entries for "user_interaction" and "LangGraph" operations from 2025-10-23. Each LangGraph entry shows JSON input data. The center panel shows a tree structure with performance metrics: "LangGraph" at 5.47s, nested "LangGraph" at 5.47s, "planner" at 0.00s, "extract_implicit_knowledge" at 0.08s, and "ChatGoogleGenerativeAI" at 1.60s with token counts (932 prompt + 137 completion, 1,069 total). Below this is a workflow graph showing connected nodes: "start" at top, flowing to "planner", which branches to "generate_evaluation_questions", "extract_implicit_knowledge", and "end". The right panel shows a detailed view with header information (trace ID, timestamp 2025-10-23 16:07:31.483), model configuration (gemini-2.0-flash-lite, temperature 0.7), and two content sections. The "Preview" tab displays the User prompt describing a visualization literacy research expert task, followed by "ORIGINAL AI-GENERATED CONTENT" and "FINAL USER-EDITED CONTENT" sections showing research questions and contributions about visualization tasks and literacy levels. An "EXISTING KNOWLEDGE" section lists conceptual depth changes about strategic focus shifts in visualization research.
Figure 7. Evaluation protocol interface. Key components of the controlled study we conducted for assessing context-mediated domain adaptation effectiveness. The system captures baseline quality assessments and provides standardized interaction protocols to ensure consistent evaluation of bidirectional learning mechanisms across participants and sessions. A side-by-side view of two evaluation protocol interfaces. The left panel (a) shows the evaluation session initialization screen with a header "Research Paper" displaying the title "Chart-of-Thought: Enhancing LLM Visualization Literacy Through Structured Data Extraction" with a green "FULLY ASSESSED" badge, author list, and affiliation. Below is an "Abstract" section with paper summary text, followed by an "Access Full Paper" section with a "View PDF" button. At the bottom is a "Research Question Refinement Instructions" box with purple header, containing "Your Mission" text explaining the participant's task and two instruction boxes: a blue "Direct Tool Edit" box stating "For questions that are not right track well (most major adjustments), freely clear out edit the text directly" and a pink "AI-Assisted Regeneration" box stating "For questions that aren't quite there (smaller approach or tone tweaks/range edits) guide the AI via prompt-specific questions." The right panel (b) shows the "Research Questions" interface with "1 of 3 Questions" header and a "Generate More" button. It displays "Research Question 1" with a 5-star quality rating, followed by the research question text about Chart-of-Thought prompting strategy and a "Contribution Summary" section. Below is "Research Question 2" marked as "DELETED" with similar structure showing a question about cognitive mechanisms and structured prompting.
| Part. | Paper | Duration minutes | Quality Rating 1=poor, 5=excellent | Direct Edits total sum | Prompt Edits total sum |
| P1 | Tell Me Without Telling Me | 9.0 | 2.67 | 4 | 0 |
| DRIVE-T Methodology | 5.1 | 3.0 | 1 | 0 | |
| Charts-of-Thought | 4.8 | 3.0 | 1 | 0 | |
| P2 | Tell Me Without Telling Me | 32.5 | 2.33 | 5 | 1 |
| DRIVE-T Methodology | 18.8 | 3.67 | 3 | 1 | |
| Charts-of-Thought | 14.2 | 2.67 | 3 | 1 |
Table 4. Participation data showing temporal patterns in domain adaptation. Key insight: Quality ratings improve across sequential participants (P1: 2.67-3.0 → P5: 4.0-4.33), demonstrating progressive knowledge accumulation. Duration (minutes), quality ratings (1=poor to 5=excellent), and edit counts reveal how different participants engaged with the system.
| Part. | Paper | Edit Distance Q=Question, C=Contribution | Edited fields amount per paper |
| P1 | Tell Me Without Telling Me | Q: 71 - C: 22 | 6 |
| DRIVE-T Methodology | Q: 19 - C: 0 | 4 | |
| Charts-of-Thought | Q: 50 - C: 0 | 4 | |
| P2 | Tell Me Without Telling Me | Q: 103 - C: 121 | 3 |
| DRIVE-T Methodology | Q: 82 - C: 63 | 1 | |
| Charts-of-Thought | Q: 83 - C: 74 | 1 |
Table 5. Character-level edit distances and field modifications by participant. Edit distances quantify character-level changes between initial AI generation and final versions across research questions (Q) and contribution summaries (C). Notice how P3’s extensive modifications (2893 total characters) coupled with high quality ratings suggest substantive refinements rather than surface corrections, while P4’s minimal edits reflect satisfaction with generation quality.
| Part. | Edited fields total amount | Conceptual Depth Changes | Domain Terminology Evolution | Methodological Refinements | Total |
| P1 | 14 | 2 | 3 | 1 | 6 |
| P2 | 5 | 9 | 2 | 5 | 16 |
| P3 | 15 | 12 | 5 | 3 | 20 |
| P4 | 3 | 2 | 0 | 1 | 3 |
| P5 | 2 | 1 | 0 | 0 | 1 |
| 39 | 26 | 10 | 10 | 46 |
Table 6. Distribution of extracted knowledge entries by participant and category. Notice how P3 contributed the most knowledge entries (n=20) despite benefiting from accumulated knowledge, demonstrating that the system enables rather than replaces expert contribution. The uneven distribution (P1: 6, P2: 16, P3: 20, P4: 3, P5: 1) reflects varying engagement levels and expertise expression styles.
More in this topic
Human-AI Interaction
Human-AI interaction studies how people and AI systems communicate, collaborate, make decisions, and shape one another over time.
See all 328 papers