Raven: The Harness of Harnesses for Composable Agentic Intelligence

EverMind AI

cs.AISep 27, 2026

Raven: The Harness of Harnesses for Composable Agentic Intelligence

Authors: EverMind AI

Organizations: EverMind AI

Abstract

As large language models advance, AI agents are moving beyond isolated, domain-specific tasks toward long-horizon, cross-domain workflows. This transition exposes two challenges: increasing harness complexity makes manual design difficult to scale, while tighter coupling to specific domains limits the generality of a single harness. The central question thus shifts from how to engineer a stronger harness for one domain to how to autonomously construct specialized harnesses, improve them through experience, and orchestrate them across domains. We introduce Raven, \emph{The Harness of Harnesses}, an open-source multi-agent ecosystem that automatically constructs and evolves modular harnesses for specific models and domains, treating each executable model--harness pair as a composable unit of intelligence. To support an \emph{All-Domain Collaboration Network}, its Host Agent decomposes goals, matches subtasks to specialized agents, coordinates execution dependencies, and integrates results, while a host archive and EverOS preserve experience across tasks and Skill Forge makes that experience available as reusable procedures. Our theory establishes sufficient conditions for such composition to expand reliable task coverage beyond that of the available individual agents under a shared resource budget. On complex and long-horizon tasks, Raven significantly outperforms the state-of-the-art agent systems, pushing the frontier of composable agentic intelligence.

Figures & tables

Figure 2: Raven As An Open Multi-Agent Ecosystem. (a) Built-in and third-party agents expose executable model–harness pairs through execution adapters. The Host Agent uses registry capabilities to compose and assign tasks, while the runtime validates the plan, schedules ready nodes, and records artifacts for handoff and reuse. (b) An illustrative FPS game project combines gameplay development and visual design before integration, testing, and operation. Nodes denote agent calls, and directed edges denote artifact dependencies coordinated by Raven. Persistent context, reusable skills, and harness evolution support adaptation across tasks.

Figure 3: Harness composition. (a) A model and its harness form a callable execution unit. (b) A host assigns and coordinates these units. Solid arrows carry artifacts and dashed arrows denote control. The research–code–design chain is an illustrative plan with explicit final synthesis by a specialist. All operations share one budget. Model labels need not identify different foundation models.

Figure 4: Graph planning and admission. (a) The host context carries only a summary of the orchestration guide. The host loads the full guide when a request needs several agents and delegates directly otherwise. A rejected graph returns its first error with a pointer back to the guide. In the admitted graph, R1 and R2 are the running example’s research nodes

v_{\mathrm{alg}}

and

v_{\mathrm{bench}}

, C is

v_{\mathrm{code}}

, and D is

v_{\mathrm{design}}

. (b) A submission is a flat node list with graph-level flags. Five groups of checks run in order before any worker is dispatched ( Table 1 ).

Group	Checks
Format	Arguments parse as complete JSON. Fields have the required types and values. No unknown field is present. Identifiers and instance handles match their patterns. Required node fields are nonblank.
Graph structure	Identifiers are unique within the graph and unused in the session. Dependencies name a node of the graph or a completed earlier node with output. Every output placeholder is backed by a dependency. Input values are well formed, and declared and referenced inputs agree. Path forms apply only to file or node inputs. File and reference paths stay within the permitted roots. The graph is acyclic.
Agent capability	Shared instances use stateful agents. Skill overrides appear only at the head of an instance chain. Path forms are not sent to agents without local-file access.
Agent status	Each named agent is registered and enabled.
Environment	The call does not come from inside a sub-agent run. Delegation is not paused, and the dispatch budget is available. The user approves the graph when confirmation is requested.

Table 1: Admission checks applied before dispatch, grouped as in Figure 4 b. Validation returns the first failure, followed by a pointer to the orchestration guide.

Figure 5: Node execution and host intervention. (a) A node runs once all predecessors have completed and settled. Returned output and raised errors both reach the judge. A failed verdict suspends the node for a host decision when continuations remain. Memory recording runs asynchronously and does not delay successors. (b) Top: the judge reads the rendered prompt, output, and transcript tail, and a negative verdict becomes an exception report for the host. Bottom: a worker’s clarification request is answered from the host’s conversation and memory when possible, and otherwise reaches the user.

Figure 6: Artifact and memory flow. (a) A node’s prompt is rendered from its template, upstream outputs, upstream memory paths, and files. The runtime writes the prompt, output, and transcript to disk, and a memory record follows asynchronously. Successors read these files on demand. (b) The host receives a run report with paths and terminal outputs. Completed nodes enter a session-wide registry, so a later graph can reference their outputs and a single delegation can continue their instance. In (b), R3 and R4 are research nodes of a follow-up graph whose slide-deck node P references earlier outputs, and the spawn call continues the coding instance C.

Figure 7: Shared group memory. (a) The host mediates access to owner-partitioned EverOS libraries. It retrieves experience before planning, deposits round summaries and outcomes in its shared library, and writes agent assessments to the corresponding agent libraries. Dispatch prefetch combines shared experience with recent assessments from a provenance ledger, with respective caps

k_{\rm group}

and

k_{\rm verdict}

. Personal user memory is excluded, while workers retain their ordinary memory access. Offset outlines indicate one library and worker per mapped agent. (b) Assessment has two phases: a close verdict after execution and a feedback verdict on the next user turn. Attribution is extracted from the host’s reply and omitted from the user-visible response and session log.

Actor	$\mathsf{L}_{\mathrm{host}}$	$\mathsf{L}_{\mathrm{user}}$	Agent Libraries
Host	read, write	none	read and write every mapped library
Mapped worker	read via prefetch	none	its own verdicts via prefetch
Unmapped agent	none	none	none

Table 2: Access within the group memory layer. The host performs all retrievals and deposits. Ordinary host and worker memory calls are outside this policy.

Figure 8: Harness self-evolution. (a) Four strategy interfaces expose decisions around a frozen model. The shared execution shell handles dispatch and accounting. (b) The best training-evaluated harness supplies failure traces for diagnosis and candidate generation. The bank stores one complete harness per edit-category/pathology cell. P/K/R/C denote prompt, knowledge, runtime, and configuration edits. The first

N_{\rm keep}

candidates passing validity, activation, and paired-gain screening receive full-training evaluation before cell competition, where

N_{\rm keep}

is the per-round survivor cap. The final training-selected harness is frozen before held-out testing.

Figure 9: Memory formation and consolidation in EverOS. (a) Semantic boundaries group dialogue and tool interactions into a source segment

\mathsf{seg}_{i}

. Its episode

\mathsf{ep}_{i}

anchors atomic facts. The dashed branch derives optional foresight with validity interval

[t_{p,0},t_{p,1}]

. (b) Across interaction history

\mathsf{Hist}

, episodes, shown as points, form MemScenes around centroids marked by crosses. These scenes select evidence for profile updates. The agent track consolidates execution cases

c_{1},c_{2},c_{3}

into reusable procedures, depicted by a small dependency graph. The diagram shows representation and data dependencies. Persistence and indexing proceed separately.

Figure 10: Raven’s user-memory retrieval through EverOS hybrid . (a) BM25 and dense search retrieve episodes. Reciprocal rank fusion (RRF) determines their expansion order. A separate calibrated score ranks them for final selection. (b) The query scores facts belonging to each expanded episode. Episodes and facts compete within a bounded candidate set. A selected fact is returned under its parent episode, which restores its context. The positive integer

k_{\rm mem}

is the backend candidate-set cap.

Figure 11: Skill selection and experience-driven updates. (a) Ranked source hits are grouped by name and fused before body retrieval, runtime policy checks, and LLM selection. The normal gate selects a bounded list, while a gate failure uses a separate ranked fallback. Resource resolution is attempted after selection. (b) A case enters an intent cluster and conditions skill operations together with existing skills and a bounded set of supporting cases. Persisted revisions become candidates for later EverOS retrieval after indexing. The positive integer

k_{\rm pool}

is the fused skill-pool cap. Numeric labels show the implementation defaults in Section B.3 .

Domains per task	Subsets covered	Tasks
2	6 / 6	66
3	4 / 4	47
4	1 / 1	27
Total	11 / 11	140

Table 3: Coverage of domain combinations. Every subset containing at least two domains from the four-domain roster is represented.

Figure 12: MAOB composition. (a) Number of domains per task and the share of tasks whose reference graph contains parallel branches. (b) Number of tasks that use each specialist domain.

Figure 13: Multi-Agent Orchestration Benchmark results with backbone (a) Qwen3.8-27B and (b) DeepSeek-V4-Flash-0731 .

Figure 14: Held-out Pass@1 reported in HarnessBank [ 39 ] for the initial and evolved harnesses with a frozen Qwen3.6-27B backbone. Gray denotes SWE-bench Verified. Table 11 lists the values.

Qwen3.6-35B-A3B
System	Accuracy	Input (M)	Output (k)	Cost (USD)
MiroFlow	47.7	1.493	11.3	–
DeepSeek-Harness	49.7	4.587	16.2	–
Raven-Research	56.3	1.706	30.0	–
Qwen3.5-397B-A17B
MiroFlow	48.0	1.799	15.3	–

Table 4: DeepResearch Mixed results. Accuracy (%) is pooled over all questions. Input tokens (including cache reads), output tokens, and cost are means per question. A dash marks a quantity that was not measured. Bold marks the best accuracy within each shared-backbone group.

Figure 15: Raven-Research results on DeepResearch Mixed. (a) Accuracy per source under DeepSeek-V4-Flash . (b) Paired accuracy gain of Raven-Research over each comparison on the same backbone, with stratified bootstrap 95% intervals over all questions. The hollow marker denotes the comparison with DeepSeek-Harness on Qwen3.5-397B-A17B , with exact McNemar

p=0.21

. (c) Accuracy against mean cost per question, with filled markers for the DeepSeek-V4-Flash harnesses and hollow markers for the commercial services, whose costs follow each provider’s own accounting.

Figure 16: Coding results. Each row is one system, grouped by benchmark and backbone. SWE-bench reports the resolved rate, WorkBuddy-Code the reward, and SWE-Refactor the mean task score. Hollow markers are public leaderboard values.

Figure 17: DataAgentBench Pass@1 on August 24, 2026, with the horizontal axis starting at 0.76. The comparison entries are the public leaderboard values on that date, and the right column gives their models.

Figure 18: PresentBench overall scores (0–100). Gold bars are Raven-Design and dark bars Claude Code, with the backbone in parentheses. Light bars are public leaderboard entries evaluated by the benchmark authors, whose model configurations are not given.

Figure 19: Visual-design scores (0–100) under (a) GPT-5.6 Luna and (b) Claude Opus 5 . Scores come from an internal grader with GPT-5.6 Luna as the judge and are not comparable with the official leaderboards.

Panel of Figure 20 — Figure 20: Deliverables of the Raven-Design case studies. Panels (a)–(d) show the cover, twelve interior slides, and the closing slide of each deck. Panel (e) shows the complete board, which compares six frameworks on four dimensions. Panel (f) shows the key visual above its four derived formats, and panel (g) shows the live page above the position fix with two, three, and four satellites. Table 5 gives the task graph of each run.

Case	Deliverable	Task Graph	Design Time (min:s)
(a) Song dynasty	Deck (Chinese)	$3\mathrm{R}\to\mathrm{R}\to\mathrm{D}$	18:45
(b) Greek polychromy	Deck (English)	$3\mathrm{R}\to\mathrm{R}\to\mathrm{D}$	55:34
(c) Pop music	Deck (English)	$3\mathrm{R}\to\mathrm{R}\to\mathrm{D}$	47:03
(d) Abstract art	Deck (English)	$3\mathrm{R}\to\mathrm{R}\to\mathrm{D}$	64:50
(e) Orchestration frameworks	Comparison board	$2\mathrm{R}\to\mathrm{D}$	6:57
(f) Light pollution	Poster series	$[(3\mathrm{R}\to\mathrm{C})\parallel\mathrm{D}]\to\mathrm{D}$	2:58, 13:57

Table 5: Task graphs of the case studies in Figure 20 . R, C, O, and D denote Raven-Research, Raven-Code, Raven-Oncall, and Raven-Design nodes, a numeral counts parallel nodes of one kind,

\to

marks a dependency, and

\parallel

marks branches that run concurrently. Design time is the wall-clock time of each Raven-Design node in the task graph, with the image-plate node first where a graph has two.

Figure 21: AI4AI nanochat pretraining. (a) Final validation BPB of each campaign, starting from 1.108 (dashed line). (b) Runtime, reported tokens, and USD cost. Lower is better throughout. The lighter bars use the DeepSeek backbone.

Figure 22: AI4S internal benchmark. Success rate on the 17 tasks, and runtime, tokens, and USD cost averaged per task. The lighter bars use the DeepSeek backbone.

Figure 23: Effect of the curated skill library reported by SkillCorpus [ 64 ] . Each arrow runs from the no-skill baseline (hollow marker) to the condition with SkillCorpus skills for one harness–backbone cell. Values are three-run means. Horizontal axes start at different values. Table 12 lists the values.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$a_{i},M_{i},h_{i}$	Execution policy, underlying model, and harness.
$\mathcal{H}_{i},\mathcal{U}_{i},\Delta$	Permitted histories, actions, and probability distributions.
$H,a_{0},\mathcal{A}$	Host controller, its nondelegating executor, and the pool including $a_{0}$ .
$S,S_{H}$	A generic system, either the composed system or a standalone agent, and the composed system $H[\mathcal{A}]$ .
$t,r_{t},\mu_{t},\mathsf{E}_{t},\mathsf{V}_{t}$	Task, request, initial law, response kernel, and objective correctness.
$\omega,\operatorname{ref},s,\ell,Y$	Reference conditions, their read-only state projection, full state, immutable-record ledger, and final record.

Appendix

Table 6: Principal notation. States and correctness predicates are analytical objects. An executor receives only its permitted observations.

Symbol	Meaning
$G_{\rm rt},G_{\rm ann}$	Submitted graph and its analytical expansion.
$\mathsf{Tpl}_{v},\operatorname{Dep}(v),\operatorname{Bind}_{v}$	Prompt template, dependency identifiers, and slot bindings.
$\operatorname{Cfg}_{v},\mathrm{inst}_{v}$	Invocation capability overrides and stateful-instance handle.
$j_{\rm rt},z_{j_{\rm rt}}(v)$	Scheduler-event index and node status at that event.
$x_{v}^{\rm task},x_{v}^{\rm base},x_{v}$	Rendered task, task plus memory paths, and final dispatched input.
$\mathsf{Paths}_{v},\operatorname{MemPath}(u)$	Advertised ancestor-memory paths and a node’s reserved memory-file location.

Appendix

Table 7: Runtime and Memory Notation.

Symbol	Meaning
$\mathrm{id}_{h},\operatorname{Exec}(h)$	Candidate identity and executable components.
$K_{\rm att},N_{\rm keep},R_{\rm round}$	Attempts per task, survivor cap, and round cap.
$\mathsf{cell},\operatorname{desc}(h)$	Archive cell and the candidate’s assigned descriptor.
$\Pi_{\rm eval},\lambda_{\rm screen}$	Fixed evaluation protocol and paired-gain screening threshold.
$\mathcal{X}_{\rm edit},\mathcal{K}_{\rm fixed}$	Editable and protected harness locations.
$\Delta^{\rm patch}$	Executable harness patch.

Appendix

Table 8: Harness Evolution and Skill Notation.

Benchmark	Initial	Evolved	Gain
Terminal-Bench-2	36.1	45.4	+9.3
LiveCodeBench	58.1	71.8	+13.7
Omni-MATH	54.3	66.0	+11.7
BrowseComp+	16.9	30.8	+13.9
GDPval	43.7	52.9	+9.2
AppWorld	41.3	56.7	+15.4

Appendix

Table 11: Held-out Pass@1 (%) before and after harness self-evolution with frozen Qwen3.6-27B . Scores and percentage-point gains are reproduced as reported by HarnessBank.

	SkillsBench			GDPval			QwenClawBench
Cell	None	Skills	Gain	None	Skills	Gain	None	Skills	Gain
Raven $\times$ Qwen3.5-27B	10.0	16.5	+6.5	82.6	83.8	+1.2	66.9	70.8	+3.9
Raven $\times$ Qwen3.5-397B-A17B	9.2	22.6	+13.4	84.0	85.2	+1.2	68.8	73.2	+4.4
OpenClaw $\times$ Qwen3.5-27B	8.8	13.0	+4.2	81.2	83.1	+1.9	65.2	66.7	+1.5
OpenClaw $\times$ Qwen3.5-397B-A17B	11.1	16.9	+5.8	82.2	84.0	+1.8	65.7	67.0	+1.3
Pooled gain	$+7.5\pm 2.3$			$+1.51\pm 0.49$			$+2.79\pm 0.70$

Appendix

Table 12: Effect of SkillCorpus skills on each harness–backbone cell. Each benchmark gives the three-run mean without skills (None), with skills (Skills), and their difference in points. SkillsBench reports Pass@1 (%), and GDPval and QwenClawBench report mean rewards multiplied by 100. The last two rows give the task-clustered pooled gain with one standard error and its

z

-score.

Catalog	Retrieval Stack	Pass@1	Gain
No skills		9.2	–
Curated	Fine-tuned	22.6	+13.4
Curated	Off-the-shelf Qwen3	13.8	+4.6
Raw crawl	Fine-tuned	14.9	+5.7

Appendix

Table 13: Single-run ablation on Raven

\times

Qwen3.5-397B-A17B , SkillsBench Pass@1 (%) over 87 tasks. Each ablation replaces one component of the full pipeline, and the gain is measured against the no-skill baseline.

Explore similar work

Apr 22, 2026cs.AI

The Last Harness You'll Ever Build

AI agents are increasingly deployed on complex, domain-specific workflows -- navigating enterprise web applications that require dozens of clicks and form fills, orchestrating multi-step research pipelines that span search, extraction, and synthesis, automating code review across unfamiliar repositories, and handling customer escalations that demand nuanced domain knowledge. \textbf{Each new task domain requires painstaking, expert-driven harness engineering}: designing the prompts, tools, orchestration logic, and evaluation criteria that make a foundation model effective. We present a two-level framework that automates this process. At the first level, the \textbf{Harness Evolution Loop} optimizes a worker agent's harness

\mathcal{H}

for a single task: a Worker Agent

W_{\mathcal{H}}

executes the task, an Evaluator Agent

V

adversarially diagnoses failures and scores performance, and an Evolution Agent

E

modifies the harness based on the full history of prior attempts. At the second level, the \textbf{Meta-Evolution Loop} optimizes the evolution blueprint

Λ= (W_{\mathcal{H}}, \mathcal{H}^{(0)}, V, E)

itself across diverse tasks, \textbf{learning a blueprint

Λ^{(\text{best})}

that enables rapid harness convergence on any new task -- so that adapting an agent to a novel domain requires no human harness engineering at all.} We formalize the correspondence to meta-learning and present both algorithms. The framework \textbf{shifts manual harness engineering into automated harness engineering}, and takes one step further -- \textbf{automating the design of the automation itself}.

Haebin Seong, Li Yin, Haoran Zhang +1

Evolving Harness Self-Evolving Agents

Ask PDF

Jun 12, 2026cs.AI

HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundry

AI agent performance depends critically on the runtime harness, comprising the prompts, tools, memory, and control flow that mediate how a model observes, reasons, and acts. Yet today's harnesses remain largely hand-crafted and static: each new model or task still demands bespoke scaffolding, and the rich traces produced during execution are rarely distilled back into systematic improvement. We introduce HarnessX, a foundry for composable, adaptive, and evolvable agent harnesses. HarnessX assembles typed harness primitives via a substitution algebra, adapts them through AEGIS, a trace-driven multi-agent evolution engine grounded in an operational mirror between symbolic adaptation and reinforcement learning, and closes the harness-model loop by turning trajectories into both harness updates and model training signal. Across five benchmarks (ALFWorld, GAIA, WebShop, tau^3-Bench, and SWE-bench Verified), HarnessX yields an average gain of +14.5% (up to +44.0%), with gains largest where baselines are lowest. These results suggest that agent progress need not come from model scaling alone: composing and evolving runtime interfaces from execution feedback is an actionable and complementary lever. The complete codebase will be open-sourced in a future release.

Tingyang Chen, Shuo Lu, Kang Zhao +11

Darwin Agent Team

Agent Harness Multi-Agent Evolution

Ask PDF

Aug 3, 2026cs.LG

HarnessCompass: Guiding Automatic Harness Evolution toward Generalizable and Effective Agent Harnesses

Harness design plays a critical role in agent performance by shaping how large language models (LLMs) perceive, reason over, and act within executable environments. Recent work has proposed automatic harness evolution, which iteratively improves the harness from agent--environment interactions. However, existing methods often overfit to the evolution tasks, rely exclusively on trajectory-derived signals, and optimize harness components jointly, causing interference across components. We propose HarnessCompass, a novel automatic harness evolution framework built around constrained evolution, proactive feedback, and component-wise optimization. HarnessCompass first enforces global constraints on evolution, restricting modifications to task-agnostic harness changes that generalize beyond the evolution tasks. It then augments trajectory-derived evidence with proactive first-person feedback from the agent about harness usage, yielding richer signals for evolution. Finally, it decouples the optimization of different harness components before consolidating them into a unified harness, reducing cross-component interference while preserving component synergy. On SWE-bench Verified with GPT-5.4, HarnessCompass improves Pass@1 from 54% to 66% in only 5 evolution iterations, outperforming AHE in both effectiveness and evolution efficiency. In addition, the evolved harness transfers effectively to held-out tasks and other models, demonstrating substantially stronger generalization than prior automatic harness evolution methods.

Luan Zhang, Ruochen Zhou, Dandan Song +9

Beijing Institute of Technology, China · City University of Hong Kong, China · Independent, China

Evolving Harness Agent Harness Evolutionary Optimization Methods

Ask PDF