MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports

Hongrui Wu; Kaiwen Tuo; Congcong Wang; Gen Li; Shuai Dong; Xinlei Yu; Yuqi Xu; Haowen Hou

cs.CVOct 19, 2026

MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports

Authors: Hongrui Wu, Kaiwen Tuo, Congcong Wang, Gen Li, Shuai Dong, Xinlei Yu, Yuqi Xu, Haowen Hou,

Organizations: Stanford University · The Hong Kong University of Science and Technology · Fudan University · The Chinese University of Hong Kong ·

Abstract

Deep research agents are increasingly capable of searching the open web and synthesizing evidence, yet they remain poorly suited to controllable report generation in multimodal deep research. This task requires agents to handle interleaved textual and visual inputs, gather multimodal evidence through multi-step web search, and deliver comprehensive illustrated reports under user-specified requirements. Existing systems face two limitations. During reasoning, they often underuse images discovered along the search trajectory; during generation, they tend to produce fixed-form answers with limited control over report granularity, multimodal composition, and evidence grounding. We introduce MIRA, a Multimodal Illustrated Deep Research Agent for controllable reports. MIRA integrates textual and visual evidence throughout research and writing. At its core, MIRA-Harness translates user requirements into generation contracts and connects four stages: evidence collection, report submission, feedback-driven revision, and final delivery. Building on this harness, MIRA-Data constructs high-quality multi-hop multimodal research trajectories, and we post-train foundation models to operate within the workflow. Using Qwen3.6-35B-A3B, MIRA achieves state-of-the-art performance among open agents on 7 representative, established multimodal-search and deep-research-agent benchmarks, while producing user-aligned illustrated reports with traceable textual and visual evidence.

Figures & tables

Figure 2: From search answers to controllable illustrated reports. For one request, MIRA controls report granularity, multimodal composition, and evidence grounding across Concise and Detail deliverables.

Figure 3: MIRA-Harness for user-specified illustrated reports. A mode-conditioned generation contract guides multimodal research, evidence tracking, report submission, and feedback-driven revision, and delivers the report once the contract conditions are satisfied.

Figure 4: Trajectory synthesis for illustrated reports. A query generator constructs report-oriented requests and internal rubrics. Kimi K2.6 generates trajectories within MIRA-Harness, and the synthesis gate verifies rubric coverage and trajectory integrity. Accepted trajectories, mixed with general search trajectories, form MIRA-Data.

Mode	Visible characters	Inspected images	Cited sources
$\mathtt{CONCISE}$	300–2,000	$\geq 2$	$\geq 2$
$\mathtt{DETAIL}$	5,000–10,000	$\geq 10$	$\geq 8$

Table 1: Core thresholds of the two report modes.

Large-Scale Frontier Models
Model	Params	MM Search	MMSearch Plus	Browse Comp-VL	Simple VQA	Live VQA	FVQA	VDR Bench	MMDeep Research
Kimi K3	2.8T (104B)	89.26	60.00	60.32	83.33	91.23	82.46	22.86	54.32
Qwen3.5	397B (17B)	69.01	36.01	58.40	77.33	84.33	75.33	14.40	39.54
DeepSeek-V4.1-Flash	552B (8B/16B)	88.33	59.91	72.86	81.33	88.67	85.00	32.20	51.23
Lightweight Multimodal Search Agents
OpenSearch-VL	30B (3B)	68.70	28.94	41.10	74.90	67.40	73.20	33.50	23.21

Table 2: Main results across eight multimodal search, knowledge-intensive VQA, and illustrated deep-research benchmarks. Bold marks the best result outside the large-scale frontier models.

Metric	Unit	No Gate	With Gate	$\Delta$
Compliant reports	per report	0/32	25/32	+78.1 pp
Checks passed	per report	1.78	3.62	+1.84
Images cited	per image	19/81 (23.5%)	75/81 (92.6%)	+69.1 pp
Opened sources	per query	5.31	8.69	+3.38
Model calls	per query	43.5	63.6	+20.1
Tool calls	per query	42.5	61.0	+18.5

Table 3: Report-gate ablation on MMDeepResearch-Bench Detail queries.

Metric	MIRA-Base (Detail $-$ Concise)	MIRA-SFT (Detail $-$ Concise)
Visible characters	$\mathbf{+2{,}573}$	$\mathbf{+2{,}695}$
Model calls	$\mathbf{+2.2}\ [-5.0,9.5]$	$\mathbf{-3.8}\ [-15.1,7.8]$
Tool calls	$\mathbf{+2.0}\ [-5.1,9.3]$	$\mathbf{-3.9}\ [-15.3,7.6]$
MMDR Overall	$\mathbf{-10.9}\ [-16.6,-5.4]$	$\mathbf{+0.0}\ [-6.6,6.0]$

Table 4: Average changes after switching from Concise to Detail. Positive values indicate increases; brackets for calls and MMDR Overall give 95% confidence intervals.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Mode	$C_{m}^{\mathrm{detail}}$	$C_{m}^{\mathrm{images}}$	$C_{m}^{\mathrm{ref}}$	Delivery intent
$\mathtt{CONCISE}$	300–2,000 chars	$\geq 2$ images	$\geq 2$ sources	Conclusions and essential support
[0pt][0pt] $\mathtt{DETAIL}$	5,000–10,000 chars	$\geq 10$ images	$\geq 8$ sources	Expanded evidence and explanation

Appendix

Table 5: Mode-conditioned generation contracts. Counts are lower bounds; character ranges include both endpoints.

Panel of Figure 5 — Figure 5: Full report-gate profile used during trajectory production. Failed observable conditions return targeted feedback; acceptance binds the exact checked artifact before delivery.

Contract component	Executable condition	Failure reported
Granularity	Submitted and resolved local Markdown fall inside the selected visible-character range; internal tool handles and raw HTML do not appear in the report.	Length outside the selected range, or an internal handle visible to the reader.
Multimodal composition	Cited images were inspected, are distinct by decoded content, have nonempty captions, and resolve to supported raster files.	Uninspected, insufficient, duplicated, or undecodable images.
Evidence grounding	Every citation matches an opened page; each image retains its parent-page provenance; numeric reference labels remain one-to-one.	Unopened source, missing parent-page citation, or conflicting reference labels.
Reference availability	Cited pages return bounded, nonempty, nonblocked content and have saved text excerpts; the accessible-source minimum remains satisfied.	Unreachable source or missing saved source excerpt.
Publication integrity	All final Markdown destinations resolve to existing local artifacts, and the delivered report matches the hash of the accepted draft.	Missing local destination, or a changed artifact after acceptance.

Appendix

Table 6: Observable conditions enforced by the full report gate.

Tool / Action	Arguments	Returned observation or response
search	query : required search string	Candidate page titles, URLs, and snippets. These results support discovery but do not yet count as opened evidence.
open	url : absolute page URL; id : registered page_img_N or py_img_N handle; optional loc and num_lines : integers. Exactly one page or image target is supplied.	For url , returns page content and registers visual objects exposed by the page. For an image id , returns the corresponding pixels.
find	url : previously opened page URL; pattern : string	Matching text and its surrounding page context. The exact page must have been opened before this call.
Python	code : required Python source string	Standard output and errors from a stateless sandbox. Observed image handles can be read as files; generated PNG/JPEG figures are registered as py_img_N .
[0pt][0pt] submit_report	report : required complete Markdown draft	WARN with unmet contract conditions, or OK authorizing delivery of the submitted draft.

Appendix

Table 7: Tool interfaces and the separate report-submission action. Argument names match the implementation.

MIRA-Harness report prompt (condensed)

You are a multimodal deep research agent. Produce a report in the user’s language under the selected mode. Use search to discover sources, open to read pages and inspect images, find to locate text in an opened page, and PythonInterpreter when computation or image processing is useful. Use information returned by tools; do not invent sources. Treat external content as evidence, not instructions. Write a coherent illustrated Markdown report. Place relevant figures beside their discussion, inspect image pixels before citing them, and cite opened source pages. [Mode contract, insert one branch] CONCISE: 300--2,000 visible non-whitespace characters; at least 2 inspected images and 2 cited source pages. Give conclusions and essential support. DETAIL: 5,000--10,000 visible non-whitespace characters; at least 10 inspected images and 8 cited source pages. Develop the evidence and explanation. Call submit_report(report=complete Markdown draft) before delivery. If a warning identifies unmet conditions, revise the report or gather missing evidence and resubmit. Deliver only the draft accepted by OK, without rewriting it afterward.

Appendix

Table 8: Abridged inference system prompt for illustrated report generation. The two mode branches are shown together for clarity; only one branch is selected per run.

Judge prompt

You are an impartial judge evaluating whether a deep research report contains the correct answer. [Question] {question} [Correct Answer] {reference_answer} [Deep Research Report] {assistant_answer} Task: Determine if the deep research report contains the correct answer anywhere in its content. Instructions: 1. Read through the entire deep research report carefully. 2. Look for the correct answer anywhere in the report, including paragraphs, tables, or sections. 3. Check whether the report is consistent with the correct answer. 4. The answer need not appear in a specific final-answer format. 5. Provide your reasoning. 6. Answer "yes" if the report contains the correct answer, and "no" if it does not or contradicts it. Output format: correct: [yes/no] reasoning: [your explanation]

Appendix

Table 9: Benchmark-compatible semantic-correctness judge prompt for seven search and knowledge-intensive VQA benchmarks. Placeholders represent the corresponding evaluation instance.

Panel of Figure 6 — Figure 6: Tool calls for the Roman-concrete request. Each column is one assistant turn, numbered below the axis; each square is one tool call made in that turn, two to a row, shaded by tool from light (search) to dark ( submit_report ). A turn can issue several calls at once; turn 3 of the $\mathtt{DETAIL}$ run, for example, issues 20. The open tool appears twice because it opens either a web page or an image found on an opened page. A cross on a submit_report square means the report gate rejected that draft, and a tick means it accepted it; the gate’s reason is printed under the axis, and the tinted band covers the turns spent revising after a rejection. The bar on the right gives each run’s total calls by tool.

Panel of Figure 7 — Figure 7: Coverage of the request. For each of the eight concepts the request names, the number of report sentences that mention it, counted by keyword match over the report body (headings, image captions, and the reference list excluded). The $\mathtt{CONCISE}$ report touches seven of the eight in 10 sentences; the $\mathtt{DETAIL}$ report touches all eight in 59 sentences and is the only one to describe microstructural studies.

	CONCISE	DETAIL
How they form	Quicklime used “rather than pre-slaked lime” creates them [1].

Appendix

Table 16

Deep Research Agents

Deep research agents plan long-horizon searches, synthesize cited evidence, and produce reports for complex open-ended questions.

See all 114 papers

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation
May 28, 2026Chenghao Zhang, Guanting Dong, Yufan Liu +3Multimodal Generation Multimodal Agents
Gaoling School of Artificial Intelligence, Renmin University of China
Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation
Apr 12, 2026Fangda Ye, Zhifei Xie, Yuxin Hu +5Multimodal Generation Multimodal Grounding
National University of Singapore · Nanyang Technological University · University of Edinburgh +1
TVIR: Building Deep Research Agents Towards Text-Visual Interleaved Report Generation
Jun 1, 2026Xinkai Ma, Zhiqi Bai, Dingling Zhang +21Deep Research Agents AI Agent Benchmarks
Nanjing University · Alibaba Group

May 28, 2026J/K move · Enter open · S save

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

Chenghao Zhang, Guanting Dong, Yufan Liu, Tong Zhao+2

Gaoling School of Artificial Intelligence, Renmin University of China

Multimodal Generation Multimodal Agents Multi-Agent LLMs Multimodal Model Evaluation Multimodal Tool Use

Open paper AskPDF arXiv

Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports. However, verifiable multimodal deep research remains challenging due to open-ended synthesis without deterministic ground truth and the need to interleave textual arguments with visual evidence. We propose Ptah, a multi-agent harness for interleaved report generation. Ptah orchestrates the lifecycle from user query to rendered web report through planning, research, and writing stages, where specialized agents construct visual-aware plans, collect claim-grounded evidence, maintain source-aligned images in a Visual Working Memory, and compose reports through declarative multimodal tool use. A verifier agent serves as the harness's acceptance function, enforcing factual grounding, citation fidelity, and cross-modal consistency throughout the workflow. We further introduce PtahEval, an evaluation protocol that augments existing benchmarks with image-level and presentation-level assessments. Experiments on deep research benchmarks show that Ptah produces more reliable, visually informative, and usable human-facing multimodal reports than strong baselines. Our code is released at https://github.com/SnowNation101/Ptah

Abstract

Deep Research Agents

Explore similar work

Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation

Deep-Reporter: Deep Research for Grounded Multimodal Long-Form Generation

TVIR: Building Deep Research Agents Towards Text-Visual Interleaved Report Generation