MIRA: A Multimodal Illustrated Deep Research Agent for Controllable Reports
Authors: Hongrui Wu, Kaiwen Tuo, Congcong Wang, Gen Li, Shuai Dong, Xinlei Yu, Yuqi Xu, Haowen Hou, Zelin Peng, Jiaqi Wang
Organizations: Stanford University · The Hong Kong University of Science and Technology · Fudan University · The Chinese University of Hong Kong · Shanghai Jiao Tong University
Deep research agents are increasingly capable of searching the open web and synthesizing evidence, yet they remain poorly suited to controllable report generation in multimodal deep research. This task requires agents to handle interleaved textual and visual inputs, gather multimodal evidence through multi-step web search, and deliver comprehensive illustrated reports under user-specified requirements. Existing systems face two limitations. During reasoning, they often underuse images discovered along the search trajectory; during generation, they tend to produce fixed-form answers with limited control over report granularity, multimodal composition, and evidence grounding. We introduce MIRA, a Multimodal Illustrated Deep Research Agent for controllable reports. MIRA integrates textual and visual evidence throughout research and writing. At its core, MIRA-Harness translates user requirements into generation contracts and connects four stages: evidence collection, report submission, feedback-driven revision, and final delivery. Building on this harness, MIRA-Data constructs high-quality multi-hop multimodal research trajectories, and we post-train foundation models to operate within the workflow. Using Qwen3.6-35B-A3B, MIRA achieves state-of-the-art performance among open agents on 7 representative, established multimodal-search and deep-research-agent benchmarks, while producing user-aligned illustrated reports with traceable textual and visual evidence.
Figures & tables
Figure 2: From search answers to controllable illustrated reports. For one request, MIRA controls report granularity, multimodal composition, and evidence grounding across Concise and Detail deliverables.
Figure 3: MIRA-Harness for user-specified illustrated reports. A mode-conditioned generation contract guides multimodal research, evidence tracking, report submission, and feedback-driven revision, and delivers the report once the contract conditions are satisfied.
Figure 4: Trajectory synthesis for illustrated reports. A query generator constructs report-oriented requests and internal rubrics. Kimi K2.6 generates trajectories within MIRA-Harness, and the synthesis gate verifies rubric coverage and trajectory integrity. Accepted trajectories, mixed with general search trajectories, form MIRA-Data.
Mode
Visible characters
Inspected images
Cited sources
CONCISE
300–2,000
≥2
≥2
DETAIL
5,000–10,000
≥10
≥8
Table 1: Core thresholds of the two report modes.
Model
Params
MM Search
MMSearch Plus
Browse Comp-VL
Simple VQA
Live VQA
FVQA
VDR Bench
MMDeep Research
Large-Scale Frontier Models
Kimi K3
2.8T (104B)
89.26
60.00
60.32
83.33
91.23
82.46
22.86
54.32
Qwen3.5
397B (17B)
69.01
36.01
58.40
77.33
84.33
75.33
14.40
39.54
DeepSeek-V4.1-Flash
552B (8B/16B)
88.33
59.91
72.86
81.33
88.67
85.00
32.20
51.23
Lightweight Multimodal Search Agents
OpenSearch-VL
30B (3B)
68.70
28.94
41.10
74.90
67.40
73.20
33.50
23.21
Table 2: Main results across eight multimodal search, knowledge-intensive VQA, and illustrated deep-research benchmarks. Bold marks the best result outside the large-scale frontier models.
Metric
Unit
No Gate
With Gate
Δ
Compliant reports
per report
0/32
25/32
+78.1 pp
Checks passed
per report
1.78
3.62
+1.84
Images cited
per image
19/81 (23.5%)
75/81 (92.6%)
+69.1 pp
Opened sources
per query
5.31
8.69
+3.38
Model calls
per query
43.5
63.6
+20.1
Tool calls
per query
42.5
61.0
+18.5
Table 3: Report-gate ablation on MMDeepResearch-Bench Detail queries.
Metric
MIRA-Base (Detail − Concise)
MIRA-SFT (Detail − Concise)
Visible characters
+2,573
+2,695
Model calls
+2.2[−5.0,9.5]
−3.8[−15.1,7.8]
Tool calls
+2.0[−5.1,9.3]
−3.9[−15.3,7.6]
MMDR Overall
−10.9[−16.6,−5.4]
+0.0[−6.6,6.0]
Table 4: Average changes after switching from Concise to Detail. Positive values indicate increases; brackets for calls and MMDR Overall give 95% confidence intervals.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Mode
Cmdetail
Cmimages
Cmref
Delivery intent
CONCISE
300–2,000 chars
≥2 images
≥2 sources
Conclusions and essential support
[0pt][0pt] DETAIL
5,000–10,000 chars
≥10 images
≥8 sources
Expanded evidence and explanation
Appendix
Table 5: Mode-conditioned generation contracts. Counts are lower bounds; character ranges include both endpoints.
Figure 5: Full report-gate profile used during trajectory production. Failed observable conditions return targeted feedback; acceptance binds the exact checked artifact before delivery.
Contract component
Executable condition
Failure reported
Granularity
Submitted and resolved local Markdown fall inside the selected visible-character range; internal tool handles and raw HTML do not appear in the report.
Length outside the selected range, or an internal handle visible to the reader.
Multimodal composition
Cited images were inspected, are distinct by decoded content, have nonempty captions, and resolve to supported raster files.
Uninspected, insufficient, duplicated, or undecodable images.
Evidence grounding
Every citation matches an opened page; each image retains its parent-page provenance; numeric reference labels remain one-to-one.
Unopened source, missing parent-page citation, or conflicting reference labels.
Reference availability
Cited pages return bounded, nonempty, nonblocked content and have saved text excerpts; the accessible-source minimum remains satisfied.
Unreachable source or missing saved source excerpt.
Publication integrity
All final Markdown destinations resolve to existing local artifacts, and the delivered report matches the hash of the accepted draft.
Missing local destination, or a changed artifact after acceptance.
Appendix
Table 6: Observable conditions enforced by the full report gate.
Tool / Action
Arguments
Returned observation or response
search
query : required search string
Candidate page titles, URLs, and snippets. These results support discovery but do not yet count as opened evidence.
open
url : absolute page URL; id : registered page_img_N or py_img_N handle; optional loc and num_lines : integers. Exactly one page or image target is supplied.
For url , returns page content and registers visual objects exposed by the page. For an image id , returns the corresponding pixels.
find
url : previously opened page URL; pattern : string
Matching text and its surrounding page context. The exact page must have been opened before this call.
Python
code : required Python source string
Standard output and errors from a stateless sandbox. Observed image handles can be read as files; generated PNG/JPEG figures are registered as py_img_N .
[0pt][0pt] submit_report
report : required complete Markdown draft
WARN with unmet contract conditions, or OK authorizing delivery of the submitted draft.
Appendix
Table 7: Tool interfaces and the separate report-submission action. Argument names match the implementation.
MIRA-Harness report prompt (condensed)
You are a multimodal deep research agent. Produce a report in the user’s language under the selected mode. Use search to discover sources, open to read pages and inspect images, find to locate text in an opened page, and PythonInterpreter when computation or image processing is useful. Use information returned by tools; do not invent sources. Treat external content as evidence, not instructions. Write a coherent illustrated Markdown report. Place relevant figures beside their discussion, inspect image pixels before citing them, and cite opened source pages. [Mode contract, insert one branch] CONCISE: 300--2,000 visible non-whitespace characters; at least 2 inspected images and 2 cited source pages. Give conclusions and essential support. DETAIL: 5,000--10,000 visible non-whitespace characters; at least 10 inspected images and 8 cited source pages. Develop the evidence and explanation. Call submit_report(report=complete Markdown draft) before delivery. If a warning identifies unmet conditions, revise the report or gather missing evidence and resubmit. Deliver only the draft accepted by OK, without rewriting it afterward.
Appendix
Table 8: Abridged inference system prompt for illustrated report generation. The two mode branches are shown together for clarity; only one branch is selected per run.
Judge prompt
You are an impartial judge evaluating whether a deep research report contains the correct answer. [Question] {question} [Correct Answer] {reference_answer} [Deep Research Report] {assistant_answer} Task: Determine if the deep research report contains the correct answer anywhere in its content. Instructions: 1. Read through the entire deep research report carefully. 2. Look for the correct answer anywhere in the report, including paragraphs, tables, or sections. 3. Check whether the report is consistent with the correct answer. 4. The answer need not appear in a specific final-answer format. 5. Provide your reasoning. 6. Answer "yes" if the report contains the correct answer, and "no" if it does not or contradicts it. Output format: correct: [yes/no] reasoning: [your explanation]
Appendix
Table 9: Benchmark-compatible semantic-correctness judge prompt for seven search and knowledge-intensive VQA benchmarks. Placeholders represent the corresponding evaluation instance.
Figure 6: Tool calls for the Roman-concrete request. Each column is one assistant turn, numbered below the axis; each square is one tool call made in that turn, two to a row, shaded by tool from light (search) to dark ( submit_report ). A turn can issue several calls at once; turn 3 of the DETAIL run, for example, issues 20. The open tool appears twice because it opens either a web page or an image found on an opened page. A cross on a submit_report square means the report gate rejected that draft, and a tick means it accepted it; the gate’s reason is printed under the axis, and the tinted band covers the turns spent revising after a rejection. The bar on the right gives each run’s total calls by tool.
Figure 7: Coverage of the request. For each of the eight concepts the request names, the number of report sentences that mention it, counted by keyword match over the report body (headings, image captions, and the reference list excluded). The CONCISE report touches seven of the eight in 10 sentences; the DETAIL report touches all eight in 59 sentences and is the only one to describe microstructural studies.
CONCISE
DETAIL
How they form
Quicklime used “rather than pre-slaked lime” creates them [1].
Large Language Models (LLMs) have advanced autonomous agents from deep search, which retrieves concise factual answers, to deep research, which synthesizes scattered evidence into long-form reports. However, verifiable multimodal deep research remains challenging due to open-ended synthesis without deterministic ground truth and the need to interleave textual arguments with visual evidence. We propose Ptah, a multi-agent harness for interleaved report generation. Ptah orchestrates the lifecycle from user query to rendered web report through planning, research, and writing stages, where specialized agents construct visual-aware plans, collect claim-grounded evidence, maintain source-aligned images in a Visual Working Memory, and compose reports through declarative multimodal tool use. A verifier agent serves as the harness's acceptance function, enforcing factual grounding, citation fidelity, and cross-modal consistency throughout the workflow. We further introduce PtahEval, an evaluation protocol that augments existing benchmarks with image-level and presentation-level assessments. Experiments on deep research benchmarks show that Ptah produces more reliable, visually informative, and usable human-facing multimodal reports than strong baselines. Our code is released at https://github.com/SnowNation101/Ptah
Chenghao Zhang, Guanting Dong, Yufan Liu +3
Gaoling School of Artificial Intelligence, Renmin University of China
Recent agentic search frameworks enable deep research via iterative planning and retrieval, reducing hallucinations and enhancing factual grounding. However, they remain text-centric, overlooking the multimodal evidence that characterizes real-world expert reports. We introduce a pressing task: multimodal long-form generation. Accordingly, we propose Deep-Reporter, a unified agentic framework for grounded multimodal long-form generation. It orchestrates: (i) Agentic Multimodal Search and Filtering to retrieve and filter textual passages and information-dense visuals; (ii) Checklist-Guided Incremental Synthesis to ensure coherent image-text integration and optimal citation placement; and (iii) Recurrent Context Management to balance long-range coherence with local fluency. We develop a rigorous curation pipeline producing 8K high-quality agentic traces for model optimization. We further introduce M2LongBench, a comprehensive testbed comprising 247 research tasks across 9 domains and a stable multimodal sandbox. Extensive experiments demonstrate that long-form multimodal generation is a challenging task, especially in multimodal selection and integration, and effective post-training can bridge the gap.
Fangda Ye, Zhifei Xie, Yuxin Hu +5
National University of Singapore · Nanyang Technological University · University of Edinburgh +1
Deep Research Agents have shown strong capability in multi-step information retrieval, reasoning, and long-form report generation, but existing benchmarks and systems remain predominantly text-centric, with limited evaluation of whether visual elements are factually reliable and well aligned with the surrounding analysis. To address this gap, we introduce TVIR (Text-Visual Interleaved Report Generation), which includes TVIR-Bench, a benchmark of 100 expert-curated multimodal deep research tasks that require visual elements to serve specific analytical sub-goals, and TVIR-Agent, a hierarchical multi-agent framework that serves as a strong baseline for constructing outlines, retrieving images, generating charts with traceable sources, and composing reports through context-aware sequential writing. We further develop a dual-path evaluation framework that combines Textual Assessment and Visual Assessment. Experiments across nine deep research systems show that TVIR-Agent achieves strong overall performance, underscoring the importance of explicit multimodal design and evaluation for evidence-driven report generation.