TOMORI: A Tool-Orchestrated Mixing Operator for REAPER Interaction

Yining Wang; Tianjiao Li

cs.SDOct 18, 2026

TOMORI: A Tool-Orchestrated Mixing Operator for REAPER Interaction

Authors: Yining Wang, Tianjiao Li

Abstract

Code-based agents can operate creative software, but mixing requests often describe an intended sound without specifying the edits needed to achieve it. We present TOMORI, a code-based mixing-agent harness for native digital audio workstation (DAW) control. Custom plugins provide REAPER project inspection, optional audio measurement, and execution of model-generated Lua code. All observations return as text; delivery includes an editable project and rendered audio. TOMORI-BENCH pairs 110 human-reviewed reference edits with three instruction forms and evaluates native DAW execution end to end through operation structure, parameter precision, and rendered-audio outcomes. Across 990 cases from three primary models, explicit instructions yield high reference agreement, while abstract requests reduce project agreement and audio improvement. Paired ablations show that the full workflow improves explicit execution over a single submission. Prompted audio measurement succeeds in all 28 selected abstract cases but yields no clear overall improvement on the reference-based metrics and increases tool calls. These results expose the gap between executable DAW control and precise realization of perceptual mixing goals.

Figures & tables

Figure 1: Sequence diagram of Tomori . Within the loop, the model selects project inspection, audio analysis, or code-based project editing ( alt ), then uses returned results to decide whether to continue ( loop ). The editing code modifies the current REAPER project; delivery includes the editable project and rendered audio.

Figure 2: Data construction. Audio supports reference-edit construction, while real mixing requests inform the instructions. The reference mix endpoint and instructions are reviewed by humans.

Form	Step 2: English translations of the archived Chinese instructions
IF0	On the left main microphone’s ReaEQ, raise band 2 (around 950 Hz) by 1.0 dB.
IF1	The left main microphone’s EQ was cut too much around 950 Hz; bring a little back.
IF2	The left main microphone sounds too hollow in the middle; restore some body.

Table 1: One repair transition expressed in three forms. The common reference moves band-2 gain from

-5.4

-4.4

dB. The latter two instructions do not uniquely specify that numerical target.

kimi-k3
	Graph F1 $\uparrow$	Param. MAE $\downarrow$	Error reduction $\uparrow$ (%)	Edits in range $\uparrow$		Excess change $\downarrow$ (%)
				MR-STFT (%)	MERT (%)
IF0	1.0000	0.0352	83.8	95.2	95.2	2.9
IF1	0.9508	0.0811	47.1	57.7	56.7	21.2
IF2	0.8011	0.1112	33.9	49.0	45.2	28.8
claude-fable-5

Table 2: Results by instruction form. Graph F1 pools matched edit nodes and changed chain links; node matching requires joint agreement in edit kind, operation/control, track, and all required parameter directions. Param. MAE averages finite observed normalized-control errors; missing controls receive no tolerance-match credit. Error reduction is the mean percentage of MR-STFT error relative to the reference removed by editing, compared with leaving the project unchanged; negative reductions are set to zero. Edits in range is the percentage of cases whose change along the reference direction is 0.5–1.5 times the reference change, estimated separately with MR-STFT and MERT. Excess change is the percentage exceeding 1.5 times the reference change under MR-STFT. Both progress estimates use the same 104 cases per form whose reference edit changes the scored audio (

a_{i}\geq 10^{-3}

). Scores use final selected deliveries; see Appendix C.5 for the DeepSeek audio exception.

Same 36 reference steps
Condition	Form	Graph F1 $\uparrow$	Param. match $\uparrow$ (%)	Error reduction $\uparrow$ (%)	MERT dir. $\uparrow$
Full workflow	IF0	1.000	97.7	85.4	0.894
Single submission (S)	IF0	0.875	72.1	61.3	0.711
Full workflow	IF2	0.701	20.9	30.2	0.753
Single submission (S)	IF2	0.657	16.3	28.7	0.640
No analysis (A)	IF2	0.754	20.9	36.7	0.744

Table 3: Same-case ablations with DeepSeek v4 Pro. S allows one Lua edit submission with no returned readback or subsequent tools; A and L prohibit audio access and analysis. M prompts measurement before editing; all 28 cases execute measurements. Graph F1 uses joint direction matching; parameter tolerance is 0.01; error reduction uses MR-STFT; MERT direction is edit-displacement cosine similarity.

Model	Graph F1 $\uparrow$	Param. match $\uparrow$ (%)	Error reduction $\uparrow$ (%)
K3	0.870	57.0	58.4
Claude	0.878	52.3	55.0
Pro	0.851	59.3	57.8
Flash	0.851	60.5	58.6
k2.6	0.838	47.7	49.5

Table 4: Five-model comparison on 36 steps, pooling IF0 and IF2. Parameter tolerance: 0.01; error reduction: MR-STFT. Per-form results: Appendix F .

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Operation family	Steps
EQ	50
Gain/pan	29
Reverb	14
Compression	11
Gate	6
Stereo width	4

Appendix

Table 5: Reference operation-family coverage. Counts overlap: five steps cover two families. Each step is tested at three instruction forms.

kimi-k3
	Graph F1 $\uparrow$	Param. MAE $\downarrow$	Param. match $\uparrow$ (%)	Error reduction $\uparrow$ (%)	MERT dir. $\uparrow$	Seconds $\downarrow$
IF0	1.0000	0.0413	93.0	80.3	0.868	145.5
IF2	0.7399	0.1253	20.9	36.6	0.739	154.3
claude-fable-5
IF0	0.9931	0.0853	79.1	75.7	0.841	157.0
IF2	0.7630	0.1271	25.6	34.3	0.745	116.3

Appendix

Table 6: Same-case subset comparison: each model is evaluated under IF0 and IF2 on the same 36 steps. The first three models reuse existing deliveries; Flash and k2.6 add 72 cases each. All Graph F1 scores use joint node and direction matching. Parameter match uses tolerance 0.01; error reduction uses MR-STFT; MERT direction is the cosine between reference and model edit displacements. Seconds average the selected final attempt, including API waiting but excluding earlier failed attempts; they do not measure total inference cost.

Flash – Pro
	Graph F1	Param. match (pp)	Error reduction (pp)
IF0	-0.006	-2.3	-2.1
IF2	+0.007	+4.7	+3.9
Pooled	+0.001	+1.2	+0.9
k2.6 – K3
IF0	-0.028	-9.3	-10.0

Appendix

Table 7: Paired differences, first model minus second; pooled rows combine IF0 and IF2. Exploratory bootstrap analysis with 10,000 step draws supports the pooled decreases in parameter matching and error reduction for k2.6, while its Graph F1 difference and the differences on the reference-based metrics between Flash and Pro remain uncertain. Comparisons concern the selected subset and are not adjusted for multiple comparisons.

	Node F1	Link F1	Opp.	Invalid	Unch.	Unmat.	[email protected]
kimi-k3
IF0	1.000	1.000	0	0	0	0	0.934
IF1	0.921	0.985	0	0	9	2	0.339
IF2	0.822	0.945	0	0	11	3	0.207
claude-fable-5
IF0	0.964	1.000	4	1	0	0	0.826

Appendix

Table 8: Diagnostics by instruction form. Node F1 uses joint edit identity and direction matching; Link F1 compares changed chain links. Graph F1 pools their match counts and denominators, rather than averaging these two scores. Opp., Invalid, Unch., and Unmat. count required parameters changed in the opposite direction, set to out-of-range or non-finite values, left unchanged, or not matched to a control. [email protected] is the share of normalized parameters within 0.01 of the target.

Core signal-dependent tasks (12 cases)
Condition	Form	Graph F1 $\uparrow$	Param. match $\uparrow$ (%)	Error reduction $\uparrow$ (%)	MERT dir. $\uparrow$
Full workflow	IF2	0.592	29.2	10.8	0.410
No analysis (L)	IF2	0.672	20.8	11.5	0.441
Forced measurement (M)	IF2	0.585	33.3	10.0	0.492
Secondary signal-dependent tasks (15 cases)
Full workflow	IF2	0.828	50.0	26.1	0.787

Appendix

Table 9: Stratified audio-access and forced-measurement comparisons with DeepSeek v4 Pro. The first two strata use reference operations; the last two use historical analysis behavior and overlap. Historical-analysis subgroups are exploratory.

Tool-Using Agents

Tool-using agents plan and act through external tools, APIs, software interfaces, and environments to complete tasks beyond language generation.

See all 277 papers

RIME: Enabling Large-Scale Agentic Music Post-Production
Jul 21, 2026Noah Schaffer, Nikhil SinghAudio Editing Multimodal Large Language Models
Dartmouth College
TimberAgent: Gram-Guided Retrieval for Executable Music Effect Control
Mar 10, 2026Shihao He, Yihan Xia, Fang Liu +2Music Information Retrieval Audio Representation Learning
College of Electronics and Information Engineering, Shenzhen University, Shenzhen, China
Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects
Aug 5, 2026Yisu Zong, Jinjie Shi, Joshua ReissControllable Music Generation Audio Diffusion Models
Centre for Digital Music, Queen Mary University of London, UK

Jul 21, 2026J/K move · Enter open · S save

RIME: Enabling Large-Scale Agentic Music Post-Production

Noah Schaffer, Nikhil Singh

Dartmouth College

Audio Editing Multimodal Large Language Models Synthetic Data Generation for Language Models

Open paper AskPDF arXiv

Almost every piece of recorded music you have ever heard was modified before it reached you; commercial releases rarely spring fully formed from a musician's mind. Despite the promise of music generation models for one-shot output, such fine-grained iterative refinement workflows are a complementary problem and largely out of their reach. There is also a gap for musicians: while they can express what they want to hear, not all have the facility with studio production tools to implement the complex set of actions needed to realize these intuitions. We formalize this task as agentic post-production, wherein individual aspects of a track are targeted, refined, and combined into a final version. We argue the bottleneck is data: existing corpora do not reflect how realistic post-production chains map onto the vocabulary musicians and engineers actually use. We observe that there is a language for modifying recorded music that is dense, consistent, and learnable. To leverage this, we introduce the Rule-based Instructions for Music Editing (RIME) framework, which generates realistic paired edit-instruction data from any baseline music dataset grounded in canonical methods, design patterns, and constraints derived from real production workflows. RIME leverages POEMS, a toolkit that combines stem separation, mixing, and common studio effects for use by multimodal agents. We use POEMS and RIME to generate 15,000 pairs of edit instructions and ground-truth audio, then use this data to evaluate existing multimodal LLMs as agents on this task, revealing persistent limitations in current models. We also demonstrate RIME's ability to improve post-production agent performance via supervised fine-tuning on synthetic data. We see RIME as a step towards iterative musical agents, collaborative systems that could transform music production much as interactive coding agents have reshaped software engineering.

Abstract

Tool-Using Agents

Explore similar work

RIME: Enabling Large-Scale Agentic Music Post-Production

TimberAgent: Gram-Guided Retrieval for Executable Music Effect Control

Diff2Mix: Controllable Music Mixing via Diffusion Models and Differentiable Audio Effects