Alignment in Time: Peak-Aware Orchestration for Long-Horizon Agentic Systems

Hanjing Shi; Dominic DiFranzo

cs.AIFeb 20, 2026

Alignment in Time: Peak-Aware Orchestration for Long-Horizon Agentic Systems

Authors: Hanjing Shi, Dominic DiFranzo

Organizations: Lehigh University, Bethlehem, Pennsylvania, USA

Abstract

Traditional AI alignment primarily focuses on individual model outputs; however, autonomous agents in long-horizon workflows require sustained reliability across entire interaction trajectories. We introduce APEMO (Affect-aware Peak-End Modulation for Orchestration), a runtime scheduling layer that optimizes computational allocation under fixed budgets by operationalizing temporal-affective signals. Instead of modifying model weights, APEMO detects trajectory instability through behavioral proxies and targets repairs at critical segments, such as peak moments and endings. Evaluation across multi-agent simulations and LLM-based planner--executor flows demonstrates that APEMO consistently enhances trajectory-level quality and reuse probability over structural orchestrators. Our results reframe alignment as a temporal control problem, offering a resilient engineering pathway for the development of long-horizon agentic systems.

Figures & tables

Figure 1. APEMO as a temporal-affective overlay for long-horizon workflows. Uniform allocation optimizes mean-step performance but allows negative peaks to persist. APEMO reallocates effort toward peak repair and ending stabilization under fixed compute budgets, improving trajectory-level robustness.

Figure 2. APEMO internal control loop. At each turn $t$ , a frustration signal $S_{f}(t)$ is computed from behavioral proxies. If a negative peak is detected, a precision repair module reallocates compute toward peak and ending turns. Coordination cost accumulates under a fixed budget constraint $C\leq C_{max}$ .

Block	Models	Turns	Episodes	Runs (n)	Policies	no_fallback_rate
Strict Long-Horizon	qwen2.5:1.5b, gemma2:2b	8	2	20	task_affect, task_peak_end, apemo	1.0
Short-Horizon Boundary	llama3.2:1b, qwen2.5:1.5b, gemma2:2b	2	2	21	task_affect, task_peak_end, apemo	1.0
Trap Recovery (strict)	gemma2:2b	8	1	20	task_peak_end, apemo	1.0
Multi-Agent Flow	qwen2.5:1.5b, gemma2:2b	8	1	16	flow_plain, flow_temporal, apemo	1.0
L3 Plan-Execute-Class	llama3.2:1b, qwen2.5:1.5b, gemma2:2b	2	2	30	plan_execute, plan_execute_reflect, task_peak_end, apemo	1.0
L3 Plan-Execute-Class (Long-Horizon)	llama3.2:1b, qwen2.5:1.5b, gemma2:2b	8	1	18	plan_execute, plan_execute_reflect, task_peak_end, apemo	1.0

Table 1. Primary LLM block configuration summary used in main Results.

Baseline	Mean Quality	Reuse Prob.	Avg. Frust.
task_peak_end	$+0.0791$ [0.0525, 0.1055]	$+0.0609$ [0.0383, 0.0826]	$-0.0172$ [-0.0306, -0.0071]
task_affect	$+0.1874$ [0.1596, 0.2159]	$+0.1161$ [0.0861, 0.1433]	$-0.0543$ [-0.0805, -0.0356]

Table 2. Strict long-horizon evaluation (

T=8

n=20

). Values are mean deltas with 95% bootstrap CIs.

Figure 3. Forest-style effect plot for key deltas (APEMO minus baseline) with 95% bootstrap CIs.

Figure 4. Trap recovery dynamics in the $n=20$ LLM trap block, derived from episode-level trap drop/rebound metrics. Left: quality change around trap; right: frustration change around trap. Shaded areas are 95% bootstrap CIs.

Setting	Quality Gain	Cost Increase
Long-Horizon ( $T{=}8$ , vs peak-end)	$+14.49\%$	$+6.28\%$
Long-Horizon ( $T{=}8$ , vs affect)	$+42.86\%$	$+8.15\%$
Short-Horizon ( $T{=}2$ , vs peak-end)	$+8.48\%$	$+3.39\%$
Trap Endpoint (vs peak-end)	$+29.90\%$	$+5.33\%$
Multi-Agent (vs flow_plain)	$+41.25\%$	$+7.14\%$

Table 3. Coordination frontier summary (large-

n

blocks). Percentages are relative deltas. All rows use mean-quality gain except Trap Endpoint, which uses endpoint-quality gain by design.

Figure 5. Coordination frontier (quality gain vs cost increase). Each point is one large- $n$ block/comparison.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 446 papers

Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning
May 4, 2026Wenyi Wu, Sibo Zhu, Kun Zhou +1Reinforcement Learning Multi-Agent Collaboration
University of California, San Diego
Proactive Agentic Orchestration
Oct 9, 2026Xueyang Feng, Chaozhuo Li, Xiaohe Bo +4LLM Agent Orchestration LLM Agent Workflow Optimization
Gaoling School of Artificial Intelligence, Renmin University of China · Beijing Academy of Artificial Intelligence · The Chinese University of Hong Kong Shenzhen
LemonHarness Technical Report
Jun 23, 2026Kailong Ren, Fubo Sun, Jiachen Liu +18LLM Agent Orchestration Long-Horizon LLM Agents

May 4, 2026J/K move · Enter open · S save

Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning

Wenyi Wu, Sibo Zhu, Kun Zhou, Biwei Huang

University of California, San Diego

Reinforcement Learning Multi-Agent Collaboration Long-Horizon LLM Agents LLM Agent Planning RL for LLM Agents

Open paper AskPDF arXiv

Language model (LM)-based agents have demonstrated promising capabilities in automating complex tasks from natural language instructions, yet they continue to struggle with long-horizon planning and reasoning. To address this, we propose an enhanced multi-agent framework that decomposes automation into three roles: a planner for high-level decision-making, an actor for task execution, and a memory manager for contextual reasoning. While this modular decomposition aligns with established design patterns, our core contribution lies in a systematic compute-allocation analysis, revealing that planning is the dominant factor influencing task performance. Execution and memory management require significantly less compute and model capacity to achieve competitive results. Building on these insights, we introduce a planner-centric reinforcement learning approach, which exclusively optimizes the planner using trajectory-level rewards from a VLM-as-judge, while freezing the other components. Extensive experiments on benchmarks spanning web navigation, OS control, and tool use demonstrate that concentrating model capacity and learning on high-level planning yields robust and compute-efficient improvements in long-horizon agent automation. Our code is publicly released.

Abstract

Long-Horizon LLM Agents

Explore similar work

Planner Matters! An Efficient and Unbalanced Multi-agent Collaboration Framework for Long-horizon Planning

Proactive Agentic Orchestration

LemonHarness Technical Report