DuplexLedger: Benchmarking Transactional Memory in Full-Duplex Speech Agents

Yuxuan Li; Xiangchen Guan; Kunrong Li; Jin Wang; Kaiwen Luo; Yang Xiao; Yufei Guo; Zhisheng Chen

cs.AIOct 20, 2026

DuplexLedger: Benchmarking Transactional Memory in Full-Duplex Speech Agents

Authors: Yuxuan Li, Xiangchen Guan, Kunrong Li, Jin Wang, Kaiwen Luo, Yang Xiao, Yufei Guo, Zhisheng Chen,

Organizations: Nanyang Technological University · Peking University · Beijing University of Technology · Xidian University ·

Abstract

Full-duplex speech agents must update persistent memory while listening and speaking, yet it remains unclear which events should be stored, revised, or excluded. Existing benchmarks isolate interaction timing from memory outcomes, obscuring the state transitions behind a correct answer. We introduce DuplexLedger, a bilingual benchmark of 1,094 persistent sessions aligning four full-duplex scenarios with event-level Write/Update/Ignore/Retain supervision, explicit pre/post memory states, and 6,564 same-session questions and checks. Evaluating five full-duplex models reveals an interaction-memory gap: no system combines strong boundary control with coherent final-state memory, and the best final-state fact coverage is 2.01%. We propose ACT-Mem, an 8.92M-parameter sidecar comprising an event controller, a versioned transactional ledger, and a typed retriever coupled with a frozen MiniCPM-o 4.5 backbone. On 219 held-out sessions, ACT-Mem increases latest-value accuracy from 4.11% to 8.68% while reducing stale assertions and distractor adoption. It improves interaction boundaries but slows interruption stopping. These results motivate selective, auditable full-duplex memory rather than indiscriminate transcript retention. Code is available at https://github.com/lyx-researcher/DuplexLedger.

Figures & tables

Figure 1: From isolated interaction and memory to transactional full-duplex memory.

			Duplex		Memory
Dataset	Scale	Hrs	Adaptive interaction	FD scenarios	State / Memory eval.	Update QA	Distractor reject	Event state
FDB v1/v1.5	727/499	NR	✗	4/4	✗	✗	✗	✗
HumDial-FDBench	15,698	$>100$	✗	9	✗	✗	✗	✗
MultiTalkBench	104	56.5	✗	N/A	✓	✗	✗	✗
FDB v2	200	N/A	✓	4	✓	✗	✗	✗
EchoChain	200	NR	✓	1	✓	✗	✗	✗

Table 1: DuplexLedger versus representative full-duplex and conversational-memory benchmarks.

Figure 2: Overview of DuplexLedger . A persistent session aligns adaptively triggered full-duplex challenges, event-level memory transitions, and spoken memory questions.

Scenario	ZH	EN	Sess.	Events	Chal.	QA
Pause handling	149	127	276	1,932	552	1,656
Speech to others	150	126	276	2,208	1,104	1,656
User backchannel	146	137	283	2,545	1,130	1,698
User interruption	132	127	259	1,554	518	1,554
Total	577	517	1,094	8,239	3,304	6,564

Table 2: Composition and supervision statistics of DuplexLedger .

(a) Full-duplex interaction
Model	RCR †	cFD $\uparrow$	Early $\downarrow$	Pre-end $\downarrow$	Win-Intr. $\downarrow$	[email protected] $\uparrow$	Stop (s) $\downarrow$
MiniCPM -o 4.5	73.51	40.23	18.70	44.38	59.47	55.69	1.032
Freeze-Omni	86.17	28.40	49.76	91.82	73.64	45.75	1.239
Moshi	44.63	37.36	66.59	72.25	45.11	60.79	0.777
Lychee-FD	75.64	47.32	24.68	57.14	53.89	90.12	0.312
PersonaPlex	93.54	22.74	46.64	72.73	90.62	16.40	1.689

Table 3: Full-duplex interaction and memory fidelity on DuplexLedger .

Figure 3: Overview of the ACT-Mem framework for transactional memory in full-duplex agents.

Figure 4: ACT-Mem versus its frozen MiniCPM -o 4.5 backbone on 219 held-out sessions; error bars show 95% paired-bootstrap CIs for Latest and Final.

	Full-duplex interaction			Memory fidelity
Variant	cFD $\uparrow$	Early $\downarrow$	Pre-end $\downarrow$	Latest $\uparrow$	Stale $\downarrow$	D-Adopt $\downarrow$	Final $\uparrow$
FIFO memory	34.63	37.21	48.65	7.08	2.28	12.33	1.61
w/o addressee gating	40.14	31.48	46.36	6.39	0.91	17.35	1.75
Untyped retrieval	40.33	29.98	44.61	9.59	1.60	19.63	1.82
ACT-Mem	41.19	16.03	41.82	8.68	0.46	11.42	2.00

Table 4: Ablations on 219 sessions. Best and second-best scores are bold and underlined.

Figure 5: ACT-Mem cases across four full-duplex scenarios, showing controller decisions, memory transitions, and paired outcomes.

$\tau_{\mathrm{write}}$	0.35	0.45	0.50	0.55	0.65	0.75
Recall (%)	55.26	55.26	54.74	53.68	50.53	48.95
F1 (%)	71.19	71.19	70.75	69.86	67.13	65.72

Table 5: Threshold sensitivity on 40 calibration sessions.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Quantity	Value
Sessions	1,094
Chinese / English	577 / 517
Interruption / backchannel sessions	259 / 283
Pause / talking-to-other sessions	276 / 276
Development / test sessions	237 / 857
Memory questions	6,564

Appendix

Table 6: Composition of DuplexLedger . Fixed-input duration refers to the compatibility waveform; event-source duration is the total duration of the individual audible speech segments used by the adaptive protocol.

Stage	Purpose	Execution semantics
play_audio	Context delivery	Plays the bounded audio window associated with a user event and preserves the event’s speaker, addressee, and dialogue-act metadata.
wait_for_assistant_ state	State-triggered synchronization	Waits for decoded assistant speech to satisfy a target state and a minimum active-speech duration. The stage terminates on satisfaction or timeout.
inject_interruption	Interruption challenge	Delivers the interruption audio only after the preceding speaking-state trigger has been satisfied.
inject_backchannel	Backchannel challenge	Delivers a backchannel only after the assistant has produced the required amount of active speech.
play_side_talk	Non-addressee speech	Plays speech addressed to another participant and then observes whether the assistant responds during the observation interval.
wait	Annotated pause or observation	Advances the scheduler for a fixed duration without fabricating user speech. A pause inside a turn is not treated as a semantic turn boundary.

Appendix

Table 7: Execution stages generated from annotated ledger events.

Parameter	Value	Parameter	Value
Input chunk	$80\,\mathrm{ms}$	Scheduler poll	$20\,\mathrm{ms}$
Event padding	$120\,\mathrm{ms}$	Assistant-start timeout	$10\,\mathrm{s}$
Interruption trigger speech	$500\,\mathrm{ms}$	Backchannel trigger speech	$800\,\mathrm{ms}$
Minimum response speech	$160\,\mathrm{ms}$	Assistant-end timeout	$12\,\mathrm{s}$
Post-event silence	$240\,\mathrm{ms}$	Stable assistant silence	$2\,\mathrm{s}$
Side-talk observation	$1.2\,\mathrm{s}$	QA pre-input quiet	$1.2\,\mathrm{s}$

Appendix

Table 8: Default timing and execution parameters.

Metric	Direction	Denominator or interpretation
protocol_valid	$\uparrow$	Attempted samples.
behaviorally_observable	$\uparrow$	Attempted samples; requires usable decoded assistant speech.
challenge_realization_rate	$\uparrow$	Distinct planned challenge identifiers.
conditional_cfd	$\uparrow^{\dagger}$	Challenge instances eligible for conditional behavioral scoring.
adaptive_fd	$\uparrow^{\dagger}$	Sample-level adaptive full-duplex score; auxiliary summary.
trigger_timeout_rate	$\downarrow$	Distinct planned challenges.

Appendix

Table 9: Aggregate runtime metrics and their sample-level denominators.

Metric	Direction	Denominator or interpretation
response_rate	$\uparrow$	All memory questions.
no_response_rate	$\downarrow$	All memory questions; complementary to response rate.
latest_value_accuracy	$\uparrow$	All latest-value questions, including unanswered questions.
superseded_value_error	$\downarrow$	All latest-value questions.
distractor_adoption_rate	$\downarrow$	All distractor-ignore questions.
distractor_rejection_accuracy	$\uparrow$	All distractor-ignore questions; rejection additionally requires a response.

Appendix

Table 10: Aggregate same-session memory metrics and their sample-level denominators.

Model	Native interface	Input / output	Chunk or inference window	Model-specific condition	GPU
MiniCPM -o 4.5	Native streaming worker	$16/24$ kHz	$80$ ms input; $1000$ ms native window	Greedy decoding, fixed seed, fixed reference audio	1
Freeze-Omni	Socket.IO-based streaming service	$16/24$ kHz	$20$ ms native input block	$p=0$ , $k=1$ , temperature $=1.0$ ; neutral system prompt	1
Moshi	WebSocket streaming service	$24/24$ kHz	$80$ ms input block	Native server decoding; no evaluator-side task-specific sampling override	1
Lychee-FD	vLLM-backed streaming service	$16/24$ kHz	$80$ ms input; $400$ ms native inference window	Fixed prompt voice; native backend decoding	2
PersonaPlex -7B-v1	Native WebSocket streaming service	$24/24$ kHz	$80$ ms input block	Fixed voice prompt and fixed random seed	1

Appendix

Table 11: Native model interfaces and model-specific inference settings.

Parameter	Value
Reference input chunk	$80\,\mathrm{ms}$
Evaluator polling interval	$20\,\mathrm{ms}$
Event-window padding	$120\,\mathrm{ms}$
Assistant-start timeout	$10\,\mathrm{s}$
Assistant-end timeout	$12\,\mathrm{s}$
Interruption trigger speech	$500\,\mathrm{ms}$

Appendix

Table 12: Shared evaluator-side inference and timing configuration.

Question type	Retrieval behavior	Maximum items
Latest-value QA	Selects the target entity’s current active value after ranking candidate facts.	2 candidates, 1 final value
Ordinary memory QA	Retrieves active facts relevant to the question.	4
Final-summary QA	Aggregates active facts across entities and memory types.	8
Distractor-ignore QA	Routes directly to rejection behavior without memory retrieval.	0

Appendix

Table 13: Question routing and retrieval scope.

Configuration	Value
Session split	$660/175/40/219$ train/validation/calibration/held-out
Trainable parameters	$8.92$ M
Controller parameters	$4.59$ M
Retriever parameters	$4.33$ M
Optimization stages	Joint training followed by retriever adaptation
Optimizer	AdamW

Appendix

Table 14: Training configuration.

Model	Protocol valid $\uparrow$	Behaviorally observable $\uparrow$	Challenge realization $\uparrow$	Conditional cFD $\uparrow$	Adaptive FD $\uparrow$	Trigger timeout $\downarrow$	Response coverage $\uparrow$	Speech duration (s)
MiniCPM -o 4.5	100.00	100.00	73.51	40.23	35.28	6.04	28.38	78.35
Freeze-Omni	100.00	100.00	86.17	28.40	30.41	0.53	76.03	203.16
Moshi	100.00	100.00	44.63	37.36	13.08	27.67	12.61	29.23
Lychee-FD	99.82	100.00	75.64	47.32	48.19	1.09	35.30	99.19
PersonaPlex -7B-v1	100.00	100.00	93.54	22.74	35.41	1.73	32.43	74.20

Appendix

Table 15: Runtime validity, challenge exposure, and response coverage on all

1{,}094

sessions. Values are percentages except assistant speech duration.

Model	Premature onset $\downarrow$	Carry- in $\downarrow$	Pre-end overlap $\downarrow$	Pause intrusion $\downarrow$	Stop P50 $\downarrow$	Stop P90 $\downarrow$	Stop@ 0.8 s $\uparrow$
MiniCPM -o 4.5	18.70	25.86	44.38	59.47	1.032	1.446	55.69
Freeze-Omni	49.76	42.44	91.82	73.64	1.239	1.802	45.75
Moshi	66.59	5.49	72.25	45.11	0.777	0.864	60.79
Lychee-FD	24.68	32.96	57.14	53.89	0.312	0.440	90.12
PersonaPlex -7B-v1	46.64	26.23	72.73	90.62	1.689	2.104	16.40

Appendix

Table 16: Turn-boundary and interruption-handling results. Rates are percentages and stopping latencies are measured in seconds.

Model	Response rate $\uparrow$	No-response rate $\downarrow$	Latest-value accuracy $\uparrow$	Superseded-value error $\downarrow$	Accuracy given response $\uparrow$
MiniCPM -o 4.5	98.90	1.10	4.30	0.59	4.32
Freeze-Omni	99.94	0.06	11.61	3.38	11.61
Moshi	54.31	45.69	2.19	0.32	3.60
Lychee-FD	92.41	7.59	25.50	1.65	30.80
PersonaPlex -7B-v1	96.40	3.60	5.85	2.29	5.93

Appendix

Table 17: Response availability and factual-update results. All values are percentages.

Model	Distractor adoption $\downarrow$	Distractor rejection $\uparrow$	Summary fact coverage $\uparrow$	Legacy token $\bm{F_{1}}$ $\uparrow$
MiniCPM -o 4.5	14.17	85.19	2.01	29.45
Freeze-Omni	65.90	34.10	0.00	15.42
Moshi	44.15	8.32	0.04	7.64
Lychee-FD	48.17	51.55	1.06	21.36
PersonaPlex -7B-v1	79.71	17.46	0.24	7.70

Appendix

Table 18: Distractor handling, final-state coverage, and answer-overlap results. All values are percentages.

Model	Attempted sessions	Protocol-valid sessions	Observable sessions	Memory questions	Latest-value questions
MiniCPM -o 4.5	1,094	1,094	1,094	6,564	2,188
Freeze-Omni	1,094	1,094	1,094	6,564	2,188
Moshi	1,094	1,094	1,094	6,564	2,188
Lychee-FD	1,094	1,092	1,094	6,564	2,188
PersonaPlex -7B-v1	1,094	1,094	1,094	6,564	2,188

Appendix

Table 19: Principal denominators for the complete benchmark evaluation.

Scenario	Expected behavior	Observed evidence	Diagnosis
Interruption	Stop the current response, replace Saturday evening with Saturday morning , and replace twenty-five minutes with ten minutes .	Both interruptions were realized, with median stopping latency $0.368$ s and [email protected] of $1.00$ . The model recovered Saturday morning , but did not recover ten minutes . When asked about the side information, it answered that the distractor should be saved.	Fast interruption stopping was accompanied by only partial factual revision and incorrect admission. This separates acoustic interruption recovery from transactional memory updating.
Pause handling	Remain silent during within-turn pauses, then commit the completed correction: Thursday morning and fifteen minutes .	For the same session, both examined models intruded into every evaluated pause. One model rejected the distractor but recovered neither scored latest value. The other recovered Thursday morning , but answered “I will save all the details into the final memory” for the distractor question and later included a travel toothbrush in its summary.	The first behavior is a missed update; the second is an admission failure followed by summary contamination. Both occur together with incorrect pause-boundary control.
Backchannel	Continue the current response without treating the backchannel as a new turn or a persistent fact.	All three challenges were realized, but session cFD was $0.33$ . The model nevertheless recalled the stable requirement to avoid hazelnut flavoring and retain whole beans, and correctly stated that the kitchen-clutter detail should not be saved. Its final summary remained incomplete.	Correct retention and distractor rejection can coexist with poor continuation behavior. Backchannel handling and final-state consolidation therefore require separate attribution.
Speech to others	Do not answer the non-addressed speech and do not admit its content into memory.	The session obtained RCR $0.75$ and cFD $0.25$ . The model explicitly rejected the side information, but failed both latest-value questions and produced an incomplete final summary.	Rejecting a queried distractor does not establish that the system maintained the intended active state. Addressee control, supersession, and later answer generation remain distinct stages.

Appendix

Table 20: Representative qualitative examples from protocol-valid sessions. RCR and cFD are reported as proportions for the selected session.

Stage	Gold transition	Predicted transaction	Consequence
Initial request	Write provisional review time and budget constraints.	The controller predicted Write and committed extracted entity–value slots.	The initial values became active and were available to subsequent retrieval.
Correction	Update the review time to this Saturday morning and the electricity cap to $150 ; supersede both provisional values.	The controller predicted Update , but the extracted correction was represented as one compound slot with an entity key different from the provisional slots.	A new item was created, while the original items remained active and no supersession link was formed. The failure therefore occurred after the operation decision, during slot normalization and ledger resolution.
Distractor-bearing event	Commit the relevant condition and Ignore the explicitly excluded side detail.	The relevant condition was written, but the exclusion clause “just a side detail and should not be saved” was also extracted as an active fact.	The ledger became contaminated even though the user’s utterance explicitly expressed non-persistence. This is an incorrect-admission failure rather than a retrieval-only failure.
Latest-value QA	Retrieve the active corrected electricity cap and answer $150 .	No spoken answer was produced.	The case remains incorrect under the fixed denominator. The trace localizes the outcome to unresolved state construction followed by answer realization.
Distractor QA	Produce an explicit rejection without exposing the side detail as evidence.	No spoken answer was produced.	Neither adoption nor successful rejection is credited; silence is kept separate from selective exclusion.

Appendix

Table 21: An ACT-Mem transaction trace illustrating both successful control decisions and downstream state-construction failures.

Failure	Observable criterion
Trigger timeout	A speech-dependent challenge is planned but cannot be injected because the required assistant-speech condition is not reached.
Boundary violation	The assistant begins before a completed user turn, speaks during a protected pause, or intrudes into a non-response window.
Slow interruption stop	Assistant speech continues beyond the stopping threshold after an interruption is delivered.
Missed write	An admissible new fact produces no committed memory item.
Incorrect admission	Non-addressed, incomplete, backchannel, or explicitly excluded content is committed as persistent memory.
Failed supersession	A valid correction is detected, but the prior active value is not linked and marked as superseded.

Appendix

Table 22: Failure taxonomy used for qualitative attribution.

Artifact group	Representative records	Reproducibility role
Session inputs	source.json , session audio, event audio	Defines language, scenario, event order, speaker roles, timing, and the rendered input waveform.
Interaction manifests	interactive_manifest.json , challenge and timing manifests	Specifies challenge conditions, observation windows, trigger rules, and the fixed execution order.
Memory supervision	memory_qa.json , reference-state records, update links	Provides question types, accepted and forbidden values, expected operations, supporting evidence, and final-state supervision.
Reference behavior	reference_behavior.json	Stores the intended interaction response for each challenge and the corresponding state-transition constraints.
Runtime execution	runtime_events.jsonl , VAD and timing records	Records challenge realization, response boundaries, interruption stopping, overlap, pauses, and execution failures.
Generated outputs	Assistant waveform, raw predictions, QA windows, frozen-ASR transcripts	Preserves the model output and the temporal windows used for answer attribution and transcription.

Appendix

Table 23: Artifact manifest for reproducing the benchmark and experiments.

Memory Management for LLM Agents

See all 88 papers

Oct 6, 2026

PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue

Modern voice assistants may be shared by multiple users and should be able to answer questions about earlier conversations such as "When did I originally plan to leave?" or adapt their behavior to individual users based on past interactions. This requires more than retrieving a topically similar passage: the assistant must identify the current speaker, recover the relevant past state, and distinguish it from later revisions. We present PERSIST, a persistent memory system for multi-session, multi-speaker spoken dialogue that explicitly models Who, What, and When. PERSIST structures cross-session histories into readable event records and retrieves them with a 3W joint scoring mechanism that combines semantic content, acoustic speaker identity, and temporal state. For real-time full-duplex interaction, PERSIST further reuses intermediate representations from the dialogue backbone, avoiding query-audio re-encoding and reducing retrieval latency from 578.42 ms to 7.03 ms. We also introduce SpokenTrace, a diagnostic benchmark that factorizes evaluation along memory tasks and speaker-query types, exposing failures in recall, speaker attribution, and temporal-state tracking. On SpokenTrace, PERSIST achieves 85.08% end-to-end task accuracy and improves all-support EM@3 from 49.01% with BGE-large to 82.10%.

Achira Lin, Siyuan Hou, Wenyi Yu +7

Tsinghua University · Huawei Technologies Ltd.

Conversational Memory Persistent Memory for Language Models Long-Term Conversational Memory Full-Duplex Spoken Dialogue Systems

Ask PDF

Jul 30, 2026

MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memory

Persistent memory lets long-running large language model agents reuse information across sessions and tasks. Yet errors in writable memory can persist and corrupt future behavior. Existing systems improve storage and retrieval, but they do not provide a transaction boundary for reliable updates and recovery. We therefore propose MemTxn, a governance layer outside the answer model. MemTxn verifies whether an update is supported by its source. It also selects the visible version when facts conflict and restores the application-visible state after a fault. The system uses Ordered PatchTest to validate writes, a Temporal Resolver to select versions, and a durable snapshot journal to recover state. On an item-disjoint audit, MemTxn accepts all 60 supported originals and rejects all 179 hard negatives. Under persistent multi-key faults on LongMemEval-S and LoCoMo states, it restores the complete declared active map without knowing the actual physical write set. On MemoryAgentBench FactConsolidation, MemTxn achieves the highest average F1 across all twelve answer-model configurations. It outperforms Dense by 17.06--24.07 points in five representative settings.

Hanshuai Cui, Zhiqing Tang, Zhi Yao +3

Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai 519087, China · School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China

Agent Memory Management

Ask PDF

Sep 26, 2026

VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models

Spoken conversational systems must recover information from prior interactions (i.e., memory), yet relevant information in speech extends beyond what was said to who said it, how it was spoken, and what was audible, information that exists only in the audio signal and cannot be recovered from a transcript. Beyond what to remember, memory also demands diverse operations: retrieving a single fact, integrating evidence across turns, tracking an evolving state. Real interactions further unfold across sessions, meaning information accumulates across distinct episodes rather than a single continuous recording. Existing benchmarks fall short on all three dimensions: they focus primarily on lexical content, adopt limited and ad hoc memory operations, and treat memory as a single-session problem. We argue that principled memory evaluation requires jointly characterizing the acoustic evidence to be retained and the operations applied to it, and introduce a taxonomy along these two axes. Building on this taxonomy, we present VoxMem: 3,196 evaluation instances over 34,743 spoken sessions (177 hours) crossing four acoustic evidence types (speech semantics, speaker identity, paralinguistic cues, environmental sound) with four memory operations (information extraction, multi-session reasoning, temporal tracking, and answer refusal), grounded in multi-session histories and stratified across context budgets from 8K to 64K tokens. Evaluating 15 LALMs, no model exceeds 40% at 32K. Models retain what was said far better than who said it, how, or what was audible, a gap that widens for complex operations, grows with history length, and manifests as qualitatively distinct failure modes across evidence types. VoxMem aims to provide a foundation to measure and drive progress on the full scope of spoken conversational memory.

Yang Xiao, Vidhyasaharan Sethu, Eun-Jung Holden +1

University of Melbourne · University of New South Wales

Audio-Language Model Evaluation Long-Term Conversational Memory Audio-Language Models Long-Context Language Model Evaluation

Ask PDF

Abstract

Memory Management for LLM Agents

Explore similar work

PERSIST: Who-What-When Memory Across Sessions for Full-Duplex Spoken Dialogue

MemTxn: A Transaction Boundary for Source-Supported Updates and Complete-State Recovery in Agent Memory

VoxMem: Benchmarking Multimodal Memory in Large Audio Language Models