Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing

Tianxiang Fei; Jiazheng Kang; Mao Zheng; Xingyan He; Xiang Yu

cs.CLOct 15, 2026

Hy-Memory-30B-A3B: Attributed Reinforcement Learning for Long-Term Memory Writing

Authors: Tianxiang Fei, Jiazheng Kang, Mao Zheng, Xingyan He, Xiang Yu

Organizations: Tencent Hunyuan

Abstract

External-memory systems extend language-model applications beyond a context window, but their quality depends critically on the model executing the write path: deciding what to retain, how precisely to represent it, and how new evidence should update persistent state. Existing frameworks typically place a general-purpose large language model behind framework-specific prompts. This raises serving cost, and a writer specialized for one protocol does not transfer directly to another; a unified model that can operate across established memory frameworks remains largely unexplored. We present Hy-Memory-30B-A3B, a compact long-term-memory writer trained across Mem0, MemOS, and Hy-Memory. Memory On-Policy Distillation teaches framework-specific write protocols on student-generated states; Memory RL then optimizes downstream answerability using dense per-chunk probes, attributed cross-chunk rewards, over-storage controls, and chunk-level GRPO. Carrying policy-written banks across windows keeps training aligned with deployment. On fixed 20% benchmark subsamples, Hy-Memory-30B-A3B ranks first on all four evaluated harnesses, including held-out EverMemOS, with a 55.49% macro average versus 54.62% for Claude Opus 5.

Figures & tables

Method	Learned policy and environment	Reward and distinction
Memory-R1	Manager and answer agent over a structured external bank	Downstream answer outcome; writer is confounded with the learned reader
MemAgent	Recurrent writer and answer policy over model-native fixed memory	Terminal score broadcast over the trajectory; not a drop-in writer
Mem- $\alpha$	Construction policy over a custom multi-component bank	Global QA plus local checks; global correctness is shared across actions
MemCoE	Guideline and evolution policy for personalization memory	Process plus final-answer reward; checks guideline adherence rather than evidence contribution
Hy-Memory-30B-A3B	Writer only, operating in Mem0, MemOS, and Hy-Memory	Per-chunk recall, attributed outcome, and growth penalty; localizes evidence to writes

Table 1: How Hy-Memory-30B-A3B differs from prior memory RL.

Family	Configuration	Role
Mem0	General	Incremental extraction and additive writing
MemOS	General	Multi-stage extraction and reconciliation
Hy-Memory	Chat	Personal conversational memory
Hy-Memory	Working	Tasks, decisions, and project state
Hy-Memory	Coding	Code, commands, bugs, and technical decisions

Table 2: Harness families and scenario configurations.

Figure 1: Hy-Memory-30B-A3B is trained inside the same deployed harness configurations in both stages. OPD supplies dense teacher supervision on student-written state chains; RL examines the resulting banks, assigns chunk-level credit, and carries the selected bank across windows. In the rollout grid, each cell is the bank after chunk

t

of rollout

i

, shaded by its chunk score

s_{i}[t]

. Advantages are normalized within each column (dashed: one chunk-level group), so the best rollout can differ from chunk to chunk. The outlined cell is the final bank with the highest

\sum_{t}s_{i}[t]

, which becomes the parent of the next window.

Pass	Annotator context	Output and target
Per-chunk probe (PRM)	Full text of chunk $t$ plus facts of chunks $t{-}1$ and $t{-}2$	Atomic facts, probes, traps, and rubrics; tests whether chunk $t$ was written precisely and remains recoverable
Cross-chunk outcome (ORM)	Earlier-sequence facts plus all facts in the current window	Questions, evidence chunks, gold facts, and rubrics; tests relations spanning chunks or sessions

Table 3: The two annotation passes and the reward each one feeds.

Figure 2: Construction of the Memory RL reward artifact. Each session is split independently into chunks of about ten turns, so chunks never cross session boundaries. The chronological chunk stream is then grouped into RL windows of at most five chunks; a window may span sessions. The per-chunk probe pass writes the PRM items for chunk

t

. It reads the full text of chunk

t

plus the extracted facts of chunks

t{-}1

and

t{-}2

, used only to resolve references and decide what is new. The cross-chunk outcome pass writes ORM items from the facts of the whole window and earlier history. Structural filters and a synthetic-answer discriminability test produce the final artifact. The bottom row lists the rules that keep answer information out of the reward path (Section 5.6 ).

Probe type	What the reward tests
Specificity	Exact names, numbers, titles, versions, paths, and entities survive writing
Semantic precision	Negation, scope, and event meaning are not inverted
Plain recall	A durable explicit fact remains answerable
Secondary dimension	A dominant topic does not erase a smaller detail
Transition	The new state and the state it supersedes are both represented
Should-not-store	Greetings, vague praise, and generic assistant text are absent

Table 4: Per-chunk failure-surface probes.

Configuration	Processed units	Reward items
Hy-Memory / Chat	3,182 windows; 14,519 chunks	41,990 recall; 2,769 traps; 15,281 ORM
Hy-Memory / Working	1,971 windows; 8,902 chunks	25,272 recall; 1,537 traps; 9,740 ORM
Hy-Memory / Coding	1,154 windows; 5,187 chunks	14,348 recall; 178 traps; 5,337 ORM
Mem0	3,182 windows; 14,516 chunks	41,416 recall; 1,969 traps; 15,228 ORM
MemOS	3,182 windows; 14,492 chunks	41,550 recall; 2,146 traps; 15,241 ORM
Total	12,671 windows; 57,616 chunks	164,576 recall; 8,599 traps; $\sim$ 60.8k ORM

Table 5: Harness-level statistics of the RL reward artifact.

Figure 3: Cross-harness macro-average scores on the fixed 20% benchmark matrix. Each harness score averages six metrics equally; Overall averages the four harnesses equally. EverMemOS is held out from training.

Memory writer	Mem0	Hy-Mem.	MemOS	EverMem.	Avg.
Hy-Memory-30B-A3B	54.02	60.55	50.51	56.89	55.49
Claude Opus 5	53.43	60.53	50.50	54.04	54.62
DeepSeek V4.1 Flash	49.73	60.20	49.17	55.47	53.64
Hy3	45.99	58.06	50.19	52.30	51.63
Qwen3.6-35B-A3B	47.51	55.87	47.36	50.75	50.37
Hy3-A3B Base	40.20	50.39	49.05	50.98	47.66

Table 6: Six-metric means on fixed 20% benchmark subsamples. The macro average weights the four harnesses equally.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Writer	LoCoMo	LME	PM-v2	Travel	Shop.	CMB	Avg.
Hy-Memory-30B-A3B	59.54	73.00	51.62	54.51	18.33	67.13	54.02
Claude Opus 5	69.47	66.00	53.58	42.98	17.78	70.74	53.43
DeepSeek V4.1 Flash	58.78	69.39	49.56	35.07	16.67	68.92	49.73
Hy3	41.22	62.00	44.44	46.87	17.22	64.16	45.99
Qwen3.6-35B-A3B	50.00	69.00	49.38	36.39	20.00	60.30	47.51
Hy3-A3B Base	50.00	47.00	40.82	37.91	14.44	51.03	40.20

Appendix

Table 7: Detailed results on the Mem0 harness.

Writer	LoCoMo	LME	PM-v2	Travel	Shop.	CMB	Avg.
Hy-Memory-30B-A3B	73.28	89.00	51.54	57.65	21.11	70.73	60.55
Claude Opus 5	74.43	89.69	48.71	51.34	23.33	75.70	60.53
DeepSeek V4.1 Flash	72.90	85.86	49.00	53.46	22.78	77.18	60.20
Hy3	69.47	89.00	52.76	49.01	21.67	66.44	58.06
Qwen3.6-35B-A3B	70.99	84.85	43.56	49.82	20.00	65.98	55.87
Hy3-A3B Base	66.41	87.00	49.84	31.36	17.78	49.97	50.39

Appendix

Table 8: Detailed results on the Hy-Memory harness.

Writer	LoCoMo	LME	PM-v2	Travel	Shop.	CMB	Avg.
Hy-Memory-30B-A3B	62.98	73.00	51.92	24.15	21.11	69.91	50.51
Claude Opus 5	63.74	79.38	52.84	22.14	19.44	65.46	50.50
DeepSeek V4.1 Flash	61.07	73.00	50.68	18.61	21.67	69.97	49.17
Hy3	62.98	81.00	49.64	18.87	18.89	69.78	50.19
Qwen3.6-35B-A3B	55.34	79.00	51.34	20.08	15.56	62.82	47.36
Hy3-A3B Base	58.40	73.00	50.86	26.05	21.11	64.88	49.05

Appendix

Table 9: Detailed results on the MemOS harness.

Writer	LoCoMo	LME	PM-v2	Travel	Shop.	CMB	Avg.
Hy-Memory-30B-A3B	73.28	76.00	53.28	56.56	19.05	63.19	56.89
Claude Opus 5	73.28	80.41	51.19	45.91	23.89	49.53	54.04
DeepSeek V4.1 Flash	72.52	82.83	49.74	59.47	18.89	49.38	55.47
Hy3	68.70	83.00	48.68	46.52	19.44	47.44	52.30
Qwen3.6-35B-A3B	69.08	79.00	50.60	43.53	18.77	43.52	50.75
Hy3-A3B Base	64.50	81.00	52.14	41.74	20.56	45.93	50.98

Appendix

Table 10: Detailed results on the held-out EverMemOS harness.

	Stage 1: OPD	Stage 2: RL
Model	Hy3-30B-A3B	Hy3-30B-A3B
Init.	Instruction-tuned	OPD step 375
Harnesses	Mem0, MemOS, Hy-Memory	Same
Ordering	Snapshot chain	Chronological windows
Unit	One chunk write	One window, $\leq$ 5 chunks
Rollouts	2 per row	8 per window

Appendix

Table 11: Training hyperparameters for the two stages.

Memory-Augmented Language Models

Memory-augmented language models combine persistent or temporary memory with neural computation to retain, retrieve, and use information beyond the active context.

See all 291 papers

MemCode: Discrete Semantic Representations for Long-Term Agent Memory
Oct 14, 2026Xin Li, Liang Hu, Duoqian Miao +2LLM Agent Memory Memory Management for LLM Agents
Tongji University · WorldMind
All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents
Oct 15, 2026Bingbing Wang, Jingjie Lin, Weiming Qiao +2Persistent Memory for Language Models Memory-Augmented Language Model Agents
Harbin Institute of Technology, Shenzhen · The Hong Kong Polytechnic University · Fudan University +1
MEMAUDIT: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing
May 4, 2026Nishant Bhargava, Rodrigo Sobral BarrentoPersistent Memory for Language Models LLM Agent Evaluation
Purdue University

Oct 14, 2026J/K move · Enter open · S save

MemCode: Discrete Semantic Representations for Long-Term Agent Memory

Xin Li, Liang Hu, Duoqian Miao, Ming Peng+1

Tongji University · WorldMind

LLM Agent Memory Memory Management for LLM Agents Long-Term Conversational Memory Agent Memory Management Associative Memory

Open paper AskPDF arXiv

Long-term memory remains a fundamental bottleneck for large language model (LLM) agents across sessions. Existing systems typically adopt a write-then-retrieve paradigm, storing context-dependent dialogue fragments and retrieving them via nearest-neighbor search in a continuous embedding space. This design yields unstable access units and systematically misses logically related but lexically distant evidence. We propose MemCode, a long-term agent memory framework that couples retrieval-oriented writing with discrete associative retrieval. MemCode rewrites dialogue into self-contained atomic memory units (AMUs), maps AMUs into a learnable codebook with isomorphic semantic quantization (ISQ), and uses codebook co-occurrence topology as a retrieval path complementary to vector search. For long-term memory stores, MemCode separates online atomic writes from asynchronous offline consolidation. On LongMemEval-S and LoCoMo with GPT-4o-mini and Qwen3-30B-A3B-Instruct-2507 backbones, MemCode achieves the best overall accuracy while keeping per-dialogue memory construction within roughly 40~~130 seconds, making it 3~~21 times faster than LightMem and over an order of magnitude faster than A-MEM and MemoryOS.

Abstract

Memory-Augmented Language Models

Explore similar work

MemCode: Discrete Semantic Representations for Long-Term Agent Memory

All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents

MEMAUDIT: An Exact Package-Oracle Evaluation Protocol for Budgeted Long-Term LLM Memory Writing