HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

Zihong He; Junxiao Shen; Chen Liang; Hai-Ning Liang

cs.AISep 25, 2026

HasMem: Hard-Origin Adaptively Softened Memory for Long-Term LLM Agents

Authors: Zihong He, Junxiao Shen, Chen Liang, Hai-Ning Liang

Organizations: The Hong Kong University of Science and Technology (Guangzhou) · University of Bristol · The Hong Kong University of Science and Technology

Abstract

Text-based memory and context compression support reuse of past interactions. Resizing continuous memory changes the input to a frozen LLM, coupling capacity allocation with readout. We propose Hard-Origin Adaptively Softened Memory (HasMem). Frozen hard-prompt embeddings provide a verifiable initial state. A controller adjusts memory widths, a Writer re-encodes resized entries, and Reader and Global provide readout adaptation and cross-turn state. On all $535$ questions in a reconstruction probe derived from the Multi-Session Chat (MSC) development split, the main configuration achieves lexical F1 of $95.3$ ( $+4.4$ percentage points) at $93.6\%$ of the hard reference's framed memory positions. With approximately matched per-question target body budgets, six configurations at mean per-entry retention around $0.83$ -- $0.91$ exceed rule-based re-encoding by $8.0$ -- $23.6$ exact-match (EM) percentage points. With fixed model parameters and rule target width ratio $0.75$ , Global's EM gain passes a user-level exact paired test with Bonferroni correction over eight comparisons. On all $500$ LongMemEval-S questions, local lexical F1 rises from the hard reference's $3.4$ to $8.9$ , and answer negative log-likelihood (NLL) falls from $12.257$ to $5.274$ . F1 gains accompany lower EM on both evaluations.

Figures & tables

Figure 1: HasMem architecture. Records begin as frozen token embeddings and are maintained as variable-width continuous slots. The Controller selects width changes, the Writer re-encodes affected slots, and Reader and Global provide readout increments. Global also maintains a recurrent state across records.

Backbone	$k$	F1 $\uparrow$ HasMem / hard	EM $\uparrow$ HasMem / hard	NLL $\downarrow$ HasMem / hard	$R_{\mathrm{all}}\downarrow$ HasMem / hard
Qwen2.5-7B	2	86.7 / 87.6	69.6 / 84.7	0.463 / 0.416	0.900 / 1.000
Qwen2.5-7B	6	95.4 / 91.0	65.7 / 88.2	0.154 / 0.074	0.879 / 1.000
Qwen2.5-7B	8	95.3 / 90.9	84.3 / 88.1	0.115 / 0.074	0.936 / 1.000
Qwen2.5-7B	12	92.3 / 90.9	77.4 / 88.1	0.173 / 0.074	0.921 / 1.000

Table 1: Reconstruction and position cost on MSC-derived 535 (Qwen2.5-7B). Cells give HasMem / hard reference. F1 and EM are percentages.

R_{\mathrm{all}}

includes role framing.

Backbone	$k$	F1 $\uparrow$ HasMem / hard	EM $\uparrow$ HasMem / hard	NLL $\downarrow$ HasMem / hard	Cover $\uparrow$ HasMem / hard	Brief $\uparrow$ HasMem / hard	Local judge $\uparrow$ HasMem / hard
Qwen2.5-0.5B	8	5.4 / 5.6	0.00 / 1.00	4.663 / 3.649	8.4 / 8.8	0.6 / 8.4	10.8 / 10.6
Qwen2.5-1.5B	4	7.0 / 3.2	0.80 / 1.20	5.058 / 4.962	9.8 / 7.4	3.4 / 7.4	8.6 / 7.6
Qwen2.5-1.5B	8	1.3 / 3.6	0.40 / 1.40	5.931 / 4.911	1.4 / 8.2	1.0 / 8.2	5.0 / 8.4
Qwen2.5-3B	8	6.8 / 3.6	0.00 / 0.20	4.842 / 10.413	15.4 / 10.0	0.4 / 7.6	9.8 / 10.6
Qwen2.5-7B	2	0.5 / 1.2	0.00 / 0.20	8.805 / 13.200	0.0 / 7.6	0.0 / 7.0	2.2 / 7.6
Qwen2.5-7B	4	2.0 / 2.7	0.00 / 1.20	5.328 / 12.985	5.4 / 9.0	2.0 / 8.4	3.8 / 9.2

Table 2: Answer quality on all 500 LongMemEval-S questions. Cells give HasMem / hard reference. Metrics except NLL are percentages. The local judge is Qwen-judge. Training settings appear in Table E1 .

$\lambda_{\mathrm{band}}$	0.5B EM $\uparrow$ / $R_{\mathrm{all}}\downarrow$	1.5B EM $\uparrow$ / $R_{\mathrm{all}}\downarrow$	3B EM $\uparrow$ / $R_{\mathrm{all}}\downarrow$	7B EM $\uparrow$ / $R_{\mathrm{all}}\downarrow$	14B EM $\uparrow$ / $R_{\mathrm{all}}\downarrow$
$0.0$	12.3 / 0.670	9.0 / 0.602	77.2 / 0.889	75.2 / 0.887	45.1 / 0.814
$0.6$ (default)	7.1 / 0.604	17.0 / 0.665	72.0 / 0.892	38.4 / 0.788	49.8 / 0.814
$1.5$	9.9 / 0.632	28.2 / 0.714	77.8 / 0.915	80.0 / 0.917	84.3 / 0.916
$3.0$	9.9 / 0.637	49.4 / 0.909	57.8 / 0.909	90.5 / 0.992	93.3 / 1.000
$6.0$	63.2 / 0.924	79.9 / 0.957	80.6 / 0.973	86.0 / 0.970	91.8 / 0.974
Hard reference	51.1 / 1.000	80.8 / 1.000	74.6 / 1.000	88.1 / 1.000	93.3 / 1.000

Table 3: Target-band penalty sweep (MSC-derived 535,

k=8

). Cells give owner-weighted EM (percent) /

R_{\mathrm{all}}

. Single-seed runs target

5465

updates. 1.5B at

\lambda_{\mathrm{band}}=0

and 14B at

0.6

complete

5132

and

4386

, respectively. Full metrics appear in Table B1 .

Trained configuration	Learned $\bar{r}$	Learned F1 / EM	Rule-based $\bar{r}$	Rule-based F1 / EM	$\Delta$ EM learned $-$ rule-based	Position gap (%) mean / maximum
Seed 2026091703	$0.952$	93.7 / 79.6	$0.952$	93.8 / 79.8	$-0.2$	0.05 / 2.40
Seed 2026091401	$0.946$	96.1 / 82.2	$0.946$	96.5 / 82.6	$-0.4$	0.05 / 2.40
Main run	$0.911$	95.5 / 84.5	$0.910$	93.9 / 66.5	$+17.9$	1.06 / 5.41
Seed 2026091702	$0.884$	93.6 / 79.3	$0.883$	94.3 / 65.4	$+13.8$	1.35 / 5.95
Seed 2026091701	$0.883$	93.7 / 77.6	$0.882$	92.1 / 54.0	$+23.6$	1.22 / 5.41
full (fixed steps)	$0.847$	94.1 / 75.0	$0.841$	93.9 / 55.5	$+19.4$	1.64 / 5.13

Table 4: Learned versus rule-based re-encoding at approximately matched per-question target body budgets (MSC-derived 535, frozen Qwen2.5-7B,

k=8

unless specified). Scores are question-weighted. The hard reference has F1/EM of

90.9/88.0

. Rows follow learned retention

\bar{r}

. The main run is Table 1 ’s seed

2026091331

, final update

5465

, with a wall-clock curriculum. The position gap is learned minus rule-based body positions as a percentage of the hard reference’s body positions (question mean / maximum).

\Delta

EM is learned minus rule-based, before rounding.

Table 6

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	MSC F1	MSC EM	Writing and readout
Verbatim window	93.04	86.19	Verbatim text
BM25 retrieval ( $k=8$ )	87.91	78.73	Verbatim text
Hard embedding retrieval	87.86	77.99	Verbatim text
A-MEM adaptation	85.57	75.75	One write per event
Mem0 adaptation	5.65	0.00	One write per event
LLMLingua-2 (official, single pass)	19.67	0.00	One token-pruning pass before readout at nominal retention $0.33$

Appendix

Table A1: Diagnostic results of external-system adaptations on MSC-derived 535 with frozen Qwen2.5-7B and full-sentence answers.

Method	Cover	Brief	Lexical F1	Protocol
Hard reference, same retrieval width	10.60	9.80	3.39	Retrieval settings shared with HasMem
HasMem (main)	14.20	5.60	8.94	Learned softening with the configuration in Table 2
LLMLingua-2 (official, single pass)	26.20	1.80	9.23	Whole-context compression at nominal ratio $0.33$

Appendix

Table A2: Descriptive comparisons on all 500 LongMemEval-S questions with frozen Qwen2.5-7B.

Method	Backbone	Memory form	Position ratio	MSC F1	$\Delta$ F1	MSC EM	$\Delta$ EM
Hard reference (HasMem)	Qwen2.5-7B	Token embeddings	$1.00\times$	90.87	—	88.06	—
HasMem (main)	Qwen2.5-7B	Continuous slots	$0.936\times$	95.29	$+4.42$	84.33	$-3.73$
Hard embedding retrieval	Qwen2.5-7B	Text tokens	$1.00\times$	87.86	—	77.99	—
LLMLingua-2 (official, single pass)	Qwen2.5-7B	Text tokens	$0.33\times$ (nominal)	19.67	$-68.19$	0.00	$-77.99$
Hard embedding retrieval	Mistral-7B	Text tokens	$1.00\times$	72.11	—	31.34	—
ICAE v2, one bank per context	Mistral-7B	Continuous slots	$1.48\times$	28.45	$-43.66$	10.26	$-21.08$

Appendix

Table A3: Sentence reconstruction by HasMem and external compressors on MSC-derived 535. Differences are relative to the hard reference within each group.

Backbone ( $k=8$ )	$\lambda_{\mathrm{band}}$	F1	EM	NLL	$R_{\mathrm{all}}$	Hard reference F1 / EM / NLL	$\Delta$ F1 / $\Delta$ EM
0.5B	$0.0$	69.8	12.3	1.065	$0.670$	59.4 / 51.1 / 0.187	$+10.4$ / $-38.8$
0.5B	$0.6$	58.9	7.1	1.598	$0.604$	59.4 / 51.1 / 0.187	$-0.4$ / $-44.0$
0.5B	$1.5$	62.5	9.9	1.349	$0.632$	59.4 / 51.1 / 0.187	$+3.1$ / $-41.2$
0.5B	$3.0$	61.1	9.9	1.417	$0.637$	59.4 / 51.1 / 0.187	$+1.7$ / $-41.2$
0.5B	$6.0$	91.5	63.2	0.197	$0.924$	59.4 / 51.1 / 0.187	$+32.1$ / $+12.1$
1.5B	$0.0$	63.0	9.0	1.362	$0.602$	90.4 / 80.8 / 0.039	$-27.4$ / $-71.8$

Appendix

Table B1: Target-band penalty sweep on MSC-derived 535, with a target of

5465

training updates per configuration.

Seed	Selected update	$R_{\mathrm{all}}$	F1	EM	Answer NLL
$2026091331$	385	$1.000$	90.9	88.1	0.074
$2026091401$	411	$0.964$	96.1	82.3	0.110
$2026091701$	4906	$0.916$	93.5	77.4	0.161
$2026091702$	3801	$0.918$	93.6	79.3	0.134
$2026091703$	417	$0.968$	93.7	79.7	0.095
$2026091711$	422	$0.966$	96.0	86.6	0.072

Appendix

Table C1: Eight-seed evaluation on MSC-derived 535 (7B,

k=8

). Each run completes

5465

updates, including

5081

policy updates. Checkpoints are selected by in-band training loss.

Configuration	All questions $\Delta$ EM [95% interval]	Subset questions / owners	Equal-position subset $\Delta$ EM [95% interval]
Seed 2026091703	-0.19 [-1.12, 0.75]	511 / 256	+0.00 [-0.59, 0.59]
Seed 2026091401	-0.37 [-1.12, 0.37]	512 / 256	-0.20 [-0.59, 0.00]
Original configuration	+17.94 [14.55, 21.39]	236 / 118	+8.05 [3.81, 12.29]
Seed 2026091702	+13.83 [10.09, 17.54]	248 / 124	+6.45 [1.61, 11.29]
Seed 2026091701	+23.55 [19.66, 27.43]	253 / 127	+17.79 [12.20, 23.62]
full (fixed update count)	+19.44 [15.54, 23.26]	177 / 89	+12.43 [5.17, 19.66]

Appendix

Table D1: Paired EM differences and equal-body-position subsets for all configurations in Table 4 . Differences are learned minus rule-based EM, in percentage points.

Backbone	$k$	Training updates
Qwen2.5-0.5B	8	5683
Qwen2.5-1.5B	4	2907
Qwen2.5-1.5B	8	4949
Qwen2.5-3B	8	5065
Qwen2.5-7B	2	5879
Qwen2.5-7B	4	5825

Appendix

Table E1: Completed training updates for the backbone and retrieval-width configurations in Tables 1 and 2 .

Configuration	Seed	Updates evaluated / completed	F1	EM	NLL	$R_{\mathrm{all}}$
full + always-on readout gate	2026091331	5465 / 5465	95.1	80.0	0.135	$0.914$
Additional hard anchoring	2026091401	426 / 5465	94.3	81.0	0.100	$0.966$
Additional hard anchoring	2026091701	428 / 5465	92.5	78.2	0.114	$0.959$
Additional hard anchoring	2026091702	422 / 5465	97.0	89.7	0.059	$0.967$
Additional hard anchoring	2026091703	433 / 5465	93.7	82.6	0.091	$0.968$
Final target frequency $10\%$	2026091701	4896 / 5465	92.7	84.0	0.096	$0.962$

Appendix

Table E2: Seed and hyperparameter configurations on MSC-derived 535 (Qwen2.5-7B,

k=8

, unless specified). Updates are listed as evaluated checkpoint / completed training.

Method	Action rule	$R_{\mathrm{all}}$	F1	EM
HasMem (original main run)	Controller-selected actions	$0.936$	95.3	84.3
Hard reference	Original token embeddings	$1.000$	90.9	88.1
Fixed	SHRINK at every step	$0.75$	80.0	22.9
Random	Uniform sampling over three actions	$0.98$	90.6	50.4

Appendix

Table E3: Action-policy comparisons on MSC-derived 535 with frozen Qwen2.5-7B and

k=8

Configuration	Training and deployment	MSC F1	MSC EM	MSC NLL	Memory vectors	LME-S F1	LME-S NLL
Hard reference	Original token embeddings, without softening	90.9	88.1	0.074	101.1	3.4	12.257
full (fixed update count)	HasMem, $5465$ updates including $5081$ policy updates, final checkpoint	94.0	74.8	0.172	89.8	9.8	5.260
Gaussian-input training	Gaussian inputs during training and hard initialization at evaluation	93.6	58.4	0.250	87.9	8.7	5.481
Fixed-ratio rule training ( $0.91$ )	Rule-based training with action imitation and Controller at evaluation	98.5	78.4	0.067	91.9	9.1	4.794
Retention-based rule training ( $0.91$ )	Highest-retention entry selection during training and Controller at evaluation	97.7	73.7	0.099	91.9	9.5	5.065

Appendix

Table E4: Common-deployment evaluation of four training recipes with frozen Qwen2.5-7B and

k=8

Configuration	Training intervention	NLL
Full HasMem	Full curriculum and all modules	0.115
No forced softening or target band	Remove forced softening and the target-band penalty	0.621
No forced softening	Remove forced softening	0.571
No target band	Remove the target-band penalty on non-KEEP prediction frequency	0.285
Frozen Writer without hinge	Freeze the warmed-up Writer and disable Writer hinge, retaining forward re-encoding	0.488
No Writer hinge	Disable Writer hinge, with an all-KEEP evaluation trajectory	0.074

Appendix

Table E5: Training interventions and answer NLL for the ablations in Table 6 (MSC-derived 535, frozen Qwen2.5-7B,

k=8

). NLL is owner-weighted. Configurations follow the order in Table 6 , which reports training updates,

R_{\mathrm{all}}

, F1, and EM.

Category	Setting	Default
Architecture	Global/shared feature dimension $m$ and Controller MLP hidden dimension	$64$ and $64$
	Reader and Global readout adaptation	Attention output projections in final $4$ layers, rank $r=4$
	Added parameters and policy-phase trainable parameters	$4{,}402{,}627$ and $478{,}210$
Maintenance	Width adjustment $\eta$ and minimum body width $K_{\min}$	$0.1$ ( $10\%$ before rounding) and $4$
	KEEP-streak clipping $s_{\max}$ and round-robin interval $c_{\mathrm{rev}}$	$8$ and $2$
	Post-write maintenance steps	$12$ for Qwen2.5-0.5B/1.5B, $6$ for Qwen2.5-3B/7B/14B and Mistral-7B

Appendix

Table G1: Default architecture, maintenance, and training settings. Progress

p

follows the training schedule in Appendix F . Fixed-update variants are specified in Appendix G.2 .

Rule target ratio $\rho$	$R_{\mathrm{all}}$	Readout	F1	EM	NLL	Wrong $\to$ right / right $\to$ wrong	$\Delta$ EM [95% interval]	Adjusted $p$
$0.91$	$0.909$	All enabled	95.8	60.1	0.182	Reference	—	—
$0.91$	$0.909$	No Reader	95.0	58.2	0.224	$4$ / $14$	$+1.87$ [ $0.56,3.36$ ]	0.1702
$0.91$	$0.909$	No Global	95.0	58.2	0.234	$5$ / $15$	$+1.87$ [ $0.37,3.38$ ]	0.2471
$0.85$	$0.826$	All enabled	91.1	41.6	0.438	Reference	—	—
$0.85$	$0.826$	No Reader	90.7	40.5	0.565	$4$ / $10$	$+1.12$ [ $-0.19,2.43$ ]	1.0000
$0.85$	$0.826$	No Global	90.5	40.5	0.588	$5$ / $11$	$+1.12$ [ $-0.19,2.44$ ]	1.0000

Appendix

Table G2: Readout-pathway ablations under rule-based allocation (MSC-derived 535, frozen Qwen2.5-7B,

k=8

). All rows share the main model parameters. Per-question position counts are identical within each ratio. F1, EM, NLL, and budget are owner-weighted. Wrong

\to

right / right

\to

wrong count questions whose EM changes after disabling a pathway.

\Delta

EM is the question-weighted gain from enabling the pathway. Intervals use paired owner bootstrap. Bold marks adjusted

p<0.05

. Exact

p

values are Bonferroni-adjusted over eight comparisons (Section 4.1 ).

Readout at inference	F1	EM	NLL	Wrong $\to$ right / right $\to$ wrong	$\Delta$ EM [95% interval]	Adjusted $p$
All enabled	95.46	84.49	0.1123	Reference	—	—
No Reader	95.46	84.11	0.1274	$2$ / $4$	$+0.37$ [ $-0.56,1.31$ ]	1.0000
No Global	95.25	83.36	0.1315	$1$ / $7$	$+1.12$ [ $0.19,2.24$ ]	0.1406

Appendix

Table G3: Readout-pathway ablations under learned allocation (MSC-derived 535). All rows share the main parameters, Controller actions, Bank / Global state trajectories, and per-question budgets, with relative budget

0.936

. Metrics are question-weighted, whereas Table 1 is owner-weighted. Change counts and

\Delta

EM follow Table G2 . Exact owner-level

p

values are Bonferroni-adjusted over two comparisons.

Memory Management for LLM Agents

See all 58 papers

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory
Jul 17, 2026Jing Yu, Yibo Zhao, Jiaming Zhang +1Persistent Memory for Language Models Long-Term Conversational Memory
School of Data Science and Engineering, East China Normal University
AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents
Jun 19, 2026Xingyu Chen, Rui Wang, Zhaopeng Tu +1LLM Agent Memory Persistent Memory for Language Models
Shanghai Jiao Tong University, Tencent · Shanghai Jiao Tong University · Tencent
ElasticMem: Latent Memory as a Learnable Resource for LLM Agents
May 29, 2026Tao Feng, Chongrui Ye, Tianyang Luo +5LLM Agent Memory Persistent Memory for Language Models
University of Illinois Urbana-Champaign · Nanyang Technological University

Jul 17, 2026J/K move · Enter open · S save

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

Jing Yu, Yibo Zhao, Jiaming Zhang, Xiang Li

School of Data Science and Engineering, East China Normal University

Persistent Memory for Language Models Long-Term Conversational Memory Agent Memory Management Memory Management for LLM Agents

Open paper AskPDF arXiv

Long-term memory enables LLM agents to leverage past interactions, but dialogue histories quickly exceed the context window, forcing agents to retrieve relevant subsets at query time. Because useful evidence is sparse and scattered across verbose conversations, retrieval faces a fundamental tension: broadening recall improves coverage but floods downstream reasoning with noise, while compressing memories at write time eases retrieval but irreversibly discards details that future queries may need. We introduce LazyMem, which resolves this tension by deferring all memory construction to query time. Given a retrieved candidate pool, a lightweight model processes it in overlapping parallel windows, selectively retaining and compressing only query-relevant content. The model is trained with supervised fine-tuning followed by reinforcement learning, using a reward that jointly encourages the identification of relevant messages and the generation of compressions that are faithful to the source and useful for answering the query. On LongMemEval, LazyMem-4B achieves an LLM-judge accuracy of 0.85, outperforming the strongest non-oracle baseline while using only 213 answer-context memory tokens, 21.0 times fewer than the baseline. It further generalizes to LoCoMo without target-domain training and reduces mean latency relative to the prior query-time baseline. Code is available at https://github.com/allacnobug/LazyMem.

Abstract

Memory Management for LLM Agents

Explore similar work

LazyMem: Retrieve Broadly, Construct Selectively for Efficient Long-Term Agent Memory

AdaMem: Learning What to Remember with Adaptive Memory Policies for Personalized Agents

ElasticMem: Latent Memory as a Learnable Resource for LLM Agents