cs.CLNov 5, 2026

Axis-LoRA: Probing and Adapting Embedding Models for Personal-Memory Retrieval

Authors: Dohoon Kim, Junho Park, Hyunwoong Bae, Seyoon Kim, Taesup Moon

Organizations: Department of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in Artificial Intelligence, Seoul National University · Mobile eXperience Business, Samsung Electronics · ASRI/INMC/AIIS, Seoul National University

Abstract

As LLMs are increasingly asked questions about a user's personal experiences, retrieval must distinguish memories by time, place, person, and event. Current personal-memory pipelines rely on frozen dense embedding models to embed memories for retrieval, but we find that these models do not capture the four axes well. Swapping a memory's date or place barely shifts its embedding, and a trained projection on top of the frozen embedding does not recover the missing signal. The problem lives inside the model, where each axis is encoded by a distinct attention-head pattern. Guided by these findings, we propose Axis-LoRA, a per-axis adapter that places four LoRA branches inside the embedding model and combines them via a per-query gate. Across three standard embedding models evaluated on LongMemEval, ATM-Bench, and LoCoMo, Axis-LoRA improves 4-benchmark mean R@10 by +1.28+1.28 to +1.56+1.56~pp while training only a small fraction of the base parameters on a small axis-counterfactual training set. Code is available at https://github.com/dohoonkim-ai/axis-lora.

Figures & tables

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Latent Personal Memory: Represent personal memory as dynamic soft prompts

    Jun 18, 2026Debrup Das, Avinash Amballa, Yashas Malur Saidutta +3LLM Inference EfficiencyMemory-Efficient Fine-Tuning

  2. Beyond the Final Embedding: Retrieving Memories from Token-Level Evidence

    Oct 9, 2026Han Luo, Zhan Zhuang, Xu He +5Late Interaction RetrievalMulti-Vector Retrieval

  3. MemORAI: Memory Organization and Retrieval via Adaptive Graph Intelligence for LLM Conversational Agents

    May 2, 2026Hung Pham Van, Nguyen Manh Hieu, Khang Pham Tran Tuan +4Conversational MemoryGraph Retrieval-Augmented Generation