Navigation-Informed Embeddings: Dense-Retriever Adaptation from Agent Search Traces
Abstract
Agentic retrieval workflows produce query, retrieval, and stopping traces as a byproduct of answering questions. We study how these traces can adapt a deployed dense retriever to changing workflow distributions without new relevance labels, synthetic queries, or LLM judgments. We introduce Navigation-Informed Embeddings (NIE), a family of trace-derived objectives. NIE-Stop turns the stopping document into a soft positive; NIE-Path additionally uses preceding path documents as hard comparisons and imposes ordinal constraints with geometric decay. A BGE encoder adapted from retained source trajectories improves support Recall@20 on an independent target benchmark from 72.2 to 78.0 overall. NIE-Stop reaches 76.9 overall and 52.3 on long paths; NIE-Path raises long-path performance to 55.4, compared with 46.7 for the unadapted encoder. A shuffled-order control under the full path objective loses 3.2 points. Without public-benchmark training, the same adapter also improves nDCG@10 by 1.9 points on standard BEIR HotpotQA. NIE therefore provides a lightweight adaptation channel for settings where trajectories are already retained, with zero incremental labeling cost.
Figures & tables
| Step | Retrieved document and agent action | NIE supervision |
|---|---|---|
| Broad product overview; continue searching | Lower-ranked path contrast | |
| Manufacturer announcement; continue searching | Contrast; | |
| Approval notice naming the regulator; continue searching | Contrast; | |
| Safety notice specifying the monitoring rule; answer and stop | Soft positive; | |
| Unrelated document sampled from the corpus | Random negative |
| Approach | Data Source | Explicit Labels | Relative Cost |
|---|---|---|---|
| Human annotation | Curated | Human | High |
| Crowdsourcing | Curated | Crowd | Medium |
| Synthetic (InPars) | Generated | LLM generation | Low |
| NIE (ours) | Existing traces | None | Low incremental |
Dense Retrieval
Dense retrieval uses learned vector representations to find relevant documents, passages, products, or tools by semantic similarity.
See all 148 papers