Conditional information Retrieval for Longitudinal Survey Harmonisation

Wing Yan Li; Zeqiang Wang; Michelle Lin; Jon Johnson; Suparna De

cs.CLOct 19, 2026

Conditional information Retrieval for Longitudinal Survey Harmonisation

Authors: Wing Yan Li, Zeqiang Wang, Michelle Lin, Jon Johnson, Suparna De

Organizations: University of Surrey · University College London

Abstract

Identifying conceptually equivalent questions across different surveys is a challenging information retrieval task where surface-level textual similarity is insufficient. Two questions may be lexically similar yet assess entirely different sociological sub-concepts, or lexically different yet conceptually equivalent. We introduce a conditional information retrieval framework that addresses this by decomposing the task into two steps: concept extraction, which identifies the underlying sociological concept of a survey question using a fine-tuned generative model, and concept-aware retrieval, which augments question representations with the extracted concept as an explicit conditioning signal for finer-grained alignment. To reflect the realistic data scarcity conditions of specialised social science annotation, we use CLOSER as a case study. Our framework consistently outperforms baselines by 2-3 points in both MAP and Recall. We further analyse fine-tuning strategies including SFT, DPO, and dynamic DPO, finding that SFT delivers the most stable gains while dynamic DPO interacts differently with instruction-tuned versus distilled reasoning backbones. Finally, we demonstrate that a model fine-tuned on a general social science thesaurus (ELSST) transfers robustly to task-specific survey data, offering a practical solution under data scarcity.

Figures & tables

Figure 1: Two-step conditional retrieval framework for retrieving conceptually equivalent items. Given a survey question

q_{i}

, (1) Concept Extraction uses an LLM to extract the underlying sociological concept

\hat{t}_{i}

; (2) Concept-Conditioned Encoding concatenates the extracted concept with the question as a conditioning signal before encoding; (3) Conceptual Comparison computes pairwise cosine similarity between query and candidate representations, returning items above threshold

\tau

Split	CLOSER-ID	CLOSER-OOD
Train	611	-
Test	405	3479

Table 1: Final count of instances per dataset. CLOSER-ID is deduplicated and split into train and test splits. CLOSER-OOD serves as the out-of-distribution test set only.

Table 2: Results on both CLOSER-ID and CLOSER-OOD. Highest metric within each backbone model group is highlighted in bold . The ELSST model uses Qwen2.5-7B-Instruct as the backbone model. Note that other than the baseline, as it is not applicable, all models use the conditioned representations.

Table 4: Ablation study results obtained using models trained with dynamic DPO training. Quantitative results (a) and qualitative examples (b) illustrate the impact of dynamic sampling on metrics and conceptual term granularity.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Table 5: Top-three retrieved items for the query “ About how often does your child do the following: How often does he: Help in the house 1|Nearly every day. ... 6|Not at all ” (ground-truth concept: Extra school activities: housework ). Bold entries show the model-predicted concept used for conditioning; italicised entries show the annotated gold concept of each retrieved item.

Table 6: Top-three retrieved items for the query ” He/She does not seem to feel sorry after he/she misbehaves. 1|Often true. 2|Sometimes true. 3|Not true “ (ground-truth concept: childdev ). Extracted conceptual terms are in bold and ground-truth conceptual terms are in italics .

Information Retrieval

Information Retrieval studies how to find, rank, and organize relevant information in large collections of data.

See all 474 papers

SCBO: Semantically Coherent Batching and Ordering for LLM-Based Social Surveys
Sep 28, 2026Yuanzi Li, Lingjie Wang, Zihang Tian +2LLM Inference Efficiency LLM Prompting
Renmin University of China
Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models
Sep 28, 2026Ji Huang, Mengfei Li, Shuai ShaoLarge Language Model-Based Social Simulation Conditional Generative Modeling
School of Computer Science, University of Science and Technology of China · School of Management, Fudan University
Silicon Sampling via Cross-Survey Transfer
Jul 3, 2026Chan-Tung Ku, Chan Hsu, Pei-Cing Huang +3LLM Evaluation Transfer Learning
Department of Information Management National Sun Yat-sen University Kaohsiung, Taiwan · Institute of Political Science National Sun Yat-sen University Kaohsiung, Taiwan · Graduate Institute of Library and Information Science National Chung Hsing University Taichung, Taiwan

Sep 28, 2026J/K move · Enter open · S save

SCBO: Semantically Coherent Batching and Ordering for LLM-Based Social Surveys

Yuanzi Li, Lingjie Wang, Zihang Tian, Lei Wang+1

Renmin University of China

LLM Inference Efficiency LLM Prompting Large Language Model-Based Social Simulation

Open paper AskPDF arXiv

Large Language Models (LLMs) offer a scalable way to simulate survey respondents using demographic profiles and observed reference responses. However, the conventional approach of predicting one question per prompt repeatedly encodes the same context, limits each target to a narrow set of reference responses, and prevents later predictions from using information in earlier answers. Predicting multiple questions in one prompt can reduce these costs, share a broader pool of references, and let later predictions build on earlier ones. This requires forming coherent batches, selecting shared references, and ordering questions and references effectively. We propose Semantically Coherent Batching and Ordering (SCBO), a training-free framework that addresses these challenges. SCBO first uses an LLM to extract compact semantic representations from survey items and filter out template noise. It then groups related questions into batches and builds a shared reference bank using target-specific retrieval and centroid-based completion. Finally, it orders target questions from easy to hard and arranges references according to their semantic alignment with those questions. Experiments on four large-scale survey datasets and four LLMs show that SCBO substantially reduces token consumption and inference time while generally improving prediction accuracy over a non-batched baseline. Code is available at https://anonymous.4open.science/r/SCBO-41D8.

Abstract

Information Retrieval

Explore similar work

SCBO: Semantically Coherent Batching and Ordering for LLM-Based Social Surveys

Simulating Respondents, Not Single Questions: Coherent Survey Generation with Large Language Models

Silicon Sampling via Cross-Survey Transfer