cs.DBSep 16, 2026

Efficiently Linking Unstructured Data for Multi-step Reasoning

Authors: Jiaming LiangHaydn JonesJacob R. GardnerMark YatskarZachary Ives

Organizations: University of Pennsylvania Philadelphia, PA, USA

Abstract

Modern LLMs and AI agents increasingly support data engineering workflows that integrate evidence from unstructured sources. Such pipelines typically do data retrieval, integration, and ranking before proceeding to more complex agentic reasoning or actions, e.g., for scientific discovery. The core retrieval problem in these workflows jointly executes multi-attribute filtering, multi-vector search, exact relational joins, and thresholded embedding-similarity joins. Given a planned query and monotone scoring function, our DASE query engine constructs and ranks candidate evidence tuples. It comprises (i) a multi-step reasoning query model over structured predicates, multiple vectors, and relational links; (ii) SemJI, a sparse materialized embedding-similarity join index for rare near-neighbor pairs; and (iii) a co-designed execution layer that combines predicate-aware ANN traversal, batched access, and threshold-based score aggregation. On scientific-discovery workloads, DASE retrieves candidate evidence for multi-step reasoning queries 6x to 46x faster than strong RDBMS, rerank, and vector-database baselines at comparable recall; and for tasks that require semantic-operator post-processing, DASE acts as a high-recall prefilter that makes downstream LLM evaluation both cheaper and more accurate -- e.g., on SemBench E-Commerce it improves BigQuery quality from 0.67 to 0.80 while cutting cost from 2.42to2.42 to 0.54.

Explore similar work

CardsList
  1. SemJoin: Semantic Join Optimization

    Jun 28, 2026Christopher Gou, Aditya Banerjee, Jiaxuan Wang +1Data LakesNatural Language Queries

  2. Baikal: Structured Search for Deep Research over Data Lakes

    Jul 30, 2026Dhruv Agarwal, Rishitha Guttapalle Mohan, Aarti Kumari +5Data LakesDeep Research