Towards Effective Orchestration of AI x DB Workloads

Naili Xing; Haotian Gao; Zhanhao Zhao; Shaofeng Cai; Zhaojing Luo; Yuncheng Wu; Zhongle Xie; Meihui Zhang

cs.DBMar 4, 2026

Towards Effective Orchestration of AI x DB Workloads

Authors: Naili Xing, Haotian Gao, Zhanhao Zhao, Shaofeng Cai, Zhaojing Luo, Yuncheng Wu, Zhongle Xie, Meihui Zhang, +1 more

Organizations: National University of Singapore · Beijing Institute of Technology · Renmin University of China · Zhejiang University

Abstract

AI-driven analytics are increasingly crucial to data-centric decision-making. Executing relational and AI operators in separate runtimes prevents the database optimizer and runtime from coordinating operator ordering, model placement, batching, and state reuse. Integrating AI operators into database engines enables such coordination but raises challenges in jointly optimizing query processing and model execution, scheduling under resource contention, and reusing relational intermediates and AI artifacts. This paper formalizes AIxDB workloads as iterative, concurrent, and shareable executions that interleave relational and AI operators. We then advocate database-native orchestration as a paradigm for redesigning database engines for these workloads and distill two design principles: holistic AIxDB co-optimization and unified AIxDB cache management. We present NeurEngine as a proof-of-concept prototype and report preliminary results illustrating the performance benefits of database-native orchestration

Explore similar work

Larch: Learned Query Optimization for Semantic Predicates
Jun 6, 2026Fuheng Zhao, Pawel Liskowski, Zihan Li +5Text-To-Sql Symbolic Predicates
Snowflake Inc. USA · Snowflake Inc. Poland
Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries
Aug 10, 2026Matthew Russo, Yash Agarwal, Tianyu Li +5Data Lakes Taxonomy-Driven Dataflow Model
MIT
SpecDB: LLM-Generated Customized Databases via Feature-Oriented Decomposition
May 29, 2026Yunkai Lou, Longbin Lai, Shunyang Li +2Large Databases Workload
Alibaba Group Hangzhou, China · Zhejiang GongshangYing ZhangUniversity Hangzhou, China

Jun 6, 2026, cs.DBJ/K move · Enter open · S save

Larch: Learned Query Optimization for Semantic Predicates

Fuheng Zhao, Pawel Liskowski, Zihan Li, Benjamin Han+4

Snowflake Inc. USA · Snowflake Inc. Poland

Text-To-Sql Symbolic Predicates Unstructured Data

Open paper AskPDF arXiv

With the advent of Large Language Models (LLMs), many database systems introduced semantic operators that enabled analytical queries over unstructured data (e.g. text, images, videos). Semantic operators typically incur high inference costs and latencies making semantic (AI) SQL queries challenging to apply on large scale datasets. At the same time, their semantic nature leads database engines to treat them as black boxes, making AISQL queries difficult to optimize. In this paper, we introduce Larch, a framework for optimizing the execution of semantic filters in AI SQL queries. Larch was inspired by two key observations: i) the high latency of semantic operators leaves significant room for computationally-heavy runtime optimization techniques, ii) unstructured data are typically accompanied by semantic information in the form of embeddings allowing for efficient semantic comparisons between AI_FILTER prompts and data values. Based on these two key observations, we present two Larch variants: Larch-A2C and Larch-Sel. Larch-A2C encodes arbitrary semantic filters expression tree using an embedding-augmented Gated Graph Neural Network and formulates the filter evaluation order as a Markov decision process. In contrast, Larch-Sel leverages a supervised learning model to predict filter selectivities, subsequently applying dynamic programming to find a near-optimal evaluation order for each input row. Evaluated across diverse real-world datasets and comprehensive synthetic workloads, both Larch variants always outperform existing semantic filter optimization techniques in terms of token usage. Our results demonstrate that Larch is robust across diverse workloads, reducing total token cost overhead by 3x-19x compared to Palimpzest and Quest.

Abstract

Explore similar work

Larch: Learned Query Optimization for Semantic Predicates

Carnot: Interpretable, Interactive, and Optimized Execution of Deep Research Queries

SpecDB: LLM-Generated Customized Databases via Feature-Oriented Decomposition