Generative retrieval models have been increasingly adopted in search and recommendation systems. However, industrial generative retrieval models are commonly specialized to a fixed online scenario: they learn from scenario-specific interactions, support a narrow set of query forms and retrieval objectives. We study how to remove these couplings and build a unified end-to-end multimodal search system for open-ended image--text queries, heterogeneous shopping intents, and online scenarios with different resource constraints. We present PailitaoMMSearch, which directly generates product Semantic IDs from a raw query image and free-form instruction and scales along three dimensions. Data scaling uses a reusable toolset that can be composed into intent- and lifecycle-specific workflows for cold-start synthesis, quality control, real-traffic refinement, and continuous data production. Intent scaling progressively equips one model with e-commerce product knowledge, multiple retrieval relations, and adjustable reasoning budgets through continual pre-training (CPT), mixed-budget supervised fine-tuning (SFT), and unified reinforcement learning (RL). The RL stage combines intent-balanced optimization and Sample--Beam rollouts aligned with multi-candidate serving. Scenario scaling exposes quality--cost operating points across model capacity and reasoning budget. The system is deployed in multimodal search applications on Taobao Pailitao. A controlled online A/B test reports statistically significant improvements, including item page views +17.90%, transactions +27.04%, and gross merchandise value +8.94%.
Figures & tables
Figure 1. PailitaoMMSearch2.0 overview. Data scaling expands supervision, intent scaling learns a unified policy, and scenario scaling exposes the learned capability under different resource envelopes.
Workflow
Coarse
Fine
QTG
CoT
QC
Cold-start synthesis
✓
✓
✓
∘
✓
Real-traffic refinement
∘
✓
∘
∘
✓
Continuous production
✓
✓
∘
∘
✓
Table 1. Tool composition for the three data-production workflows. QTG and QC denote Query-Text-Gen and Quality-Control.
Similar
Editing
Matching
Proxy/cold-start training
24M
24M
12M
Refined real-traffic training
1M
8M
1M
Automatic test
502
772
371
Evaluator calibration
40,000
20,000
40,000
Table 2. Dataset statistics for training, automatic evaluation, and evaluator calibration.
Intent
Good Prec. ( ≥2 )
Excellent Prec. ( =3 )
Similar
0.924
0.936
Editing
0.935
0.939
Matching
0.932
0.936
Table 3. Automatic-evaluator precision against disjoint human annotations.
System
Similar
Editing
Matching
Macro
Avg. RT (ms)
GPU devices
NDCG
GR
ER
NDCG
GR
ER
NDCG
GR
ER
NDCG
GR
ER
Qwen3.8-Flash-Next
0.885
74.5%
47.5%
0.894
66.7%
47.9%
0.892
74.4%
61.1%
0.890
71.9%
52.2%
1,075
4
GPT6-luna
0.883
71.3%
44.7%
0.859
60.0%
43.5%
0.930
80.4%
67.3%
0.891
70.6%
51.8%
3,646
–
Internal recall-and-ranking pipeline
0.874
70.6%
54.1%
0.903
64.0%
50.1%
0.745
51.4%
44.2%
0.841
62.0%
49.5%
780
–
PailitaoMMSearch2.0-0.8B low-think
0.887
64.3%
44.6%
0.877
51.2%
35.5%
0.944
81.1%
70.9%
0.903
65.5%
50.3%
187
1
PailitaoMMSearch2.0-2B low-think
0.907
70.6%
50.5%
0.891
57.4%
40.6%
0.955
84.7%
74.3%
0.918
70.9%
55.1%
278
1
Table 4. Offline retrieval quality and serving cost across the three intents. Higher NDCG, GR, and ER are better; lower average response time (RT) and fewer GPU devices are better. Best and second-best quality and RT results are shown in bold and underlined, respectively.
Data volume
Data setting
Sim. ER
Edit ER
Match ER
Macro ER
30%
–
40.48%
33.73%
64.76%
46.32% (−2.71pp)
70%
–
43.91%
35.29%
64.24%
47.81% (−1.22pp)
100%
–
45.56%
35.43%
66.09%
49.03%
100%
w/o real traffic
44.46%
34.57%
65.71%
48.25% (−0.78pp)
100%
w/o real traffic w/o quality control
44.88%
34.27%
64.77%
47.97% (−1.06pp)
Table 5. Effect of training-data scale and data-production components on the 2 B SFT model.
Initialization
Macro GR
Macro ER
Base VLM + SFT
62.5% (−0.8pp)
47.8% (−1.2pp)
CPT + SFT
63.3%
49.0%
Table 6. Effect of continual pre-training under a fixed downstream SFT configuration.
SFT intent mixture
Similar ER
Editing ER
Matching ER
Editing only
–
33.11%
–
Similar + Editing
44.63%
35.88%
–
Similar + Editing + Matching
45.56%
35.43%
66.09%
Table 7. Cross-intent transfer as the SFT supervision mixture expands.
Training setting
Similar ER
Editing ER
Matching ER
Macro ER
SFT
45.56%
35.43%
66.09%
49.03% (−5.97pp)
Standard RL
46.79%
37.37%
73.93%
52.70% (−2.3pp)
Sample–Beam RL
49.65%
40.48%
74.88%
55.00%
Table 8. Effects of unified RL and Sample–Beam rollouts under matched initialization, intent mixture, and reward evaluation.
Figure 2. Case study of the three reasoning budgets on representative queries. No-think answers directly, Low-think isolates the decisive visual and textual constraints, and High-think adds a longer trace. The examples illustrate the effect of the extra reasoning.
Params
Budget
Similar ER
Editing ER
Matching ER
Macro ER
Avg. tokens
RT (ms)
0.8 B
No-think
46.22%
35.67%
70.45%
50.78%
0
136
Low-think
44.55%
35.51%
70.87%
50.31%
9.0
187
High-think
43.51%
35.09%
70.21%
49.60%
51.7
326
2 B
No-think
49.71%
40.70%
73.45%
54.62%
0
210
Low-think
50.54%
40.60%
74.25%
55.13%
8.6
278
High-think
49.65%
40.48%
74.88%
55.00%
54.4
544
Table 9. Effects of model scale and inference-time reasoning budget on retrieval quality, reasoning tokens, and response time. Quality columns report ER.
The evolution of e-commerce has fundamentally transformed how users search for products, shifting from simple text-based keyword queries to complex multimodal interactions that seamlessly combine product images, natural language descriptions, and mixed-intent instructions. However, existing approaches face a critical dilemma: single-modal specialist models, deployed independently for text retrieval, visual search, and voice recognition, operate in isolation and cannot handle cross-modal queries, while general-purpose vision-language models lack the domain-specific knowledge necessary for fine-grained product understanding, user behavior modeling, and commercial intent reasoning. In this work, we present Pailitao-MMSearch, one native e-commerce multimodal search foundation model designed to bridge this gap. Our approach introduces three key innovations: (1)HybSID (Hybrid Semantic ID);(2)a two-stage continual pre-training strategy; and (3)a hybrid reasoning post-training pipeline. Built upon Qwen and deployed on Taobao's Pailitao multimodal search platform, Pailitao-MMSearch achieves substantial improvements in online A/B testing, including up to +13.61% in Gross Merchandise Volume (GMV) and +8.21% in transaction volume compared to traditional multi-modal search pipeline, demonstrating the effectiveness of our native e-commerce multimodal search large language models.
Generative retrieval has demonstrated strong performance by directly generating product semantic identifiers (SIDs). Extending this paradigm to image search, however, is nontrivial because real-world query images contain diverse information, including the search target, useful auxiliary evidence, and irrelevant visual content. This requires the model to identify and focus on the search target while selectively utilizing auxiliary evidence. In this paper, we propose PailitaoGR, a Latent Think-with-Images method for generative image retrieval, which internalizes target-focused perception and selective auxiliary-evidence utilization into a the generative retrieval model, enabling Zooming without Cropping and Reading without OCR. Specifically, we design a target-focused perception mechanism that identifies and enhances visual tokens of the search target, consisting of a target Enhancer and a learning strategy based on on-policy distillation and attention guidance loss, enabling the model to focus on search-target regions. We also design a selective auxiliary-evidence utilization mechanism that identifies and enhances visual tokens of auxiliary evidence, including an auxiliary enhancer and an in-capacity incremental contrastive distillation strategy, enabling the model to exploit auxiliary evidence. We construct training and validation sets sampled from real-world online image-search logs. Experiments show that our method outperforms existing baselines by an average of 13.8%, validating its effectiveness.
Modern retrieval systems must both be automated and interactive, allowing users to search and refine results in real time. We present AiSearch, a flexible multimodal retrieval framework that leverages the zero shot capabilities of Vision Language Models (VLMs) for natural language search over images and videos. AiSearch supports interactive search refinement through user feedback to tailor results to the user's intent, and allows visual benchmarking across multiple VLMs, enabling users to select the most suitable model for their task.
Ali Koksal, Mei Chee Leong, Vicky Sintunata +2
Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore