Large language models offer transformative potential for e-commerce search by enabling intent-aware recommendations. However, their industrial deployment is hindered by two critical challenges: (1) knowledge hallucination due to insufficient encoding of dynamic, fine-grained product knowledge, and (2) security vulnerabilities under jailbreak attacks that threaten compliance. To address these issues, we propose SIA--a Synthesize-Inject-Align framework for building knowledgeable and secure e-commerce search LLMs. Our approach first synthesizes high-quality natural language corpus by combining structured knowledge graphs with unstructured behavioral logs, augmented with reasoning chains and safety-aware data. We then introduce a parameter-efficient pre-training strategy based on Depth Up-Scaling to inject domain knowledge while preserving general capabilities. Finally, a dual-path alignment method via multi-task instruction tuning and adversarial training strengthens both task performance and safety robustness. The framework has been deployed at JD.com, China's largest self-operated e-commerce platform, where A/B tests across five core search scenarios demonstrate significant improvements in key business metrics, validating its industrial effectiveness and scalability.
Figures & tables
Figure 1. SIA Overview
Figure 2. safety scenario
Model
EKQA
PAU
EQEC
C-EVAL
CMMLU
MMLU
GSM8K
MMLU-Pro
Qwen2.5-14B-Instruct (baseline)
23.87
50.65
39.04
83.35
84.09
78.69
94.8
63.7
SIA-15B (Ours)
58.01
91.72
74.29
83.98
84.94
78.45
94.9
66.82
deepseek-v3
25.82
57.72
51.20
/
/
/
/
/
Table 1. E - commerce Domain and General Competence Evaluation Results
Type
Qwen-14B
dp-V3
gpt4o
SIA-15B
SP
34.16
33.15
30.58
28.60
SE
13.16
10.73
9.32
10.48
Prohibited
10.85
10.11
9.97
7.81
RI
13.94
12.21
10.28
14.08
CNC
11.61
14.65
13.45
9.34
Religious
19.59
18.85
16.32
11.11
Table 2. Safety and Compliance Evaluation Results(lower is better)
Variants
EKQA
PAU
EQEC
C-Eval
CMMLU
MMLU
GSM8K
MMLU-Pro
JDSec-avg
Qwen2.5-14B-Instruct (baseline)
23.87
50.65
39.04
83.35
84.09
78.69
94.8
63.7
16.68
SIA-15B (Full)
58.01
91.72
74.29
83.98
84.94
78.45
94.9
66.82
13.65
w/o Context-Aware Synthesis
41.89
74.56
54.72
83.52
84.21
78.03
94.7
64.51
15.22
w/o Depth Up-Scaling
44.73
77.89
57.96
82.54
83.02
77.51
94.6
63.05
15.51
w/o Common Replay
59.12
92.15
75.33
81.07
82.05
76.02
94.5
61.03
14.03
w/o CPT
3015
64.88
47.89
83.56
84.13
78.54
94.8
63.82
16.01
Table 3. Ablation Results
Model
CSR
AIGCS
PE
SOCA
TS
SIA-15B
72.03
80.71
91.43
86.67
75.47
baseline
42.71
60.18
59.46
61.25
48.81
Table 4. Online Evaluation Results for Security and Compliance
Modern e-commerce search is evolving to resolve complex user intents. While Large Language Models (LLMs) offer strong reasoning, existing LLM-based paradigms face a fundamental blindness-latency dilemma: query rewriting is agnostic to retrieval capabilities and real-time inventory, yielding invalid plans; conversely, deep search agents rely on iterative tool calls and reflection, incurring seconds of latency incompatible with industrial sub-second budgets. To resolve this conflict, we propose Environment-Aware Search Planning (EASP), reformulating search planning as a dynamic reasoning process grounded in environmental reality. EASP introduces a Probe-then-Plan mechanism: a lightweight Retrieval Probe exposes the retrieval snapshot, enabling the Planner to diagnose execution gaps and generate grounded search plans. The methodology comprises three stages: (1) Offline Data Synthesis: A Teacher Agent synthesizes diverse, execution-validated plans by diagnosing the probed environment. (2) Planner Training and Alignment: The Planner is initialized via Supervised Fine-Tuning (SFT) to internalize diagnostic capabilities, then aligned with business outcomes (conversion rate) via Reinforcement Learning (RL). (3) Adaptive Online Serving: A complexity-aware routing mechanism selectively activates planning for complex queries, ensuring optimal resource allocation. Extensive offline evaluations and online A/B testing on JD.com demonstrate that EASP significantly improves relevant recall and achieves substantial lifts in UCVR and GMV. EASP has been successfully deployed in JD.com's AI-Search system.
LLM-based search agents are widely used for information-seeking tasks, but their reliance on external tool returns introduces a critical security risk: web content retrieved during execution is untrusted, exposing agents to prompt injection and goal hijacking. Prior work on search-agent safety primarily focuses on static web-content injection, but modern agents issue follow-up queries and cross-check competing sources, so a single injected page is often diluted or rejected. We show that the channel delivering search and page observations is a fragile security boundary: beyond exposing the agent to a single poisoned page, a mediated search interface can repeatedly steer how the agent gathers evidence and forms its final answer. Under a constrained tool-intermediary threat model, appending only one controlled result per query can substantially increase attack success when the evidence is coordinated across the agent's trajectory. We study this setting with a strategy-driven long-horizon attack system and introduce Authority-Chain Hijack (ACH), an expert-refined strategy that turns isolated search-result and page-content manipulations into a coherent evidence chain across seemingly corroborating sources. ACH achieves the highest Overall ASR among all baselines, reaching 55.9% / 83.3% ASR / MaxN ASR on the full SafeSearch test split. We further introduce Trace-Guided Strategy Evolution (TGSE), which automatically improves attacker strategies from execution traces, replacing manual redesign with trace-driven refinement; its strongest single setting reaches 71.4% / 95.0% in held-out evaluation.
Xuebin Li, Hanqing Zhao, Siyuan Liang +4
School of Cyber Science and Technology, University of Science and Technology of China · College of Computing and Data Science, Nanyang Technological University
Large Language Model (LLM)-based agents show promise for e-commerce conversational shopping, yet existing implementations lack the interaction depth and contextual breadth required for complex product research. Meanwhile, the Deep Research paradigm, despite advancing information synthesis in web search, suffers from domain gaps when transferred to e-commerce. We propose ProductResearch, a multi-agent framework that synthesizes high-fidelity, long-horizon tool-use trajectories for training robust e-commerce shopping agents. The framework employs a User Agent to infer nuanced shopping intents from behavioral histories, and a Supervisor Agent that orchestrates iterative collaboration with a Research Agent to generate synthetic trajectories culminating in comprehensive, insightful product research reports. These trajectories are rigorously filtered and distilled through a reflective internalization process that consolidates multi-agent supervisory interactions into coherent single-role training examples, enabling effective fine-tuning of LLM agents for complex shopping inquiries. Extensive experiments show that a compact MoE model fine-tuned on our synthetic data achieves substantial improvements over its base model in response comprehensiveness, research depth, and user-perceived utility, approaching the performance of frontier proprietary deep research systems and establishing multi-agent synthetic trajectory training as an effective and scalable paradigm for enhancing LLM-based shopping assistance.