SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment
Organizations: JD.com Beijing, China
Abstract
Large language models offer transformative potential for e-commerce search by enabling intent-aware recommendations. However, their industrial deployment is hindered by two critical challenges: (1) knowledge hallucination due to insufficient encoding of dynamic, fine-grained product knowledge, and (2) security vulnerabilities under jailbreak attacks that threaten compliance. To address these issues, we propose SIA--a Synthesize-Inject-Align framework for building knowledgeable and secure e-commerce search LLMs. Our approach first synthesizes high-quality natural language corpus by combining structured knowledge graphs with unstructured behavioral logs, augmented with reasoning chains and safety-aware data. We then introduce a parameter-efficient pre-training strategy based on Depth Up-Scaling to inject domain knowledge while preserving general capabilities. Finally, a dual-path alignment method via multi-task instruction tuning and adversarial training strengthens both task performance and safety robustness. The framework has been deployed at JD.com, China's largest self-operated e-commerce platform, where A/B tests across five core search scenarios demonstrate significant improvements in key business metrics, validating its industrial effectiveness and scalability.
Figures & tables
| Model | EKQA | PAU | EQEC | C-EVAL | CMMLU | MMLU | GSM8K | MMLU-Pro |
|---|---|---|---|---|---|---|---|---|
| Qwen2.5-14B-Instruct (baseline) | 23.87 | 50.65 | 39.04 | 83.35 | 84.09 | 78.69 | 94.8 | 63.7 |
| SIA-15B (Ours) | 58.01 | 91.72 | 74.29 | 83.98 | 84.94 | 78.45 | 94.9 | 66.82 |
| deepseek-v3 | 25.82 | 57.72 | 51.20 | / | / | / | / | / |
| Type | Qwen-14B | dp-V3 | gpt4o | SIA-15B |
|---|---|---|---|---|
| SP | 34.16 | 33.15 | 30.58 | 28.60 |
| SE | 13.16 | 10.73 | 9.32 | 10.48 |
| Prohibited | 10.85 | 10.11 | 9.97 | 7.81 |
| RI | 13.94 | 12.21 | 10.28 | 14.08 |
| CNC | 11.61 | 14.65 | 13.45 | 9.34 |
| Religious | 19.59 | 18.85 | 16.32 | 11.11 |
| Variants | EKQA | PAU | EQEC | C-Eval | CMMLU | MMLU | GSM8K | MMLU-Pro | JDSec-avg |
|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-14B-Instruct (baseline) | 23.87 | 50.65 | 39.04 | 83.35 | 84.09 | 78.69 | 94.8 | 63.7 | 16.68 |
| SIA-15B (Full) | 58.01 | 91.72 | 74.29 | 83.98 | 84.94 | 78.45 | 94.9 | 66.82 | 13.65 |
| w/o Context-Aware Synthesis | 41.89 | 74.56 | 54.72 | 83.52 | 84.21 | 78.03 | 94.7 | 64.51 | 15.22 |
| w/o Depth Up-Scaling | 44.73 | 77.89 | 57.96 | 82.54 | 83.02 | 77.51 | 94.6 | 63.05 | 15.51 |
| w/o Common Replay | 59.12 | 92.15 | 75.33 | 81.07 | 82.05 | 76.02 | 94.5 | 61.03 | 14.03 |
| w/o CPT | 3015 | 64.88 | 47.89 | 83.56 | 84.13 | 78.54 | 94.8 | 63.82 | 16.01 |
| Model | CSR | AIGCS | PE | SOCA | TS |
|---|---|---|---|---|---|
| SIA-15B | 72.03 | 80.71 | 91.43 | 86.67 | 75.47 |
| baseline | 42.71 | 60.18 | 59.46 | 61.25 | 48.81 |
E-Commerce Search
E-commerce search studies how to match shoppers’ queries, preferences, and product signals with useful items in online marketplaces.
See all 42 papers