Modern recommendation systems face substantial memory pressure, largely from the sparse embedding layer that maps high-cardinality categorical features into dense vectors. In-training embedding pruning (ITEP) removes less important embedding rows during training, yielding end-to-end memory savings but sometimes degrading accuracy because per-table pruning ratios are set by size-based heuristics. We use feature importance (FI) to guide these ratios (FI-ITEP), assigning higher pruning to tables tied to less important features. Across three large-scale recommendation models, FI-ITEP achieves 20--40% total memory reduction while matching or improving accuracy relative to percentile-based ITEP. Because the feature-importance signal requires minimal resources to compute, these memory savings translate into concrete systems benefits: the same models train on fewer GPU trainers (e.g., 112 to 80) with higher per-GPU throughput, saving 1,000+ GPU-hours per model launch. We reproduce these trends on the open-source DLRM (Criteo 1TB) and quantify the end-to-end training-time and GPU-cost tradeoff---including the burden of computing FI---offering practical guidance for embedding pruning in real-world systems.
Figures & tables
Model
# of Sparse Features
Raw Emb Size
# of GPUs
FI-ITEP (Shuffle)
(Approximate Value)
(Approximate Value)
Memory Saving
Avg. Eval NE Gain (%)
Rec. Model 1
555
390 GB
32
≈ 13.6%
36.4%
Rec. Model 2
1012
1.1 TB
128
≈ 14.2%
51.3%
Rec. Model 3
423
80 GB
32
≈ 8.3%
32.0%
Table 1: A summary of the impact of FI-ITEP on DLRMs of varying sizes. The "Memory Saving" and "Avg. Eval NE Gain (%)" columns indicate relative percentage improvements over the percentile-based ITEP for these models.
Figure 1: Example pruning schedules for Recommendation Model 1 under the percentile, shuffle-FI, and SVS-FI strategies; color encodes each table’s assigned pruning ratio. Embedding tables with low feature importance above a predefined minimum table size are assigned higher pruning ratios.
Memory Reduction
Batches
FI-ITEP AUROC
Percentile-ITEP AUROC
Δ AUROC
50%
200
0.6395
0.6176
+0.0218
50%
1000
0.6920
0.6851
+0.0068
80%
200
0.6268
0.6223
+0.0045
80%
1000
0.6932
0.6932
+0.0001
Table 2: FI-ITEP vs. Percentile-ITEP on the open-source DLRM (Criteo 1TB) at matched memory reduction. “Memory Reduction” is the fraction of sparse-embedding memory removed.
Configuration
Memory
Training Time
Speedup vs. Baseline
Baseline
1,476 MB
130.7s
—
FI-ITEP 50%
738 MB
107.1s
18% faster
Percentile-ITEP 50%
738 MB
107.7s
18% faster
FI-ITEP 80%
295 MB
93.4s
29% faster
Percentile-ITEP 80%
295 MB
112.2s
14% faster
Table 3: Training-time speedup on the open-source DLRM (1000 batches) at matched memory footprint.
Metric
Baseline (Unpruned)
FI-ITEP (96 Trainers)
FI-ITEP (80 Trainers)
Number of trainers (GPUs)
112
96
80
P90 Global QPS
985K
879K
760K
P90 QPS/GPU
8,800
9,161
9,501
P90 Per-GPU QPS %
0%
4%
8%
Equivalent GPU-Hour Saving
0 hr
54 hr
108 hr
GPU Max Utilization
75%
80%
93%
Table 4: End-to-end GPU savings with FI-ITEP via trainer reduction on an industry-scale model.
Metric
Model 1
Model 2
Model 3
Total Sparse Features
555
1012
423
Total training data
55 GB
80 GB
50 GB
FI data sampled
1–2 GB
1–2 GB
1–2 GB
FI computation time
1h 37m
1h 19m
45m
Full run duration
38h
11h 58m
15h 20m
Total FI overhead
4.9%
11%
4.3%
Table 5: Feature Importance (FI) computation overhead across industry-scale models.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Example feature importance values computed using shuffle and Shapley Value Sampling (SVS) for Recommendation Model 1. A floor is applied to raise all zero feature importance values to the minimum nonzero feature importance value.
Model 1
Model 2
Model 3
Metric
1 mo
2 mo
1 mo
2 mo
1 mo
2 mo
Top-100 FI score correlation (Spearman)
0.90
0.78
0.85
0.67
0.92
0.81
% change in top-100 features
9.5%
21.2%
11.8%
34.1%
8.9%
20.5%
Appendix
Table 6: FI score drift (Shapley sampling) across one- and two-month retraining intervals.