Mapping the Concept Landscape: Structural Perception of Global Distributions for Transparent Data Pruning
Organizations: State Key Laboratory of Multispectral Information Intelligent Processing Technology, School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan, China · Ant Group · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China
Abstract
Existing data pruning methods predominantly rely on high-dimensional feature embeddings to measure sample importance. However, these compressed vectors often obscure fine-grained semantic interactions, leading to suboptimal coverage of rare semantic concepts in the pruned subsets. In this paper, we propose Mapping the Concept Landscape (MCL), a novel structural perception framework for transparent data pruning. Instead of abstract embeddings, we represent each image-caption pair as an explicit sample-level graph comprising entities, events, and attributes. By integrating these individual graphs into a comprehensive dataset-level graph, we characterize the global distribution of semantic concepts and quantify their rarity across the entire corpus. Based on this structured perception, we develop a greedy concept-coverage maximization algorithm that iteratively selects samples to maximize the marginal gain of high-value, under-represented concepts. Experimental results on various benchmarks demonstrate that our method not only achieves superior pruning efficiency compared to state-of-the-art methods but also provides a transparent and interpretable audit trail for the selection process.
Figures & tables
| Benchmarks | Valid Data | MME-P | MME-C | SEED-I | POPE | MMMU | SQA | GQA | TextQA | Rel. |
| Baseline | 665k | 1476.9 | 267.9 | 67.4 | 86.4 | 32.8 | 70.0 | 63.0 | 58.2 | 100.0% |
| Random | 50k | 1387.5 | 287.5 | 59.7 | 85.7 | 32.2 | 68.4 | 55.0 | 53.1 | 95.7% |
| Length | 50k | 1357.0 | 265.7 | 47.0 | 82.6 | 33.9 | 60.9 | 55.5 | 45.2 | 89.1% |
| EL2N [ 35 ] | 50k | 1077.3 | 252.5 | 59.3 | 80.8 | 33.6 | 71.0 | 61.0 | 41.7 | 90.1% |
| GradN [ 35 ] | 50k | 1275.4 | 303.6 | 58.3 | 75.7 | 32.4 | 70.9 | 44.9 | 46.0 | 90.5% |
| IFD [ 24 ] | 50k | 1113.4 | 301.8 | 55.1 | 76.7 | 33.0 | 48.2 | 41.9 | 43.6 | 83.6% |
| Method | ||||||
| Baseline | 40.06 | 61.12 | 42.03 | 19.27 | 43.39 | 59.24 |
| Random | 38.26 1.80 | 58.91 | 39.77 | 17.83 | 41.19 | 56.69 |
| EL2N | 34.35 5.71 | 53.29 | 36.14 | 16.58 | 37.64 | 48.54 |
| InfoBatch | 34.95 5.11 | 53.86 | 36.55 | 17.00 | 38.74 | 49.05 |
| DivBS | 39.74 0.32 | 60.84 | 41.98 | 18.07 | 43.14 | 58.78 |
| PFB | 39.69 0.37 | 60.85 | 41.74 | 18.45 | 43.25 | 59.07 |
Dataset Pruning
Dataset pruning selects compact, informative training subsets to reduce computation while preserving or improving model performance.
See all 22 papers