cs.LGSep 12, 2026

Leakage-Safe and Scheduler-Aware Machine Learning for Grid Job Runtime Prediction

Authors: Ashfaq Ali Shafin, Khandaker Mamun Ahmed

Organizations: Augustana College, USA · Florida International University, USA · Dakota State University, USA

Abstract

Accurate job runtime prediction can improve scheduling-aware resource management in grid and distributed computing environments, but prediction models must be evaluated under realistic deployment constraints. This paper revisits CPU burst time prediction on the GWA-T-4 AuverGrid workload trace and reformulates it as leakage-safe pre-execution job runtime prediction. We define the target as job-level runtime, use only submission-time attributes, exclude post-execution variables, and evaluate models under temporal and cold-start settings rather than relying only on random cross-validation. We compare standard regressors, chronological historical baselines, categorical encoding strategies, and CatBoost with native categorical handling. We further add temporal hyperparameter tuning, runtime predictability analysis, feature ablation, error analysis by job length, and a minimal scheduling simulation. After temporal-validation tuning, CatBoost achieves the strongest deployment-oriented result with R^2=0.239, MAE=27,019, RMSE=46,587, and LogMAE=2.646 on the held-out temporal test set. A single-server simulation over all 69,523 held-out temporal test jobs shows that prediction-informed SJF reduces average waiting time by 50.92% relative to FCFS. The results show that random-split evaluation overestimates performance, categorical-native boosting improves temporal generalization, and long-job underestimation remains a scheduler-relevant challenge.

Figures & tables

Explore similar work

CardsList
  1. Duration-Informed Workload Scheduler

    Mar 7, 2026Daniela Loreti, Davide Leone, Andrea BorghesiHigh-Performance Computing

  2. SafeWall: A Risk-Aware Job Walltime Recommendation for HPC Scheduling

    Oct 19, 2026Kanglin Xu, Michael Papka, Zhiling LanHigh-Performance Computing

  3. Personalized Execution Time Optimization for Billion-Scale Scheduled Jobs

    Mar 11, 2022Yang Liu, Juan Wang, Idris Malik +7Learning to Rank