Delay-Guided Attention for Quality Prediction in Multistage Manufacturing

Jiarui Zhang; Zhaoyang Guan; Zhixuan Xia; Naichen Shi

cs.LGOct 21, 2026

Delay-Guided Attention for Quality Prediction in Multistage Manufacturing

Authors: Jiarui Zhang, Zhaoyang Guan, Zhixuan Xia, Naichen Shi

Organizations: Department of Engineering Sciences and Applied Mathematics, Northwestern University, Evanston, IL, USA · Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, IL, USA

Abstract

Quality prediction in multistage manufacturing is challenging when inter-stage lags vary over time, as measurements taken at the same timestamp may reflect different process states. Existing approaches often preserve production order but fuse timestamp-aligned features or model delayed dependencies implicitly, without identifying the upstream history relevant to each downstream observation. We propose Delay-Guided Attention for Multi-stage Forecasting (DAMF), which explicitly estimates inter-stage lag distributions and uses them to guide cross-stage information transfer for final quality prediction. For each adjacent stage pair, a lag predictor compares the current downstream representation with candidate lagged upstream representations. An occurrence-magnitude factorization separately estimates the probability of a positive lag and its conditional magnitude, yielding a sample- and stage-pair-specific distribution over discrete lag states without requiring plant-specified delays at inference. DelayAlignment incorporates this soft distribution as a temporal prior into differentiable cross-stage attention, while gated residual fusion regulates information propagation across production stages. For interpretation, segment-wise Viterbi decoding converts pointwise lag distributions into temporally coherent lag paths, whereas forecasting retains the soft distributions for alignment. On the real-world LiquidSugar dataset, DAMF improves final quality prediction over the compared baselines. Evaluations on two controlled-delay benchmarks show more accurate and stable lag recovery, and ablation studies support the contributions of lag prediction, delay-guided attention, and structured temporal decoding.

Figures & tables

Figure 1: Schematic illustration of the sequential multi-stage Liquid Sugar production process.

Figure 2: Overall architecture of DAMF. Edge-wise lag distributions guide DelayAlignment and gated residual fusion across successive processing stages.

Figure 3: Illustration of an inter-stage lag. An upstream process feature observed at $t-k$ corresponds to a downstream feature observed at $t$ , giving a lag of $k$ sampling steps.

Figure 4: Transition-score structure for lag-path decoding under the experimental parameter settings. Darker cells indicate higher scores and smaller penalties.

Figure 5: Lag-guided alignment and fusion for an adjacent-stage pair. DelayAlignment combines content-based matching with the estimated lag distribution to compute attention weights over upstream candidates. The resulting upstream message is combined with the current-stage representation through gated residual fusion.

Figure 6: Lag-recovery comparison on the two controlled-delay benchmarks. Each panel reports one lag-recovery metric for MNet, WCC, and DAMF. MNet uses the hidden-lag output of MultistageNet, WCC combines window-based cross-correlation with a conventional Viterbi decoder, and DAMF denotes the proposed lag-recovery pipeline and is highlighted in red.

Figure 7: Representative lag-recovery examples on the controlled-delay benchmarks. The blue curves denote the ground-truth lag paths, while the orange curves and shaded regions represent the mean decoded paths and their ranges across random seeds, respectively. Each benchmark contains both block-shaped and bump-shaped delay patterns.

Setting	Occ F1 $\uparrow$	Lag MAE $\downarrow$	$\pm 1$ Acc $\uparrow$	Large-lag Recall $\uparrow$	Collapse to 0 $\downarrow$
Full lag-prediction objective	0.91 $\pm$ 0.02	0.65 $\pm$ 0.13	0.90 $\pm$ 0.02	0.76 $\pm$ 0.05	0.03 $\pm$ 0.03
Occurrence + positive CE only	0.90 $\pm$ 0.02	0.82 $\pm$ 0.15	0.87 $\pm$ 0.02	0.68 $\pm$ 0.05	0.20 $\pm$ 0.09
w/o relaxed-TopN loss	0.89 $\pm$ 0.04	0.94 $\pm$ 0.15	0.86 $\pm$ 0.02	0.76 $\pm$ 0.13	0.10 $\pm$ 0.08
w/o positive-lag CE loss	0.64 $\pm$ 0.11	2.77 $\pm$ 0.08	0.60 $\pm$ 0.01	0.14 $\pm$ 0.02	0.65 $\pm$ 0.04
w/o large-occurrence recall loss	0.83 $\pm$ 0.07	1.32 $\pm$ 0.23	0.82 $\pm$ 0.03	0.62 $\pm$ 0.16	0.23 $\pm$ 0.17
w/o selector loss	0.86 $\pm$ 0.05	0.91 $\pm$ 0.39	0.85 $\pm$ 0.05	0.65 $\pm$ 0.15	0.12 $\pm$ 0.09

Table 8

Setting	Acc $\uparrow$	$\pm 1$ Acc $\uparrow$	Lag MAE $\downarrow$	Large-lag Recall $\uparrow$	Collapse to 0 $\downarrow$	Collapse to small $\downarrow$
Raw argmax	0.65 $\pm$ 0.04	0.79 $\pm$ 0.04	1.03 $\pm$ 0.12	0.55 $\pm$ 0.04	0.02 $\pm$ 0.02	0.44 $\pm$ 0.03
Viterbi	0.76 $\pm$ 0.05	0.90 $\pm$ 0.02	0.65 $\pm$ 0.13	0.76 $\pm$ 0.05	0.03 $\pm$ 0.03	0.22 $\pm$ 0.03

Table 2: Effect of Viterbi decoding on lag recovery. The best numerical result for each metric is shown in bold.

Figure 8: Path-level comparison between raw pointwise argmax predictions and Viterbi decoding on representative test regions. The left column shows raw argmax predictions before decoding, whereas the right column shows the lag paths after Viterbi decoding. The blue curve denotes the ground-truth lag path, the orange curve denotes the prediction mean across random seeds, and the shaded region denotes the corresponding prediction range.

Figure 9: Sensitivity analysis of the lag prior weight. The four panels report RMSE, $\Delta$ RMSE, MAE, and $R^{2}$ under different prior-weight settings.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 12 — Figure A1: Lag recovery results of the edge-level lag predictor on two controlled-delay datasets. The blue line denotes the ground-truth lag path, the orange line denotes the mean Viterbi-decoded lag path across seeds, and the shaded region denotes the corresponding range across seeds 42, 43, and 44.

Figure B1: Effect of Viterbi decoding on lag-path recovery across five test regions. The left and right columns show the raw point-wise lag predictions and the corresponding Viterbi-decoded lag paths, respectively. Solid colored curves and shaded bands denote the mean and range across three random seeds, while blue curves represent the ground-truth lag paths. Vertical dashed lines delimit the evaluation intervals, and the reported MAE is computed within each interval.

Manufacturing

Manufacturing research develops methods and systems to design, plan, automate, monitor, and improve industrial production.

See all 77 papers

Quantile-Led Feature Extraction for Multi-Horizon Predictive Maintenance in Industrial Manufacturing Systems
Sep 7, 2026David J Poland, Daniele Ravi, Na HelianQuantile Regression Predictive Maintenance
Department of Computer Science, University of Hertfordshire, College Lane, Hatfield, AL10 9AB, United Kingdom
Long Horizon Transformer Quantile Fault Prediction for Multi Site Industrial Predictive Maintenance
Sep 4, 2026David J Poland, Daniele Ravi, Na HelianQuantile Regression Multivariate Time Series Classification
Department of Computer Science, University of Hertfordshire, Hatfield AL10 9AB, U.K.
LLM4Delay: Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation
Oct 24, 2025Thaweerath Phisannupawong, Joshua Julian Damanik, Han-Lim ChoiAir Traffic Management Intelligent Transportation Systems
Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea.

Sep 7, 2026J/K move · Enter open · S save

Quantile-Led Feature Extraction for Multi-Horizon Predictive Maintenance in Industrial Manufacturing Systems

David J Poland, Daniele Ravi, Na Helian

Department of Computer Science, University of Hertfordshire, College Lane, Hatfield, AL10 9AB, United Kingdom

Quantile Regression Predictive Maintenance Time Series Representation Learning

Open paper AskPDF arXiv

In data-driven predictive maintenance (PdM), feature extraction is usually treated as fixed preprocessing: a descriptor set is chosen once and reused while the downstream model or forecasting horizon changes. This paper isolates the representation-learning stage and presents a quantile-led feature-extraction framework based on a dual-stage MLP-QRNN hierarchy. QRNN1 learns a broad ten-quantile conditional distribution for each sensor channel, while skip-connected QRNN2 refines a retained mid-tail quantile set into compact, channel-resolved, distribution-aware features. A fixed thirteen-pipeline ablation spans 1-hour, 70-hour, and 30-day regimes across 72 machines in 9 industrial facilities, with the downstream temporal classifier held fixed within each regime. Increasing the retained mid-tail set from two to four quantiles improves 30- and 60-minute F1-score, reaching 75.92% and 72.44% with attention enabled. The results also show that representations do not transfer reliably beyond their design horizon unless feature capacity, temporal embedding, activation strategy, and sensor breadth are scaled with the forecasting task. The unmodified short-horizon extractor falls to 42.90% F1 at 70 hours, whereas horizon-conditioned extractors reach 60.38% at 70 hours and 79.97% at 30 days. The framework therefore supports treating PdM feature extraction as a horizon-dependent representational stage rather than fixed preprocessing.

Abstract

Manufacturing

Explore similar work

Quantile-Led Feature Extraction for Multi-Horizon Predictive Maintenance in Industrial Manufacturing Systems

Long Horizon Transformer Quantile Fault Prediction for Multi Site Industrial Predictive Maintenance

LLM4Delay: Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation