Operationalization of Machine Learning with Serverless Architecture: An Industrial Operationalization of Machine Learning with Serverless Architecture: An Industrial Implementation for Harmonized System Code Prediction

Sai Vineeth Kandappareddigari; Santhoshkumar Jagadish; Gauri Verma; Ilhuicamina Contreras; Christopher Dignam; Anmol Srivastava; Benjamin Demers

cs.LGFeb 19, 2026

Operationalization of Machine Learning with Serverless Architecture: An Industrial Operationalization of Machine Learning with Serverless Architecture: An Industrial Implementation for Harmonized System Code Prediction

Authors: Sai Vineeth Kandappareddigari, Santhoshkumar Jagadish, Gauri Verma, Ilhuicamina Contreras, Christopher Dignam, Anmol Srivastava, Benjamin Demers

Organizations: Governance Schneider Electric U.S.A Boston, USA · Tax and Customs Schneider Electric Mexico Mexico city, Mexico · Governance Schneider Electric Paris, France

Abstract

This paper presents a serverless MLOps framework orchestrating the complete ML lifecycle from data ingestion, training, deployment, monitoring, and retraining to using event-driven pipelines and managed services. The architecture is model-agnostic, supporting diverse inference patterns through standardized interfaces, enabling rapid adaptation without infrastructure overhead. We demonstrate practical applicability through an industrial implementation for Harmonized System (HS) code prediction, a compliance-critical task where short, unstructured product descriptions are mapped to standardized codes used by customs authorities in global trade. Frequent updates and ambiguous descriptions make classification challenging, with errors causing shipment delays and financial losses. Our solution uses a custom text embedding encoder and multiple deep learning architectures, with Text-CNN achieving 98 percent accuracy on ground truth data. Beyond accuracy, the pipeline ensures reproducibility, auditability, and SLA adherence under variable loads via auto-scaling. A key feature is automated A/B testing, enabling dynamic model selection and safe promotion in production. Cost-efficiency drives model choice; while transformers may achieve similar accuracy, their long-term operational costs are significantly higher. Deterministic classification with predictable latency and explainability is prioritized, though the architecture remains extensible to transformer variants and LLM-based inference. The paper first introduces the deep learning architectures with simulations and model comparisons, then discusses industrialization through serverless architecture, demonstrating automated retraining, prediction, and validation of HS codes. This work provides a replicable blueprint for operationalizing ML using serverless architecture, enabling enterprises to scale while optimizing performance and economics.

Figures & tables

Fig. 1 : Distribution of assurance levels in the training data

FEATURES	DESCRIPTION
Short Description	Technical description of each item or product along with information about product hierarchy and product name
Medium Description	Description about product family that contains similar products, and also includes product dimensions and technical specifications
ETIM	International classification standard for technical products

TABLE I : Features used in various models

Fig. 2 : Text length (number of characters) for combined description in training data

Fig. 3 : Class Imbalance as seen on training data

Panel of Figure 5 — Fig. 4 : Top to Bottom: Architecture Diagram for LSTM, DNN and Text-CNN Model (Specifications in the diagram are the same as those used in the experimentation)

Hyper-parameters	Range	DNN base model	DNN up-sampled model	LSTM (base & up-sampled)
Initial neurons	11- 174	11.01	168.01	25.0
Neuron Pct	0.35 - 1.0	0.44	0.95	1.0
Dropout	0.10 - 0.75	0.37	0.1	0.1
Neuron Shrink	0.25 - 0.95	0.31	0.466	0.95
Embedding Dim	11.0 - 87.0	65.74	43.3	78.0
N_layer	1.0 - 15.0	8.02	7.89	3.0

TABLE II : Hyper-parameters for DNN and LSTM base and up-sampled models

Model	Accuracy		Precision						Recall						F1
			$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80
	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U
DNN	89%	95%	15	19	5	1	3	3	12	20	5	2	6	1	14	19	5	2	4	2
LSTM	93%	97.50%	19	20	2	0	2	3	16	22	5	1	2	0	18	21	3	1	2	2
Text-CNN	98%	98%	19	20	2	1	2	2	20	21	2	2	1	0	19	20	2	2	2	1

TABLE III : Accuracy, Precision, Recall and F1 metrics for DNN, LSTM and Text-CNN on training data

Model	Accuracy		Precision						Recall						F1
			$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80
	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U
DNN	89%	95%	15	19	5	1	3	3	12	20	5	2	6	1	14	19	5	2	4	2
LSTM	92%	97.00%	19	20	2	0	2	3	16	22	5	1	2	0	18	21	3	1	2	2
Text-CNN	98%	98%	19	20	2	1	2	2	20	21	2	2	1	0	19	20	2	2	2	1

TABLE IV : Accuracy, Precision, Recall and F1 metrics for DNN, LSTM and Text-CNN on test data

Hyper-parameters	Range	Values
Filter Size	[64,128,256]	128
Kernel Size	1 - 10	[5]
Embedding Dim	50 - 150	[100]
N_layer	1.0 - 5.0	1.0

TABLE V : Hyper-parameters for Text-CNN Model

Precision	Recall	Evaluation	Type of F Score
High	High	Best case scenario - model detects HS code and predicts well with less misclassifications performs well on train and test	F0
High	Low	Model detects HS codes but with some misclassifications model performs better on training but not that well on tests	between F1-F2
Low	High	Over fitting -model predicts well on train but not on test set	between F0-F1
Low	Low	Model is not learning as Precision and recall are low in both train and test	between F1-F2

TABLE VI : F Score estimation to balance bias between Precision and Recall

Model	Hyper-parameter Tuning		Training		Total Time Taken
	Base Model	up-sampled Model	Base Model	up-sampled Model	Base Model	up-sampled Model
DNN	36 mins	2 hours 55 mins	15 mins	27 mins	51 mins	3 hours 36 mins
LSTM	4 hours	4.2 hours	1.5 hours	1.2 hours	5.5 hours	5.4 hours
Text-CNN	2 hours 20 min	1 hour 50 mins	55 mins	1 hour 20 mins	3 hours 15 mins	3 hours 10 mins

TABLE VII : Time taken for training proposed models

Fig. 5 : Machine learning model life-cycle

Text Classification

Text classification assigns text to categories such as topic, sentiment, intent, stance, or risk, supporting search, moderation, and decisions.

See all 226 papers

Jun 20, 2026

Load Testing for Machine Learning Model Serving Systems at Scale

Machine learning (ML) model serving has become a dominant consumer of GPU infrastructure, yet capacity planning in these systems remains largely ad hoc. Under-provisioning leads to service-level objective (SLO) violations and production incidents, while over-provisioning results in substantial resource waste. This paper presents \sys, an industrial load testing framework for ML serving systems that systematically estimates serving capacity through an adaptive, feedback-driven search strategy. The approach leverages real-time performance signals, incorporating dampening, spike tolerance, and convergence detection to efficiently identify maximum sustainable throughput under SLO constraints. We evaluate \sys through a longitudinal analysis of 14 industrial case studies spanning four ML architecture classes: recommendation, ranking, vision, and NLP. This study demonstrates that systematic load testing leads to substantial improvements in GPU resource efficiency and operational reliability. Prior to adopting \sys, a significant fraction of model launches were under-provisioned, resulting in recurring incidents; these issues were substantially reduced after deployment. Our results show that ML-specific design decisions are critical to accurate capacity estimation: workload calibration using recorded traffic reduces estimation error from approximately 30% to 2--6%, while proper warmup handling yields a 22.2% improvement in accuracy. Further analysis reveals key factors influencing prediction error, including model size and co-location effects. This paper distills six lessons and derive architectural guidelines for ML load testing, offering actionable insights for building reliable and efficient ML serving systems.

Amr S. Abdelfattah, Nakul Tirumalai, Indu Mohanan +4

Meta Inc., Menlo Park, USA

Automated Software Testing

Ask PDF

Mar 24, 2026

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

Providing reliable predictive maintenance is a critical industrial AI service essential for ensuring the high availability of manufacturing devices. Existing deep-learning methods present competitive results on such tasks but lack a general service-oriented framework to capture complex dependencies in industrial IoT sensor data. While Transformer-based models show strong sequence modeling capabilities, their direct deployment as robust AI services faces significant bottlenecks. Specifically, streaming sensor data collected in real-world service environments often exhibits multi-scale temporal correlations driven by machine working principles. Besides, the datasets available for training time-to-failure predictive services are typically limited in size. These issues pose significant challenges for directly applying existing models as robust predictive services. To address these challenges, we propose MsFormer, a lightweight Multi-scale Transformer designed as a unified AI service model for reliable industrial predictive maintenance. MsFormer incorporates a Multi-scale Sampling (MS) module and a tailored position encoding mechanism to capture sequential correlations across multi-streaming service data. Additionally, to accommodate data-scarce service environments, MsFormer adopts a lightweight attention mechanism with straightforward pooling operations instead of self-attention. Extensive experiments on real-world datasets demonstrate that the proposed framework achieves significant performance improvements over state-of-the-art methods. Furthermore, MsFormer outperforms across industrial devices and operating conditions, demonstrating strong generalizability while maintaining a highly reliable Quality of Service (QoS).

Jiahui Zhou, Dan Li, Ruibing Jin +5

School of Software Engineering, Sun Yat-Sen University, China · Agency for Science, Technology and Research (A*STAR), Singapore · School of the Art Institute of Chicago, Chicago +1

Industrial Internet of Things Predictive Maintenance RUL Estimation Time Series Transformers Multiscale Time-Series Modeling Internet of Things

Ask PDF

Jan 16, 2026

OpFML: Pipeline for ML-based Operational Inference

Machine learning models for climate and Earth science are becoming increasingly capable, yet model deployment into operational use remains a largely unaddressed challenge: general-purpose model-serving tools, such as MLflow and KServe, assume input data availability at the inference node, while data acquisition, failure handling, and preprocessing are trusted to a separate workflow. We present OpFML: Operational Forecasting with Machine Learning - a configurable pipeline integrating the four steps of operational inference into a single TOML-configured workflow: data consumption, contingency handling, preprocessing, and model inference. By consolidating these steps, OpFML removes the significant boilerplate code required for each new deployment. We demonstrate the pipeline on the operational forecasting of daily fire activity over southern Italy.

Shahbaz Alvi, Giusy Fedele, Gabriele Accarino +3

CMCC Foundation - Euro-Mediterranean Center on Climate Change, Lecce, 73100, Italy · Columbia University, Department of Earth and Environmental Engineering, New York, USA · University of Salento,Department of Innovation Engineering, Lecce, 73100, Italy

Wildfire Forecasting Scientific ML

Ask PDF

Abstract

Text Classification

Explore similar work

Load Testing for Machine Learning Model Serving Systems at Scale

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

OpFML: Pipeline for ML-based Operational Inference