Operationalization of Machine Learning with Serverless Architecture: An Industrial Operationalization of Machine Learning with Serverless Architecture: An Industrial Implementation for Harmonized System Code Prediction

Sai Vineeth Kandappareddigari; Santhoshkumar Jagadish; Gauri Verma; Ilhuicamina Contreras; Christopher Dignam; Anmol Srivastava; Benjamin Demers

cs.LGFeb 19, 2026

Operationalization of Machine Learning with Serverless Architecture: An Industrial Operationalization of Machine Learning with Serverless Architecture: An Industrial Implementation for Harmonized System Code Prediction

Authors: Sai Vineeth Kandappareddigari, Santhoshkumar Jagadish, Gauri Verma, Ilhuicamina Contreras, Christopher Dignam, Anmol Srivastava, Benjamin Demers

Organizations: Governance Schneider Electric U.S.A Boston, USA · Tax and Customs Schneider Electric Mexico Mexico city, Mexico · Governance Schneider Electric Paris, France

Abstract

This paper presents a serverless MLOps framework orchestrating the complete ML lifecycle from data ingestion, training, deployment, monitoring, and retraining to using event-driven pipelines and managed services. The architecture is model-agnostic, supporting diverse inference patterns through standardized interfaces, enabling rapid adaptation without infrastructure overhead. We demonstrate practical applicability through an industrial implementation for Harmonized System (HS) code prediction, a compliance-critical task where short, unstructured product descriptions are mapped to standardized codes used by customs authorities in global trade. Frequent updates and ambiguous descriptions make classification challenging, with errors causing shipment delays and financial losses. Our solution uses a custom text embedding encoder and multiple deep learning architectures, with Text-CNN achieving 98 percent accuracy on ground truth data. Beyond accuracy, the pipeline ensures reproducibility, auditability, and SLA adherence under variable loads via auto-scaling. A key feature is automated A/B testing, enabling dynamic model selection and safe promotion in production. Cost-efficiency drives model choice; while transformers may achieve similar accuracy, their long-term operational costs are significantly higher. Deterministic classification with predictable latency and explainability is prioritized, though the architecture remains extensible to transformer variants and LLM-based inference. The paper first introduces the deep learning architectures with simulations and model comparisons, then discusses industrialization through serverless architecture, demonstrating automated retraining, prediction, and validation of HS codes. This work provides a replicable blueprint for operationalizing ML using serverless architecture, enabling enterprises to scale while optimizing performance and economics.

Figures & tables

Fig. 1 : Distribution of assurance levels in the training data

FEATURES	DESCRIPTION
Short Description	Technical description of each item or product along with information about product hierarchy and product name
Medium Description	Description about product family that contains similar products, and also includes product dimensions and technical specifications
ETIM	International classification standard for technical products

TABLE I : Features used in various models

Fig. 2 : Text length (number of characters) for combined description in training data

Fig. 3 : Class Imbalance as seen on training data

Panel of Figure 5 — Fig. 4 : Top to Bottom: Architecture Diagram for LSTM, DNN and Text-CNN Model (Specifications in the diagram are the same as those used in the experimentation)

Hyper-parameters	Range	DNN base model	DNN up-sampled model	LSTM (base & up-sampled)
Initial neurons	11- 174	11.01	168.01	25.0
Neuron Pct	0.35 - 1.0	0.44	0.95	1.0
Dropout	0.10 - 0.75	0.37	0.1	0.1
Neuron Shrink	0.25 - 0.95	0.31	0.466	0.95
Embedding Dim	11.0 - 87.0	65.74	43.3	78.0
N_layer	1.0 - 15.0	8.02	7.89	3.0

TABLE II : Hyper-parameters for DNN and LSTM base and up-sampled models

Model	Accuracy		Precision						Recall						F1
			$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80
	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U
DNN	89%	95%	15	19	5	1	3	3	12	20	5	2	6	1	14	19	5	2	4	2
LSTM	93%	97.50%	19	20	2	0	2	3	16	22	5	1	2	0	18	21	3	1	2	2
Text-CNN	98%	98%	19	20	2	1	2	2	20	21	2	2	1	0	19	20	2	2	2	1

TABLE III : Accuracy, Precision, Recall and F1 metrics for DNN, LSTM and Text-CNN on training data

Model	Accuracy		Precision						Recall						F1
			$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80		$\geq$ 90		80 - 90		$\leq$ 80
	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U	B	U
DNN	89%	95%	15	19	5	1	3	3	12	20	5	2	6	1	14	19	5	2	4	2
LSTM	92%	97.00%	19	20	2	0	2	3	16	22	5	1	2	0	18	21	3	1	2	2
Text-CNN	98%	98%	19	20	2	1	2	2	20	21	2	2	1	0	19	20	2	2	2	1

TABLE IV : Accuracy, Precision, Recall and F1 metrics for DNN, LSTM and Text-CNN on test data

Hyper-parameters	Range	Values
Filter Size	[64,128,256]	128
Kernel Size	1 - 10	[5]
Embedding Dim	50 - 150	[100]
N_layer	1.0 - 5.0	1.0

TABLE V : Hyper-parameters for Text-CNN Model

Precision	Recall	Evaluation	Type of F Score
High	High	Best case scenario - model detects HS code and predicts well with less misclassifications performs well on train and test	F0
High	Low	Model detects HS codes but with some misclassifications model performs better on training but not that well on tests	between F1-F2
Low	High	Over fitting -model predicts well on train but not on test set	between F0-F1
Low	Low	Model is not learning as Precision and recall are low in both train and test	between F1-F2

TABLE VI : F Score estimation to balance bias between Precision and Recall

Model	Hyper-parameter Tuning		Training		Total Time Taken
	Base Model	up-sampled Model	Base Model	up-sampled Model	Base Model	up-sampled Model
DNN	36 mins	2 hours 55 mins	15 mins	27 mins	51 mins	3 hours 36 mins
LSTM	4 hours	4.2 hours	1.5 hours	1.2 hours	5.5 hours	5.4 hours
Text-CNN	2 hours 20 min	1 hour 50 mins	55 mins	1 hour 20 mins	3 hours 15 mins	3 hours 10 mins

TABLE VII : Time taken for training proposed models

Fig. 5 : Machine learning model life-cycle

Text Classification

Text classification assigns text to categories such as topic, sentiment, intent, stance, or risk, supporting search, moderation, and decisions.

See all 226 papers

Load Testing for Machine Learning Model Serving Systems at Scale
Jun 20, 2026Amr S. Abdelfattah, Nakul Tirumalai, Indu Mohanan +4Automated Software Testing
Meta Inc., Menlo Park, USA
MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices
Mar 24, 2026Jiahui Zhou, Dan Li, Ruibing Jin +5Industrial Internet of Things Predictive Maintenance
School of Software Engineering, Sun Yat-Sen University, China · Agency for Science, Technology and Research (A*STAR), Singapore · School of the Art Institute of Chicago, Chicago +1
OpFML: Pipeline for ML-based Operational Inference
Jan 16, 2026Shahbaz Alvi, Giusy Fedele, Gabriele Accarino +3Wildfire Forecasting Scientific ML
CMCC Foundation - Euro-Mediterranean Center on Climate Change, Lecce, 73100, Italy · Columbia University, Department of Earth and Environmental Engineering, New York, USA · University of Salento,Department of Innovation Engineering, Lecce, 73100, Italy

Jun 20, 2026J/K move · Enter open · S save

Load Testing for Machine Learning Model Serving Systems at Scale

Amr S. Abdelfattah, Nakul Tirumalai, Indu Mohanan, Xiao Li+3

Meta Inc., Menlo Park, USA

Automated Software Testing

Open paper AskPDF arXiv

Machine learning (ML) model serving has become a dominant consumer of GPU infrastructure, yet capacity planning in these systems remains largely ad hoc. Under-provisioning leads to service-level objective (SLO) violations and production incidents, while over-provisioning results in substantial resource waste. This paper presents \sys, an industrial load testing framework for ML serving systems that systematically estimates serving capacity through an adaptive, feedback-driven search strategy. The approach leverages real-time performance signals, incorporating dampening, spike tolerance, and convergence detection to efficiently identify maximum sustainable throughput under SLO constraints. We evaluate \sys through a longitudinal analysis of 14 industrial case studies spanning four ML architecture classes: recommendation, ranking, vision, and NLP. This study demonstrates that systematic load testing leads to substantial improvements in GPU resource efficiency and operational reliability. Prior to adopting \sys, a significant fraction of model launches were under-provisioned, resulting in recurring incidents; these issues were substantially reduced after deployment. Our results show that ML-specific design decisions are critical to accurate capacity estimation: workload calibration using recorded traffic reduces estimation error from approximately 30% to 2--6%, while proper warmup handling yields a 22.2% improvement in accuracy. Further analysis reveals key factors influencing prediction error, including model size and co-location effects. This paper distills six lessons and derive architectural guidelines for ML load testing, offering actionable insights for building reliable and efficient ML serving systems.

Abstract

Text Classification

Explore similar work

Load Testing for Machine Learning Model Serving Systems at Scale

MsFormer: Enabling Robust Predictive Maintenance Services for Industrial Devices

OpFML: Pipeline for ML-based Operational Inference