Organizations: Department of Informatics and Telematics, Harokopio University of Athens, Athens, 17778, Greece · School of Electrical and Computer Engineering, National Technical University of Athens, Athens, 15773, Greece
Deployed machine learning models in real-world IoT and edge/cloud systems face a critical reliability bottleneck: the requirement for online adaptation to non-stationary data streams while operating without access to the full original training dataset. Performance is severely degraded when models encounter Out-of-Distribution (OOD) inputs or suffer from catastrophic forgetting. Addressing these challenges requires adaptation mechanisms capable of learning from evolving data streams while preserving previously acquired knowledge. This paper proposes an edge-cloud methodology for integrating Continual Learning (CL) with Out-of-Distribution (OOD) Detection and Concept Drift Detection in IoT applications. The methodology incorporates ODIN and ECDD as alternative mechanisms for identifying informative samples for annotation and triggering rehearsal-based model updates. Experimental results across automotive, healthcare, and agricultural datasets demonstrate the applicability of the proposed methodology and its potential to support model adaptation while mitigating catastrophic forgetting.
Figures & tables
Figure 1: Overview of the proposed continual learning pipeline using edge-cloud interactions. The edge device includes a mechanism for identifying novel or ambiguous samples, which are stored for deferred processing in the cloud. Labels are provided or verified by a human annotator, and the model is updated through a continual learning mechanism before being redeployed to the edge.
Algorithm
Complexity
Requires ground truth?
ECDD
O(f)+O(1)
Yes
ODIN
O(f)+O(g)
No
Table 1: ECDD and ODIN comparison.
Training methodology
Abbr.
Description
Edge training*
TRN
The model is being trained only with new samples and the user/annotator labeling. This type of continual learning usually induces catastrophic forgetting.
Edge/cloud rehearsal
RH
The model is being trained with rehearsing new samples (with their respective labels/annotations), alongside older samples from a factory dataset subset (memory buffer). This operation can either reside in an edge device or a cloud infrastructure
Cloud annotation and dataset expansion
ANDE
The model is being trained with the newly annotated samples and an extended dataset which is composed of factory samples and older annotated images from another edge device
Sample selection method
Abbr.
Description
Random
RAND
The samples are being loaded in a buffer with reservoir sampling regardless of conflict or not.
Drift detection*
ECDD
Drift detection method as described in 3.1 . The samples are being stored, based on the disagreements between the operator and the model. The samples are being used for rehearsal when the method is triggered or when the buffer is full. This method requires user input
Table 2: List of all methodologies with abbreviations and a short description. Note that, in the case of edge training there is no sample selection. The methods in asterisk require user input to the edge device/model
Figure 2: Accuracy results for the custom Belgium Traffic Signs dataset with cloud annotation and dataset expansion as described in Table 2 . The dotted lines are the factory and stream accuracy in the first timestep. The results are the mean and standard deviation between three drivers
Strategy
AˉFactory
AˉStream
Initial state
0.9533
0.7125
TRN
0.8124 ±0.0216
0.6188 ±0.0148
RH+RAND
0.9584 ±0.0008
0.7044 ±0.0035
RH+ODIN
0.9447 ±0.0009
0.7494 ±0.0146
RH+ECDD
0.9422 ±0.0025
0.6952 ±0.0088
ANDE+RAND
0.9257 ±0.0048
0.7444 ±0.0074
Table 3: Average accuracy on the Belgium Traffic Signs custom dataset with all method combinations. The maximum average is shown in teal.
Figure 3: Accuracy results for a stream of 1000 timesteps of the nuScenes objects dataset with ODIN and RAND methods as described in Table 2 . The dotted lines are the factory and stream accuracy in the first timestep. The results are the mean and standard deviation between three drivers
Class name
#Stream samples
#Factory samples
truck
2916
589
motorcycle
1354
273
construction
472
95
car
20078
4056
bus
647
130
bicycle
1357
274
Table 4: Number of samples per class for the nuScenes experiment stream
Strategy
AˉFactory
AˉStream
Initial state
0.7659
0.6976
RH+RAND
0.7708 ±0.0018
0.7268 ±0.0083
RH+ODIN
0.7727 ±0.0006
0.7332 ±0.0016
ANDE+RAND
0.7799 ±0.0024
0.7478 ±0.0048
ANDE+ODIN
0.8035 ±0.0006
0.7546 ±0.0003
Table 5: Accuracy results for the nuScenes objects dataset with all available method combinations. The maximum average is shown in teal.
Figure 4: Accuracy results for a stream of 1000 timesteps of the Malaria objects dataset with edge-based methods, as described in Table 2 . The dotted lines are the factory and stream accuracy in the first timestep. The results are the mean and standard deviation between three users
Strategy
AˉFactory
AˉStream
Initial state
0.9408
0.9027
TRN
0.949 ±0.0075
0.9433 ±0.0021
RH+RAND
0.9588 ±0.0018
0.9537 ±0.0011
RH+ODIN
0.9574 ±0.0026
0.9459 ±0.0005
RH+ECDD
0.9595 ±0.0029
0.9512 ±0.0017
Table 6: Accuracy results for the Malaria infected cells dataset with all available method combinations. The maximum average is shown in teal.
Figure 5: Accuracy results for a stream of 1000 timesteps of the Plant Village objects dataset with edge-based methods, as described in Table 2 . The dotted lines are the factory and stream accuracy in the first timestep. The results are the mean and standard deviation between three users
Strategy
AˉFactory
AˉStream
Initial state
0.8468
0.7873
TRN
0.6943 ±0.1645
0.81 ±0.0947
RH+RAND
0.9082 ±0.0087
0.8974 ±0.0031
RH+ODIN
0.9168 ±0.0059
0.9145 ±0.0025
RH+ECDD
0.916 ±0.009
0.9168 ±0.0027
Table 7: Accuracy results for the Plant Village dataset with all available method combinations. The maximum average is shown in teal.
Faculty of Electrical Engineering and Information Technologies, Ss. Cyril and Methodius University in Skopje, 1000 Skopje, North Macedonia · Dept. of Electronic Systems, Aalborg University, Aalborg, Denmark
School of Computing and Information Systems, The University of Melbourne, Australia · School of Computer Science Engineering, University of New South Wales, Australia