From Simulation to Deep Learning: Survey on Network Performance Modeling Approaches
Organizations: Universitat Politècnica de Catalunya (UPC), Barcelona, Catalonia, Spain
Abstract
Network performance modeling is a field that predates early computer networks and the beginning of the Internet. It aims to predict the traffic performance of packet flows in a given network. Its applications range from network planning and troubleshooting to feeding information to network controllers for configuration optimization. Traditional network performance modeling has relied heavily on Discrete Event Simulation (DES) and analytical methods grounded in mathematical theories such as Queuing Theory and Network Calculus. However, as of late, we have observed a paradigm shift, with attempts to obtain efficient Parallel DES, the surge of Machine Learning models, and their integration with other methodologies in hybrid approaches. This has resulted in a great variety of modeling approaches, each with its strengths and often tailored to specific scenarios or requirements. In this paper, we comprehensively survey the relevant network performance modeling approaches for wired networks over the last decades. With this understanding, we also define a taxonomy of approaches, summarizing our understanding of the state-of-the-art and how both technology and the concerns of the research community evolve over time. Finally, we also consider how these models are evaluated, how their different nature results in different evaluation requirements and goals, and how this may complicate their comparison.
Figures & tables
| Acronym | Description |
|---|---|
| ADNC | Algebraic-based Deterministic Network Calculus |
| AI | Artificial Intelligence |
| AQM | Active Queue Management |
| CVAE | Conditional Variational Auto-Encoder |
| [P]DES | [Parallel] Discrete Event Simulation |
| DCQCN | Data Center Quantized Congestion Notification [ 151 ] |
| RON | Resilient Overlay Networks [ 120 ] |
|---|---|
| RTT | Round Trip Time |
| SDE | (System of) Stochastic Differential Equations |
| SFA | Separate Flow Analysis [ 93 ] |
| SNC | Stochastic Network Calculus |
| SVR | Support Vector Regression |
| [DC]TCP | [Data Center [ 5 ] ] Transmission Control Protocol |
| Model | Type | Input scope | Output scope | Traffic type | Performance metrics | Evaluation |
|---|---|---|---|---|---|---|
| REAL [ 82 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | |
| ns [ 136 ] , -2, -3 [ 121 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | |
| OMNeT, OMNeT++ [ 137 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | |
| OPNET [ 27 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | |
| CaSiNo [ 61 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | |
| WNS [ 22 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information |
| pd-gem5 [ 4 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | Testbed data |
|---|---|---|---|---|---|---|
| SimBricks [ 100 ] | Modular DES | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| SplitSim [ 99 ] | Modular DES | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| DONS [ 50 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| Unison [ 15 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| NSX [ 83 ] | DES | Full network scenarios | Packet-level | Any | Full packet path information | Only evaluates cost, not accuracy |
| Parsimon [ 150 ] | Link-level DES | Full network scenarios | Flow-level | TCP (Focuses on DCTCP, but generalizable) | Tail FCT (90th percentile or higher) | Simulated data |
| Model | Type | Input scope | Output scope | Traffic type | Performance metrics | Evaluation |
|---|---|---|---|---|---|---|
| [ 88 ] | M/M/1, M/G/ , M/G/1 queues | Single-flow scenarios | Flow-level | Non-specific | Average packet delay, buffer overflow probability | Analytical evaluation |
| [ 117 ] | M/G/ queue | Traffic-matrix scenarios | Flow-level | Non-specific | Loss rate | Simulated data evaluation |
| [ 46 ] | GI/G/1 queue | Single-flow scenarios | Flow-level | Non-specific | Upper and lower bounds, average packet delay | Simulated data evaluation |
| [ 51 ] | System of M/M/ queues | Traffic-matrix scenarios | Flow-level | TCP (Tahoe) | Average packet [loss probability, RTT] | Simulated data evaluation |
| [ 52 ] | System of M/G/ queues | Traffic-matrix scenarios | Flow-level | TCP (Tahoe) | Average packet [loss probability, RTT] | Simulated data evaluation |
| [ 147 ] | Modified M/M/1 queues | Traffic-matrix scenarios | Flow-level | Non-specific | Flow throughput, average packet delay | Simulated data evaluation |
| [ 109 ] | SDE fluid queuing model | Full network scenarios | Flow-level (temporal) | TCP (Generic w/ RED) | Flow throughput, average RTT (but evaluation only evaluates queue-length) | Simulated data evaluation |
|---|---|---|---|---|---|---|
| [ 103 ] | SDE fluid queuing model | Full network scenarios | Flow-level (temporal) | TCP (Sack, Reno, and NewReno) | Flow throughput, average RTT (but evaluation only evaluates queue-length and window size) | Simulated data evaluation |
| multi-AIMD [ 13 ] | Fluid/ discrete hybrid queuing model | Full network scenarios | Flow-level (temporal) | TCP and UDP | Flow throughput, average RTT | Simulated data evaluation |
| [ 18 ] , [ 94 ] | Fluid/ discrete hybrid queuing model | Full network scenarios | Flow-level (temporal) | TCP and UDP | Average flow throughput and RTT (only drop rate and window size retain a temporal component) | Simulated data evaluation |
| [ 105 ] | PDE fluid queuing model | Traffic-matrix scenarios | Flow-level (temporal) | TCP (Generic) | Average flow throughput, FCT and loss rate | Simulated data evaluation |
| [ 11 ] | PDE fluid queuing model | Traffic-matrix scenarios | Flow-level (temporal) | TCP (Scalable TCP) | Flow throughput | Simulated data evaluation |
| [ 23 ] | ODE fluid queuing model | Full network scenarios | Flow-level (temporal) | UDP 2 and TCP (Reno) | Flow RTT, throughput and loss rate | Simulated data evaluation |
| [ 35 ] | Discrete and fluid queuing model | Traffic-matrix scenarios | Flow-level (temporal) | TCP | Flow packet delay (evaluation focuses in average queue lengths) | Simulated data evaluation |
|---|---|---|---|---|---|---|
| TFA [ 33 , 34 ] | ADNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Analytical evaluation |
| D-BIND [ 87 ] | ADNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Real data evaluation |
| [ 2 ] | ADNC | Full network scenarios | Flow-level (temporal) | UDP and traffic with throughput guarantees | Worst-bound packet delay | Analytical evaluation |
| SFA [ 93 ] | ADNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Analytical evaluation |
| PMOO [ 44 ] | ADNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Simulated data evaluation |
| [ 92 ] | ADNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Analytical evaluation |
| [ 84 ] | ODNC | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Simulated data evaluation |
|---|---|---|---|---|---|---|
| [ 29 ] | SNC | Traffic-matrix scenarios | Flow-level (temporal) | Non-specific | Stochastic worst-bound packet delay | Analytical evaluation |
| [ 79 ] | SNC | Traffic-matrix scenarios | Flow-level (temporal) | Non-specific | Stochastic worst-bound packet delay | Analytical evaluation |
| [ 7 ] | SNC | Traffic-matrix scenarios | Flow-level (temporal) | Non-specific | Stochastic worst-bound packet delay | Analytical evaluation |
| [ 91 ] | Analytical model from RFC definitions | Single-flow scenarios | Flow-level (temporal) | TCP (generic with phase effects and RED) | Flow throughput | Simulated data evaluation |
| [ 113 ] | Stochastic analytical model from observations | Single-flow scenarios | Flow-level (temporal) | TCP (Reno) | Flow packet rate and throughput | Real data evaluation |
| [ 106 ] | Utility maximization model | Traffic-matrix scenarios | Flow-level | TCP | Flow throughput | Analytical evaluation |
| Model | Type | Input scope | Output scope | Traffic type | Performance metrics | Evaluation |
|---|---|---|---|---|---|---|
| [ 67 ] | MA, EWMA, Holt-Winters | Single-flow scenarios | Flow-level (temporal) | TCP (generic) | Flow throughput | Testbed data |
| PathPerf [ 108 ] | SVR | Single-flow scenarios | Flow-level (temporal) | TCP (generic) | Flow throughput | Tesbed data and real data |
| WISE [ 135 ] | Causal graph | Single-flow scenarios | Flow-level | HTTP connections | Network response time | Real data |
| [ 69 ] | EVT | Multi-flow scenarios | Flow-level | UDP | 50th, 75th, 90th, 99th, 99.9th, 99.99th, 99.999th and 100th percentiles for packet delay and jitter | Testbed data |
| CLAAP [ 110 ] | RBFNN | Single-flow scenarios | Flow-level (temporal) | UDP | Flow latency | Real data |
| [ 107 ] | DNN | Multi-flow scenarios | Flow-level | UDP | Average packet delay | Simulated data |
| RouteNet-Erlang [ 42 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP | Average packet delay and jitter, loss rate | Simulated data |
| RouteNet-Fermi [ 41 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP | Average packet delay and jitter, loss rate | Simulated data and testbed data |
| RouteNet-Gauss [ 60 ] | Custom-designed MPNN with temporal component | Full network scenarios | Flow-level (temporal) | UDP | [Average, median, and 90th, 95th, and 99th percentile] packet [delay, jitter] | Testbed data |
| [ 37 ] | Custom-designed MPNN with graph attention | Full network scenarios | Flow-level | UDP | Average packet delay | Simulated data |
| [ 32 ] | Custom-designed MPNN with graph attention | Full network scenarios | Flow-level | UDP | Average packet delay | Simulated data |
| [ 80 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP | Average packet delay | Testbed data |
| [ 58 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP | Average packet delay | Testbed data |
| DeepComNet [ 56 ] | GGNN [ 101 ] | Full network scenarios | Flow-level | UDP and TCP | TCP flow throughput, UDP average packet delay | Simulated data |
|---|---|---|---|---|---|---|
| [ 75 ] | GGNN [ 101 ] | Full network scenarios | Flow-level | TCP (Reno, Cubic, Bic, Illinois, Veno, Vegas and Ledbat) | Flow RTT and throughput | Simulated data |
| [ 134 ] | GCN [ 86 ] | Full network scenarios | Flow-level | UDP and TCP | Average packet delay | Simulated data |
| EAGLE [ 102 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP and TCP | Average packet delay and loss rate | Simulated data |
| xNet [ 71 ] | Custom-designed MPNN with temporal component | Full network scenarios | Flow-level (temporal) | UDP and TCP (DTCP) | Average packet delay, flow throughput and FCT | Simulated data |
| GLANCE [ 96 ] | Custom-designed MPNN | Full network scenarios | Flow-level | UDP | Average packet delay and jitter, flow throughput, loss rate | Simulated data |
| FlowSeer [ 39 ] | Ensemble of GCN [ 86 ] , DNN with attention and DNN | Full network scenarios | Flow-level | UDP | Average packet delay | Simulated data |
| xWeaver [ 141 ] | CNN | Full network scenarios | Flow-level | UDP | Flow FCT and throughput | Simulated data and testbed data |
|---|---|---|---|---|---|---|
| Deep-Q [ 143 ] | CVAE [ 85 ] + LSTM [ 70 ] | Multi-flow scenarios | Flow-level (temporal) | UDP | Packet delay and loss rates | Testbed data |
| [ 38 ] | Transformer [ 138 ] | Single-flow scenarios | Packet-level | UDP | Packet delay | Simulated data |
| DeepQueueNet [ 145 ] | Transformer [ 138 ] + BiLSTM | Full network scenarios | Packet-level | UDP | Full packet path information | Simulated data |
| [ 64 , 65 ] | NP [ 53 ] | Multi-flow scenarios | Flow-level | Non-specific | Average packet delay, flow throughput and loss rates | Testbed data |
| Model | Main Type | Secondary Type | Input Scope | Output Scope | Traffic Type | Performance Metrics | Evaluation |
| Pantheon [ 144 ] | Network emulator (Mahimahi [ 112 ] ) | Bayesian optimization | Full network scenarios | Packet-level | Any | Full packet path information | Real data |
| iBox [ 10 ] | Network emulator with a queue model | Bayesian optimization | Full network scenarios | Packet-level | Any | Full packet path information | Real data |
| Prophet [ 148 ] | NUM analytical model [ 81 ] | Gradient descent | Full network scenarios | Flow-level | TCP 1 | Flow throughput | Simulated data |
| DeepTMA [ 54 , 55 ] | NC (TMA [ 20 ] ) | GGNN [ 101 ] | Full network scenarios | Flow-level (temporal) | Non-specific | Worst-bound packet delay | Simulated data |
| [ 68 ] | GraphSage [ 62 ] | NC (TFA [ 33 , 34 ] , SFA [ 93 ] ) | Full network scenarios | Flow-level | UDP | Mean, min, max, 90th and 99th percentile packet delays | Testbed data |
| QT-RouteNet [ 36 ] | RouteNet [ 123 ] | M/M/1/B queue model | Full network scenarios | Flow-level | UDP | Average packet delay | Simulated data |
| QINN [ 66 ] | DNN | M/G/1 queue model | Traffic-matrix scenarios | Flow-level | UDP and TCP (non-specific) | Avg. packet delay, throughput | Simulated data |
|---|---|---|---|---|---|---|---|
| [ 146 ] | ns-2 [ 136 ] | Flow queueing model [ 103 ] | Full network scenarios | Packet-level | UDP and TCP 2 | Full packet path information | Simulated data |
| MimicNet [ 149 ] | ns-3 [ 121 ] | LSTM [ 70 ] | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| m3 [ 97 ] | flowSim [ 111 ] | Transformer (LLama2 [ 73 ] ) | Full network scenarios | Flow-level | UDP and TCP (DCTCP, TIMELY, DCQCN, HPCC) | 99th percentile FCT | Simulated + Real data |
| CausalSim [ 6 ] | Trace simulation | Causal DNN | Full network scenarios | Packet-level | Any | Full packet path information | Simulated data |
| Sim2HW [ 130 ] | OMNET++ [ 137 ] | GraphSage [ 62 ] | Full network scenarios | Flow-level | UDP | Min, 25th, 50th, 75th, 90th, 99th, 99.9th, 99.99th, 99.999th percentile packet delays | Testbed data |
| Approach | Accuracy | Expressiveness | Applicability | Computational Cost | |
| Simulation | DES | High | Packet-level | General | High (and sequential) |
| PDES | High | ||||
| QT | Discrete | Low | Flow-level | Specific TCP version | Low |
| Fluid | Flow-level | ||||
| (temporal) | |||||
| NC | ADNC | Medium | Flow-level | Feed-foward networks | Medium |