CrowdCC: Jointly Benchmarking Corruption Robustness and Deployment Quantization in Crowd Counting
Organizations: Tulane University
Abstract
Crowd-counting models deployed on edge devices face two shifts: inputs are degraded along the capture and transmission pipeline, while models are quantized to satisfy resource constraints. Yet corruption robustness and deployment quantization are typically evaluated separately. We introduce CrowdCC, a controlled corruption benchmark that jointly evaluates these effects using 23 deployment-motivated corruptions at three severity levels, four crowd-counting datasets, and 26 models using A100 TensorRT engines at FP32, FP16, and post-training INT8 precision, paired with throughput and energy measurements from corresponding Jetson Orin NX configurations. The corruptions span six families, with head annotations adjusted when geometry or visibility changes. Our evaluation shows that capture-chain corruptions consistently produce the largest degradation across the evaluated models, while crowd density affects absolute and relative error differently. Clean accuracy does not fully predict corrupted performance. INT8 alters corruption robustness substantially more than FP16, with no systematic direction, and does not consistently improve latency or energy. INT8 inference efficiency is strongly associated with reformat overhead in the compiled engine, while slowdowns are also observed under limited INT8 coverage. Precision changes corrupted-error rankings and robustness-cost frontiers, showing why model selection must consider corruption robustness and measured deployment cost together. The code is available at https://github.com/Tulane-Computer-Architecture-Systems/CrowdCC-Release.
Figures & tables
| (a) base datasets and the instances they generate | |||||
| Dataset | Inst. | #test-image | Res. (MP) | count min / max | mean |
| ShanghaiTech A | SHA-C | 182 | – | 66 / 2,256 | 433.3 |
| ShanghaiTech B | SHB-C | 316 | 9 / 539 | 123.8 | |
| UCF-QNRF | QNRF-C | 334 | – | 65 / 4,535 | 718.9 |
| JHU-Crowd++ | JHU-C | 1,600 | – | 0 / 9,059 | 326.2 |
| realized engine | |||||
| Model | (M) | Output type | FP16 or FP32 ops. | INT8 | |
| CNN–based | |||||
| CHF [ 39 ] | 21.51 | D | – | 1.99 | 0.50 |
| DM-Count [ 45 ] | 21.50 | D | – | 1.98 | 0.48 |
| GCHF [ 39 ] | 21.51 | D | – | 1.98 | 0.49 |
| CSRNet [ 24 ] | 16.26 | D | – | 1.97 | 0.49 |
| Model | SHA | SHB | QNRF | JHU |
|---|---|---|---|---|
| CHF | 57.2/57.3/ 55.7 | 7.3/7.3/7.9 | 84.2/84.1/85.7 | 60.4/60.4/ 60.3 |
| DM-Count | 65.7/65.7/66.0 | 7.8/7.8/7.9 | 91.5/91.5/91.8 | 62.4/62.4/62.4 |
| GCHF | 59.2/59.2/ 58.1 | 7.1/7.1/8.3 | 81.8/81.8/84.1 | 60.4/60.3/60.5 |
| CSRNet | 67.4/67.4/68.2 | 11.4/11.4/ 11.1 | 95.8/95.9/96.7 | 68.9/68.9/ 68.8 |
| P2PNet | 57.8/57.8/ 57.5 | 8.0/8.0/8.1 | 101.0/101.1/102.0 | 66.9/66.9/ 65.8 |
| APGCC | 57.9/57.9/ 57.3 | 8.3/8.3/8.6 | 110.4/110.4/ 105.5 | 72.7/72.8/73.4 |
| mcMAE (FP32/INT8) | FP32 | INT8 | FP32 RD % by corruption family | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | SHA | SHB | QNRF | JHU | RD % | mCE | RD % | mCE | Env | Opt | Sen | Enc | Clu | Vis |
| CHF | 74.8/73.6 | 17.4/16.8 | 107.0/107.0 | 72.2/72.8 | 28.9 | 0.821 | 28.5 | 0.824 | 62.6 | 77.5 | 78.6 | 38.2 | 11.5 | 35.3 |
| DM-Count | 81.0/81.1 | 16.3/15.9 | 110.0/109.5 | 73.5/73.0 | 21.7 | 0.840 | 21.1 | 0.838 | 38.2 | 60.4 | 68.7 | 32.7 | 12.8 | 22.4 |
| GCHF | 73.4/72.5 | 14.2/14.6 | 104.9/105.9 | 74.7/75.1 | 26.1 | 0.797 | 25.3 | 0.812 | 39.2 | 60.4 | 65.6 | 33.0 | 18.3 | 33.1 |
| CSRNet | 87.2/87.9 | 26.5/26.1 | 120.8/120.1 | 86.2/85.8 | 27.8 | 1.000 | 26.8 | 1.000 | 66.8 | 71.6 | 87.9 | 38.0 | 7.8 | 20.5 |
| P2PNet | 80.7/82.1 | 18.7/18.4 | 117.7/118.4 | 77.5/77.2 | 28.1 | 0.887 | 30.1 | 0.892 | 49.9 | 62.7 | 96.8 | 35.1 | 15.3 | 21.8 |
| Output representation | RD (%) | mCE | ||
|---|---|---|---|---|
| D Density map | 15 | 23.6 | 0.857 | 2.55 |
| B Blockwise count map | 5 | 22.9 | 0.916 | 2.74 |
| P Localization-oriented | 4 | 25.9 | 0.913 | 1.97 |
| C Direct count | 2 | 14.0 | 0.859 | 0.75 |
Appendix figures & tables27 assets
Supplementary material from the paper’s appendix.
Appendix
| Corruption | Mechanism | Parameter | Light | Mod. | High | |
|---|---|---|---|---|---|---|
| Sensor & exposure | ||||||
| fixed_pattern_sensor_noise | stuck/hot pixels, column FPN [ 13 ] | column | 0.002 | 0.008 | 0.012 | |
| stuck_fraction | 0.00005 | 0.0002 | 0.0003 | |||
| row_sigma | 0.002 | 0.008 | 0.012 | |||
| temporal_sigma | 0.002 | 0.008 | 0.012 | |||
| gaussian_noise | sensor read noise [ 13 ] | 0.010 | 0.025 | 0.050 | ||
| magnitude per dataset (px) | ||||||
| Corruption | Level | spacing frac. | SHA | SHB | QNRF | JHU |
| median head spacing (px) | 17.23 | 33.11 | 34.78 | 32.99 | ||
| camera motion blur | L | 0.050 | 0.9 | 1.7 | 1.7 | 1.6 |
| M | 0.150 | 2.6 | 5.0 | 5.2 | 4.9 | |
| H | 0.250 | 4.3 | 8.3 | 8.7 | 8.2 | |
| far-field defocus | L | 0.050 | 0.9 | 1.7 | 1.7 | 1.6 |
| point | heads removed (%) | |||
|---|---|---|---|---|
| Corruption | rule | min | med | max |
| field-of-view loss | crop | 12.3 | 25.1 | 80.3 |
| foreground occlusion | occlusion | 5.5 | 16.2 | 30.7 |
| object distractors | occlusion | 6.3 | 11.9 | 21.5 |
| packet-loss concealment | occlusion | 5.8 | 11.9 | 26.3 |
| crowd-like distractors | occlusion | 3.6 | 10.4 | 29.8 |
| ShanghaiTech A | ShanghaiTech B | UCF-QNRF | JHU-Crowd++ | |||||
| Model | Ours | Paper | Ours | Paper | Ours | Paper | Ours | Paper |
| CNN-based | ||||||||
| FFNet-MobileNetV3 | 70.68/118.90 | 64.30/109.30 | 12.64/23.91 | – | 107.94/198.67 | – | 65.14/233.29 | – |
| ZIP-Pico | 79.88/125.90 | 71.18/109.60 | 9.73/15.35 | 8.23/12.62 | 111.62/173.88 | 96.29/161.82 | 75.96/285.50 | – |
| ZIP-Nano | 70.49/109.17 | 58.86/94.63 | 8.50/13.69 | 7.74/12.14 | 114.44 / 196.29 | 86.46/147.64 | 73.53/275.65 | – |
| MobileCount | 86.66 / 139.69 | 89.40/146.00 | 9.24/ 14.94 | 9.00/15.40 | 114.69 / 189.84 | 131.10/222.60 | 137.04/397.79 | – |
| clean MAE | mcMAE | RD (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | FP32 | FP16 | INT8 | FP32 | INT8 | FP32 | FP16 | INT8 | |
| APGCC | 57.9 | 57.9 | 57.3 | 75.4 | 76.1 | +30.4 | +30.2 | +32.7 | +2.3 |
| CHF | 57.2 | 57.3 | 55.7 | 74.8 | 73.6 | +30.7 | +30.7 | +32.2 | +1.5 |
| GCHF | 59.2 | 59.2 | 58.1 | 73.4 | 72.5 | +24.0 | +24.0 | +24.8 | +0.7 |
| CCST | 63.6 | 63.6 | 63.2 | 72.3 | 72.0 | +13.8 | +13.7 | +13.9 | +0.2 |
| CCTrans-Full | 57.3 | 57.4 | 59.7 | 70.5 | 73.2 | +23.0 | +22.8 | +22.6 | -0.3 |
| clean MAE | mcMAE | RD (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | FP32 | FP16 | INT8 | FP32 | INT8 | FP32 | FP16 | INT8 | |
| APGCC | 8.3 | 8.3 | 8.6 | 18.5 | 18.8 | +122.5 | +122.9 | +118.9 | -3.6 |
| CHF | 7.3 | 7.3 | 7.9 | 17.4 | 16.8 | +139.4 | +139.4 | +112.8 | -26.6 |
| GCHF | 7.1 | 7.1 | 8.3 | 14.2 | 14.6 | +100.2 | +100.3 | +76.8 | -23.4 |
| CCST | 8.1 | 8.1 | 8.3 | 13.8 | 13.7 | +69.4 | +69.7 | +65.4 | -4.0 |
| CCTrans-Full | 7.5 | 7.5 | 8.1 | 12.5 | 13.7 | +66.0 | +66.2 | +68.8 | +2.7 |
| clean MAE | mcMAE | RD (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | FP32 | FP16 | INT8 | FP32 | INT8 | FP32 | FP16 | INT8 | |
| APGCC | 110.4 | 110.4 | 105.5 | 136.1 | 131.0 | +23.3 | +23.2 | +24.1 | +0.9 |
| CHF | 84.2 | 84.1 | 85.7 | 107.0 | 107.0 | +27.1 | +27.1 | +24.8 | -2.3 |
| GCHF | 81.8 | 81.8 | 84.1 | 104.9 | 105.9 | +28.2 | +28.1 | +25.9 | -2.3 |
| CCST | 96.3 | 96.2 | 96.7 | 106.5 | 107.5 | +10.6 | +10.7 | +11.2 | +0.6 |
| CCTrans-Full | 92.3 | 92.3 | 91.2 | 99.5 | 101.2 | +7.8 | +7.8 | +10.9 | +3.1 |
| clean MAE | mcMAE | RD (%) | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Model | FP32 | FP16 | INT8 | FP32 | INT8 | FP32 | FP16 | INT8 | |
| APGCC | 72.7 | 72.8 | 73.4 | 80.9 | 81.2 | +11.3 | +11.2 | +10.7 | -0.5 |
| CHF | 60.4 | 60.4 | 60.3 | 72.2 | 72.8 | +19.6 | +19.7 | +20.6 | +1.0 |
| GCHF | 60.4 | 60.3 | 60.5 | 74.7 | 75.1 | +23.7 | +23.6 | +24.1 | +0.5 |
| CCST | 61.4 | 61.4 | 61.2 | 67.5 | 67.9 | +9.9 | +9.9 | +11.0 | +1.1 |
| CCTrans-Full | 56.8 | 56.8 | 59.1 | 63.8 | 67.8 | +12.3 | +12.3 | +14.7 | +2.4 |
| SHA | SHB | QNRF | JHU | |||||
|---|---|---|---|---|---|---|---|---|
| Corruption | F32 | I8 | F32 | I8 | F32 | I8 | F32 | I8 |
| far-field defocus | 22 | 24 | 320 | 303 | 29 | 26 | 24 | 23 |
| highlight clipping | 40 | 42 | 227 | 223 | 49 | 48 | 30 | 30 |
| depth-aware haze | 39 | 37 | 252 | 244 | 28 | 27 | 18 | 16 |
| Gaussian noise | 49 | 51 | 183 | 182 | 35 | 38 | 58 | 59 |
| lens flare | 23 | 22 | 206 | 196 | 26 | 25 | 10 | 9 |
| Corruption | SHA | SHB | QNRF | JHU |
|---|---|---|---|---|
| clean (no corruption) | 67.4 | 11.4 | 95.8 | 68.9 |
| Sensor & exposure | ||||
| fixed-pattern noise | 102.6 | 28.5 | 141.0 | 103.9 |
| Gaussian noise | 99.8 | 33.4 | 155.6 | 117.5 |
| highlight clipping | 121.1 | 38.3 | 179.5 | 109.4 |
| low light | 93.0 | 38.0 | 141.5 | 101.0 |
| Output representation | Environment | Optics & motion | Sensor & exposure | Encoding & transport | Scene clutter | Visibility loss | |
|---|---|---|---|---|---|---|---|
| D Density map | 15 | 38.2 | 60.4 | 68.7 | 38.0 | 14.9 | 22.4 |
| B Blockwise count map | 5 | 44.2 | 56.9 | 53.6 | 26.7 | 19.9 | 25.5 |
| P Localization-oriented | 4 | 51.0 | 66.3 | 86.8 | 32.0 | 11.9 | 21.8 |
| C Direct count | 2 | 27.7 | 37.3 | 20.8 | 18.2 | 17.4 | 22.5 |
| Data | Q | clean | mean | med. | p90 | RD | nMAE |
|---|---|---|---|---|---|---|---|
| SHA | S | 24.1 | 6.3 | 2.0 | 27.9 | 26.2 | 4.4 |
| M | 43.8 | 7.7 | 2.9 | 42.8 | 17.7 | 3.2 | |
| D | 60.0 | 13.0 | 5.0 | 66.1 | 21.7 | 3.4 | |
| VD | 125.8 | 33.4 | 12.0 | 142.9 | 26.5 | 3.7 | |
| SHB | S | 2.7 | 3.8 | 1.3 | 12.0 | 141.1 | 10.1 |
| M | 5.2 | 5.5 | 2.0 | 19.0 | 105.0 | 8.0 |
| clean | degradation | FP32 drop | ||||
|---|---|---|---|---|---|---|
| Dataset | FP32 | FP16 | INT8 | precision | recall | |
| SHA | 0.726 | 0.034 | 0.033 | 0.032 | 0.030 | 0.036 |
| SHB | 0.826 | 0.051 | 0.051 | 0.050 | 0.003 | 0.088 |
| QNRF | 0.739 | 0.027 | 0.027 | 0.027 | 0.014 | 0.037 |
| JHU | 0.685 | 0.029 | 0.029 | 0.029 | 0.005 | 0.047 |
| degradation | FP32 drop | ||||
|---|---|---|---|---|---|
| Family | FP32 | FP16 | INT8 | precision | recall |
| Sensor & exposure | 0.054 | 0.054 | 0.052 | 0.015 | 0.085 |
| Optics & motion | 0.034 | 0.034 | 0.034 | -0.001 | 0.062 |
| Environment | 0.033 | 0.033 | 0.033 | 0.018 | 0.043 |
| Encoding & transport | 0.030 | 0.030 | 0.030 | 0.007 | 0.050 |
| Scene clutter | 0.026 | 0.026 | 0.025 | 0.030 | 0.021 |
| FP16 | INT8 | |||||
| Corruption | med | med | med | p10 | p90 | largest comparison |
| Sensor & exposure | ||||||
| Gaussian noise | -0.03 | +0.58 | 5.91 | -13.2 | +20.4 | ZIP-Small / JHU |
| vignetting | +0.01 | -0.55 | 2.80 | -16.1 | +4.6 | ZIP-Small / SHB |
| highlight clipping | +0.01 | -0.52 | 3.66 | -12.8 | +9.9 | ZIP-Pico / SHB |
| low light | -0.01 | -0.23 | 3.67 | -14.1 | +8.5 | GCHF / SHB |
| FP32 | FP16 | INT8 | ||||||
| Model | Par. | INT8 | J/img | img/s | J/img | img/s | J/img | img/s |
| (M) | %lyr † | |||||||
| CNN–based | ||||||||
| APGCC | 18.7 | 94 | 2.49 | 9.6 | 1.17 | 18.3 | 0.66 | 23.6 |
| CHF | 21.5 | 100 | 2.36 | 10.4 | 1.07 | 20.7 | 0.52 | 29.1 |
| GCHF | 21.5 | 100 | 2.37 | 10.4 | 1.07 | 20.7 | 0.52 | 29.1 |
| FP32 | FP16 | INT8 | ||||||
| Model | Par. | INT8 | J/img | img/s | J/img | img/s | J/img | img/s |
| (M) | %lyr † | |||||||
| CNN–based | ||||||||
| APGCC | 18.7 | 94 | 3.35 | 7.3 | 1.53 | 14.1 | 0.86 | 18.1 |
| CHF | 21.5 | 100 | 3.54 | 7.1 | 1.59 | 14.5 | 0.73 | 20.9 |
| GCHF | 21.5 | 100 | 3.54 | 7.1 | 1.59 | 14.6 | 0.73 | 20.9 |
| FP32 | FP16 | INT8 | ||||||
| Model | Par. | INT8 | J/img | img/s | J/img | img/s | J/img | img/s |
| (M) | %lyr † | |||||||
| CNN–based | ||||||||
| APGCC | 18.7 | 94 | 12.35 | 1.6 | 6.46 | 2.4 | 4.28 | 2.8 |
| CHF | 21.5 | 100 | 14.75 | 1.5 | 7.29 | 2.4 | 4.19 | 2.9 |
| GCHF | 21.5 | 100 | 14.74 | 1.5 | 7.23 | 2.4 | 4.16 | 2.9 |
| FP32 | FP16 | INT8 | ||||||
| Model | Par. | INT8 | J/img | img/s | J/img | img/s | J/img | img/s |
| (M) | %lyr † | |||||||
| CNN–based | ||||||||
| APGCC | 18.7 | 94 | 4.98 | 4.5 | 2.40 | 7.8 | 1.46 | 9.5 |
| CHF | 21.5 | 100 | 6.94 | 3.5 | 3.17 | 6.6 | 1.61 | 8.8 |
| GCHF | 21.5 | 100 | 7.01 | 3.5 | 3.18 | 6.6 | 1.59 | 9.0 |
| layers | plan layers (%) | |||||||
|---|---|---|---|---|---|---|---|---|
| Model | all | comp. | INT8 | cov. (%) | INT8 | FP16 | FP32 | build (s) |
| CHF | 27 | 19 | 19 | 100.0 | 93 | 4 | 4 | 44 |
| GCHF | 27 | 19 | 19 | 100.0 | 93 | 4 | 4 | 40 |
| CCST | 349 | 63 | 63 | 100.0 | 43 | 47 | 10 | 136 |
| CSRNet | 23 | 17 | 17 | 100.0 | 91 | 4 | 4 | 11 |
| DM-Count | 34 | 19 | 19 | 100.0 | 75 | 21 | 4 | 14 |
Crowd Counting
Crowd counting estimates how many people appear in images or video, supporting occupancy monitoring, safety planning, and crowd management.
See all 23 papers