Phys-3D: Physics-Constrained Real-Time Crowd Tracking and Counting on Railway Platforms

Bin Zeng; Johannes Künzel; Anna Hilsmann; Peter Eisert

cs.CVFeb 26, 2026

Phys-3D: Physics-Constrained Real-Time Crowd Tracking and Counting on Railway Platforms

Authors: Bin Zeng, Johannes Künzel, Anna Hilsmann, Peter Eisert

Organizations: Visual Computing Group, Humboldt University Berlin, Unter den Linden 6, 10099 Berlin, Germany · Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI, Einsteinufer 37, 10587 Berlin, Germany

Abstract

Accurate, real-time crowd counting on railway platforms is essential for safety and capacity management. We propose to use a single camera mounted in a train, scanning the platform while arriving. While hardware constraints are simple, counting remains challenging due to dense occlusions, camera motion, and perspective distortions during train arrivals. Most existing tracking-by-detection approaches assume static cameras or ignore physical consistency in motion modeling, leading to unreliable counting under dynamic conditions. We propose a physics-constrained tracking framework that unifies detection, appearance, and 3D motion reasoning in a real-time pipeline. Our approach integrates a transfer-learned YOLOv11m detector with EfficientNet-B0 appearance encoding within DeepSORT, while introducing a physics-constrained Kalman model (Phys-3D) that enforces physically plausible 3D motion dynamics through pinhole geometry. To address counting brittleness under occlusions, we implement a virtual counting band with persistence. On our platform benchmark, MOT-RailwayPlatformCrowdHead Dataset(MOT-RPCH), our method reduces counting error to 2.97%, demonstrating robust performance despite motion and occlusions. Our results show that incorporating first-principles geometry and motion priors enables reliable crowd counting in safety-critical transportation scenarios, facilitating effective train scheduling and platform safety management.

Figures & tables

Panel of Figure 1 — Figure 1: Two frames from a tracked video sequence. Each bounding box is annotated with a unique track IDs; the bottom-left overlay shows the unique-ID count from the virtual counting zone (yellow area).

Figure 2: Overview of the proposed physics-constrained detect–track–count pipeline. The Phys-3D model integrates geometric and ego-motion priors to achieve stable identity tracking under camera motion.

Figure 3: Virtual Counting Band Illustration. The diagram shows the virtual counting zones positioned near the left and right image borders. The band is defined by start and end boundaries as proportions of image width (Start=0.05, End=0.20), creating a buffer region that tolerates brief occlusions and detection jitter. A target is counted only when it remains continuously within the band for a preset number of frames, providing robust counting under challenging conditions.

Dataset	Images	Instances	Box(Precision)	Recall	mAP50	mAP50–95
CrowdHuman	4370	98568	0.862	0.715	0.794	0.547
Custom Human	600	4779	0.965	0.946	0.980	0.816

Table 1: Performance of the YOLOv11m network. Upper row: after pre-training on the CrowdHuman Dataset. Lower row: fine-tuning results on the Custom Head Dataset.

Figure 4: ReID Input Size Ablation Experiment Results

Input Size (pix)	mAP (%)	Rank-1 (%)	FPS	FLOPs (G)	Parameter quantity	Marginal benefits
64x64	90.74	95.7	94.8	0.002	206240	0
96x96	92.64	96.77	95	0.005	206240	+1.90 mAP
128x128	94.24	100	69.5	0.009	206240	+1.60 mAP
160x160	95.49	100	70.2	0.014	206240	+1.25 mAP
192x192	95.63	100	73.3	0.02	206240	+0.14 mAP
256x256	95.36	98.92	76.7	0.035	206240	–0.27 mAP

Table 2: ReID Ablation Experiment Results.

Model	MAE	RMSE	MAPE(%)	ME
CV-8D	3.4	6.1514	14.592	1.4
CA-12D	2.4	4.5837	6.99	0.7
Phys-3D	0.9	1.3601	2.97	-0.2

Table 3: Counting performance comparison of the three models. Our Phys-3D approach clearly outperforms standard constant velocity or acceleration models.

Metric	MOTA	IDF1	IDSW	Precision	Recall	FAF
Phys-3D	67.19%	76.32%	24.5	89.06%	77.43%	0.32

Table 4: Overall multi-object tracking performance of the Phys-3D model.

Confidence	0.3	0.5	0.7	0.9
IoU	0.3	0.5	0.7	0.9
Image Size	640	736	832	960

Table 5: YOLO detection parameter values tested, yielding 64 combinations.

Conf	IoU	Imgsz	mAP50	mAP50-95	Prec.	Recall	Inf. Time(ms)
0.3	0.3	736	0.9224	0.6871	0.9193	0.8707	16.86

Table 6: Optimal YOLOv11m detector configuration.

Start	End	MAE	RMSE	MAPE(%)	ME
0.05	0.20	0.90	1.38	2.97	-0.20

Table 7: Optimal virtual counting band configuration (Start=0.05, End=0.2).

Method	MAE	RMSE	MAPE(%)	ME
Line-crossing	31.28	43.66	93.43	-31.28
Counting zones	3.13	4.75	10.87	0.81

Table 8: Comparison of average counting metrics lines vs. zones.

Video	MOTA	MOTP	IDF1	IDP	IDR	IDSW	Matches	FP	Misses	FAF	Precision	Recall	MT	PT	ML	LC	RC	TC	TV	MAPE
1	54.26	19.63	72.96	81.73	65.89	2	86	16	41	0.16	84.62	68.22	1	3	0	0	4	4	4	0
2	57.35	17.64	75.65	87.65	66.55	3	375	51	187	0.22	88.11	66.9	3	5	0	8	0	8	8	0
3	57.25	17.66	73.19	77.53	69.32	7	993	215	359	0.52	82.3	73.58	12	10	3	24	0	25	25	0
4	78.33	16.74	83.16	81.49	84.9	1	610	86	58	0.29	87.66	91.33	9	1	0	0	10	10	10	0
5	64.42	16.5	78.33	79.56	77.14	3	874	175	208	0.36	83.37	80.83	13	4	1	19	0	19	18	5.56
6	81.1	16.11	82.98	84.91	81.13	8	2440	196	319	0.29	92.59	88.47	13	3	0	16	0	16	16	0

Table 9: Complete Phys-3D Detailed Performance Results Across All 20 Test Videos

No.	Conf	IoU	Imgsz	mAP50	mAP50-95	Precision	Recall	Inference_Time(ms)
1	0.3	0.3	640	0.9248	0.6749	0.9272	0.8741	13.18
2	0.3	0.3	736	0.9224	0.6871	0.9193	0.8707	16.86
3	0.3	0.3	832	0.8997	0.6545	0.9195	0.8238	21.79
4	0.3	0.3	960	0.8664	0.6144	0.8947	0.7786	29.42
5	0.3	0.5	640	0.9247	0.6748	0.9246	0.8745	14.63
6	0.3	0.5	736	0.9224	0.6869	0.9177	0.8712	18.18

Table 10: Complete YOLOv11m Detector Ablation Study Results Across All 64 Parameter Combinations

Start	End	MAE	RMSE	MAPE(%)	ME
0.05	0.20	0.90	1.38	2.97	-0.20
0.05	0.15	1.15	1.75	4.07	-0.95
0.00	0.15	1.25	1.77	4.67	-0.15
0.10	0.20	1.40	2.10	5.28	-1.10
0.00	0.20	1.50	2.14	4.97	0.60
0.15	0.30	1.50	2.37	8.58	0.20

Table 11: Complete Virtual Counting Band Ablation Study Results Across All Parameter Combinations

Model	Top-1 Acc.	Top-5 Acc.	#Params	Ratio-to-EfficientNet	#FLOPs	Ratio-to-EfficientNet
EfficientNet-B0	77.1%	93.3%	5.3M	1.0×	0.39B	1.0×
ResNet-50	76.0%	93.0%	26M	4.9×	4.1B	11×
DenseNet-169	76.2%	93.2%	14M	2.6×	3.5B	8.9×
EfficientNet-B1	79.1%	94.4%	7.8M	1.0×	0.70B	1.0×
ResNet-152	77.8%	93.8%	60M	7.6×	11B	16×
DenseNet-264	77.9%	93.9%	34M	4.3×	6.0B	8.6×

Table 12: ImageNet performance versus computational complexity across EfficientNet and other representative CNN architectures [ 22 ] .

Video	Resolution	FPS	Duration (s)	Frames	Frame Range	BBox Count	Objects	MinH (px)	MaxH (px)	MinW (px)	MaxW (px)	AvgH (px)	AvgW (px)	Format
1	1920×1080	25	8.2	205	98–196	129	4	27.24	57.35	20.38	52.67	37.75	30.39	.mp4
2	1920×1080	25	11.44	286	41–267	565	8	20.19	86.36	16.63	69.53	40.08	31.74	.mp4
3	1920×1080	25	18.12	453	36–451	1359	25	14.64	153.88	16.36	106.39	52.36	41.48	.mp4
4	2304×1296	59.94	7.84	470	44–427	670	10	34.95	130.56	24.32	129.21	63.99	55.43	.mp4
5	2304×1296	59.94	11.29	677	73–639	1085	18	24.82	197.75	17.58	227.34	70.60	65.51	.mp4
6	2304×1296	59.94	11.18	670	2–666	2911	16	18.00	289.02	20.16	259.88	77.29	64.89	.mp4

Table 13: Twenty video sequences selected from MOT-RPCH for multi-object tracking and counting evaluation.

Video	Resolution	FPS	Duration (s)	Frames	Frame Range	BBox Count	Objects	MinH (px)	MaxH (px)	MinW (px)	MaxW (px)	AvgH (px)	AvgW (px)	Format
1	1920×1080	25	7.24	181	67–166	437	8	22.44	103.24	17.95	87.47	46.75	36.23	.mp4
2	1920×1080	25	7.16	179	64–151	181	3	22.85	58.42	14.31	49.01	36.74	27.28	.mp4
3	2304×1296	59.94	11.54	692	51–664	1914	53	15.91	194.65	14.42	221.60	57.45	51.93	.mp4
4	2560×1440	59.94	6.99	419	90–419	905	18	12.72	127.42	10.28	117.93	26.46	24.48	.mp4
5	2560×1440	59.94	45.71	2740	1–2697	7415	93	10.84	122.27	8.69	127.90	29.51	29.76	.mp4
6	2304×1296	59.94	15.01	900	22–898	1171	9	26.70	214.66	21.27	179.46	68.55	55.99	.mp4

Table 14: Seven video sequences for ReID training and ablation studies.

Crowd Counting

Crowd counting estimates how many people appear in images or video, supporting occupancy monitoring, safety planning, and crowd management.

See all 23 papers

Closed-Circuit Television Data as an Emergent Data Source for Urban Rail Platform Crowding Estimation
Aug 3, 2025Riccardo Fiorista, Awad Abdelhalim, Anson F. Stewart +3Intelligent Transportation Systems Crowd Counting
Transit Lab Department of Urban Studies and Planning Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Washington Metropolitan Area Transit Authority 300 7th Street SW, Washington, DC 20024, USA
CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments
Sep 7, 2026Antonius Bima Murti Wijaya, Paul Henderson, Marwa MahmoudPedestrian Trajectory Prediction Multi-Object Tracking
School of Computing Science University of Glasgow University Avenue G12 8QQ, Glasgow, United Kingdom
IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline
Apr 2, 2026Sebastian-Ion Nae, Radu Moldoveanu, Alexandra Stefania Ghita +1Instance Segmentation Multi-Object Tracking
National University of Science and Technology Politehnica Bucharest, Romania · Expleo, Romania

Aug 3, 2025J/K move · Enter open · S save

Closed-Circuit Television Data as an Emergent Data Source for Urban Rail Platform Crowding Estimation

Riccardo Fiorista, Awad Abdelhalim, Anson F. Stewart, Gabriel L. Pincus+2

Transit Lab Department of Urban Studies and Planning Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Washington Metropolitan Area Transit Authority 300 7th Street SW, Washington, DC 20024, USA

Intelligent Transportation Systems Crowd Counting Video Object Detection

Open paper AskPDF arXiv

Accurately estimating urban rail platform occupancy can support more informed operational decision-making by transit agencies, particularly during periods of crowding. However, sensing real-time platform occupancy remains challenging and often relies on indirect proxies, such as automatic fare collection data or staff observations. Recently, Closed-Circuit Television (CCTV) footage has emerged as a promising data source for accurate, real-time occupancy estimation. The present study investigates this approach by comparing three state-of-the-art computer vision approaches for extracting crowd-related features from platform CCTV imagery: (a) object detection and counting using YOLOv11, RT-DETRv2, and APGCC; (b) crowd-level classification via a custom-trained Vision Transformer (Crowd-ViT); and (c) semantic segmentation using DeepLabV3. Additionally, we present a novel and efficient convex ridge regression over mean-pooled segmentation features to extract counts from the generated segmentation maps while accounting for image depth and passenger dispersion along a platform. Tested on a privacy-preserving dataset, created in collaboration with the Washington Metropolitan Area Transit Authority (WMATA), and including more than 600 hours of video material, our results demonstrate that computer vision approaches can provide substantive value for crowd estimation. This work demonstrates that CCTV image data, independent of other data sources available to transit agencies, can enable real-time crowding estimation and, eventually, timely operational responses to mitigate platform crowding.

Abstract

Crowd Counting

Explore similar work

Closed-Circuit Television Data as an Emergent Data Source for Urban Rail Platform Crowding Estimation

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments

IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline