IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline

Sebastian-Ion Nae; Radu Moldoveanu; Alexandra Stefania Ghita; Adina Magda Florea

cs.CVApr 2, 2026

IndoorCrowd: A Multi-Scene Dataset for Human Detection, Segmentation, and Tracking with an Automated Annotation Pipeline

Authors: Sebastian-Ion Nae, Radu Moldoveanu, Alexandra Stefania Ghita, Adina Magda Florea

Organizations: National University of Science and Technology Politehnica Bucharest, Romania · Expleo, Romania

Abstract

Understanding human behaviour in crowded indoor environments is central to surveillance, smart buildings, and human-robot interaction, yet existing datasets rarely capture real-world indoor complexity at scale. We introduce IndoorCrowd, a multi-scene dataset for indoor human detection, instance segmentation, and multi-object tracking, collected across four campus locations (ACS-EC, ACS-EG, IE-Central, R-Central). It comprises $31$ videos ( $9{,}913$ frames at $5$ fps) with human-verified, per-instance segmentation masks. A $620$ -frame control subset benchmarks three foundation-model auto-annotators: SAM3, GroundingSAM, and EfficientGroundingSAM, against human labels using Cohen's $κ$ , AP, precision, recall, and mask IoU. A further $2{,}552$ -frame subset supports multi-object tracking with continuous identity tracks in MOTChallenge format. We establish detection, segmentation, and tracking baselines using YOLOv8n, YOLOv26n, and RT-DETR-L paired with ByteTrack, BoT-SORT, and OC-SORT. Per-scene analysis reveals substantial difficulty variation driven by crowd density, scale, and occlusion: ACS-EC, with $79.3\%$ dense frames and a mean instance scale of $60.8$ px, is the most challenging scene. The project page is available at https://sheepseb.github.io/IndoorCrowd/.

Figures & tables

Dataset	Year	Indoor	Frames	Scenes	BBox	Mask	MOT	Real
Caltech [ 11 ]	2009	—	250K	1	$\checkmark$	—	—	$\checkmark$
MOT17 [ 26 ]	2016	—	11K	7	$\checkmark$	—	$\checkmark$	$\checkmark$
CityPersons [ 42 ]	2017	—	35K	1	$\checkmark$	—	—	$\checkmark$
CrowdHuman [ 34 ]	2018	—	15K	1	$\checkmark$	—	—	$\checkmark$
JTA [ 14 ]	2018	$\circ$	460K	1	$\checkmark$	—	$\checkmark$	—
WiderPerson [ 43 ]	2019	—	13K	1	$\checkmark$	—	—	$\checkmark$

Table 1: Comparison of pedestrian detection and tracking datasets.

\checkmark

: available;

\circ

: partial or indirect; —: not available. IndoorCrowd is one of the real-world indoor datasets providing all three annotation types across multiple scenes.

Figure 1: Representative frames from the four dataset scenes (left to right): ACS-EC ground-level view showing dense seating and circulation areas; ACS-EC elevated view providing a top-down perspective of the same atrium; ACS-EG narrow corridor with strong near-to-distal scale variance; IE-Central entrance hall captured from an elevated angle; and R-Central central atrium with prominent structural columns and overhead viewpoint.

Scene	Instances	Persons / Frame	Density (% frames)			Occlusion
Scene	Instances	Mean $\pm$ Std	Sparse	Medium	Dense	Rate
ACS-EC	2,128	$12.23\pm 3.80$	2.9%	17.8%	79.3%	27.5%
ACS-EG	1,260	$5.41\pm 1.94$	18.5%	81.5%	0.0%	38.3%
IE-Central	1,083	$7.96\pm 3.63$	0.0%	76.5%	23.5%	31.9%
R-Central	391	$6.86\pm 1.52$	1.8%	98.2%	0.0%	26.9%
Overall	4,862	$8.10\pm 4.18$	5.8%	62.0%	32.2%	30.3%

Table 2: Per-scene crowd statistics for the detection and segmentation subset. Density bins are defined as sparse (

\leq 3

), medium (4–10), and dense (

>10

) persons per frame. ACS-EC is a challenging scene, with 79.3% of frames classified as dense and a mean of

12.23\pm 3.80

persons per frame. ACS-EG and R-Central are predominantly medium-density, while IE-Central spans the widest per-frame range (4–17 persons, 23.5% dense frames). Occlusion rates are estimated via bounding-box overlap (IoU

>

0.1); ACS-EG shows the highest rate (38.3%) despite its lower density, reflecting its corridor geometry and ground-level viewpoint rather than crowd size alone.

Scene	Relative Scale		Abs. Scale (px)	Size Distribution (% instances)
Scene	Mean $\pm$ Std	AR Mean $\pm$ Std	Mean $\pm$ Std	Small	Medium	Large
ACS-EC	$0.063\pm 0.039$	$2.02\pm 0.89$	$60.8\pm 37.7$	17.2%	73.5%	9.4%
ACS-EG	$0.141\pm 0.081$	$3.28\pm 1.03$	$135.6\pm 78.2$	1.0%	40.6%	58.4%
IE-Central	$0.095\pm 0.040$	$2.48\pm 0.66$	$91.2\pm 38.6$	3.3%	49.8%	46.9%
R-Central	$0.062\pm 0.017$	$2.26\pm 0.73$	$59.1\pm 16.4$	4.9%	93.6%	1.5%

Table 3: Per-scene instance scale and aspect-ratio statistics. Relative scale is

\sqrt{A_{\text{box}}}/\sqrt{A_{\text{frame}}}

; absolute scale is the mean bounding-box side length in pixels. Size bins follow COCO convention: small

{<}32^{2}

px, medium

32^{2}

–

96^{2}

px, large

{>}96^{2}

px. ACS-EG is dominated by large instances (

58.4

%) due to its ground-level corridor viewpoint, while ACS-EC and R-Central present predominantly small-to-medium instances (mean

{\approx}60

px), posing a challenge for instance segmentation. The high aspect-ratio variance in ACS-EC (

2.02\pm 0.89

) reflects diverse poses, partial occlusions, and frame-boundary truncations in its dense atrium setting.

Panel of Figure 2 — Figure 2: Spatial density heatmaps showing the normalised distribution of person bounding-box centres across all annotated frames per scene. Colour encodes relative density (yellow $\to$ dark red = low $\to$ high). The heatmaps reveal four distinct spatial regimes: a dominant horizontal band in ACS-EC driven by circulation traffic and stationary occupants in the common area; a concentrated mid-depth cluster in ACS-EG reflecting strong scale variance along its linear corridor axis; three discrete zones in IE-Central spanning the entry, corridor junction, and seating area; and a diffuse, column-interrupted spread in R-Central where the overhead viewpoint collapses ascending and descending pedestrian flow into a single projection. These spatial patterns directly inform the per-scene variation in crowd density, occlusion rate, and detection difficulty reported in Sections 4.3 and 4.4 .

Figure 3: Qualitative comparison of auto-labelling methods across ACS-EC, IE-Central, and R-Central (rows, top to bottom). Columns show the raw image, SAM3, GroundingSAM, and human ground truth, with per-frame instance counts (n). SAM3 produces false positives on ACS-EC (row 1); GroundingSAM misses occluded persons across all scenes.

Scene	Method	$N_{\text{pred}}$	$N_{\text{GT}}$	AP $\uparrow$		Prec. $\uparrow$ / Rec. $\uparrow$ @ 0.5		Mask IoU $\uparrow$	Cohen’s $\uparrow$
Scene	Method	$N_{\text{pred}}$	$N_{\text{GT}}$	@0.5	@0.75	P	R	(mean)	$\kappa$
ACS-EC	SAM3	3596	2128	0.783	0.599	0.522	0.882	0.802	0.849
	G-SAM	1214		0.435	0.324	0.831	0.474	0.831	0.762
	EG-SAM	1212		0.433	0.323	0.829	0.472	0.803	0.758
ACS-EG	SAM3	1643	1260	0.963	0.862	0.749	0.976	0.859	0.926
	G-SAM	1359		0.922	0.824	0.870	0.939	0.869	0.931

Table 4: Auto-labelling quality against human ground truth across all four scenes.

N_{\text{pred}}

and

N_{\text{GT}}

denote predicted and ground-truth instance counts; Cohen’s

\kappa

follows [ 19 ] . SAM3 maximises recall at the cost of precision; GroundingSAM and EfficientGroundingSAM behave near-identically with the inverse trade-off. All methods degrade most on ACS-EC.

Model	Task	Box mAP $\uparrow$		Mask mAP $\uparrow$		Mean	Detection $\uparrow$			Latency	Size	GFLOPs
Model	Task	@0.5	@0.50:95	@0.5	@0.50:95	IoU	P	R	F1	(ms)	(MB)
YOLOv8n	detect	0.864	0.616	—	—	—	0.876	0.766	0.817	2.28	6.22	8.1
YOLOv26n	detect	0.796	0.560	—	—	—	0.749	0.715	0.732	2.81	5.36	5.2
RT-DETR-L	detect	0.911	0.704	—	—	—	0.898	0.834	0.865	27.36	66.21	103.4
YOLOv8n-seg	segment	0.864	0.620	0.833	0.541	0.630	0.915	0.757	0.829	1.89	6.76	11.3
YOLOv26n-seg	segment	0.808	0.572	0.787	0.512	0.599	0.773	0.741	0.757	2.61	6.51	9.0

Table 5: Detection and segmentation benchmark results. RT-DETR-L achieves the highest accuracy, but at a high compute cost; YOLOv8n-seg offers the best accuracy–efficiency trade-off for the combined detection and segmentation task. Latency measured on an NVIDIA RTX 4060Ti (16 GB, batch size 1).

Method	Uniq. IDs	Tracks	Mean	Med.	Min	Max
ACS-EC
Human	260	260	32.51	31.0	1	102
SAM3-Native	321	321	40.90	39.0	1	82
SAM3-BotSort	507	507	24.36	17.0	1	82
ACS-EG
Human	122	122	33.49	26.5	1	136

Table 6: Per-scene track statistics comparing human-annotated ground truth against SAM3-Native (raw SAM3 tracklets) and SAM3-BotSort (SAM3 detections linked by BoT-SORT). Mean , Med. , Min , and Max refers to the track length in frames. Human tracks are consistently fewer and longer than automatic counterparts, reflecting the consolidation of fragmented and spurious tracklets during human review.

ACS-EC
	Track Changes			Refinement Actions			Interp.
Method	$\Delta$ IDs	$\Delta$ Trk	$\Delta$ Len	Ghosts	Merged	ID Sw.	Frames
SAM3-Native	61	61	$-$ 8.39	41	85	85	1067
SAM3-BotSort	247	247	8.15	164	118	118	596
ACS-EG
SAM3-Native	71	71	4.27	36	49	49	336

Table 7: Per-scene human correction actions applied to SAM3-Native and SAM3-BotSort tracklets during human review.

\Delta

IDs and

\Delta

Trk are the net reductions in unique identities and track count after review.

\Delta

Len is the mean change in track length (frames); negative values indicate that human tracks are shorter on average than the automatic source. Ghosts are fully deleted spurious tracks; Merged and ID Sw. count identity-switch corrections; Interp. Frames is the number of missing detections filled by linear interpolation. SAM3-BotSort consistently required more structural edits (

\Delta

IDs, Ghosts), while SAM3-Native demanded heavier interpolation, reflecting their complementary failure modes.

Scene	Detector	Tracker	MOTA $\uparrow$	IDF1 $\uparrow$	MT% $\uparrow$	ML% $\downarrow$	IDS $\downarrow$	FPS $\uparrow$
Overall	YOLOv8n	ByteTrack	48.5	66.9	49.0	35.4	161	108.2
	YOLOv8n	BotSort	51.5	68.6	52.9	34.6	143	118.5
	YOLOv8n	OC-SORT	51.4	68.6	45.3	35.8	138	119.5
	RT-DETR-L	ByteTrack	51.4	69.3	55.5	27.2	172	31.4
	RT-DETR-L	BotSort	55.5	71.6	60.6	26.2	161	32.0
	RT-DETR-L	OC-SORT	56.2	71.8	54.7	27.4	166	32.7

Table 8: Multi-object tracking results per scene. Best value per metric per scene is bolded . MT%: mostly tracked; ML%: mostly lost; IDS: identity switches.

Explore similar work

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments
Sep 7, 2026Antonius Bima Murti Wijaya, Paul Henderson, Marwa MahmoudPedestrian Trajectory Prediction Multi-Object Tracking
School of Computing Science University of Glasgow University Avenue G12 8QQ, Glasgow, United Kingdom
A Smart Classroom Behavior Analysis Framework with a New Highly Congested Classroom Dataset
Jun 19, 2026Wei Xu, Maoxiang Chu, Yuelong Fan +5Human Activity Recognition Object Detection
Depth-Guided Video Object Counting in Crowded Scenes
Aug 6, 2026Yuanjing Xu, Xinyan Liu, Weidong Chen +5Video Object Detection Object Detection
Harbin Institute of Technology (Weihai) Weihai, Shandong, China · City University of Hong Kong Hong Kong, China · University of Science and Technology of China Hefei, Anhui, China +1

Sep 7, 2026J/K move · Enter open · S save

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments

Antonius Bima Murti Wijaya, Paul Henderson, Marwa Mahmoud

School of Computing Science University of Glasgow University Avenue G12 8QQ, Glasgow, United Kingdom

Pedestrian Trajectory Prediction Multi-Object Tracking

Open paper AskPDF arXiv

In real-world applications, pedestrian trajectory prediction models rely on inputs from detection and tracking systems. Prior trajectory prediction benchmarks either contain relatively sparse pedestrian interactions, assume perfect tracking inputs, or rely on overhead viewpoints that minimize occlusion and perspective distortion, limiting evaluation in realistic dense-crowd scenarios. We present CrowdTraj, a benchmark for pedestrian trajectory prediction in natural dense crowd scenes. Unlike previous datasets, CrowdTraj supports end-to-end evaluation from detection through tracking to trajectory prediction under severe occlusion in CCTV views. It also captures diverse, natural pedestrian behaviours, including abrupt directional changes rarely observed in existing benchmarks. CrowdTraj includes five diverse scenes, with an average of 1,146 unique pedestrians per scene, maximum frame-level densities ranging from 114 to 372 pedestrians, and over 3.2 million annotated head bounding boxes. CrowdTraj provides pixel and real-world coordinates via per-scene homography matrices for physically meaningful analysis. Our experimental results show that tracking accuracy (IDF1) drops to 0.68 to 0.70 in the densest scenes, compared with approximately 0.90 in less crowded scenes. Trajectory prediction training also becomes substantially more computationally expensive in dense scenes, with training times increasing by up to 8 times. These findings show that CrowdTraj exposes limitations in current trajectory prediction pipelines that remain hidden on existing sparse-crowd benchmarks, particularly in robustness to tracking noise and computational scalability.

Abstract

Explore similar work

CrowdTraj: A Benchmark for Dense Crowd Trajectory Prediction in Realistic Crowded Environments

A Smart Classroom Behavior Analysis Framework with a New Highly Congested Classroom Dataset

Depth-Guided Video Object Counting in Crowded Scenes