cs.CVOct 2, 2026

DABACO: A Multi-Camera Dataset and Benchmark for Screen Localization and Pointing Estimation

Authors: Óscar Gómez-Cárdenes, José Gil Marichal-Hernández, Juan Manuel Martín-Doñas

Organizations: Universidad de La Laguna (ULL)

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Screen localization and pointing estimation are key to low-cost interactive devices. Yet developing and evaluating these algorithms requires realistic data: synthetic captures cannot fully reproduce the optical distortion, rolling shutter, motion blur, and display processing of a physical acquisition, and most existing datasets provide static images rather than the video needed to assess continuous pointing. In this paper, we introduce the DABACO Dataset. Developed within the DABACO (Dispositivo Apuntador de BAjo COste, or Low-Cost Pointing Device) project, this dataset supports the development and evaluation of screen detection and camera-based pointing algorithms for embedded systems. It comprises video sequences captured with multiple low-cost camera sensors, including monochrome global-shutter and color rolling-shutter modules, on embedded platforms such as the Raspberry Pi 4B and ESP32-S3. We present a systematic annotation pipeline combining temporary visual watermarking, optical-flow tracking, manual verification, and marker removal. Both the original marked captures and the marker-free images, together with explicit corner annotations and modification masks, are released to support auditing and the study of potential reconstruction bias. In addition, we release an open-source evaluation toolkit with two reference baselines: a classical screen-detection pipeline based on edge and contour geometry, and a closed-vocabulary, off-the-shelf YOLOv8 segmentation model evaluated without dataset-specific training. Both are evaluated on a general-purpose computer using Intersection over Union, corner error, and pointing error, establishing initial reference results for future embedded implementations. Overall, DABACO addresses a gap in existing resources and is intended to support the development and evaluation of new low-cost screen-localization and pointing systems.

Figures & tables

DatasetVideoTemporalDisplayHomo-CornersEmbeddedPointingEval.
    graphy   code
HPatches [ 15 ]SequenceYesNoYesNoNoNoYes
HEB/Pi3D [ 16 ]NoNoNoYesNoNoNoYes
Screen Detection YOLO [ 17 ]NoNoYesNoNoNoNoNo
Screen Detection v2 [ 18 ]NoNoYesNoNoNoNoNo
PlanarTrack [ 19 , 20 ]YesYesPartialDerivableYesNoNoYes
Table 1: Comparison between related resources and DABACO. Column criteria are defined in the text.
Figure 1: Overview of the Ground Truth Generation Pipeline.
Figure 2: Visual progression of a frame through the four stages of the Ground Truth Generation Pipeline.
Figure 3: The Interactive Verification editor’s graphical user interface. The left panel shows the system’s output during automatic pre-annotation. The right panel illustrates the manual review process, where annotators utilize the integrated 4.5×4.5\times zoom loupe to perform fine-grained coordinate corrections.
Annotation SourceFramesPercentage
Direct Marker Detection (Anchors)16,99352.08%
Optical Flow Propagation14,15843.39%
Manual Correction4891.50%
Discarded / Invalid Geometry9893.03%
Total Processed Frames32,629100.00%
Table 2: Frame annotation origins in the initial release.
Figure 4: Visual comparison of Telea inpainting across representative capture scenarios. Top row: original crops containing the ArUco markers. Bottom row: corresponding inpainted regions. Left (a, d): Seamless marker removal on uniform background ( esp32_ov3660_scrA_v2_m2vga , frame 170). Middle (b, e): Stretched pixel structure forming a square of four converging triangles ( rpi4_ov5647_scrA_v1_m1fhd , frame 94). Right (c, f): Inward bleeding of the dark screen bezel creating an artificial vignetting corner cut under reduced scale ( rpi4_ov9281_scrC_v1_m3hifps , frame 267).
CameraTypeHigh-Quality (HQ)Medium-Quality (MQ)High-Speed (HS)
OV9281Mono, Global1280×8001280\times 800 @30, Auto1280×8001280\times 800 @30, 1/1000s640×400640\times 400 @100, Auto
OV5647Color, Rolling1080p@30, Auto960p@30, 1/500s720p@60, 1/1000s
ESP32-S3IoT, Color800×600800\times 600 @17.4, Auto640×480640\times 480 @24, 1/125s320×240320\times 240 @27.6, Auto
Table 3: Operational modes per camera: the three exposure and resolution configurations (HQ, MQ, HS).
CameraDisplayV1 (Animation)V2 (Gameplay)V3 (Presentation)
OV9281MonitorHQMQHS
 TVMQHSHQ
 LaptopHSHQMQ
OV5647MonitorHQMQHS
 TVMQHSHQ
 LaptopHSHQMQ
Table 4: Balanced Latin Square assignment of the three modes to each display–content combination.
Figure 5: Block diagram of the classical computer vision baseline for screen detection.
AlgorithmCoverage (%)False-Lock (%)Corner Error (%)IoU (%)Pointing Error (% – all)Pointing Error (% – valid)Time-to-Lock (frames)
Classical92.6358.7912.52 (11.44)52.98 (53.24)28.74 (23.30)23.0739.7
YOLOv8 Seg.86.9752.317.89 (6.39)58.84 (63.16)90.56 (30.72)89.1525.4
Table 5: Aggregate Baseline Performance on the DABACO Dataset (Mean values, with medians in parentheses). The “all frames” column reports the mean over all evaluated frames, including the default penalty for missing predictions. The “valid preds.” column reports the mean restricted to frames for which a valid prediction was returned. Median and P95 values for both aggregation protocols are available in the accompanying benchmark report.
DisplayMethodCoverage (%)False-Lock (%)IoU (%)Pointing Error (% – all frames)
MonitorClassical96.7162.0958.7621.09
MonitorYOLOv897.1335.3576.999.83
TVClassical89.8637.0867.5619.23
TVYOLOv884.3238.3368.5920.10
LaptopClassical91.3177.1932.6445.91
LaptopYOLOv879.4783.2730.93241.76
Table 6: Baseline performance by display type (Mean values reported). Pointing error can exceed 100% for valid but geometrically unstable homography predictions.

Explore similar work

CardsList
Jun 30, 2026

Fusing Complementary Multi-view Features for Screen-Based Eye Tracking

Current multi-view gaze estimation remains limited by existing datasets, insufficient exploitation of complementary cross-view information, and evaluation focused primarily on average gaze error. We address these limitations through a more systematic study of multi-view gaze estimation. First, we introduce PrismGaze, a new dataset with over three million images, capturing continuous headpose variation for the same gaze targets. Second, we propose PrismFusion, a multi-view feature fusion framework based on region partitioning, which masks complementary image regions across views during training to encourage effective cross-view information integration. Third, we develop a broader evaluation framework that examines the effects of camera number and placement, target location,and viewing depth. Our experiments show that the primary benefit of multi-view gaze estimation comes from compensating for poorly observed views with cameras providing more favorable viewpoints. PrismFusion remains robust to changes in viewing depth. Together, our dataset, method, and evaluation provide a more comprehensive foundation for studying multi-view gaze estimation in realistic settings.

Chang Liu, Jiaqi Liu, Chengwen Zhang +6

Tsinghua University · Georgia Institute of Technology

Gaze EstimationMulti-View Feature FusionMulti-View LearningEye Tracking
AskPDF
Sep 1, 2026

TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views

Multi-camera systems are increasingly practical for robotics, AR/VR, and autonomous driving because complementary views reduce depth ambiguity and preserve visibility under occlusion. Existing point-tracking benchmarks, however, focus on a single video or static multi-camera rigs. None test long-term 3D point tracking across several synchronized views under camera motion. We introduce TAPVid-MV (Tracking Any Point in Video across Multiple Views), the first benchmark for this setting. It contains a curated set of 238 sequences, 1,004 calibrated camera streams, and 74,147 point tracks across seven subsets spanning indoor and outdoor domains, from robotics and human activity to driving and synthetic procedural scenes. We obtain these trajectories using dataset-specific auxiliary modalities: sensor depth, LiDAR, SLAM and SfM points, human meshes, posed object meshes, and simulation. Every sequence and trajectory is visually verified by human annotators. Across more than 30 baselines, no method comes close to solving the task. Surprisingly, existing multi-view point trackers do not consistently outperform monocular point trackers. By evaluating reconstruction and point tracking on the same datasets, TAPVid-MV helps distinguish errors in recovered geometry from errors in point correspondence. Through this joint analysis, we identify geometry recovery as a major bottleneck for accurate 3D point tracking. Beyond multi-view 3D point tracking, our released annotations support monocular 2D and 3D point tracking, future-trajectory prediction, and 4D reconstruction. Project page: https://tapvidmv.github.io/

Skanda Koppula, Frano Rajic, Abdullah Faiz Ur Rahman +9

Google DeepMind · University College London · ETH Zürich +1

Point Tracking3D Point TrackingMulti-View 3D Reconstruction
AskPDF
Sep 25, 2026

Low Cost Eye Tracking for Vision Screening

This engineering study assesses gaze-assisted data acquisition during visual screening with a Gaze Quest implementation via webcam and GC308 near infrared camera with an Orlosky eye tracking pipeline. Ten subjects performed under both acquisition conditions. Accuracy of Tumbling E orientation, reported application logMAR value,and response latency were determined from keyboard responses and hence are behavioral outputs as opposed toeye tracker screening results. Frame-to-frame change in theangle between successive normalized gaze vectors (28.2degrees) served as the GC308 eye tracker derived diagnostic with an effective sampling rate of roughly 8 FPS. No screen target calibration or gaze to screen translation was done on the GC308 eye tracker output data for the current analysis and hence, these data represent acquisition stability and not gaze accuracy in relation to the target. The webcam pipeline relied on 25-point screen calibration. Because the display resolution and pixel pitch could not be retained in the records of the study, the reported application logMAR values cannot be viewed as measures of visual acuity in any way.

Maissa Abir Smaili, Yaman Ismaiel, Zafer Iscan

Istanbul Medipol University

Eye Tracking
AskPDF