cs.CVNov 11, 2026

FoveaDet: Software-Hardware Co-Design of Foveated CNN-Transformer for Object Detection

Authors: Deepak Mewada, Apporva Saraf, Ashwin Lele

Organizations: Centre for Machine Intelligence and Data Science (CMInDS), IIT Bombay, India · Department of Electrical Engineering, IIT Delhi, India

Abstract

Deploying real-time object detection on power-constrained edge platforms is challenging as processing full-resolution frames is often computationally redundant. Inspired by biological foveation which assigns variable resolution based on visual saliency, we propose a novel software-hardware co-design framework called FoveaDet. On the software side, FoveaDet dynamically identifies salient regions to predict fixation points, followed by aggressive patch pruning in Vision Transformers (ViT) to discard irrelevant background data. To mitigate slowdowns from sequential patch generation, we implement optimized interpolation and patch selection algorithms. Evaluated on standard GPUs, this foveated approach reduces floating-point operations by 65.4% to 92.2% and accelerates inference by 4.76x, while suffering a negligible mAP drop compared to dense baselines. To eliminate hardware bottlenecks, we complement the algorithm with a custom RISC-V core featuring specialized instructions to accelerate the foveated processing pipeline. FoveaDet provides a scalable, efficient architecture capable of unlocking advanced vision processing on severely constrained mobile platforms.

Figures & tables

Explore similar work

CardsList
  1. RT-DETRv4: Painlessly Furthering Real-Time Object Detection with Vision Foundation Models

    Oct 29, 2025Zijun Liao, Yian Zhao, Xin Shan +5Real-Time Object DetectionEfficient Object Detection

  2. MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers

    May 11, 2026Novendra Setyawan, Chi-Chia Sun, Mao-Hsiu Hsu +2Structural ReparameterizationEnergy-Efficient ML

  3. FlexViT: A Flexible FPGA-based Accelerator for Edge Vision Transformers

    Jun 30, 2026Hubert Dymarkowski, Xingjian Fu, Rappy Saha +2Hybrid CNN-Transformer ArchitecturesEdge Inference