cs.CVMar 27, 2026
SaveComputer Vision with a Superpixelation Camera
Organizations: Portland State University · Willamette University
Abstract
Conventional cameras generate a lot of data that can be challenging to process in resource-constrained applications. Usually, cameras generate data streams on the order of the number of pixels in the image. However, most of this captured data is redundant for many downstream computer vision algorithms. We propose a novel camera design, which we call SuperCam, that adaptively processes captured data by performing superpixel segmentation on the fly. We show that SuperCam performs better than current state-of-the-art superpixel algorithms under memory-constrained situations. We also compare how well SuperCam performs when the compressed data is used for downstream computer vision tasks. Our results demonstrate that the proposed design provides superior output for image segmentation, object detection, and monocular depth estimation in situations where the available memory on the camera is limited. We posit that superpixel segmentation will play a crucial role as more computer vision inference models are deployed in edge devices. SuperCam would allow computer vision engineers to design more efficient systems for these applications.
Figures & tables
Figure 2: SuperCam pseudoscode. SuperCam stores scene information as a collection of segment boundaries and their associated intensity values. Segment information is updated after a new measurement is available. We envision steps 4–7 of the algorithm to run on-sensor (in or near the camera pixels) and step 9–10 to be implemented off-sensor. The SuperCam algorithm is simple, does not require heavy compute and can be implemented with minimal hardware modifications. The nearest segment operation is parallelizable and can be implemented to run real-time off sensor.
Figure 3: Quantitative comparison with non-learning based methods. We compare the superpixel segmentation quantitatively using the BSD500, NYUV2, SBD and SUNRGBD datasets. (a) Under Segmentation Error Under Segmentation Error comparison for SuperCam and SNIC for all datasets. SuperCam does at least twice as well as SNIC when using the same amount of memory. (b) Superpixel Performance Precision vs recall plot for the SuperCam and SNIC algorithms. SuperCam has better recall than SNIC on all datasets, although it has a lower precision than SNIC due to the higher quantity of superpixels used for the same amount of memory.
Figure 4: Comparison of memory-constrained SNIC and SuperCam superpixel images. Superpixel images generated from the BSD500 dataset for different memory settings in kilobytes (KB), shown here without Gaussian blur applied for either method. The SuperCam results show higher fidelity, better textural details, and less aliasing.
Figure 5: Visual comparison of SuperCam and memory constrained SNIC for three different computer vision tasks. (a) Image Segmentation results using SAM2 on the BSD500 dataset. The florescent green color refers to regions of the image that were not given any mask by the SAM2 model. (b) Object Detection results using the YOLOv12 model on the COCO dataset. (c) Monocular Depth Estimation results using the DepthAnythingV2 model on the NYUV2 dataset. Rows compare across algorithms and columns show different memory settings (in kilobytes (KB)). SuperCam produces visually better results than SNIC using the same amount of memory.
Figure 6: Quantitative evaluation of image segmentation. We compare the quality of image segmentation results produced by the SegmentAnythingV2 model using the mIOU metric on publicly available (a) NYUV2 and (b) BSD500 datasets. Observe that our proposed SuperCam method achieves a lower mIOU mean error than SNIC. The error approaches that of an unsegmented image as the number of superpixels is increased.
Figure 7: Quantitative evaluation of object detection. We compare object detection performance for the superpixel images of SNIC and SuperCam with downsampled images. We use mAP(50-95) values for the ground truth bounding boxes produced by the YOLOv12 on the COCO dataset. Observe that SuperCam detections are better than memory constrained SNIC and comparable to results for resolution-matched images. Missed detections in SuperCam are primarily smaller objects lost due to sampling limitations. Both SNIC and SuperCam converge toward the results produced by an unsegmented image as the available memory is increased.
Figure 8: Quantitative evaluation of monocular depth estimation. We compare the (a) Absolute Relative Error (AbsRel) and (b) Threshold Accuracy ( ) error metrics for depth estimates produced by the DepthAnythingV2 model on the NYUV2 dataset. Observe that SuperCam error metrics are better when compared to SNIC using the same amount of memory. Both algorithms converge to the error values produced by the unsegmented image as memory increases.
Figure 9: Qualitative comparison of SPAD camera results. We show experimental results of SuperCam on real world SPAD data from the SWISS SPAD sensor used in the Burst-Vision work. (a) Image Segmentation results using SAM2. The florescent green color refers to regions of the image that were not given any mask by SAM2 model. (b) Object Detection results using the YOLOv12 model. (c) Monocular Depth Estimation results using the DepthAnythingV2 model. Observe that results get better as we increase the memory used.
Supplementary Figure 1 : Comparison of memory-restricted SNIC and SuperCam superpixel images. Images taken from the BSD500 dataset, shown at different memory settings in kilobytes (KB). These are raw output images without Gaussian blur applied. SuperCam results show better visual details and less aliasing.
Supplementary Figure 2 : Comparisons of SuperCam with other superpixel algorithms. Quantitative comparisons of SuperCam with other learning based and non-learning based superpixel algorithms. (a) Comparisons of SuperCam with learning based superpixel algorithms: We show comparisons of SuperCam with two recent learning based algorithms SPAM and LNS-Net. (b) Comparisons of SuperCam with non-learning based superpixel algorithms: Here we show comparisons of SuperCam with memory restricted SNIC, SLIC and ERS on the BSD500 dataset. We can see that the overall trend is similar for all the memory restricted algorithms and SuperCam performs better that all algorithms.
Supplementary Figure 3 : Results of downstream computer vision applications for other non-learning based superpixel algorithms. We show a snapshot of qualitative comparisons for SuperCam against memory restricted SNIC, ERS and SLIC. SuperCam performs better that all the other algorithms on all computer vision tasks.
Supplementary Figure 4 : SuperCam Algorithm. We show a diagram of the proposed SuperCam algorithm. The pixels are first adaptively sampled to create an initial superpixel image that has holes. The holes are filled using nearest neighbor interpolation and a Gaussian Blur is applied to it to generate the final SuperCam image. The circles with the green border are the pixels that are exposed adaptively. The ellipses are the calculated optimal 2D Gaussian blur kernels for the pixel locations marked with blue borders.
Supplementary Figure 5 : Quantitative evaluation of image segmentation. We compare the quality of image segmentation results produced by the SegmentAnythingV2 model using the mIOU metric on publicly available SBD dataset. Observe that our proposed SuperCam method achieves a lower mIOU mean error than SNIC. The error approaches that of an unsegmented image as the number of superpixels is increased.
Supplementary Figure 6 : Comparison of qualitative Image Segmentation results for memory-restricted SNIC and SuperCam. Image segmentation results produced using the SAMV2 model on superpixel images for SuperCam, raw SNIC, and SNIC with the optimal blur kernel applied. Images are drawn from the (a) NYUV2 and (b) SBD datasets for different memory settings in kilobytes (KB). SuperCam results look visually better and converge to the ground truth as we increase the memory used.
Supplementary Figure 7 : Comparison of memory-constrained SNIC and SuperCam superpixel results for Object Detection. Results on the YOLOv12 model applied to superpixel images for SuperCam, raw SNIC, and SNIC with the optimal blur kernel applied. Images are drawn from the COCO dataset, and each column shows a different memory setting in kilobytes (KB). SuperCam results look visually better and converge to the ground truth as we increase the memory used.
Supplementary Figure 8 : Quantitative evaluation of monocular depth estimation on the KITTI dataset. We compare the (a) Absolute Relative Error (AbsRel) and (b) Threshold Accuracy ( ) metrics for depth estimates produced by the DepthAnythingV2 model on the KITTI dataset. Observe that SuperCam error metrics are better when compared to restricted SNIC across all memory settings.
Supplementary Figure 9 : Quantitative evaluation of monocular depth estimation on the DIODE dataset. We compare the (a) Absolute Relative Error (AbsRel) and (b) Threshold Accuracy ( ) error metrics for depth estimates produced by the DepthAnythingV2 model on the DIODE dataset. Observe that SuperCam error metrics are better when compared to restricted SNIC using the same amount of memory.
Supplementary Figure 10 : Quantitative evaluation of mean absolute error for monocular depth estimation on the Sintel dataset. We compare the (a) mean Absolute Relative Error (AbsRel) and (b) Threshold Accuracy ( ) error metrics for depth estimates produced by the DepthAnythingV2 model on the Sintel dataset. AbsRel and Threshold Accuracy values are comparable to Restricted SNIC for all memory settings.
Supplementary Figure 11 : Quantitative evaluation of median absolute error for monocular depth estimation on the Sintel dataset. We compare the median (a) Absolute Relative Error (AbsRel) and (b) Threshold Accuracy ( ) error metrics for depth estimates produced by the DepthAnythingV2 model on the Sintel dataset. AbsRel are comparable to or better than Restricted SNIC for all memory settings. Average Threshold Accuracy values are higher than Restricted SNIC across memory settings, although subsequent figures show that the underlying error distributions are comparable.
Supplementary Figure 12 : Comparison of Monocular Depth Estimation results for the DIODE dataset. Object Detection results on the DepthAnythingV2 model applied to the superpixel images for SuperCam, Restricted SNIC, and SNIC with the optimal blur kernel applied. Images are drawn from the DIODE dataset and columns show different memory settings in kilobytes (KB). SuperCam results look visually better, particularly for lower memory settings.
Supplementary Figure 13 : Comparison of Monocular Depth Estimation results for the KITTI dataset. Object Detection results on the DepthAnythingV2 model applied to superpixel images for SuperCam, Restricted SNIC, and SNIC with the optimal blur kernel applied. Images are drawn from the KITTI dataset, and columns show different memory settings in kilobytes (KB). SuperCam results look slightly better visually. The lack of significant difference visually between SuperCam and SNIC can be explained by the fact that the KITTI dataset is very sparse and this affects the conversion of relative depth to absolute depth as there are only a few valid data points to do the conversion.
Supplementary Figure 14 : Comparison of Monocular Depth Estimation results for the Sintel dataset. Object Detection results on the DepthAnythingV2 model on the superpixel images for SuperCam, Restricted SNIC, and SNIC with the optimal blur kernel applied. Images are drawn from the Sintel dataset and columns show different memory settings in kilobytes (KB). SuperCam results look visually better.
Supplementary Figure 15 : Qualitative comparison of experimental results. We show experimental results of SuperCam on real world SPAD data from the SWISS SPAD sensor used in the Burst-Vision work. (a) Image Segmentation results using SAM2 on experimental SPAD data collected from the SWISS SPAD sensor. The florescent green color refers to regions of the image that were not given any mask by SAM2 model. (b) Object Detection results using the YOLOv12 model on experimental SPAD data collected from the SWISS SPAD sensor. (c) Monocular Depth Estimation results using the DepthAnythingV2 model on experimental SPAD data collected from the SWISS SPAD sensor. Observe that results the SuperCam results are visually better than the SNIC result that uses the same amount of memory. The results improve with increase in the amount of memory used.
More in this topic
Image Segmentation
Image segmentation assigns pixels or voxels to objects and regions, supporting measurement, scene understanding, and decisions across image domains.
See all 390 papers