On the Transfer of Collinearity to Computer Vision

Frederik Beuth; Danny Kowerko

cs.CVMar 17, 2026

On the Transfer of Collinearity to Computer Vision

Authors: Frederik Beuth, Danny Kowerko

Organizations: Junior Professorship of Media Computing, Chemnitz University of Technology, 09107 Chemnitz, Germany.

Abstract

Collinearity is a visual perception phenomenon in the human brain that amplifies spatially aligned edges arranged along a straight line. However, it is vague for which purpose humans might have this principle in the real-world, and its utilization in computer vision and engineering applications even is a largely unexplored field. In this work, our goal is to transfer the collinearity principle to computer vision, and we explore the potential usages of this novel principle for computer vision applications. We developed a prototype model to exemplify the principle, then tested it systematically, and benchmarked it in the context of four use cases. Our cases are selected to spawn a broad range of potential applications and scenarios: sketching the combination of collinearity with deep learning (case I and II), using collinearity with saliency models (case II), and as a feature detector (case I). In the first use case, we found that collinearity is able to improve the fault detection of wafers and obtain a performance increase by a factor 1.24 via collinearity (decrease of the error rate from 6.5% to 5.26%). In the second use case, we test the defect recognition in nanotechnology materials and achieve a performance increase by 3.2x via collinearity (deep learning, error from 21.65% to 6.64%), and also explore saliency models. As third experiment, we cover occlusions; while as fourth experiment, we test ImageNet and observe that it might not be very beneficial for ImageNet. Therefore, we can assemble a list of scenarios for which collinearity is beneficial (wafers, nanotechnology, occlusions), and for what is not beneficial (ImageNet). Hence, we infer collinearity might be suitable for industry applications as it helps if the image structures of interest are man-made because they often consist of lines. Our work provides another tool for CV, hope to capture the power of human processing.

Figures & tables

Fig. 1 : a) Illustration of the collinearity principle. b) Transfer from psychology to the computer vision domain. b, left) Psychology, reprinted from [ 1 ] . b, right) Computer vision domain: Fault detection in semiconductor wafers [ 2 ] . Defect recognition in nanotechnology materials [ 3 ] , and ImageNet subset taken from [ 4 ] (in clock-wise direction). The green circles denote the structures of interests in [ 2 , 3 ] .

Panel of Figure 2 — Fig. 2 : Neural model of collinearity.

Fig. 3 : Neural connections which mediate the concept of collinearity in the model. The connections link neurons along a virtual line (collinearity), and their influence enhances the central neuron. They are located inside the collinearity layer. Note, only one of the multiple connection orientations is shown. In the right inset, it is shown the assumed location of the model in the human brain (primary visual cortex, V1) and the cortical layers (reprinted from [ 23 ] ).

Fig. 4 : Precise spatial extent as well as width of the collinearity connectivity. The receiving neuron is marked in gray, and a red color denotes that a neuron at this position has a connection to the receiving neuron, i.e. the color denotes the weight strength. Both dimensions are measured in $\lambda$ , with 1 $\lambda$ = size of wavelength of the Gabor, e.g. $1\,\lambda\,\hat{=}\,5\,$ pixels. The connections in the direction of the spatial extent have a minimum distance, i.e. there is a region without a collinearity influence around the neuron, and the connections stretch until a maximum distance. Note, only the connectivity matrix for a single orientation is shown, each orientation has its own connectivity pattern, which is rotated appropriately for each orientation. See main text for details (Sec. II-C and matrix $w_{x,x^{\prime},l}$ in Eq. 10 ).

Layer	Type	Output shape	Kernel size	Stride

Collinearity layer	Collinearity	$513\times 76\times 9$	/	/
Pooling layer	Pooling	$513\times 76\times 9$	$3\times 3$	$3$
Gabor Layer	Convolution	$1540\times 229\times 9$	$11\times 11$	$1$
Image	Input	$1540\times 229\times 1$	$/$	$/$

TABLE I: The collinearity model’s sizes for the first real-world scenario (semiconductor wafer - fault detector). Note, the different scenarios have different image resolutions, thus the model sizes differ.

Fig. 5 : Basic example of operation of the collinearity principle. (a) If line elements are arranged along a longer line pattern, they are enhanced. The outer line elements are denoted as flankers. (b) If the outer flankers are not aligned, i.e. by $90^{\circ}$ rotated, they are not enhanced. On the left are shown the image setups, whereas on the right are shown the neural responses, which are read out for the central position of the collinearity layer (marked by the arrow, response $r^{\mbox{\tiny Col}}$ , cf. Sec. II-B ).

Panel of Figure 7 — Fig. 6: Collinearity model in the use cases of the basic example, in the systematic evaluations, and in the psychophysics. The layers inside the collinearity model are not illustrated and its internal processing is symbolized by the dotted arrow (cf. model figure 2 ).

Fig. 7 : Evaluation of the contrast’s influence on the collinearity approach via a low-visible line element. a) Neural response of the system depending on the contrast of a central line element between two outer lines (flankers). The central line element is simulated with varying illumination in front of a black background, and the setup is shown as an inset. b) Neuronal response along a slice on the line (shown as a dotted line in the inset), illustrated at different contrasts of the middle line segment. The dotted lines in (b) illustrate the neural responses without collinearity. The middle part denotes the target element, whereas the left and right denote the flankers.

Fig. 8 : Influence of the contrast on the collinearity approach via a full line. a) Neural response of the system depending on the input’s contrast. The input stimulus is an entire line with the given illumination in front of a black background as shown by the setup inset. b) Neuronal response along a slice on the line, shown at different contrasts of the input line. The notation of the figure is similar to Fig. 7 .

Fig. 9 : Influence of the line’s length on the collinearity characteristics. a) Collinearity response dependent on the line-segment’s length in pixel. The stimulus is a line of the given length at a contrast of C= $0.5$ in front of a black background, as shown by the setup inset. b) Collinearity response along a slice on the line for three exemplary lines with the denoted lengths. The input contrast is always C= $0.5$ . The collinearity connectivity has a minimum and maximum connection length, which influences the response. The connectivity is modeled based on the psychology (Sec. II-C ).

Fig. 10 : Replication of psychophysical data. Dependency of the collinearity effect on the distance between the target stimulus and its neighbors (denoted as flankers). a) Psychophysical data of the collinearity facilitation, from [ 1 , Fig. 3a] . Note, the illustration is plotted with an inverted y-axis as there was conducted a contrast-threshold experiment in the original study, in which a lower contrast threshold denotes a better detection of the stimulus. b) Influence of the collinearity facilitation in the model. The response is compared to the Gabor pooling response, which denotes an unmodulated baseline.

Fig. 11 : Influence when the flankers do not match precisely in the orientation (rotation tuning curve). The setup is a central stimulus with fixed orientation and accompanied by two flanker stimuli with systematically-changed orientation. The gray dashed line shows a 50% value of the maximum.

Panel of Figure 13 — Fig. 12 : Semiconductor wafer domain. Overview example of a wafer (left) with chips (middle) and faulty and correct cuts (right). The image classification task is to distinguish faulty cuts from correct cuts (the fault is marked in red). The examples shown were generated synthetically from the original images to protect the intellectual property. Reprinted and adapted from [ 2 ] .

Fig. 13 : Fault detection in the semiconductor wafer use case via a feature detector. a) Example image. The correct cut lines of the wafer are collinear, whereas the faulty cuts (marked in red) are non-collinear. The goal is to distinguish correct from faulty cuts. b) Response of the Gabor layer (in its pooled version). c) Response of the collinearity layer. For each response map, the values are illustrated as the maximum over all orientations. d) Developed collinearity-based fault detection solution. A fault response map is calculated from the difference between the collinearity layer response (c) and the Gabor layer response (b). e) With appropriate thresholding, a fault detector map is created from the fault response map (d). Note, the image is highlighted for better visibility via contrast normalization, and the collinearity model receives the unpreprocessed image (a). All plots in (b)–(e) are stretched in y-direction for higher visibility.

Panel of Figure 15 — Fig. 14 : Collinearity model in the use case of fault detection in semiconductor wafers - feature detector. The neural layers inside the collinearity model are not illustrated and its internal processing is symbolized by the dotted arrow.

Panel of Figure 16 — Fig. 15 : Collinearity model in the use case of deep learning in semiconductor wafers (configuration 1). The fault response map is based on the modeling work shown in Fig. 13 d, and the notation of the figure is similar to Fig. 14 .

Model	Acc. { $C_{0},C_{1}$ }	Error rate
CNN without collinearity (IECON 2020 [ 2 , 46 ] )	$93.50\,\%\newline \pm 1.46~~$	$6.50\,\%\newline \pm 1.46~~$
CNN with collinearity (Ours) Configuration 1	$\textbf{94.74\,\%}\newline \pm\textbf{1.09}~~$	$\textbf{5.26\,\%}\newline \pm\textbf{1.09}~~$
CNN with collinearity (Ours) Configuration 2	$93.04\,\%$	$6.96\,\%$

TABLE II : Results of the fault detection in semiconductor wafers via deep learning. Two configurations were tested (Configurations 1 and 2), while the baseline refers to IECON - Beuth et al. (2021, 2020) [ 2 , 46 ] . All results were measured and averaged over 5 runs.

Fig. 16 : Saliency use case. a, b) Example image from the test set. The clumped structures denote the defects to be detected, for illustration-purposes shaded in green (b). c) Response of the Gabor layer. d) Response of the collinearity layer. For each response map, the values are illustrated as the maximum over all orientations. e) Saliency map. f) The first four extracted region-of-interests from the example image.

Panel of Figure 19 — Fig. 17: Collinearity model in the use case as a saliency model. The notation of the figure is similar to Fig. 14 . ROIs: Regions-of-interest.

	With collinearity	Without collinearity
ROIs Class “Anomalous”	$~~\,313$	$~~\,587$
ROIs Class “Normal”	$~~\,804$	$3\,814$
Total number of detected regions-of-interest (ROIs)	$1\,117$	$4\,401$

TABLE III : Results for saliency models - scanning electron microscopes (SEM).

Panel of Figure 21 — Fig. 18: Collinearity model in the use case of deep learning in the SEM images domain. As feature detector, we utilize the saliency map from before (cf. Fig. 16 e). The notation of the figure is similar to Fig. 14 .

Panel of Table 22 — TABLE IV : Data set for deep learning in the SEM domain. Examples show the two classes of “anomalous” (defects) and “normal”.

Custom CNN	$\mathbf{93.36\pm 3.9}$	$\mathbf{78.35\pm 14.2}$
	With collinearity	Without collinearity
DenseNet121 [ 58 ]	$95.10\pm 2.5$	$89.20\pm 3.9$
ResNet50 [ 59 ]	$50.00\pm 0.0$	$50.50\pm 6.3$
Xception [ 60 ]	$93.10\pm 5.7$	$82.50\pm 7.5$

TABLE V : SEM deep learning results. Accuracy in

\%

, along with the standard deviation. All the results are measured over five runs.

Panel of Figure 24 — Fig. 19 : Deep learning stability in the SEM use case. a, c) With collinearity. b, d) Without collinearity. It is analyzed over multiple runs and shows the mean of the multiple runs’ loss metrics (solid line) as well as the standard deviation (shaded area).

Fig. 20 : Interruptions and occlusions. a) Two exemplary image patches from SEM with interrupted and occluded lines. b) Gabor response. The response is shown as an overlay over all orientations, obtained by a maximum of the responses over all orientations. c) Collinearity response. d) Response when utilizing a broad Gabor filter to detect the long lines, here chosen as $5\times$ the size of the Gabor in (b). (d) is an ablation study. Both (c) and (d) are options for detecting longer lines.

Fig. 21 : ImageNet. Taken from Krizhevsky et al. (2017) [ 4 ] . Left are shown some test images and right the collinearity responses.

Apr 13, 2026

Ultra-low-light computer vision using trained photon correlations

Illumination using correlated photon sources has been established as an approach to allowing high-fidelity images to be reconstructed from noisy camera frames by taking advantage of the knowledge that signal photons are spatially correlated whereas detector clicks due to noise are uncorrelated. However, in computer-vision tasks, the goal is often not ultimately to reconstruct an image, but to make inferences about a scene -- such as what object is present. Here we show how correlated-photon illumination can be used to gain an advantage in a hybrid optical-electronic computer-vision pipeline for object recognition. We demonstrate correlation-aware training (CAT): end-to-end optimization of a trainable correlated-photon illumination source and a Transformer backend in a way that the Transformer can learn to benefit from the correlations, using a small number (<= 100) of shots. We show a classification accuracy enhancement of up to 15 percentage points over conventional, uncorrelated-illumination-based computer vision in ultra-low-light and noisy imaging conditions, as well as an improvement over using untrained correlated-photon illumination. Our work illustrates how specializing to a computer-vision task -- object recognition -- and training the pattern of photon correlations in conjunction with a digital backend allows us to push the limits of accuracy in highly photon-budget-constrained scenarios beyond existing methods focused on image reconstruction.

Mandar M. Sohoni, Jérémie Laydevant, Mathieu Ouellet +7

School of Applied and Engineering Physics, Cornell University, Ithaca, NY 14853, USA · NTT Physics and Informatics Laboratories, NTT Research Inc., Sunnyvale, CA 94085, USA · Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA +2

Computational Imaging

Ask PDF

Aug 27, 2025

Adaptive Dual-Constrained Line Aggregation for Cross-Paradigm Line Segment Detection

Line segment detection has been studied for decades, yet existing methods are typically designed for different detection paradigms. Generic line segment detectors aim to recover all meaningful line segments in an image, whereas recent deep-learning-based approaches mainly target wireframe line segments that describe salient geometric structures. Because these paradigms follow different detection objectives, methods optimized for one often perform poorly on the other. In this work, we propose Adaptive Dual-Constrained Line Aggregation (ADLA), a line extraction framework designed to operate across different line segment detection paradigms. Starting from an edge strength map, ADLA progressively aggregates pixels into candidate line segments under two complementary geometric constraints: orientation coherence and bounded orthogonal distance to an adaptively estimated line model. During aggregation, the line centroid and orientation are dynamically updated using the accumulated supporting pixels, progressively improving the geometric consistency of the estimated line. Edge strength information is further incorporated into orientation estimation, seed selection, model refinement, and segment validation, reducing the need for extensive parameter tuning. Experiments on three publicly available datasets covering generic, wireframe, and Manhattan line segment detection demonstrate consistently strong performance across substantially different annotation settings. ADLA achieves (F^H) scores of 0.8665 on YorkUrban-LineSegment dataset, 0.8720 on ShanghaiTech dataset, and 0.7297 on YorkUrban dataset. These results demonstrate the effectiveness and flexibility of ADLA across different line segment detection paradigms. The source code for this work is publicly available at https://github.com/ChenguangTelecom/adla .

Chenguang Liu, Chisheng Wang, Huilin Chen +2

Ministry of Natural Resources (MNR) Key Laboratory for Geo-Environmental Monitoring of Great Bay Area & Guangdong Key Laboratory of Urban Informatics & Shenzhen Key Laboratory of Spatial Smart Sensing and Services, School of Architecture & Urban Planning, Shenzhen University, 3688 Nanhai Avenue, Shenzhen, 518060, Guangdong Province, China

Topic pending

Ask PDF

Aug 10, 2026

Human versus Computer Vision

Computer vision saliency models predict where people will look, one map per image, and a predicted-attention industry sells those maps in place of measuring real viewers. I evaluate the leading models against every publicly available eye-tracking corpus that records viewer demographics, comprising 27,384 participants across eight datasets, including 11.4 million webcam gaze points from 2,993 US adults recruited to national quotas while viewing circulating news images. I show that a single map predicts where every group looks about equally well, but systematically ignores how long each place holds a group's attention. That time follows a clear sociodemographic and political fingerprint, so the map carries a systematic bias against the youngest and oldest viewers and against Black, lower-income, and Republican-leaning ones. I use each group's own gaze to test whether it carries a learnable pattern, across every demographic axis the sample supports. Showing which audiences these systems can learn and what determines this sets the standard for judging claims that computer vision captures diverse human vision.

Elena Sirotkina

Center for Data Science, New York University

Visual Attention Algorithmic Fairness Saliency Map Evaluation Visual Saliency Gaze Forecasting

Ask PDF

Abstract

Explore similar work

Ultra-low-light computer vision using trained photon correlations

Adaptive Dual-Constrained Line Aggregation for Cross-Paradigm Line Segment Detection

Human versus Computer Vision