Towards Minimal Focal Stack in Shape from Focus

Khurram Ashfaq; Muhammad Tariq Mahmood

cs.CVApr 2, 2026

Towards Minimal Focal Stack in Shape from Focus

Authors: Khurram Ashfaq, Muhammad Tariq Mahmood

Organizations: Korea University of Technology and Education

Abstract

Shape from Focus (SFF) is a depth reconstruction technique that estimates scene structure from focus variations observed across a focal stack, that is, a sequence of images captured at different focus settings. A key limitation of SFF methods is their reliance on densely sampled, large focal stacks, which limits their practical applicability. In this study, we propose a focal stack augmentation that enables SFF methods to estimate depth using a reduced stack of just two images, without sacrificing precision. We introduce a simple yet effective physics-based focal stack augmentation that enriches the stack with two auxiliary cues: an all-in-focus (AiF) image estimated from two input images, and Energy-of-Difference (EOD) maps, computed as the energy of differences between the AiF and input images. Furthermore, we propose a deep network that computes a deep focus volume from the augmented focal stacks and iteratively refines depth using convolutional Gated Recurrent Units (ConvGRUs) at multiple scales. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed augmentation benefits existing state-of-the-art SFF models, enabling them to achieve comparable accuracy. The results also show that our approach maintains state-of-the-art performance with a minimal stack size.

Figures & tables

Figure 1: First Row: A synthetic AiF image of size $64\times 64$ is generated and defocused images by convolving with a Gaussian with $\sigma\in\left\{0.5,1.0,1.5,2.0\right\}$ . Second Row: (left) Illustration of defocused image formation, (right) behavior of EOD and Laplacian focus measure (FM).

Figure 2: The proposed framework has three modules: Focal Stack Augmentation, Deep Focus Volume, and Depth Extraction.

Figure 3: Left : The architecture of the DFV module. For brevity, the feature encoder from which the feature volumes ${\{\mathbf{A}_{1},\mathbf{A}_{2},\mathbf{A}_{3},\mathbf{A}_{4}\}}$ are extracted is not shown. Right : The recurrent update block, where GRUs operating at coarse, medium, and fine scales exchange multi-scale information among themselves.

Dataset	Size	Input	Indices	Patch
FT	15	2 ; 3	[5,11] ; [4,8,12]	512x512
FOD	5	2 ; 3	[2,4] ; [1,3,5]	256x256
DDFF	10	2 ; 3	[4,7] ; [3,6,8]	224x224

Table 1: Training settings for each dataset, including focal stack size, number of input images used, slice indices, and patch size.

Figure 4: EOD visualization: (Row:1) GT AiF and corresponding EOD maps, (Row:2) Estimated AiF and corresponding EOD maps, (Row:3) EOD computed across the focal stack for AiFs estimated using 2 and 3 slices compared with the GT AiF.

Ablation	MAE $\downarrow$	RMS $\downarrow$	SqRel $\downarrow$	$\delta$ $\uparrow$	$\delta^{2}$ $\uparrow$
Imgs	6.61	11.82	9.05	83.22	87.39
AiF+Imgs	6.22	11.30	7.81	84.81	87.74
AiF+EODs	6.91	12.06	9.06	83.30	87.36
Imgs+EODs	6.32	11.40	7.95	84.42	87.65
Imgs+AiF+EODs	6.18	11.26	7.68	84.64	87.72

Table 2: EOD ablation with different augmentation settings.

Figure 5: Qualitative Results for EOD Ablation. The numbers on each map denote RMS error.

	FT							FOD
Method	Input	MAE $\downarrow$	RMS $\downarrow$	AbsRel $\downarrow$	$\delta$ $\uparrow$	$\delta^{2}$ $\uparrow$	$\delta^{3}$ $\uparrow$	Input	MAE $\downarrow$	RMS $\downarrow$	AbsRel $\downarrow$	$\delta$ $\uparrow$	$\delta^{2}$ $\uparrow$	$\delta^{3}$ $\uparrow$
	Using Full Focal Stack
AiFDNet	15	6.812	13.14	0.731	85.43	87.67	88.87	5	0.071	0.156	0.118	84.75	94.67	97.46
DFV-FV	15	6.332	12.09	0.899	85.09	87.60	89.52	5	0.078	0.160	0.142	80.93	94.19	97.57
DFV-Diff	15	5.509	10.65	0.615	86.18	88.09	89.93	5	0.077	0.165	0.133	81.77	93.98	97.43
DWild	15	5.542	10.44	0.611	86.35	88.21	89.84	5	0.073	0.154	0.138	83.12	94.45	97.26

Table 3: Quantitative comparison on FT and FOD datasets. The upper part shows baseline results using full focal stacks, while the lower part shows performance with the proposed EOD augmentation.

Method	Input	MAE $\downarrow$	RMS $\downarrow$	$\delta$ $\uparrow$	$\delta^{2}$ $\uparrow$
AiFDNet	3	0.0028	0.0084	98.04	98.63
DFV-FV	3	0.0035	0.0072	97.75	99.01
DFV-Diff	3	0.0040	0.0076	97.68	99.03
Ours	3	0.0031	0.0065	98.60	99.39
AiFDNet	2	0.0028	0.0082	98.12	98.71
DFV-FV	2	0.0036	0.0074	97.64	98.93

Table 4: Results on DDFF Validation set using the proposed augmentation

Figure 6: Qualitative results on the FT dataset. Comparison between baseline models trained with full focal stacks and models using the proposed EOD augmentation. Numbers on each map denote the corresponding RMS error with respect to the GT.

Figure 7: Qualitative results on the FOD dataset. Comparison between baseline models trained with full focal stacks and models using the proposed EOD augmentation. Numbers on each map denote the corresponding RMS error with respect to the GT. Our method produces sharper boundaries; zoom in for details.

Figure 8: Qualitative comparison on the Mobile dataset. Ours is only from 3 input images.

Figure 9: Pair-selection evaluation for the two-image setting. Heatmaps show different metrics for slice pairs; rows denote the first slice index and columns denote the second.

Figure 10: Activation maps across different focus regions. Despite, trained with only two defocused inputs, the model still responds strongly to all depth regions, including weakly focused areas. From left to right: background, far-mid, mid-near, and near focus areas are activated.

AiF	Input	MAE $\downarrow$	RMS $\downarrow$	SqRel $\downarrow$	$\delta\uparrow$	$\delta^{3}\uparrow$
Estimated AiF	3	1.78	4.08	1.18	97.95	99.55
GT AiF	3	1.48	3.30	0.56	98.56	99.68
Estimated AiF	2	2.18	4.65	1.61	97.21	99.37
GT AiF	2	1.68	3.64	0.60	98.39	99.39

Table 5: Comparisons including estimated or GT AiF with images from the FT dataset.

Depth Estimation

Depth estimation infers scene distance from images and sensors, enabling 3D reconstruction, navigation, and spatial understanding.

See all 198 papers

Zero-Shot Depth from Defocus
Mar 27, 2026Yiming Zuo, Hongyu Wen, Venkat Subramanian +5Metric Depth Estimation Depth Estimation
Department of Computer Science, Princeton University
Focusable Monocular Depth Estimation
May 12, 2026Yuxin Du, Tao Lin, Zile Zhong +7Depth Estimation Monocular Depth Estimation
School of Artificial Intelligence, Shanghai Jiao Tong University · 2The Hong Kong University of Science and Technology (Guangzhou) · 3King Abdullah University of Science and Technology
Depth from Defocus via Direct Optimization
Feb 18, 2026Holly Jackson, Caleb Adams, Ignacio Lopez-Francos +1Depth Estimation Computational Imaging
⋆University of California, Berkeley, California, USA · † NASA Ames Research Center, Moffett Field, California, USA · ‡ KBR, Houston, Texas, USA

Mar 27, 2026J/K move · Enter open · S save

Zero-Shot Depth from Defocus

Yiming Zuo, Hongyu Wen, Venkat Subramanian, Patrick Chen+4

Department of Computer Science, Princeton University

Metric Depth Estimation Depth Estimation

Open paper AskPDF arXiv

Depth from Defocus (DfD) is the task of estimating a dense metric depth map from a focus stack. Unlike previous works overfitting to a certain dataset, this paper focuses on the challenging and practical setting of zero-shot generalization. We first propose a new real-world DfD benchmark ZEDD, which contains 8.3x more scenes and significantly higher quality images and ground-truth depth maps compared to previous benchmarks. We also design a novel network architecture named FOSSA. FOSSA is a Transformer-based architecture with novel designs tailored to the DfD task. The key contribution is a stack attention layer with a focus distance embedding, allowing efficient information exchange across the focus stack. Finally, we develop a new training data pipeline allowing us to utilize existing large-scale RGBD datasets to generate synthetic focus stacks. Experiment results on ZEDD and other benchmarks show a significant improvement over the baselines, reducing errors by up to 55.7%. The ZEDD benchmark is released at https://zedd.cs.princeton.edu. The code and checkpoints are released at https://github.com/princeton-vl/FOSSA.

Abstract

Depth Estimation

Explore similar work

Zero-Shot Depth from Defocus

Focusable Monocular Depth Estimation

Depth from Defocus via Direct Optimization