Acquisition Geometry-Assisted Whole-Group Localization of X-ray Fluorescence Maps in Optical Microscopy Images

Xiangyu Yin; Tatjana Paunesku; Letonia Copeland-Hardin; Martina Ralle; Zichao Wendy Di; Si Chen; Gayle E. Woloschak; Barry Lai

cs.CVAug 18, 2026

Acquisition Geometry-Assisted Whole-Group Localization of X-ray Fluorescence Maps in Optical Microscopy Images

Authors: Xiangyu Yin, Tatjana Paunesku, Letonia Copeland-Hardin, Martina Ralle, Zichao Wendy Di, Si Chen, Gayle E. Woloschak, Barry Lai,

Organizations: Advanced Photon Source, Argonne National Laboratory, Lemont, Illinois, USA · Northwestern University, Chicago, Illinois, USA · University of Chicago, Chicago, Illinois, USA · Oregon Health and Science University, Portland, Oregon, USA

Abstract

X-ray fluorescence (XRF) microscopy maps elemental distributions, while optical microscopy can provide complementary morphological context. Localizing XRF fields of view (FOVs) in optical images is difficult because the two modalities differ in contrast mechanism and resolution. Most current workflows place each XRF tile independently, even when acquisition metadata already record the tiles' relative scan positions. This study formalizes XRF tile-group localization, in which one optical-frame placement is estimated for the whole group, constrained by acquisition geometry and quantified using group intersection-over-union (GroupIoU). In a controlled case study, independent localization failed with GroupIoU 0.000, whereas group localization achieved 0.931. Replacing the normalized cross-correlation (NCC) metric with mutual information (MI) gave nearly identical results, showing that the outcome is not specific to one local similarity metric. In another multiscale case study, using a coarse XRF survey scan to connect the fine-scale tile group to the optical image increased mean GroupIoU from 0.694 to 0.856. These case studies support using acquisition geometry as an explicit constraint when localizing related XRF tiles.

Figures & tables

Figure 1: Task overview. The XRF acquisition produces a group of high-resolution tiles whose relative positions are recorded by the instrument. The goal is to estimate one placement that maps the tile group into the optical coordinate frame so that all tile footprints are jointly located.

Strategy	Color	Candidate source	Geometry in score	Bridge use
Single-tile reference	navy	Independent per-tile appearance	No	None
Mosaic	beige	Fused group template	During template construction	None
Anchor-verified group	blue	Anchor candidates	Yes	None
Structural bridge	green	Bridge-to-optical plus group-in-bridge	Yes	Bridge reference frame
Bridge-prior direct	coral	Bridge-seeded optical-frame search	Yes	Bridge spatial prior
Bridge rule	purple	Structural if runtime-valid, otherwise bridge-prior direct	Yes	Runtime route selection

Table 1: Method variants used in this study. The color column defines the method palette used in the result figures.

Figure 2: Designed-ambiguity example from the isolation case study. Left: XRF tiles. Middle: independent per-tile localization places both tiles in visually similar but wrong regions (GroupIoU 0.000). Right: anchor-verified group localization correctly places the full group (GroupIoU 0.931).

Figure 3: Isolation-case-study perturbation sweeps on Tile 1 and Tile 2 under normalized cross-correlation. (a) GroupIoU as a function of Gaussian metadata-position jitter applied to the companion tile’s recorded scan center, mean of 20 trials per level. Independent per-tile localization is included as an invariant reference because it does not consume acquisition metadata; anchor-verified and mosaic localization show the soft-constraint and hard-constraint responses to metadata error. A same-colored “x” indicates that at least one trial at that level returned no-match. (b) Decoy intensity (left) and decoy count (right) sweeps under two decoy realizations, cut-and-pasted optical patch at the anchor footprint (solid) and synthetic insertion of the anchor’s normalized XRF template (dashed). Cut-and-pasted patches do not act as cross-modal decoys, while multiple synthetic insertions collapse the fused-mosaic strategy.

Figure 4: Multiscale case-study summary. (a) Per-case GroupIoU for the four placements, the italic header above each case names the route selected by the runtime-validity rule, and an “x” indicates a no-match return. (b) Coverage-aware mean GroupIoU across the four cases per route, the filled bar is the failure-aware mean (no-match counted as zero), and a hollow overlay is drawn only for the structural bridge route (coverage 2/4) to show its conditional mean over returned matches. Routes with full 4/4 coverage have failure-aware and conditional means equal by construction and are drawn as a single bar. Numeric value and coverage fraction are annotated above each bar. The full per-case route-level diagnostics are in Table 6 .

Figure 5: Selected placements for the four cases in the multiscale case study. Each panel shows the optical image with ground-truth footprints (solid outlines) and Bridge-rule predicted footprints (light-purple dashed outlines). Panel titles name the route selected by the Bridge rule: Cases 1 and 3 use structural-bridge localization, while Cases 2 and 4 use bridge-prior direct localization.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$I$	Optical reference image.
$J$	Intermediate XRF bridge image, when available.
$\mathcal{T}=\{T_{1},\dots,T_{n}\}$	XRF tile group acquired from one specimen, with each $T_{i}$ a 2D template.
$x_{i}$	Physical scan center of tile $T_{i}$ recorded by the instrument.
$a$	Index of the anchor tile (the tile whose appearance proposes the placement).
$\theta$	Candidate group placement: anchor center, scale, orientation, optional flip.

Appendix

Table 2: Notation used in the group-localization formulation.

Step	Operation
1	Prepare the optical reference image and the XRF tile templates. Read each tile center and pixel size from acquisition metadata.
2	Select an anchor tile or an eligible set of anchor tiles. Generate anchor candidates by multiscale template localization and non-maximum suppression.
3	For each anchor candidate, use Eq. 1 to predict the footprint of every companion tile from the recorded group geometry.
4	Search a local window around each predicted companion footprint. Keep the best local match and record its image score and center residual.
5	Form the matched set $M(\theta)$ from tiles that pass local score, visibility, and support checks. Reject candidates that fail minimum tile-count or match-weight requirements.
6	Score each valid candidate with Eq. 7 . Select the highest-scoring valid group placement, or report no match if no candidate passes the runtime checks.

Appendix

Table 3: Geometry-aware grouped localization. The procedure first proposes anchor placements, then asks whether the companion tiles can be verified near the locations recorded by acquisition geometry.

Step	Operation
1	Match the bridge image to the optical image to estimate the bridge-to-optical placement.
2	Run the structural bridge route by localizing the target XRF group inside the bridge image and composing the target-to-bridge and bridge-to-optical placements.
3	Independently run the bridge-prior direct route by using the bridge placement to define an optical-frame search neighborhood, then localizing the target group directly in the optical frame.
4	If the structural route returns a runtime-valid group result, select it. Otherwise select the bridge-prior direct result.
5	Use ground-truth overlap only after selection, for evaluation and reporting.

Appendix

Table 4: Bridge-informed route selection. The structural and bridge-prior routes are run as separate localization paths before the runtime-validity rule selects one returned placement.

Setting or check	Reported value	Role
XRF normalization	2–98% percentile range	Robust grayscale normalization before localization.
Isotropic XRF resampling	enabled, geomean step	Converts anisotropic scan pixels to a common isotropic grid.
Stage-1 bridge-to-optical feature and metric	gradient magnitude, NCC	Localizes the bridge image in the optical frame.
Stage-1 angle search	center 0 ∘ , range 60 ∘ , step 6 ∘	Searches bridge orientation around the nominal direction.
Stage-2 target-to-bridge feature and metric	raw XRF, NCC	Matches the target group inside the bridge image for the structural route.
Stage-2 scale search	one scale, 5% span	Uses metadata-centered scale for target-to-bridge localization.

Appendix

Table 5: Settings and runtime validity checks used for the multiscale bridge-informed results.

Case	Route	Status	Selected	GroupIoU	TileIoU ${}_{\text{min}}$	TileIoU ${}_{\text{mean}}$	TileIoU ${}_{\text{wmean}}$
Case 1	Anchor-verified	ok	no	0.502	0.000	0.167	0.498
Case 1	Structural bridge	ok	yes	0.934	0.038	0.409	0.927
Case 1	Bridge-prior direct	ok	no	0.883	0.000	0.294	0.876
Case 2	Anchor-verified	ok	no	0.788	0.048	0.444	0.711
Case 2	Structural bridge	no match	no	–	–	–	–
Case 2	Bridge-prior direct	ok	yes	0.869	0.746	0.810	0.864

Appendix

Table 6: Route-level diagnostics for the four multiscale cases. No-match entries did not return a final group prediction.

Figure 6: Stress test on the real 3-tile combination. The scenarios include baseline, synthetic decoy, metadata jitter, and two missing-tile settings in which one tile’s optical region is zeroed out and the group matcher is run under permissive (lenient) or strict matched-fraction thresholds. A same-colored “x” marks an error or no-match return.

Figure 7: Case 3 failure diagnostic. (a) The selected structural route first localizes the bridge image in the optical frame. The bridge footprint overlaps the correct tissue region but is rotated relative to the reference outline. (b) In the bridge frame, the target group placement is substantially better aligned. (c) Composing the target-to-bridge result with the bridge-to-optical placement transfers the rotation into the optical frame. The resulting union overlap remains moderate, but one tile has zero overlap with its ground-truth footprint.

Microscopy Image Analysis

Microscopy image analysis uses computational methods to detect, segment, track, reconstruct, and quantify structures in scientific images.

See all 198 papers

XRF-to-Optical Field-of-View Localization with Vision Language Models
Aug 18, 2026Xiangyu Yin, Tatjana Paunesku, Letonia Copeland-Hardin +7Cross-Modal Image Registration Microscopy Image Analysis
Advanced Photon Source, Argonne National Laboratory, Lemont, Illinois, USA · Northwestern University, Chicago, Illinois, USA · University of Chicago, Chicago, Illinois, USA +1
Localize Any Object in X-Ray Security Scans without Human Annotation
Oct 5, 2026Yaqi Cai, Mingxuan Liu, Lorenzo Vaquero +5Self-Supervised Learning Synthetic Data Generation
University of Trento · Fondazione Bruno Kessler · Dalian University of Technology +1
XRFormer: Multiscale Tokenization for XRF Representation Learning
Jul 7, 2026Sofiane Daimellah, Sylvie Le Hégarat-Mascle, Clotilde BoustRepresentation Learning Hybrid CNN-Transformer Architectures
Université Paris-Saclay, Gif-sur-Yvette, France · Centre de Recherche et de Restauration des Musées de France, Paris, France

Aug 18, 2026J/K move · Enter open · S save

XRF-to-Optical Field-of-View Localization with Vision Language Models

Xiangyu Yin, Tatjana Paunesku, Letonia Copeland-Hardin, Martina Ralle+6

Advanced Photon Source, Argonne National Laboratory, Lemont, Illinois, USA · Northwestern University, Chicago, Illinois, USA · University of Chicago, Chicago, Illinois, USA+1

Cross-Modal Image Registration Microscopy Image Analysis Medical Image Localization

Open paper AskPDF arXiv

Registering images acquired with different microscopy modalities is essential for relating complementary measurements of the same specimen. In correlative X-ray fluorescence (XRF) and optical microscopy, the XRF map often covers only a small region of an optical image acquired from the same or an adjacent tissue section. Field-of-view (FOV) localization is necessary but can be difficult when appearance and structure differ across modalities. Here we evaluate training-free vision language model (VLM) localization on two datasets representing same-section high-correspondence and adjacent-section low-correspondence imaging. We test unconstrained and metadata-constrained search and compare VLMs with geometric controls, classical template matching, and two alternative training-free approaches (DINOv2 and multiGradICON). Direct VLM prompting produced content-dependent spatial signals but was not reliable alone. Classical matching was most accurate when cross-modal structure was preserved but failed in the low-correspondence collection. A proposal-and-verify workflow used repeated VLM predictions as candidates and image-based similarity to select the final location. This workflow recovered useful localization in the low-correspondence regime.

Abstract

Microscopy Image Analysis

Explore similar work

XRF-to-Optical Field-of-View Localization with Vision Language Models

Localize Any Object in X-Ray Security Scans without Human Annotation

XRFormer: Multiscale Tokenization for XRF Representation Learning