Nyxus: A Next Generation Image Feature Extraction Library for the Big Data and AI Era

Nicholas Schaub; Andriy Kharchenko; Hamdah Abbasi; Sameeul Samee; Hythem Sidky; Nathan Hotaling

cs.CVMar 12, 2026

Nyxus: A Next Generation Image Feature Extraction Library for the Big Data and AI Era

Authors: Nicholas Schaub, Andriy Kharchenko, Hamdah Abbasi, Sameeul Samee, Hythem Sidky, Nathan Hotaling

Organizations: DSBU, Axle Research, 6116 Executive Blvd Suite 400, Rockville, 20852, MD, USA. · NGRF, NovaGen Research Fund, 6116 Executive Blvd Suite 400, Rockville, 20852, MD, USA. · NCATS, 9800 Medical Center Dr, Rockville, 20850, MD, USA.

Abstract

Modern imaging instruments can produce terabytes to petabytes of data for a single experiment. The biggest barrier to processing big image datasets has been computational, where image analysis algorithms often lack the efficiency needed to process such large datasets or make tradeoffs in robustness and accuracy. Deep learning algorithms have vastly improved the accuracy of the first step in an analysis workflow (region segmentation), but the expansion of domain specific feature extraction libraries across scientific disciplines has made it difficult to compare the performance and accuracy of extracted features. To address these needs, we developed a novel feature extraction library called Nyxus. Nyxus is designed from the ground up for scalable out-of-core feature extraction for 2D and 3D image data and rigorously tested against established standards. The comprehensive feature set of Nyxus covers multiple biomedical domains including radiomics and cellular analysis, and is designed for computational scalability across CPUs and GPUs. Nyxus has been packaged to be accessible to users of various skill sets and needs: as a Python package for code developers, a command line tool, as a Napari plugin for low to no-code users or users that want to visualize results, and as an Open Container Initiative (OCI) compliant container that can be used in cloud or super-computing workflows aimed at processing large data sets. Further, Nyxus enables a new methodological approach to feature extraction allowing for programmatic tuning of many features sets for optimal computational efficiency or coverage for use in novel machine learning and deep learning applications.

Figures & tables

Figure 1: Nyxus feature set compared to other common image feature extraction Methods. Each racetrack plot compares features from 9 different feature extraction libraries; NY - Nyxus, WC – WND-CHRM, RJ- RadiomicsJ, PR- PyRadiomics, NF- NIST Java Features, MI- MITK, MT- MatLab, IM- Imea, CP- CellProfiler. McQuin et al. (2018) Each plot compares features classified as shape, intensity, texture, volumetric, and miscellaneous. Complete information on individual features can be found in Supplemental Tables S1 (Intensity), S2 (Shape and Moments), S3 (Texture), S4 (Miscellaneous), and S5 (Volumetric).

Figure 2: Heatmap comparing feature extraction library capabilities across multiple criteria. Color coding is row-specific: green generally indicates full support or optimal capability (e.g., $>90\%$ for quantitative metrics, most permissive for licensing), blue indicates partial support or intermediate capability (e.g., $10\%-90\%$ for quantitative metrics, some restrictions for qualitative metrics), and red indicates minimal/no support or most restrictive (e.g., $<10\%$ for quantitative metrics, proprietary licensing). Supplemental Table S7 provides detailed definitions for how each row’s criteria were classified and what the color coding means in that specific context

Panel of Figure 3 — Figure 3: Nyxus hardware-specific performance. (a) Tissuenet (Nyxus untargeted), (b) Tissuenet (Nyxus targeted), (c) Decathlon (Nyxus untargeted), (d) Decathlon (Nyxus targeted) - relative Nyxus performance in untargeted and targeted mode versus competition software by feature group on various datasets (Amazon EC2 p5 node). (e) Scalability on different hardware at varying number of CPU threads and various datasets (Notebook hub, EC2, M1). (f) Scalability as a function of region size - CUDA GPU-enabled scalability on synthetic slides having various ROI area and population (Amazon EC2 p5 node using 8 CPU threads). (g) CPU (left) versus GPU (right) execution cost (bigger = slower, sizes are relative to each other) - compute cost structure of featurizing an image of 100x 100K-pixel ROIs on a GPU-disabled and GPU-enabled p5 node. The legend shows color coded broad feature categories, with fold improvement of GPU over CPU in parenthesis.

Panel of Figure 4 — Figure 4: Heatmap of fully IBSI-compliant feature extraction results by Nyxus correlated with competition software calculated on a segmented slide hosting 32 ROIs. Color coding: dark blue indicates high correlation, red/orange indicate anti-correlation, and tan/teal represents little/no correlation.

Panel of Figure 5 — Figure 5: Nyxus use cases: high-performance pipeline, Python library, and Napari backend. (a) Nyxus serving multiple workflows. (b) Nyxus architecture related to main use cases. (c) Data analysis session with Python. (d) Napari with Nyxus as data analysis backend.

Panel of Figure 6 — Figure 6: Representative datasets analyzed. (a) Representative nucleus and cell border images and labeled masks from the TissueNet 1.0 dataset. (b) Representative CT image and labeled masks and MRI image and labeled mask from the Medical Image Decathalon dataset. (c) Representative synthetic data images of labeled masks, and 9, 100, and 1000 binary ROIs overlayed on intensity images.

Panel of Figure 7 — Figure 7: Hard and soft features, and their tuning with Nyxus. (a) Typical feature set implementing targeted measurements via soft features. The figure shows all the feature classes that are hard (plain text) and soft (light blue box). (b) Tuning a 2D soft feature via varying the grey level co-occurrence matrix size. The figure illustrates tuning the GLCM feature for grey depth and offset on ROIs in 523-images from the Tissuenet dataset. Callouts A-C represent a GLCM cooccurrence matrix of varying size to represent computational complexity as more grey depth is added. (c) Contours of classification accuracy of the tissue type (yellow – high, purple low) from the Tissuenet test dataset using a 50-state random forest model as a function of feature extraction hyperparameters - grey depth and offset.

Panel of Figure 8 — Figure S1 : Heatmap of feature extraction results by Nyxus default profile correlated with competition software calculated on a segmented slide hosting 32 ROIs. Color coding: dark blue indicates high correlation, red/orange indicate anti-correlation, and tan/teal represents little/no correlation.

Feature	CP	IM	MT	MI	NF	PR	RJ	WC	NY
1 C of var	–	–	–	1	–	–	1	–	–
2 Covered img Inten Range	–	–	–	1	–	–	–	–	–
3 Edge integrated intensity	1	–	–	–	–	–	–	1	1
4 Edge intensity max	1	–	–	–	–	–	–	–	1
5 Edge intensity min	1	–	–	–	–	–	–	–	1
6 Edge intensity mean	1	–	–	–	–	–	–	–	1

Table S1 : 2-dimensional intensity features

Feature

Shape

Statistical lengths

41 Area

–

89 Feret ⌀mode

–

42 Area projection

–

90 Feret ⌀stddev

–

43 Aspect Ratio

–

91 Feret max angle

–

44 BB Height

–

92 Feret max coordinates

–

45 BB Width

–

93 Feret max ⌀

–

Table S2 : 2-dimensional shape and geometric moment features

Feature

GLCM

197 LRE

–

140 ASM

–

198 LRHGLE

–

141 ACOR

–

199 LRLGLE

–

142 AVEAMP

–

200 LGLRE

–

143 CLUPROM

–

201 RE

–

144 CLUSHADE

–

202 RLNU

–

Table S3 : 2-dimensional texture features

Feature	CP	IM	MT	MI	NF	PR	RJ	WC	NY
Intensity distribution
265 Gabor	–	–	–	–	–	–	–	1	1
266 FracAtD	1	–	–	–	–	–	–	–	1
267 MeanFrac	1	–	–	–	–	–	–	–	1
268 RadialCV	1	–	–	–	–	–	–	–	1
269 Zernike	1	–	–	–	–	–	–	1	1

Table S4 : Miscellaneous 2-dimensional features

Feature

303 Coef of var

–

353 length_3d_bb

–

304 Covered img inten range

–

354 height_3d_bb

–

305 Energy

–

355 feret_3d_max

–

306 Entropy

–

356 feret_3d_min

–

307 Excess kurtosis

–

357 x_max_3d

–

308 Hyperflatness

–

358 y_max_3d

–

Table S5 : Volume methods

	CP 1	IM 2	MT 3	MI 4	NF 5	PR 6	RJ 7	WC 8	NY ${}_{\textrm{U}}^{9}$	NY ${}_{\textrm{T}}^{10}$
tissuenet
intensity	211	–	110	446726	116	152	418842	82	65	4
shape	1516	5892	612	–	54	–	15613	106	86	35
texture	5334	–	740	450056	96	875	62034	58	153	41
misc.	1267	15409	–	–	–	–	–	63	160	129
decathlon

Table S6 : Benchmarks on Tissuenet and Decathlon datasets by feature group. All values are in seconds. NY

{}_{\textrm{U}}

refers to the Nyxus untargeted profile (all features enabled), and NY

{}_{\textrm{T}}

refers to the Nyxus targeted profile (optimized feature subset). These raw timing values correspond to the fold-change comparisons shown in Figure 3 .

Category	Remark	Green	Blue	Red
Out of Core Support	Can the Method calculate all (or the majority) of its features if an image or ROI is larger than available RAM?	$>90\%$	$10\%\leqslant x\leqslant 90\%$	$<10\%$
# Of Feature Classes	Count of unique measurements extracted by each Method. Supplemental Tables S1 - S5 display all feature classes, each class is given a unique number in these tables. Of note is that often within a given feature depending on feature hyperparameters, many different measurements can be made. For example, Zernike polynomials measure the intensity distribution from each object’s center to its boundary within a set of bins, across its radial and azimuthal degrees and therefore can have unlimited Zernike polynomial measures for a single feature class. However, for this manuscript Zernike would represent only one feature class	$>90\%$	$10\%\leqslant x\leqslant 90\%$	$<10\%$
Next Gen. Output Format	Method supports both in memory representation and saving to disk of measured feature classes in indexed columnar formats such as Apache Arrow, Feather, and/or Parquet	Supports $>1$ such format	Supports 1 such format	Does not natively support output of feature values in an indexed columnar format
GPU Support	Method supports calculation of features using a GPU	Supports calculating ¿ 1 feature classes on GPU	Supports calculating 1 feature class on GPU	Does not support calculation of any feature classes on GPU
Soft (Tunable) Features	Method supports users setting hyperparameters for features that have hyperparameters	$>90\%$	$10\%\leqslant x\leqslant 90\%$	$<10\%$
IBSI Compliant	Method can be automatically configured to output feature measurement values that fall within accepted ranges on IBSI benchmark datasets for feature classes within the IBSI standard	$>90\%$	$10\%\leqslant x\leqslant 90\%$	$<10\%$

Table S7 : Criteria behind the functionality heatmap (from Figure 2 )

Medical Image Analysis

Medical image analysis uses computational methods to interpret clinical images, measure anatomy and disease, and support diagnosis and treatment.

See all 292 papers

Aug 27, 2026

Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction

Whole-slide images (WSIs) are central to computational pathology but are prohibitively large, making patch-based processing the practical unit for foundation model inference. At scale, however, generating and handling massive numbers of patches on quickly introduces significant I/O and orchestration overhead, often dominating end-to-end performance. We present a decoupled, I/O-aware pipeline for large-scale WSI embedding extraction that decomposes the workflow into three stages: (1) patch generation and staging, (2) embarrassingly parallel embedding inference, and (3) sharded vector database ingestion. This design isolates data movement from compute, enabling efficient patch delivery, scalable multi-node inference with minimal communication. The resulting system produces a distributed vector database where embeddings are persistently coupled with rich metadata (e.g., patient, slide, and patch attributes), enabling efficient filtering, retrieval, and downstream reuse. This representation database is compact and reusable for tasks such as retrieval, classification, and few-shot learning, particularly benefiting low-resource environments. We show that decoupling I/O, computation, and ingestion enables high-throughput WSI embedding extraction at scale. By characterizing the scaling envelope, we demonstrate that storage dominates beyond moderate concurrency, reframing WSI embedding extraction as a data-centric systems problem rather than a purely compute-bound workload.

Mayanka Chandrashekar, Xi Zhang, Ethan Seefried +3

Oak Ridge National Laboratory

Whole-Slide Image Analysis Distributed Inference Computational Pathology

Ask PDF

May 10, 2026

Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration

Transforming neuroimaging data into clinically actionable biomarkers is a knowledge-intensive and labor-intensive process. Standardized workflows such as fMRIPrep have improved robustness and efficiency, but they are statically configured and cannot reason about downstream objectives, deliberate over alternative strategies, or close the loop between intermediate evidence and subsequent decisions in the way a human researcher would. This lack of closed-loop adaptation often leaves domain experts trapped in a cycle of manual trial-and-error to tune parameters and remediate pipeline failures, severely constraining the scalability of clinical biomarker development. To bridge this gap, we introduce NEXUS, an autonomous multi-agent framework that integrates neuroimaging workflow execution with scientific-objective understanding. Unlike conventional flat toolcalling agents, NEXUS adopts a code-centric execution paradigm where specialist agents collaboratively synthesize and optimize executable programs over composable domain-specific primitives. This design enables robust, long-horizon workflow construction that adapts dynamically to runtime observations. Furthermore, we propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection to drive evidence-grounded workflow remediation. Experiments on ADHD-200 and ADNI demonstrate that NEXUS outperforms standard workflow-based baselines in predictive performance while exhibiting sophisticated agentic behaviors, including strategy exploration and adaptive refinement. The code is available at https://github.com/LearningKeqi/Virtual-Neuroscientist-NEXUS.

Keqi Han, Songlin Zhao, Yao Su +4

Emory University · Lehigh University · Worcester Polytechnic Institute +2

Multi-Agent Collaboration Neuroimaging Scientific Workflow Automation Autonomous Research Agents

Ask PDF

Sep 2, 2026

Exemplar: Few-Shot Biomedical Segmentation That Refits Exactly in Seconds

Segmenting a new microscopy dataset usually means training a deep network on its annotations, and training again for every added image. Exemplar solves a segmentation readout in closed form and updates it exactly with each new mask in seconds, as accurately as networks trained for minutes. Its linear readout combines frozen DINOv3 features, read at a magnification chosen from the masks, with classical filter responses at native resolution. Its regularisation and its pixel weighting are chosen exactly by leaving out one annotated image at a time, so none is set aside for validation. On ten biomedical datasets from cells to vessels and filaments, Exemplar is more accurate than every few-shot method that only applies a pretrained model, and on average than nnU-Net trained on the same masks. A fine-tuned Cellpose-SAM, the strongest baseline, is as accurate on average, but each of its fits takes a median of 25 minutes against 8.5 seconds, and it cannot be updated exactly.

Michal Průšek, Adam Novozámský, Filip Šroubek

The Czech Academy of Sciences, Institute of Information Theory and Automation, Czechia · Czech Technical University in Prague, Faculty of Nuclear Sciences and Physical Engineering, Czechia

Image Segmentation Few-Shot Semantic Segmentation Microscopy Image Analysis

Ask PDF

Abstract

Medical Image Analysis

Explore similar work

Decoupled I/O-Dominant Pipelines for Large-Scale Whole-Slide Image Embedding Extraction

Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration

Exemplar: Few-Shot Biomedical Segmentation That Refits Exactly in Seconds