An Imbalanced Dataset with Multiple Feature Representations for Studying Quality Control of Next-Generation Sequencing
Authors: Philipp Röchner, Clarissa Krämer, Johannes U Mayer, Franz Rothlauf, Steffen Albrecht, Maximilian Sprang
Organizations: Johannes Gutenberg University Mainz · University of Southern Denmark · Department of Dermatology, University Medical Center of the Johannes Gutenberg University · Institute of Quantitative and Computational Biology, Johannes Gutenberg University Mainz · University of Auckland
Next-generation sequencing (NGS) is a key technique for studying the DNA and RNA of organisms. However, identifying quality problems in NGS data across different experimental settings remains challenging. To develop automated quality-control tools, researchers require datasets with features that capture the characteristics of quality problems. Existing NGS repositories, however, offer only a limited number of quality-related features. To address this gap, we propose a dataset derived from 37,491 NGS samples with two types of quality-related feature representations. The first type consists of 34 features derived from quality control tools (QC-34 features). The second type has a variable number of features ranging from eight to 1,183. These features were derived from read counts in problematic genomic regions identified by the ENCODE blocklist (BL features). All features describe the same human and mouse samples from five genomic assays, allowing direct comparison of feature representations. The proposed dataset includes a binary quality label, derived from automated quality control and domain experts. Among all samples, 3.2% are of low quality. Supervised machine learning algorithms accurately predicted quality labels from the features, confirming the relevance of the provided feature representations. The proposed feature representations enable researchers to study how different feature types (QC-34 vs. BL features) and granularities (varying number of BL features) affect the detection of quality problems.
Figures & tables
Figure 1: Feature generation (a) Researchers upload their experimental data as FASTQ files to the ENCODE database. (b) The experimental data is automatically reviewed based on quality metrics. If these metrics indicate insufficient quality, ENCODE quality experts manually review the samples and label the reported data as released or revoked . (c) We downloaded 37,491 FASTQ files and their associated metadata from ENCODE, then mapped the reads to the reference genomes using Bowtie 2. (d) The 34 QC-34 features are generated using quality control and bioinformatics tools. (e) For the BL features, we first integrate the quality-related and species-specific blocklisted regions to create a combined human-mouse blocklist. The integration can be done with varying degrees of restriction, based on the alignment ratio r , yielding n blocklisted regions. The feature values for the BL- n features are the number of reads in each of the n blocklisted regions.
Assay Type
Released
Revoked
Human
Mouse
Overall
#
%
#
%
#
%
#
%
#
%
ChIP-Seq
25,491
70.24
962
80.17
23,479
76.18
2,974
44.57
26,453
70.56
RNA-Seq
4,401
12.13
70
5.83
2,268
7.36
2,203
33.02
4,471
11.93
DNase-Seq
5,596
15.42
151
12.58
4,252
13.80
1,495
22.41
5,747
15.33
eCLIP
803
2.21
17
1.42
820
2.66
0
0.00
820
2.19
Overall
36,291
96.80
1,200
3.20
30,819
82.20
6,672
17.80
37,491
100.00
Table 1: Absolute (#) and relative (%) distribution of assay types in the samples. The table shows the overall distribution and the distribution for released and revoked samples, as well as for human and mouse samples.
Figure 2: Distribution of the QC-34 feature subsets (a) ordinal RAW features (b) numeric MAP, TSS, and LOC features on a logarithmic scale
Figure 3: Number of BL features depending on the alignment ratio As the alignment ratio decreases, more genomic regions with lower sequence similarity between species are included in the cross-species blocklist, resulting in more BL features.
Feature
Missing Rate
Explanation
Accession
0.0%
ID assigned to each experiment or sample; refers to the dataset within the ENCODE database
Project
0.0%
Name or ID of the ENCODE project to which the dataset belongs; helps categorize datasets by research focus or initiative
Dataset
0.0%
ID referring to the corresponding ENCODE Dataset or Experiment the sample belongs to, which contains more detailed information, see Table 3
Date created
0.0%
Date when the dataset was created or submitted to ENCODE; helps to track the timeline of data submissions and updates
Status
0.0%
Status of the dataset, such as released and revoked
Biosample name
0.0%
Name or ID of the biological sample used in the experiment; provides information about the tissue, cell line, or species from which the data were derived
Table 2: Metadata features of the ENCODE samples
Feature
Missing Rate
Explanation
ID
0.0%
Experiment accession, identical with the accessions in the Dataset and Experiment column of the metadata file
Project
0.0%
Project accession; each project can contain multiple experiments and datasets
Status
0.0%
Experiment status, such as released , revoked , or archived ; depends on the files of the experiment
Biosample summary
0.02%
Description of the specimen biology, e.g. Homo sapiens K562
Biosample accession
0.02%
ID or accession for the biosample(s) prepared for a single or multiple batches used in this experiment
Organism
0.02%
Organism investigated in the experiment
Table 3: Metadata features of the ENCODE experiments
Figure 4: Feature validation Performance of supervised classifiers for detecting revoked samples on the test set depending on the number of BL features ( ∘ ) and QC-34 ( □ ) features across different genomic assays: (a) ChIP-Seq samples, (b) RNA-Seq samples, (c) DNase-Seq samples, and (d) eCLIP samples.
We describe extensive numerical experiments assessing and quantifying how classifier performance depends on the quality of the training data, a frequently neglected component of the analysis of classifiers. More specifically, in the scientific context of metagenomic assembly of short DNA reads into "contigs," we examine the effects of degrading the quality of the training data by multiple mechanisms, and for four classifiers -- Bayes classifiers, neural nets, partition models and random forests. We investigate both individual behavior and congruence among the classifiers. We find breakdown-like behavior that holds for all four classifiers, as degradation increases and they move from being mostly correct to only coincidentally correct, because they are wrong in the same way. In the process, a picture of spatial heterogeneity emerges: as the training data move farther from analysis data, classifier decisions degenerate, the boundary becomes less dense, and congruence increases.
Alan F. Karr, Regina Ruane
Fraunhofer USA Center Mid-Atlantic, Riverdale MD 20737 · Computational Social Science Laboratory, University of Pennsylvania, Philadelphia, PA 19104
Do genomic foundation models (FMs) provide predictive information beyond simple sequence composition? We evaluate frozen representations from five DNA FMs against normalized k-mer frequencies across 57 genomic classification datasets and two exploratory gene-expression regression tasks. Simple k-mer features remain competitive with most models, whereas NTv3 shows a consistent advantage; the ranking is robust to globally fixed and multi-scale k-mer baselines and alternative downstream probes. To test incremental predictive utility directly, we compare global 6-mer features with their concatenation with each FM embedding under a matched Random Forest protocol. NTv3 improves 46 of 57 datasets (mean ΔMCC = +0.072), while Caduceus provides smaller gains and the other models do not improve mean MCC. We separately decompose embeddings into a component linearly predictable from 6-mer frequencies and a ridge residual, finding that linear recoverability does not reliably indicate downstream utility. Exact 6-mer-preserving shuffles further reveal task-relevant sequence-order sensitivity, most pronounced for NTv3, although order sensitivity alone does not guarantee predictive improvements. Our results establish compositional features as essential controls and offer complementary diagnostics for evaluating the added value of frozen genomic representations, without making claims about fine-tuning capacity.
Vincenzo Yuto Civale, Andrew David Bagdanov, Alberto Magi
Large Language Models (LLMs) are increasingly adopted as conversational assistants in genomics, where they are mainly used to reason over biological knowledge, annotations, and analysis outputs through natural language interfaces. However, existing benchmarks either focus on specialized DNA models trained for sequence prediction or evaluate biological knowledge using text-only questions, leaving the behavior of general-purpose LLMs when directly exposed to raw genome sequences underexplored. We introduce GenomeQA, a benchmark designed to provide a controlled evaluation setting for general-purpose LLMs on sequence-based genome inference tasks. GenomeQA comprises 5,200 samples drawn from multiple biological databases, with sequence lengths ranging from 6 to 1,000 base pairs (bp), spanning six task families: Enhancer and Promoter Identification, Splice Site Identification, Taxonomic Classification, Histone Mark Prediction, Transcription Factor Binding Site Prediction, and TF Motif Prediction. Across six frontier LLMs, we find that models consistently outperform random baselines and can exploit local sequence signals such as GC content and short motifs, while performance degrades on tasks that require more indirect or multi-step inference over sequence patterns. GenomeQA establishes a diagnostic benchmark for studying and improving the use of general-purpose LLMs on raw genomic sequences.
Weicai Long, Yusen Hou, Junning Feng +4
Hong Kong University of Science and Technology (Guangzhou) · The University of Hong Kong · Peking University