Data Quality Issue

Data quality issues pose a significant challenge across diverse scientific domains, hindering the reliability and accuracy of data-driven applications. Current research focuses on automated detection and correction of problems like missing values, duplicates, inconsistencies, and outliers, often employing hybrid approaches combining statistical methods with machine learning algorithms to enhance both accuracy and explainability. This work is crucial for improving the trustworthiness of AI models and ensuring the validity of scientific findings, particularly in high-stakes applications like healthcare and environmental monitoring where data quality directly impacts decision-making. The development of tools and techniques for identifying and mitigating these issues is a key area of ongoing investigation.

Papers

October 8, 2024

Data Quality Issues in Vulnerability Detection Datasets
Yuejun Guo, Seifeddine Bettaieb
Deep Learning Vulnerability Detection Software Vulnerability Vulnerability Data Code Security Data Quality Issue

September 16, 2024

Towards Explainable Automated Data Quality Enhancement without Domain Knowledge
Djibril Sarr
Inherent Interpretability Domain Knowledge Data Quality Anomalous Event Data Quality Issue

January 25, 2024

Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets
Kumar Abhishek, Aditi Jain, Ghassan Hamarneh
Quality Issue Data Quality Large Datasets Data Quality Issue Dermatological Disease Datasets

December 17, 2023

Ocean Data Quality Assessment through Outlier Detection-enhanced Active Learning
Na Li, Yiyang Qi, Ruyue Xin, Zhiming Zhao
Active Learning Ocean Data Argoverse Motion Forecasting Data Quality Issue Outlier Event

July 10, 2023

Badgers: generating data quality deficits with Python
Julien Siebert, Daniel Seifert, Patricia Kelbert, Michael Kläs, Adam Trendowicz
Data Quality Context Generation Data Quality Issue

May 19, 2023

MedLens: Improve Mortality Prediction Via Medical Signs Selecting and Regression
Xuesong Ye, Jun Wu, Chengjie Mou, Weinan Dai
Novel Regression Data Quality Area Under the ROC Curve Vital Sign Data Quality Issue

March 15, 2023

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks
Mouhamed Naby Ndiaye, El Houcine Bergou, Hajar El Hammouti
Reinforcement Learning Proximal Policy Optimization Policy Reinforcement Learning Cooperative Multi Agent Reinforcement Learning Multi Agent Proximal Policy Optimization Data Quality Issue UAV Assisted

June 28, 2022

On the Impact of Noises in Crowd-Sourced Data for Speech Translation
Siqi Ouyang, Rong Ye, Lei Li
Global Impact Speech Translation High Quality Crowd Sourced Data Real World Noise Speech Translation Corpus Data Quality Issue Independent Phone to Audio Alignment

March 19, 2022

Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems
Harald Foidl, Michael Felderer, Rudolf Ramler
Data Detection AI System Data Quality High Quality Data Experiential Consequence Suspicious Commenter Data Corruption Code Smell Data Quality Issue

March 15, 2022

Data Smells in Public Datasets
Arumoy Shome, Luis Cruz, Arie van Deursen
Data Centric Data Science Available Datasets Code Smell Data Quality Issue

Data Quality Issue

Papers

Data Quality Issues in Vulnerability Detection Datasets

Towards Explainable Automated Data Quality Enhancement without Domain Knowledge

Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets

Ocean Data Quality Assessment through Outlier Detection-enhanced Active Learning

Badgers: generating data quality deficits with Python

MedLens: Improve Mortality Prediction Via Medical Signs Selecting and Regression

Muti-Agent Proximal Policy Optimization For Data Freshness in UAV-assisted Networks

On the Impact of Noises in Crowd-Sourced Data for Speech Translation

Data Smells: Categories, Causes and Consequences, and Detection of Suspicious Data in AI-based Systems

Data Smells in Public Datasets