Data Preprocessing

Data preprocessing, the crucial initial step in machine learning pipelines, aims to transform raw data into a format suitable for model training and analysis. Current research emphasizes efficient preprocessing techniques, particularly for large datasets, exploring hardware acceleration and novel algorithms to overcome computational bottlenecks and improve model explainability, especially in sensitive domains like medicine. These advancements are vital for enhancing the accuracy, efficiency, and fairness of machine learning models across diverse applications, from medical diagnosis to cybersecurity and climate modeling.

Papers

September 23, 2024

Efficient Tabular Data Preprocessing of ML Pipelines
Yu Zhu, Wenqi Jiang, Gustavo Alonso
Machine Learning Hardware Accelerator Machine Learning Pipeline Data Preprocessing Heavy Undocumented Preprocessing

September 15, 2024

Enhancing Lesion Segmentation in PET/CT Imaging with Deep Learning and Advanced Data Preprocessing Techniques
Jiayi Liu, Qiaoyi Xue, Youdan Feng, Tianming Xu, Kaixin Shen, Chuyun Shen, Yuhang Shi
Deep Learning Computed Tomography Whole Body Data Preprocessing Preprocessing Method PET Dog Augmentation Pipeline

August 30, 2024

Common Steps in Machine Learning Might Hinder The Explainability Aims in Medicine
Ahmed M Salih
Machine Learning High Explainability Cross Over Step Data Preprocessing Outlier Removal Data Normalisation

August 15, 2024

Impact of Comprehensive Data Preprocessing on Predictive Modelling of COVID-19 Mortality
Sangita Das, Subhrajyoti Maji
Global Impact Covid 19 Prediction Model Data Preprocessing Accurate Predictive Model Preprocessing Pipeline

July 27, 2024

Data Processing Techniques for Modern Multimodal Models
Yinheng Li, Han Ding, Hang Chen
Multimodal Large Language Model Multimodal Model Data Preprocessing

July 15, 2024

Impacts of Data Preprocessing and Hyperparameter Optimization on the Performance of Machine Learning Models Applied to Intrusion Detection Systems
Mateus Guimarães Lima, Antony Carvalho, João Gabriel Álvares, Clayton Escouper das Chagas, Ronaldo Ribeiro Goldschmidt
Machine Learning Machine Learning Model System Performance Hyperparameter Optimization Intrusion Detection System Classification Model Threat Detection Data Preprocessing

July 13, 2024

A Survey of Distance-Based Vessel Trajectory Clustering: Data Pre-processing, Methodologies, Applications, and Experimental Evaluation
Maohan Liang, Ryan Wen Liu, Ruobin Gao, Zhe Xiao, Xiaocai Zhang, Hua Wang
Financial Application NCD Method Data Preprocessing Vessel Trajectory Trajectory Similarity Trajectory Clustering

February 22, 2024

Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data
Arshmeet Kaur, Morteza Sarmadi
Machine Learning Model Feature Selection Data Preprocessing Class Imbalanced Data Improved Classification Skewed Non IID

January 18, 2024

DataViz3D: An Novel Method Leveraging Online Holographic Modeling for Extensive Dataset Preprocessing and Visualization
Jinli Duan
3D Object Interactive Visualization New Method 3D Data 3D Space 3D Modeling Data Preprocessing

December 11, 2023

A method for recovery of multidimensional time series based on the detection of behavioral patterns and the use of autoencoders
Alexey Yurtin
Data Detection Time Series Recurrent Neural Network Supervised Autoencoder Prompt Recovery Deep Learning Technology Data Preprocessing Behavioral Pattern

December 4, 2023

Jellyfish: A Large Language Model for Data Preprocessing
Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada
Large Language Model Instruction Dataset Association Rule Data Preprocessing Data Distillation General Algorithm

October 10, 2023

An improved CTGAN for data processing method of imbalanced disk failure
Jingbo Jia, Peng Wu, Hussain Dawood
Adversarial Learning Data Imbalance Data Preprocessing Tabular Generative Adversarial Network Disk Failure

September 30, 2023

MFL Data Preprocessing and CNN-based Oil Pipeline Defects Detection
Iurii Katser, Vyacheslav Kozitsin, Igor Mozolin
Anomaly Detection Computer Vision Data Preprocessing Oil Pipeline Defect Detection

September 25, 2023

AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data
Jianwei Yu, Hangting Chen, Yanyao Bian, Xiang Li, Yi Luo, Jinchuan Tian, Mengyang Liu, Jiayi Jiang, Shuai Wang
Automatic Speech Recognition Text to Speech Speech Data Data Preprocessing

August 30, 2023

Large Language Models as Data Preprocessors
Haochen Zhang, Yuyang Dong, Chuan Xiao, Masafumi Oyamada
Entity Matching State of the Art Large Data Preprocessing LLM Based Framework

August 20, 2023

DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data
Peng Li, Zhiyi Chen, Xu Chu, Kexin Rong
LeArning Abstract Machine Learning Model Tabular Data Data Preprocessing Differentiable ML Pipeline

August 8, 2023

Machine Learning, Deep Learning and Data Preprocessing Techniques for Detection, Prediction, and Monitoring of Stress and Stress-related Mental Disorders: A Scoping Review
Moein Razavi, Samira Ziyadidegan, Reza Jahromi, Saber Kazeminasab, Vahid Janfaza, Ahmadreza Mahmoudzadeh, Elaheh Baharlouei, Farzan Sasangohar
Deep Learning Machine Learning Continuous Chronic Stress Data Preprocessing Anxiety Detection Stress Detection Model Stress Management

July 24, 2023

Big Data - Supply Chain Management Framework for Forecasting: Data Preprocessing and Machine Learning Techniques
Md Abrar Jahin, Md Sakib Hossain Shovon, Jungpil Shin, Istiyaque Ahmed Ridoy, M. F. Mridha
Machine Learning State of the Art Forecasting Big Data Inventory Management Data Preprocessing Supply Chain Resilience Forecasting Analysis Forecasting Strategy

June 19, 2023

Evaluating Loss Functions and Learning Data Pre-Processing for Climate Downscaling Deep Learning Models
Xingying Huang
Deep Learning Model Loss Function Earth System Climate Data Data Preprocessing Climate Downscaling

April 28, 2023

Pre-processing training data improves accuracy and generalisability of convolutional neural network based landscape semantic segmentation
Andrew Clark, Stuart Phinn, Peter Scarth
Neural Network Semantic Segmentation Deep Convolutional Neural Network Landscape Image Earth Observation Data Preprocessing Land Use Land Cover

Data Preprocessing

Papers

Efficient Tabular Data Preprocessing of ML Pipelines

Enhancing Lesion Segmentation in PET/CT Imaging with Deep Learning and Advanced Data Preprocessing Techniques

Common Steps in Machine Learning Might Hinder The Explainability Aims in Medicine

Impact of Comprehensive Data Preprocessing on Predictive Modelling of COVID-19 Mortality

Data Processing Techniques for Modern Multimodal Models

Impacts of Data Preprocessing and Hyperparameter Optimization on the Performance of Machine Learning Models Applied to Intrusion Detection Systems

A Survey of Distance-Based Vessel Trajectory Clustering: Data Pre-processing, Methodologies, Applications, and Experimental Evaluation

Comparative Analysis of Data Preprocessing Methods, Feature Selection Techniques and Machine Learning Models for Improved Classification and Regression Performance on Imbalanced Genetic Data

DataViz3D: An Novel Method Leveraging Online Holographic Modeling for Extensive Dataset Preprocessing and Visualization

A method for recovery of multidimensional time series based on the detection of behavioral patterns and the use of autoencoders

Jellyfish: A Large Language Model for Data Preprocessing

An improved CTGAN for data processing method of imbalanced disk failure

MFL Data Preprocessing and CNN-based Oil Pipeline Defects Detection

AutoPrep: An Automatic Preprocessing Framework for In-the-Wild Speech Data

Large Language Models as Data Preprocessors

DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data

Machine Learning, Deep Learning and Data Preprocessing Techniques for Detection, Prediction, and Monitoring of Stress and Stress-related Mental Disorders: A Scoping Review

Big Data - Supply Chain Management Framework for Forecasting: Data Preprocessing and Machine Learning Techniques

Evaluating Loss Functions and Learning Data Pre-Processing for Climate Downscaling Deep Learning Models

Pre-processing training data improves accuracy and generalisability of convolutional neural network based landscape semantic segmentation