In this work, we present a novel machine learning framework called Self-Directed Task Identification (SDTI), which enables models to autonomously identify the correct target variable for each dataset in a zero-shot setting without pre-training. SDTI is a minimal, interpretable framework demonstrating the feasibility of repurposing core machine learning concepts for a novel task structure. To our knowledge, no existing architectures have demonstrated this ability. Traditional approaches lack this capability, leaving data annotation as a time-consuming process that relies heavily on human effort. Using only standard neural network components, we show that SDTI can be achieved through appropriate problem formulation and architectural design. We evaluate the proposed framework on a range of benchmark tasks and demonstrate its effectiveness in reliably identifying the ground truth out of a set of potential target variables. SDTI outperformed baseline architectures by 14% in F1 score on synthetic task identification benchmarks. These proof-of-concept experiments highlight the future potential of SDTI to reduce dependence on manual annotation and to enhance the scalability of autonomous learning systems in real-world applications.
Figures & tables
Figure 1: Visual representation of Alg 1 . Within the outer loop the model randomly creates hyperparameters to be passed into the SDTI layer for the vecotrized ANN’s running simultaneously. The SDTI layer will run for S iterations and return the last cost value produced. These values are then reshaped and passed into an ArgMax function. This output is then saved in a specific data structure T that is passed through another ArgMax function after E iterations. The final output is a vector of integers indicating predicted target variable for a given dataset.
Group
Mean Cost ( μ )
Variance ( σ2 )
Correct Pairs
0.6483
0.2597
Incorrect Pairs
0.9913
0.6093
Table 1: Mean and variance of cost values for correct vs. incorrect dataset–label pairings for each individual run in abalation study with over 30 SDTI epochs.
Figure 2: Stratified representation of mean values and variances for every SDTI epoch in all ablation study results that included 30 SDTI epochs.
Figure 3: Ablation study over SDTI hyperparameters. Each point represents one configuration in the (A,B,C) parameter space, where A is the number of training records, B is the number of epochs, and C is the number of SDTI iterations. Color intensity encodes the resulting F1 score. Regions with more than 500 records and 10+ epochs achieve near-perfect alignment (F1 > 0.98), confirming the stability of SDTI under sufficient data and training time.
Field
Records ( ≥ 500 vs < 500)
Epochs ( ≥ 10 vs < 10)
SDTI-Epochs (30 vs 5)
Parameters (Improved vs Worse)
≥ 500 .
< 500
≥ 10
< 10
30
5
Imp.
Worse.
Count
2880
720
3000
600
600
600
400
20
Mean F1
0.9705
0.9352
0.9830
0.8656
0.9724
0.9315
0.9900
0.7193
F1 STD
0.0733
0.0963
0.0377
0.1398
0.0597
0.1095
0.0262
0.1712
Min F1
0.2843
0.3529
0.6863
0.2843
0.5686
0.2843
0.9216
0.3529
25th Percentile F1
1.0000
0.9216
1.0000
0.7745
0.9216
0.7745
1.0000
0.6358
Table 2: Comprehensive comparison across all parameters in abalation study. Columns correspond to the original comparisons: records, epochs, SDTI-epochs, and parameter settings. For each pair, the first column represents the higher/better condition (improved parameters: records ≥ 500, epochs ≥ 10, 30 SDTI-epochs, worse parameters: records < 500, epochs < 10, 5 SDTI-epochs) and the second column represents the lower/worse condition. In each sub-table, only the indicated parameter is varied, while all other parameters are held constant.
Algorithm
Mean F1
Standard Deviation
SDTI
0.9900
.0262
Pearson’s Correlation
0.8431
0
Mutual Information
0.8431
0
Cosine Similarity
0.8431
0
Table 3: Comparison of SDTI with benchmark algorithms.
Machine learning task type identification is essential for constructing valid ML pipelines, yet in practice it is typically specified manually. We investigate whether large language models (LLMs) can infer both the data domain and the downstream prediction task directly from dataset-level information when only the target feature is provided by the user. Together with our LLM-based system we also release an annotated benchmark comprising 625 public tabular and time series datasets. We evaluate the proposed approach in three settings: (i) tabular datasets in comparison with established AutoML heuristics, (ii) cross-domain evaluation across tabular and time series datasets, and (iii) a practical deployment scenario using smaller local models. The results show consistent advantages for LLM-based task type identification, with increasing difficulty in heterogeneous and resource-constrained settings. LLM-based approaches outperform AutoGluon in the tabular setting, reaching 0.98 F1 macro compared to 0.93. In the cross-domain setting, the best model achieves 0.90 F1 macro, while smaller locally deployable models reach 0.75, indicating a trade-off between deployment feasibility and accuracy.
Foundation model pseudo-labeling - labeling data strictly via zero-shot inference - enables massive scale, but performance is undermined by hallucinations that evade standard thresholds. To eliminate these errors, we introduce the Turing-inspired Label Imitation Game (LIG), a framework that formalizes pseudo-label pruning as an adversarial interrogation. Rather than filtering labels via isolated thresholds, we use the LIG to train a Turing Test Network (TTN), a task-agnostic "judge" that evaluates candidate pseudo-labels within a dataset-wide context. Experiments across four diverse datasets demonstrate the TTN's robustness, consistently enhancing label accuracy for three state-of-the-art vision-language models without costly supervision or retraining. Crucially, we demonstrate that learned semantic-contextual logic is a robust alternative to spatial-geometric verification, enabling a unique zero-shot task transfer capability - a TTN trained strictly on image classification datasets can effectively prune complex object detection pseudo-labels. This pruning yields F1-score gains of 28% for the worst-performing baseline categories and 44% with task-specific fine-tuning. Significantly, we also observe Category Revival, where the TTN pruning "detoxifies" the training signal for downstream models and enables them to recover from zero recall on transfer-vulnerable classes. The pre-trained TTN models and code are available at https://github.com/voxel51/ttn.
The purpose of our paper is to develop a unified multi-agent architecture that automates end-to-end machine learning (ML) pipeline generation from datasets and natural-language (NL) goals, improving efficiency, robustness and explainability. A five-agent system is proposed to handle profiling, intent parsing, microservice recommendation, Directed Acyclic Graph (DAG) construction and execution. It integrates code-grounded Retrieval-Augmented Generation (RAG) for microservice understanding, an explainable hybrid recommender combining multiple criteria, a self-healing mechanism using Large Language Model (LLM)-based error interpretation and adaptive learning from execution history. The approach is evaluated on 150 ML tasks across diverse scenarios. The system achieves an 84.7% end-to-end pipeline success rate, outperforming baseline methods. It demonstrates improved robustness through self-healing and reduces workflow development time compared to manual construction. The study introduces a novel integration of code-grounded RAG, explainable recommendation, self-healing execution and adaptive learning within a single architecture, showing that tightly coupled intelligent components can outperform isolated solutions.