cs.AISep 24, 2026
SaveThe Gold in Bias: Maturing the AI Design Process through Verification
Organizations: Pegaso University, Italy · University of Milan, Italy
Abstract
Bias in AI systems is typically framed as a flaw to be minimized, yet it also serves as a critical indicator of underlying weaknesses in data, modeling assumptions, and system design. Existing approaches often treat bias as an isolated problem rather than as evidence that can strengthen verification and governance across the AI lifecycle. This paper aims to reconceptualize bias as a diagnostic tool that supports rigorous AI verification. We seek to develop a multidimensional framework to analyze bias, demonstrate how biases emerge in both Traditional and Generative AI, and provide a structured pathway for verification-driven mitigation. We present a multidimensional framework analyzing bias across four dimensions: origin sources, emergence points throughout the AI modeling lifecycle, technical and methodological causes, and validation approaches for detection and mitigation. Through a comprehensive typology spanning traditional and generative AI systems, we demonstrate how biases manifest and propagate across development stages. Our analysis encompasses 30 distinct bias types, 16 verification methods, and 20 countermeasures, providing an actionable roadmap for practitioners. We introduce a hierarchical evidence framework that distinguishes internal validity (mechanistic integrity of AI systems) from external validity (contextual reliability in deployment environments). The framework reveals how biases manifest and propagate across modeling stages, enabling systematic mapping between bias types, verification techniques, and effective countermeasures. The proposed evidence hierarchy clarifies how different verification strategies contribute to mechanistic integrity and contextual reliability. We advocate for ''Ethics by Design'' principles that integrate bias verification throughout the development lifecycle, enabling the construction of fairer, more robust, and trustworthy AI systems.
Figures & tables
Figure 1: Categorization of different types of bias, Social Bias, Data Representation Bias, Measurement and Decision Bias, and Usage Bias, by their emergent sources: Societal Prejudices, Human Subjectivity, and Technical Limitation. For each category, a few examples of bias types are recalled. Framed blue examples demonstrate those types that, with high probability, happen in the GenAI system, while orange frames happen in both systems, and Green frames only for TAI.
| AI Modeling Stage | Description |
|---|---|
| B1. Data Collection | Gathering raw data from sensors, databases, user interactions, web scraping, or third-party providers. In Traditional AI (TAI), data often include tabular formats, images, or domain-specific corpora. In Generative AI (GenAI), large-scale multimodal datasets (text, images, audio, etc.) are collected, often from Internet-scale sources. |
| B1. Feature Engineering / Embedding | Transforms raw data into structured features or dense representations. In TAI, this includes manual feature selection (normalization, one-hot encoding, domain-specific extraction). In GenAI, it relies on automated embedding methods (tokenization, word/sentence or multimodal encoders). |
| B1. Pre-Training | A defining step in GenAI, where foundational models (e.g., LLMs like GPT, vision models like CLIP) are trained on massive unlabeled or weakly labeled datasets to learn general-purpose representations. In TAI, this stage is often absent, as models are trained directly on task-specific data. |
| B1. Fine-Tuning / Prompt Design / RLHF | Adapts or aligns generic models to specific tasks. Includes: Fine-Tuning — further training on smaller, task-specific datasets; Prompt Design — crafting inputs to steer outputs without modifying model parameters; RLHF — using human feedback to train a reward model that guides reinforcement learning for more aligned outputs. |
| B1. Model Training / Optimization | In TAI, objectives focus on accuracy, precision, or recall, optimizing parameters to minimize loss. In GenAI, objectives include likelihood maximization, adversarial (GAN) losses, or transformer-specific training criteria. |
| B1. Deployment | The trained model is integrated into real-world environments where it performs or automates decisions. |
Table 1: Stages of the AI lifecycle.
Figure 2: This Venn diagram illustrates the three perspectives in model evaluation: M (what the model has learnt), D (what is present in the training and test data) and S (what is truly required by the system in the real world). Classical evaluation focuses on the intersection of M and D. Gaps between D and S or M and S, however, highlight areas where the model may fail to generalise or overfit to irrelevant patterns.
Figure 3: Taken from ( Hüllermeier and Waegeman, 2021 ) . Illustration of systematic misclassification within the region . The left panel depicts an aleatoric condition, where class boundaries overlap and instances near the frontier are inherently ambiguous, leading to unavoidable misclassifications. The right panel shows an epistemic condition, where instances occur in regions of the decision space that are underrepresented in the training data, resulting in uncertain or systematically incorrect predictions.
Figure 4: Taken from ( Hüllermeier and Waegeman, 2021 ) . The figure shows how varying the number or scale of features affects class separability within the model’s decision space. When key features are missing or inadequately scaled, the resulting projection of the system into the data constrains the model, leading to distorted or overlapping decision boundaries.
Figure 5: Taken from ( Cordeiro and Carneiro, 2020 ) . Illustration of labeling errors and their impact on the alignment between system, data, and model. Incorrect or noisy labels can distort the learning process in two distinct ways. Examples belonging to the system are not correctly identified in the data, resulting in partial misalignment . Examples are correctly included within but are associated with the wrong class, leading the model to learn systematically incorrect decision boundaries. Both forms of error illustrate how mislabeling propagates bias even when data quantity and model capacity appear sufficient.
| Level | Description | Methods and Metrics |
|---|---|---|
| I1 – Metric-Based Verification | Evidence derived from standardized quantitative metrics applied to specific datasets, focusing on the measurement of performance related to isolated tasks. | Accuracy, F1-score, OmniAccuracy, and fairness indices such as demographic parity and equalized odds. |
| I2 – Process-Aligned Verification | Evidence resulting from the integration of verification activities across the design lifecycle. It ensures consistency among data, models and objectives, as well as compliance with quality standards. | Trace alignment, data–model conformance checking, explainability validation, compliance with ISO/IEC 25059, data consistency, and model calibration metrics. |
| I3 – Formal and Structural Verification | Evidence obtained through formal modeling, static analysis, or interpretable representations that enable reasoning about internal logic, dependencies, and component behavior. | White-box testing, symbolic reasoning, formal proofs, structural coverage analysis, and invariant satisfaction. |
| I4 – Integrated Verification | Comprehensive and multi-dimensional evidence combining performance, robustness, bias detection, and epistemic uncertainty quantification. This ensures alignment across all design and testing stages. | Causal validation, uncertainty estimation, bias–robustness trade-offs. |
Table 2: Hierarchy of Evidence Levels for Internal Validity in AI Verification
| Level | Description | Methods and Evidence |
|---|---|---|
| E1 – Simulated or Synthetic Testing | Evidence obtained from controlled or synthetic environments. The potential for generalisation is limited due to the artificial conditions and lack of contextual variability. | Simulation-based evaluation, synthetic data generation, stress tests, and sandboxed experimentation. |
| E2 – Benchmark-Based Validation | Evidence derived from standardized benchmarks or curated datasets, offers comparability, but has limited real-world generalisation. | Use of public benchmarks such as MMLU, BLEU, ROUGE, GLUE, SuperGLUE, BIG-bench, or HumanEval, and leaderboard-based validation. |
| E3 – Contextual and Task-Based Validation | Evidence is gathered from evaluations in realistic or domain-specific contexts that reflect actual usage conditions. This level includes assessing dataset completeness to ensure that the training data adequately represent the diversity of operational contexts and populations. | Task-based testing, domain-specific pilots, user-centered trials, representational diversity analysis, and coverage-based completeness metrics. |
| E4 – Transparent or Hybrid Testing (Gray/White Box) | Evidence obtained through evaluations that integrate internal transparency with external testing allows interpretable links to be established between internal mechanisms and real-world outcomes. | Gray- and white-box testing, explainability-based audits, robustness tracing, and interpretability-driven evaluations. |
| E5 – Continuous and Longitudinal Monitoring | Evidence was accumulated through real-world operation, with a focus on the system’s dynamic performance and reliability over time. | Post-deployment monitoring, incident reporting, continuous auditing, and reliability metrics such as Mean Time Between Failures (MTBF) and performance drift tracking. |
Table 3: Hierarchy of Evidence Levels for External Validity in AI Verification
| Bias Category | Verification (Bias Detection / Diagnosis) | Countermeasures (Bias Mitigation / Correction) |
|---|---|---|
| 1. Social Bias 5.1 | • D1.5. Algorithmic Transparency • D1.2. Group-wise Performance Metrics • D1.3. Group-wise Error Rates • D1.1. Fairness Metrics • D1.6. Correlation Analysis • D2.2. Longitudinal Studies • D2.4. Benchmarks • D2.3. Data Audits | • CM1. Incorporating Fully Representative Data • CM2. Synthetic Data Generation • CM3. Bias-aware and Adversarial Training • CM4. Counter-prompting/Bias-Aware RLHF and Input Curation • CM5. Explainable Interface Design • CM6. Implementing Human in the Loop Mechanism |
| 2. Data Representation Bias 5.2 | • D1.4. Cluster analysis/ Analogy tests • D2.4. Benchmarks • D1.2. Group-wise Performance Metrics • D2.6. Continuous Monitoring | • CM2. Synthetic Data Generation • CM7. Balanced Dataset Curation and Weighted Sampling • CM8. Debiasing Static Embedding • CM9. Debiasing Contextual Embedding • CM4. Counter-prompting/Bias-Aware RLHF and Input Curation |
| 3. Measurement & Decision Bias 5.3 | • D1.3. Group-wise Error Rates • D1.2. Group-wise Performance Metrics • D2.3. Data Audits • D1.6. Correlation Analysis • D1.7. Causal Inference | • CM3. Bias-aware and Adversarial Training • CM11. Thresholding • CM12. Improving the Generalizability • CM14. Diversifying Feedback Sources and Reward Signals • CM15. Multi-objective Optimization |
| 4. Usage Bias 5.4 | • D2.5. User Experience Audits • D2.6. Continuous Monitoring • D2.7. Temporal Fairness Checks | • CM5. Explainable Interface Design • CM18. Inclusive Prompt Engineering, Output Filtering, and User Education • CM19. Decoupling Feedback Input from Training Data • CM20. Incremental/ Continual Learning, RAG for Temporal Updates |
Table 4: Bias Categories with Corresponding Verification and Countermeasures.