The Gold in Bias: Maturing the AI Design Process through Verification

Samira Maghool; Paolo Ceravolo

cs.AISep 24, 2026

The Gold in Bias: Maturing the AI Design Process through Verification

Authors: Samira Maghool, Paolo Ceravolo

Organizations: Pegaso University, Italy · University of Milan, Italy

Abstract

Bias in AI systems is typically framed as a flaw to be minimized, yet it also serves as a critical indicator of underlying weaknesses in data, modeling assumptions, and system design. Existing approaches often treat bias as an isolated problem rather than as evidence that can strengthen verification and governance across the AI lifecycle. This paper aims to reconceptualize bias as a diagnostic tool that supports rigorous AI verification. We seek to develop a multidimensional framework to analyze bias, demonstrate how biases emerge in both Traditional and Generative AI, and provide a structured pathway for verification-driven mitigation. We present a multidimensional framework analyzing bias across four dimensions: origin sources, emergence points throughout the AI modeling lifecycle, technical and methodological causes, and validation approaches for detection and mitigation. Through a comprehensive typology spanning traditional and generative AI systems, we demonstrate how biases manifest and propagate across development stages. Our analysis encompasses 30 distinct bias types, 16 verification methods, and 20 countermeasures, providing an actionable roadmap for practitioners. We introduce a hierarchical evidence framework that distinguishes internal validity (mechanistic integrity of AI systems) from external validity (contextual reliability in deployment environments). The framework reveals how biases manifest and propagate across modeling stages, enabling systematic mapping between bias types, verification techniques, and effective countermeasures. The proposed evidence hierarchy clarifies how different verification strategies contribute to mechanistic integrity and contextual reliability. We advocate for ''Ethics by Design'' principles that integrate bias verification throughout the development lifecycle, enabling the construction of fairer, more robust, and trustworthy AI systems.

Figures & tables

Figure 1: Categorization of different types of bias, Social Bias, Data Representation Bias, Measurement and Decision Bias, and Usage Bias, by their emergent sources: Societal Prejudices, Human Subjectivity, and Technical Limitation. For each category, a few examples of bias types are recalled. Framed blue examples demonstrate those types that, with high probability, happen in the GenAI system, while orange frames happen in both systems, and Green frames only for TAI.

AI Modeling Stage	Description
B1. Data Collection	Gathering raw data from sensors, databases, user interactions, web scraping, or third-party providers. In Traditional AI (TAI), data often include tabular formats, images, or domain-specific corpora. In Generative AI (GenAI), large-scale multimodal datasets (text, images, audio, etc.) are collected, often from Internet-scale sources.
B1. Feature Engineering / Embedding	Transforms raw data into structured features or dense representations. In TAI, this includes manual feature selection (normalization, one-hot encoding, domain-specific extraction). In GenAI, it relies on automated embedding methods (tokenization, word/sentence or multimodal encoders).
B1. Pre-Training	A defining step in GenAI, where foundational models (e.g., LLMs like GPT, vision models like CLIP) are trained on massive unlabeled or weakly labeled datasets to learn general-purpose representations. In TAI, this stage is often absent, as models are trained directly on task-specific data.
B1. Fine-Tuning / Prompt Design / RLHF	Adapts or aligns generic models to specific tasks. Includes: Fine-Tuning — further training on smaller, task-specific datasets; Prompt Design — crafting inputs to steer outputs without modifying model parameters; RLHF — using human feedback to train a reward model that guides reinforcement learning for more aligned outputs.
B1. Model Training / Optimization	In TAI, objectives focus on accuracy, precision, or recall, optimizing parameters to minimize loss. In GenAI, objectives include likelihood maximization, adversarial (GAN) losses, or transformer-specific training criteria.
B1. Deployment	The trained model is integrated into real-world environments where it performs or automates decisions.

Table 1: Stages of the AI lifecycle.

Panel of Figure 2 — Figure 2: This Venn diagram illustrates the three perspectives in model evaluation: M (what the model has learnt), D (what is present in the training and test data) and S (what is truly required by the system in the real world). Classical evaluation focuses on the intersection of M and D. Gaps between D and S or M and S, however, highlight areas where the model may fail to generalise or overfit to irrelevant patterns.

Figure 3: Taken from ( Hüllermeier and Waegeman, 2021 ) . Illustration of systematic misclassification within the region $M\cap D\cap S$ . The left panel depicts an aleatoric condition, where class boundaries overlap and instances near the frontier are inherently ambiguous, leading to unavoidable misclassifications. The right panel shows an epistemic condition, where instances occur in regions of the decision space that are underrepresented in the training data, resulting in uncertain or systematically incorrect predictions.

Figure 4: Taken from ( Hüllermeier and Waegeman, 2021 ) . The figure shows how varying the number or scale of features affects class separability within the model’s decision space. When key features are missing or inadequately scaled, the resulting projection of the system into the data constrains the model, leading to distorted or overlapping decision boundaries.

Figure 5: Taken from ( Cordeiro and Carneiro, 2020 ) . Illustration of labeling errors and their impact on the alignment between system, data, and model. Incorrect or noisy labels can distort the learning process in two distinct ways. Examples belonging to the system are not correctly identified in the data, resulting in partial misalignment $D\cap S\neg M$ . Examples are correctly included within $M\cap D\cap S$ but are associated with the wrong class, leading the model to learn systematically incorrect decision boundaries. Both forms of error illustrate how mislabeling propagates bias even when data quantity and model capacity appear sufficient.

Level	Description	Methods and Metrics
I1 – Metric-Based Verification	Evidence derived from standardized quantitative metrics applied to specific datasets, focusing on the measurement of performance related to isolated tasks.	Accuracy, F1-score, OmniAccuracy, and fairness indices such as demographic parity and equalized odds.
I2 – Process-Aligned Verification	Evidence resulting from the integration of verification activities across the design lifecycle. It ensures consistency among data, models and objectives, as well as compliance with quality standards.	Trace alignment, data–model conformance checking, explainability validation, compliance with ISO/IEC 25059, data consistency, and model calibration metrics.
I3 – Formal and Structural Verification	Evidence obtained through formal modeling, static analysis, or interpretable representations that enable reasoning about internal logic, dependencies, and component behavior.	White-box testing, symbolic reasoning, formal proofs, structural coverage analysis, and invariant satisfaction.
I4 – Integrated Verification	Comprehensive and multi-dimensional evidence combining performance, robustness, bias detection, and epistemic uncertainty quantification. This ensures alignment across all design and testing stages.	Causal validation, uncertainty estimation, bias–robustness trade-offs.

Table 2: Hierarchy of Evidence Levels for Internal Validity in AI Verification

Level	Description	Methods and Evidence
E1 – Simulated or Synthetic Testing	Evidence obtained from controlled or synthetic environments. The potential for generalisation is limited due to the artificial conditions and lack of contextual variability.	Simulation-based evaluation, synthetic data generation, stress tests, and sandboxed experimentation.
E2 – Benchmark-Based Validation	Evidence derived from standardized benchmarks or curated datasets, offers comparability, but has limited real-world generalisation.	Use of public benchmarks such as MMLU, BLEU, ROUGE, GLUE, SuperGLUE, BIG-bench, or HumanEval, and leaderboard-based validation.
E3 – Contextual and Task-Based Validation	Evidence is gathered from evaluations in realistic or domain-specific contexts that reflect actual usage conditions. This level includes assessing dataset completeness to ensure that the training data adequately represent the diversity of operational contexts and populations.	Task-based testing, domain-specific pilots, user-centered trials, representational diversity analysis, and coverage-based completeness metrics.
E4 – Transparent or Hybrid Testing (Gray/White Box)	Evidence obtained through evaluations that integrate internal transparency with external testing allows interpretable links to be established between internal mechanisms and real-world outcomes.	Gray- and white-box testing, explainability-based audits, robustness tracing, and interpretability-driven evaluations.
E5 – Continuous and Longitudinal Monitoring	Evidence was accumulated through real-world operation, with a focus on the system’s dynamic performance and reliability over time.	Post-deployment monitoring, incident reporting, continuous auditing, and reliability metrics such as Mean Time Between Failures (MTBF) and performance drift tracking.

Table 3: Hierarchy of Evidence Levels for External Validity in AI Verification

Bias Category	Verification (Bias Detection / Diagnosis)	Countermeasures (Bias Mitigation / Correction)
1. Social Bias 5.1	• D1.5. Algorithmic Transparency • D1.2. Group-wise Performance Metrics • D1.3. Group-wise Error Rates • D1.1. Fairness Metrics • D1.6. Correlation Analysis • D2.2. Longitudinal Studies • D2.4. Benchmarks • D2.3. Data Audits	• CM1. Incorporating Fully Representative Data • CM2. Synthetic Data Generation • CM3. Bias-aware and Adversarial Training • CM4. Counter-prompting/Bias-Aware RLHF and Input Curation • CM5. Explainable Interface Design • CM6. Implementing Human in the Loop Mechanism
2. Data Representation Bias 5.2	• D1.4. Cluster analysis/ Analogy tests • D2.4. Benchmarks • D1.2. Group-wise Performance Metrics • D2.6. Continuous Monitoring	• CM2. Synthetic Data Generation • CM7. Balanced Dataset Curation and Weighted Sampling • CM8. Debiasing Static Embedding • CM9. Debiasing Contextual Embedding • CM4. Counter-prompting/Bias-Aware RLHF and Input Curation
3. Measurement & Decision Bias 5.3	• D1.3. Group-wise Error Rates • D1.2. Group-wise Performance Metrics • D2.3. Data Audits • D1.6. Correlation Analysis • D1.7. Causal Inference	• CM3. Bias-aware and Adversarial Training • CM11. Thresholding • CM12. Improving the Generalizability • CM14. Diversifying Feedback Sources and Reward Signals • CM15. Multi-objective Optimization
4. Usage Bias 5.4	• D2.5. User Experience Audits • D2.6. Continuous Monitoring • D2.7. Temporal Fairness Checks	• CM5. Explainable Interface Design • CM18. Inclusive Prompt Engineering, Output Filtering, and User Education • CM19. Decoupling Feedback Input from Training Data • CM20. Incremental/ Continual Learning, RAG for Temporal Updates

Table 4: Bias Categories with Corresponding Verification and Countermeasures.

Explore similar work

Nov 27, 2025cs.CY

ECHO: A Participatory Framework for Bias-Anchored AI Harm Anticipation

Artificial Intelligence (AI) systems increasingly shape consequential decisions, creating value but also potential harms for individuals, social groups, and society. This has prompted calls for proactive approaches that anticipate harms early in the AI lifecycle. Although prior research identifies AI biases as sources of harm, the associations between particular lifecycle biases and harms remain insufficiently understood. We introduce \texttt{ECHO}, a systematic, context-sensitive, and participatory framework that anchors early harm anticipation in lifecycle biases and elicits their perceived associations with potential harms.\texttt{ECHO} identifies domain-specific stakeholders, instantiates biases through vignettes, collects harm judgements from human participants and a large language model (LLM), and organises them into descriptive and inferential ethical matrices. Applied to disease diagnosis and hiring, \texttt{ECHO} surfaced non-uniform, context-sensitive bias--harm patterns indicating which harms were perceived as plausible consequences of particular AI biases. The theoretical interpretability of these patterns and the inferential support for specific associations strengthen the plausibility of the mappings. By linking stakeholder-specific anticipated harms to lifecycle biases, \texttt{ECHO} supports source-level harm anticipation and provides structured input to subsequent AI governance actions

Nicoleta Tantalaki, Sophia Vei, Athena Vakali

Aristotle University, Thessaloniki, Greece

Ethics Harms Artificial Intelligence Lifecycle Biases Implications

Ask PDF

Feb 27, 2026cs.AI

CIRCLE: A Framework for Evaluating AI from a Real-World Lens

This study proposes CIRCLE, a six-stage, lifecycle-based framework to bridge the reality gap between model-centric performance metrics and AI system outcomes in deployment. Current approaches such as MLOps frameworks and AI model benchmarks offer detailed insights into system stability and model capabilities, but they do not provide decision makers outside the AI stack with systematic evidence of how these systems actually behave in real world contexts or affect their organizations over time. CIRCLE operationalizes the Validation phase of TEVV (Test, Evaluation, Verification, and Validation) by translating priorities of stakeholders outside the stack into measurable signals. Unlike participatory design which often remains localized, or algorithmic audits which are often retrospective, CIRCLE provides a structured, prospective protocol for linking context sensitive qualitative insights to scalable quantitative metrics. By integrating methods such as field testing, red teaming, and longitudinal studies into a coordinated pipeline, CIRCLE produces systematic knowledge; evidence that is comparable across sites yet sensitive to local context. This can enable governance based on materialized downstream effects rather than theoretical capabilities.

Reva Schwartz, Carina Westling, Morgan Briggs +12

Civitaas Insights · Bournemouth University · Independent Researcher +11

Artificial Intelligence Evaluation Lifecycle Performance Metric Lens

Ask PDF

Apr 21, 2026cs.LO

Counting Worlds Branching Time Semantics for post-hoc Bias Mitigation in generative AI

Generative AI systems are known to amplify biases present in their training data. While several inference-time mitigation strategies have been proposed, they remain largely empirical and lack formal guarantees. In this paper we introduce CTLF, a branching-time logic designed to reason about bias in series of generative AI outputs. CTLF adopts a counting worlds semantics where each world represents a possible output at a given step in the generation process and introduces modal operators that allow us to verify whether the current output series respects an intended probability distribution over a protected attribute, to predict the likelihood of remaining within acceptable bounds as new outputs are generated, and to determine how many outputs are needed to remove in order to restore fairness. We illustrate the framework on a toy example of biased image generation, showing how CTLF formulas can express concrete fairness properties at different points in the output series.

Alessandro G. Buda, Giuseppe Primiero, Leonardo Ceragioli +1

Generative Artificial Intelligence Inference-Time Defense Post-Hoc Task Arithmetic

Ask PDF