cs.AINov 7, 2026

Too Big to Inspect? Ethically Auditing Large Datasets Through Small Samples

Authors: Giovana Marchetti, Jhessica Silva, Tiago Torrent, Carlos Caetano, Sandra Avila

Organizations: Instituto de Computação (IC), Universidade Estadual de Campinas (UNICAMP), Brazil

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Big techs and researchers increasingly claim that their training datasets are too large to inspect for bias and ethical problems. However, in the race to build ever-large models, they have relied on automated web scraping and no human curation. Datasets built this way absorb the stereotypes, exclusionary norms, and historical prejudices of their sources, and the models trained on them can reproduce and amplify those patterns. Because manually auditing billions of samples is humanly impossible, we need practical, effective methods to identify biases in these datasets. This paper studies the feasibility of ethically auditing large datasets through small samples and proposes a practical method for identifying structural biases by applying AI Ethics Tools (AIETs) to random samples. We applied Datasheets for Datasets, Data Statements, Data Nutrition Labels, and Data Cards to a 1% sample of Framed Multi30K, and Datasheets for Datasets to two 0.00006% samples of relaion2B-en-research-safe. Our results show that evaluating small samples with AIETs can reveal ethical problems, structural biases, and technical inconsistencies. Although industry practice assumes that identifying problems requires analyzing the entire dataset, we show that a random subset supports a viable qualitative analysis without overwhelming computational or human resources. Demonstrating that a dataset is biased does not require examining every instance: if a bias appears in a random sample, it is a flaw of the creation process. The AIETs' questions acted as provocateurs, evoking different perspectives and analysis methods for the same dataset, but the analysis and its results depended fundamentally on the human factor. We hope these findings contribute to urgent public discussions about ethics, transparency, accountability, and AI regulation by offering a practical pathway for responsibly auditing massive datasets.

Figures & tables

Figure 1. Proposed pipeline.
Figure 2. Example: (a) identifiable children and (b) women described as “menina” or “girl”.
Figure 3. Example: (a) original image and (b) associated description.
Figure 4. Example of an image and its respective visual caption: (a) original image and (b) associated description.
Figure 5. Sample instances from relaion2B-en-research-safe . (a) Secondary school teacher [X] at home on maternity leave with her 3-month-old daughter, [X]. A resident of New Zealand, [X] has access to paid leave, something many American mothers do not. (b) Young Woman Flirting And Dancing With Old Millionaire On Date, Escort Service Stock Photos. (c) VIDEO: 6 homes destroyed in massive Allentown blaze. (d) Horse ready to be put down and then…. (e) [X] smiling while holding his Colorado State Championship Jersey and wearing all his cycling medals. (f) Dr. [X] Featured in Ahmedabad Mirror. (g) [X] (Nepali Youtuber) Lifestyle, Income, Biography, Girlfriend, House, Car. (h) A boy pauses in front of a wall-sized poster depicting the faces of 90 survivors of the mass killings of Armenians in the Ottoman Empire. (i) The thighs are the first place of the women’s body, where the fat builds up and can be seen easily. For this reason, the women who experienced extra weight are especially unhappy with the appearanc… (j) China wheelchair specifications lightweight potable comode 24v battery electric wheelchair for disabled people. The symbol [X] indicates data intentionally anonymized to protect personal information.
Figure 6. Sample instances from relaion2B-en-research-safe . (a) Portrait of a young Hispanic electrician wearing overalls using a multimeter and smiling on a white background Banque d’images. (b) Why Kenyan school children are being taught to say ’no … (c) Plus Size S-4XL Women Medieval Cosplay Skirts Female Court-Style Gothic High Split Patchwork Long Skirts Sexy High Split Buckle Skirt(no Tops). (d) The IT Crowd. (e) Christchurch, New Zealand - March 2011: A car is crushed by a collapsed brick wall on March 20, 2011 in Christchurch following a 6.2 magnitude earthquake. (f) “[X] testifies about her prescription pill addiction in her husband’s Craigslist ‘companionship’ trial.” (g) IEMUH Winter Plus Size 90% White Duck Down Women Jackets Down Warm Coat Female Long Jacket Fat Women Parkas Loose Snow Outwear. (h) Conair HB1 The Hair Removal System (p_unsafe 0,005). (i) Only You Kindle Edition (p_unsafe 0,005). (j) [X], Webcam Peep, Gets 30-Day Jail Sentence [Update: Prosecutors Appeal]. (k) QR code with Turnagain Church Of Christ contacts. The symbol [X] indicates data intentionally anonymized to protect personal information.
More in this topic

AI Ethics

See all 33 papers

Explore similar work

CardsList
Jan 18, 2026

A Scalable Entity-Based Framework for Auditing Bias in Large Language Models

Existing approaches to bias evaluation in large language models (LLMs) trade ecological validity for statistical control, relying either on artificial prompts that poorly reflect real-world use or on naturalistic tasks that lack scale and rigor. We introduce a scalable bias-auditing framework that uses named entities as controlled probes to measure systematic disparities in model behavior. Synthetic data enables us to construct diverse, controlled inputs, and we show that it reliably reproduces bias patterns observed in natural text, supporting its use for large-scale analysis. Using this framework, we conduct the largest bias audit to date, comprising 1.9 billion data points across multiple entity types, tasks, languages, models, and prompting strategies. We find consistent patterns: models penalize right-wing politicians and favor left-wing politicians, prefer Western and wealthier countries over the Global South, favor Western companies, and penalize firms in the defense and pharmaceutical sectors. While instruction tuning reduces bias, increasing model scale amplifies it, and prompting in Chinese or Russian does not mitigate Western-aligned preferences. These findings highlight the need for systematic bias auditing before deploying LLMs in high-stakes applications. Our framework is extensible to other domains and tasks, and we make it publicly available to support future work.

Akram Elbouanani, Aboubacar Tuo, Adrian Popescu

Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France

LLM AuditingPolitical Bias in Language ModelsCultural Bias in Language ModelsLanguage Model Bias Evaluation
AskPDF
Jul 30, 2026

Benchmarks Are Not Monolithic: Sample-Level Auditing and Orchestration for LLM Evaluation

Benchmark datasets are central to evaluating Large Language Models (LLMs), yet they are typically conceived as monolithic tasks, obscuring substantial variation in the demands of individual samples. We introduce a dataset-centric meta-evaluation framework that audits benchmark datasets at the sample level along five latent dimensions: 1. Cognitive and Knowledge Demands, 2. Language and Content Quality, 3. Task Properties, 4. Context, and 5. Ethics, Safety, and Fairness. Applying this framework, we annotate five influential benchmarks -- MMLU, ARC, WinoGrande, HellaSwag, and TruthfulQA -- revealing pronounced internal heterogeneity that is not captured by aggregate accuracy scores. We show how these annotations enable criterion-driven orchestration of composite benchmark subsets across datasets, supporting targeted evaluation of model capabilities such as Reasoning Depth or Ethical Sensitivity. This approach reframes benchmark evaluation as dataset introspection, providing a principled methodology for analyzing and re-composing existing benchmarks to better reflect diverse evaluation needs.

Philipp D. Siedler, Jordan Sassoon

Aleph Alpha Research, Heidelberg, Germany.

LLM EvaluationBenchmark AuditingBenchmark ConstructionLLM Auditing
AskPDF
Jan 6, 2026

Audit Me If You Can: Query-Efficient Active Fairness Auditing of Black-Box LLMs

Large Language Models (LLMs) exhibit systematic biases across demographic groups. Auditing is proposed as an accountability tool for black-box LLM applications, but suffers from resource-intensive query access. We conceptualise auditing as uncertainty estimation over a target fairness metric and introduce BAFA, the Bounded Active Fairness Auditor for query-efficient auditing of black-box LLMs. BAFA maintains a version space of surrogate models consistent with queried scores and computes uncertainty intervals for fairness metrics (e.g., ΔΔ AUC) via constrained empirical risk minimisation. Active query selection narrows these intervals to reduce estimation error. We evaluate BAFA on two standard fairness dataset case studies: CivilComments and Bias-in-Bios, comparing against stratified sampling, power sampling, and ablations. BAFA achieves target error thresholds with up to 40×\times fewer queries than stratified sampling (e.g., 144 vs 5,956 queries at ε=0.02\varepsilon=0.02 for CivilComments) for tight thresholds, demonstrates substantially better performance over time, and shows lower variance across runs. These results suggest that active sampling can reduce resources needed for independent fairness auditing with LLMs, supporting continuous model evaluations.

David Hartmann, Lena Pohlmann, Lelia Hanslik +3

Weizenbaum Institut Berlin · Technische Universität Berlin · FIZ Karlsruhe +2

LLM AuditingAlgorithmic FairnessAdaptive SamplingDemographic Bias in Language ModelsBlack-Box Model Auditing
AskPDF