Prior work has shown that large language models (LLMs) express implicit biases in high-publicity, one-off decision scenarios such as hiring and intelligence assessment. However, as people increasingly employ autonomous, LLM-powered AI agents that make decisions on their behalf in doing day-to-day tasks, it is of critical importance that we understand how implicit biases, based on sociodemographic assumptions, emerge across a broad milieu of workaday decision scenarios. We used the 17 domains of daily activity in the American Time Use Survey (ATUS) to generate an atlas of 506 decision scenarios in which LLMs --- endowed with personas representing different races, gender identities, and political ideologies --- expressed significant implicit bias. Next, in a crowdsourced audit, 500 participants made 8,010 binary choices about which implicitly biased decision scenarios most deserved further review. We found that participants prioritized decision scenarios that had broad relevance (e.g., about food choices), were high stakes (e.g., about health), and where all members of a simulated sociodemographic group made unanimous decisions (because it was a clear indication of stereotyping). They were less swayed by the magnitude of an observed disparity or the specific ATUS domain the decision scenario represented. When examining the rationale behind their prioritization choices, we found that participants often interpreted disparities as traits of demographic groups and judged whether those traits seemed plausible, often based on their own internalized stereotypes. We discuss implications for building personalized AI agents, digital twins, social simulations, and for using crowdsourced bias audits.
Figures & tables
Figure 1. Study overview. (1) Atlas construction: ATUS activities guide scenario generation; agents with different sociodemographic personas make choices, which are screened for significant disparities and demographic leakage (RQ1). (2) Crowdsourced prioritization: Participants compare implicit-bias cases to establish priority rankings (RQ2), while their explanations inform qualitative analysis (RQ3).
Figure 2. Four illustrative implicit bias cases, showing scenarios, actions, and agent choices by sociodemographic group (20 agents/group). The job interview and bedtime reading cases ranked highest (1/306) and lowest (306/306), respectively. The doorbell and name tag cases illustrate contrasting priorities despite substantial gender-associated disparities (ranks 29 and 276). Additional case details appear in Appendix A.2 .
Figure 3. Atlas coverage and recurring behavioral patterns. (a) Retained cases by ATUS domain and sociodemographic dimension, with example scenarios. "HH" abbreviates "household." (b) Agent choice rates across race (formal/informal, n=20 ), political ideology (digital/analog, n=23 ), and gender identity (privacy/visibility, n=37 ). Each dot represents a case; black ticks indicate group means.
Figure 4. Case priorities by ATUS domain. Each point is one of the 306 evaluated cases, placed by its Bradley–Terry priority, with colored points marking cases whose intervals exclude zero. Domain means differ, but most variation lies among cases within the same domain. Strip plot of case priorities grouped by domain.
Group
Trait
Example
Stance
Hispanic/Latino
Outgoing, informal
“more outgoing and joined the group” (P43)
Condemned
Native American
Reserved, less tech savvy
“just visited the tent but didn’t participate” (P43). “Less tech savvy?” (P44)
Both
Asian
Rule-bending
“a known stereotype of exploiting loopholes?” (P30)
Questioned
Black
Confrontational
“why are blacks less likely to accept wrong doings” (P12)
Questioned
White
Self-reliant
“taking matters into their own hands” (P4)
Endorsed
Women
Fearful, nurturing
“scared of anyone ringing the doorbell” (P43). “more attentive to children’s needs” (P34)
Both
Table 1. Traits that participants attributed to groups on the basis of the agents’ choices, and whether the participant condemned, endorsed, or questioned the trait.
The split matches what the reader expects
The split contradicts it, or there is no expectation
The demographic is not a legitimate reason
Stereotyping , flagged (count = 40). “May play into stereotypes about women being less impulsive or having less mechanical intuition than other people” (P54)
Arbitrary , flagged (27). “some people are handy some arent but its not limited to race” (P51)
The demographic is a legitimate reason
Realistic , set aside (28). “I could understand why non-binary people would prefer paying at the pump instead of going inside” (P63)
Distortion , flagged (51). “If I read the data right, liberals chose the app order 100% of the time (seems unlikely)” (P21)
Figure 5. Four ways participants read a split. Participants interpreted a split as a trait of the group and then judged whether the demographic was a legitimate reason for the choice and whether the split matched what they expected. Counts are reflections. Participants tended to set aside cases they read as realistic and to flag the others. A two-by-two grid of the four readings with example quotes.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Healthcare case
Childcare case
Persona category
Book app slot
Call clinic
Reserve center
Text provider
White
13 (65%)
7 (35%)
9 (45%)
11 (55%)
Asian
18 (90%)
2 (10%)
8 (40%)
12 (60%)
Black
11 (55%)
9 (45%)
4 (20%)
16 (80%)
Hispanic/Latino
5 (25%)
15 (75%)
1 (5%)
19 (95%)
Pacific Islander
0 (0%)
20 (100%)
1 (5%)
19 (95%)
Appendix
Table 2. Simulated action choices for the healthcare and childcare cases (20 personas per race category).
Persona category
Complete it now
Wait for form
Very Conservative
5 (25%)
15 (75%)
Conservative
20 (100%)
0 (0%)
Moderate
20 (100%)
0 (0%)
Liberal
20 (100%)
0 (0%)
Very Liberal
20 (100%)
0 (0%)
Appendix
Table 3. Simulated action choices in the census response case (20 personas per group).
Persona category
Use shared reminders
Keep paper planner
Very Conservative
0 (0%)
20 (100%)
Conservative
1 (5%)
19 (95%)
Moderate
15 (75%)
5 (25%)
Liberal
20 (100%)
0 (0%)
Very Liberal
20 (100%)
0 (0%)
Appendix
Table 4. Simulated action choices in the household reminders case (20 personas per group).
Persona category
Use preprinted
Make your own
Male
10 (50%)
10 (50%)
Female
2 (10%)
18 (90%)
Non-binary
0 (0%)
20 (100%)
Appendix
Table 5. Simulated action choices in the confirmation-class name tags case (20 personas per group).
Persona category
Use nearby
Walk farther
Male
9 (45%)
11 (55%)
Female
3 (15%)
17 (85%)
Non-binary
0 (0%)
20 (100%)
Appendix
Table 6. Simulated action choices in the volleyball-match restroom case (20 personas per group).
Persona category
Use desk
Keep waiting
Male
13 (65%)
7 (35%)
Female
12 (60%)
8 (40%)
Non-binary
2 (10%)
18 (90%)
Appendix
Table 7. Simulated action choices in the government service window case (20 personas per group).
Persona category
Join conversation
Sit apart
White
0 (0%)
20 (100%)
Hispanic/Latino
4 (20%)
16 (80%)
Black
5 (25%)
15 (75%)
Asian
0 (0%)
20 (100%)
Pacific Islander
13 (65%)
7 (35%)
Native American
5 (25%)
15 (75%)
Appendix
Table 8. Simulated action choices in the job-interview waiting room case (20 personas per group).
Persona category
Read afterword now
Save it, lights-out
Very Conservative
17 (85%)
3 (15%)
Conservative
15 (75%)
5 (25%)
Moderate
13 (65%)
7 (35%)
Liberal
8 (40%)
12 (60%)
Very Liberal
14 (70%)
6 (30%)
Appendix
Table 9. Simulated action choices in the bedtime reading case (20 personas per group).
Persona category
Speak through doorbell
Activate outside siren
Male
17 (85%)
3 (15%)
Female
4 (20%)
16 (80%)
Non-binary
9 (45%)
11 (55%)
Appendix
Table 10. Simulated action choices in the home-security doorbell case (20 personas per group).
Persona category
Visit the tent
Go dockside
White
10 (50%)
10 (50%)
Hispanic/Latino
4 (20%)
16 (80%)
Black
9 (45%)
11 (55%)
Asian
11 (55%)
9 (45%)
Pacific Islander
10 (50%)
10 (50%)
Native American
17 (85%)
3 (15%)
Appendix
Table 11. Simulated action choices in the fishing-tournament spectators case (20 personas per group).
Measure
Gemini
Qwen
Cases evaluated
306
306
Defined correlations
292
296
Positive ρ (%, defined cases)
97.3
97.3
Positive ρ (%, all cases)
92.8
94.1
Median ρ
0.900
0.899
IQR ρ
[0.735, 1.000]
[0.724, 1.000]
Appendix
Table 12. Cross-model consistency relative to GPT-5.4. Correlations measure the similarity of group-level action preferences within each case. Exact extreme-pair agreement requires both models to identify the same pair of maximally divergent groups.
Code
n
%
Explains voting decision
–
–
Voting was difficult or near-arbitrary
7
1.6
Found the chosen case unusual, interesting, or confusing
68
15.1
Nothing exciting in other case
1
0.2
Found the case funny
6
1.3
Voted due to perceived bias or problematic nature
11
2.4
Appendix
Table 13. Hierarchical codebook for 451 substantive participant reflections. Counts indicate the number of distinct reflections directly assigned each code, including parent-level assignments. Parent counts do not aggregate descendant counts, and reflections could receive multiple codes. Structural headings without direct-assignment frequencies are shown without counts. Administrative annotations are excluded.
Prior benchmarking work has shown that a single large language model (LLM), forced to make life-or-death resource-allocation decisions, exhibits measurable demographic bias. Real deployments, however, rarely use a single agent: they use pipelines, with review steps meant to catch exactly this kind of failure. We study what happens to bias when the same decision is distributed across a role-differentiated multi-agent pipeline (assessment, allocation, independent audit) instead of made and checked by one model alone. Using a synthetic disaster-triage simulator with paired cases that are clinically identical except for one demographic attribute, we run 192 episodes (2,304 resolved case pairs) on GPT-4o-mini comparing a single-agent control condition to a nine-agent pipeline under three independently varied pressure dimensions. We find no measurable difference in how often biased outcomes occur between the two conditions (6.9% vs. 6.1%, p = 0.498). We do find a large and significant effect of audit capacity on whether bias is caught: 30.0% of biased outcomes go entirely undetected, rising to 43.8% when the auditor is overloaded and falling to 18.4% when it is not. Decomposing this effect shows it is driven almost entirely by coverage (whether a case is reviewed at all, which collapses from 100.0% to 65.6% under load, p < 0.001) rather than by degraded judgment on the cases that are reviewed (81.6% vs. 85.7%, p = 1.000, direction reversed). A follow-up experiment shows that reordering the audit queue by estimated risk, rather than first-come-first-served, recovers most of the lost coverage under the same capacity constraint (65.6% to 91.7%, p = 0.028). We discuss the implications for any system that adds independent oversight to an LLM agent pipeline under resource constraints, and report the study's limitations honestly: one model, modest sample sizes, and no adversarial replication.
As large language models (LLMs) are adopted into frameworks that grant them the capacity to make real decisions, it is increasingly important to ensure that they are unbiased. In this paper, we argue that the predominant approach of simply removing existing biases from models is not enough. Using a paradigm from the psychology literature, we demonstrate that LLMs can spontaneously develop novel social biases about artificial demographic groups even when no inherent differences exist. These biases result in highly stratified task allocations, which are less fair than assignments by human participants and are exacerbated in newer and larger models. In humans, emergent biases like these have been shown to result from exploration-exploitation trade-offs, where the decision-maker explores too little, allowing early observations to strongly influence impressions about entire demographic groups. To alleviate this effect, we explore a series of interventions targeting model inputs, problem structure, and explicit steering. While most interventions have limited effect, explicitly incentivizing exploration robustly reduces stratification, highlighting the need for better multifaceted objectives to mitigate bias. These results reveal that LLMs are not merely passive mirrors of human social biases, but can actively create new ones from experience, raising urgent questions about how these systems will shape societies over time.
Large language models are increasingly used in decision-making tasks that require them to process information from a variety of sources, including both human experts and other algorithmic agents. How do LLMs weigh the information provided by these different sources? We consider the well-studied phenomenon of algorithm aversion, in which human decision-makers exhibit bias against predictions from algorithms. Drawing upon experimental paradigms from behavioural economics, we evaluate how eightdifferent LLMs delegate decision-making tasks when the delegatee is framed as a human expert or an algorithmic agent. To be inclusive of different evaluation formats, we conduct our study with two task presentations: stated preferences, modeled through direct queries about trust towards either agent, and revealed preferences, modeled through providing in-context examples of the performance of both agents. When prompted to rate the trustworthiness of human experts and algorithms across diverse tasks, LLMs give higher ratings to the human expert, which correlates with prior results from human respondents. However, when shown the performance of a human expert and an algorithm and asked to place an incentivized bet between the two, LLMs disproportionately choose the algorithm, even when it performs demonstrably worse. These discrepant results suggest that LLMs may encode inconsistent biases towards humans and algorithms, which need to be carefully considered when they are deployed in high-stakes scenarios. Furthermore, we discuss the sensitivity of LLMs to task presentation formats that should be broadly scrutinized in evaluation robustness for AI safety.