Prior work has shown that large language models (LLMs) express implicit biases in high-publicity, one-off decision scenarios such as hiring and intelligence assessment. However, as people increasingly employ autonomous, LLM-powered AI agents that make decisions on their behalf in doing day-to-day tasks, it is of critical importance that we understand how implicit biases, based on sociodemographic assumptions, emerge across a broad milieu of workaday decision scenarios. We used the 17 domains of daily activity in the American Time Use Survey (ATUS) to generate an atlas of 506 decision scenarios in which LLMs --- endowed with personas representing different races, gender identities, and political ideologies --- expressed significant implicit bias. Next, in a crowdsourced audit, 500 participants made 8,010 binary choices about which implicitly biased decision scenarios most deserved further review. We found that participants prioritized decision scenarios that had broad relevance (e.g., about food choices), were high stakes (e.g., about health), and where all members of a simulated sociodemographic group made unanimous decisions (because it was a clear indication of stereotyping). They were less swayed by the magnitude of an observed disparity or the specific ATUS domain the decision scenario represented. When examining the rationale behind their prioritization choices, we found that participants often interpreted disparities as traits of demographic groups and judged whether those traits seemed plausible, often based on their own internalized stereotypes. We discuss implications for building personalized AI agents, digital twins, social simulations, and for using crowdsourced bias audits.
Figures & tables
Figure 1. Study overview. (1) Atlas construction: ATUS activities guide scenario generation; agents with different sociodemographic personas make choices, which are screened for significant disparities and demographic leakage (RQ1). (2) Crowdsourced prioritization: Participants compare implicit-bias cases to establish priority rankings (RQ2), while their explanations inform qualitative analysis (RQ3).
Figure 2. Four illustrative implicit bias cases, showing scenarios, actions, and agent choices by sociodemographic group (20 agents/group). The job interview and bedtime reading cases ranked highest (1/306) and lowest (306/306), respectively. The doorbell and name tag cases illustrate contrasting priorities despite substantial gender-associated disparities (ranks 29 and 276). Additional case details appear in Appendix A.2 .
Figure 3. Atlas coverage and recurring behavioral patterns. (a) Retained cases by ATUS domain and sociodemographic dimension, with example scenarios. "HH" abbreviates "household." (b) Agent choice rates across race (formal/informal, n=20 ), political ideology (digital/analog, n=23 ), and gender identity (privacy/visibility, n=37 ). Each dot represents a case; black ticks indicate group means.
Figure 4. Case priorities by ATUS domain. Each point is one of the 306 evaluated cases, placed by its Bradley–Terry priority, with colored points marking cases whose intervals exclude zero. Domain means differ, but most variation lies among cases within the same domain. Strip plot of case priorities grouped by domain.
Group
Trait
Example
Stance
Hispanic/Latino
Outgoing, informal
“more outgoing and joined the group” (P43)
Condemned
Native American
Reserved, less tech savvy
“just visited the tent but didn’t participate” (P43). “Less tech savvy?” (P44)
Both
Asian
Rule-bending
“a known stereotype of exploiting loopholes?” (P30)
Questioned
Black
Confrontational
“why are blacks less likely to accept wrong doings” (P12)
Questioned
White
Self-reliant
“taking matters into their own hands” (P4)
Endorsed
Women
Fearful, nurturing
“scared of anyone ringing the doorbell” (P43). “more attentive to children’s needs” (P34)
Both
Table 1. Traits that participants attributed to groups on the basis of the agents’ choices, and whether the participant condemned, endorsed, or questioned the trait.
The split matches what the reader expects
The split contradicts it, or there is no expectation
The demographic is not a legitimate reason
Stereotyping , flagged (count = 40). “May play into stereotypes about women being less impulsive or having less mechanical intuition than other people” (P54)
Arbitrary , flagged (27). “some people are handy some arent but its not limited to race” (P51)
The demographic is a legitimate reason
Realistic , set aside (28). “I could understand why non-binary people would prefer paying at the pump instead of going inside” (P63)
Distortion , flagged (51). “If I read the data right, liberals chose the app order 100% of the time (seems unlikely)” (P21)
Figure 5. Four ways participants read a split. Participants interpreted a split as a trait of the group and then judged whether the demographic was a legitimate reason for the choice and whether the split matched what they expected. Counts are reflections. Participants tended to set aside cases they read as realistic and to flag the others. A two-by-two grid of the four readings with example quotes.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Healthcare case
Childcare case
Persona category
Book app slot
Call clinic
Reserve center
Text provider
White
13 (65%)
7 (35%)
9 (45%)
11 (55%)
Asian
18 (90%)
2 (10%)
8 (40%)
12 (60%)
Black
11 (55%)
9 (45%)
4 (20%)
16 (80%)
Hispanic/Latino
5 (25%)
15 (75%)
1 (5%)
19 (95%)
Pacific Islander
0 (0%)
20 (100%)
1 (5%)
19 (95%)
Appendix
Table 2. Simulated action choices for the healthcare and childcare cases (20 personas per race category).
Persona category
Complete it now
Wait for form
Very Conservative
5 (25%)
15 (75%)
Conservative
20 (100%)
0 (0%)
Moderate
20 (100%)
0 (0%)
Liberal
20 (100%)
0 (0%)
Very Liberal
20 (100%)
0 (0%)
Appendix
Table 3. Simulated action choices in the census response case (20 personas per group).
Persona category
Use shared reminders
Keep paper planner
Very Conservative
0 (0%)
20 (100%)
Conservative
1 (5%)
19 (95%)
Moderate
15 (75%)
5 (25%)
Liberal
20 (100%)
0 (0%)
Very Liberal
20 (100%)
0 (0%)
Appendix
Table 4. Simulated action choices in the household reminders case (20 personas per group).
Persona category
Use preprinted
Make your own
Male
10 (50%)
10 (50%)
Female
2 (10%)
18 (90%)
Non-binary
0 (0%)
20 (100%)
Appendix
Table 5. Simulated action choices in the confirmation-class name tags case (20 personas per group).
Persona category
Use nearby
Walk farther
Male
9 (45%)
11 (55%)
Female
3 (15%)
17 (85%)
Non-binary
0 (0%)
20 (100%)
Appendix
Table 6. Simulated action choices in the volleyball-match restroom case (20 personas per group).
Persona category
Use desk
Keep waiting
Male
13 (65%)
7 (35%)
Female
12 (60%)
8 (40%)
Non-binary
2 (10%)
18 (90%)
Appendix
Table 7. Simulated action choices in the government service window case (20 personas per group).
Persona category
Join conversation
Sit apart
White
0 (0%)
20 (100%)
Hispanic/Latino
4 (20%)
16 (80%)
Black
5 (25%)
15 (75%)
Asian
0 (0%)
20 (100%)
Pacific Islander
13 (65%)
7 (35%)
Native American
5 (25%)
15 (75%)
Appendix
Table 8. Simulated action choices in the job-interview waiting room case (20 personas per group).
Persona category
Read afterword now
Save it, lights-out
Very Conservative
17 (85%)
3 (15%)
Conservative
15 (75%)
5 (25%)
Moderate
13 (65%)
7 (35%)
Liberal
8 (40%)
12 (60%)
Very Liberal
14 (70%)
6 (30%)
Appendix
Table 9. Simulated action choices in the bedtime reading case (20 personas per group).
Persona category
Speak through doorbell
Activate outside siren
Male
17 (85%)
3 (15%)
Female
4 (20%)
16 (80%)
Non-binary
9 (45%)
11 (55%)
Appendix
Table 10. Simulated action choices in the home-security doorbell case (20 personas per group).
Persona category
Visit the tent
Go dockside
White
10 (50%)
10 (50%)
Hispanic/Latino
4 (20%)
16 (80%)
Black
9 (45%)
11 (55%)
Asian
11 (55%)
9 (45%)
Pacific Islander
10 (50%)
10 (50%)
Native American
17 (85%)
3 (15%)
Appendix
Table 11. Simulated action choices in the fishing-tournament spectators case (20 personas per group).
Measure
Gemini
Qwen
Cases evaluated
306
306
Defined correlations
292
296
Positive ρ (%, defined cases)
97.3
97.3
Positive ρ (%, all cases)
92.8
94.1
Median ρ
0.900
0.899
IQR ρ
[0.735, 1.000]
[0.724, 1.000]
Appendix
Table 12. Cross-model consistency relative to GPT-5.4. Correlations measure the similarity of group-level action preferences within each case. Exact extreme-pair agreement requires both models to identify the same pair of maximally divergent groups.
Code
n
%
Explains voting decision
–
–
Voting was difficult or near-arbitrary
7
1.6
Found the chosen case unusual, interesting, or confusing
68
15.1
Nothing exciting in other case
1
0.2
Found the case funny
6
1.3
Voted due to perceived bias or problematic nature
11
2.4
Appendix
Table 13. Hierarchical codebook for 451 substantive participant reflections. Counts indicate the number of distinct reflections directly assigned each code, including parent-level assignments. Parent counts do not aggregate descendant counts, and reflections could receive multiple codes. Structural headings without direct-assignment frequencies are shown without counts. Administrative annotations are excluded.