cs.CLApr 14, 2026
SaveThe role of System 1 and System 2 semantic memory structure in human and LLM biases
Organizations: Department of Computer Science, University of Pisa, Italy · Institute of Information Science and Technologies - National Research Council, Italy · CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento, Italy
Abstract
Implicit biases in both humans and large language models (LLMs) pose significant societal risks. Dual process theories propose that biases arise primarily from associative System 1 thinking, while deliberative System 2 thinking mitigates bias, but the cognitive mechanisms that give rise to this phenomenon remain poorly understood. To better understand what underlies this duality in humans, and possibly in LLMs, we model System 1 and System 2 thinking as semantic memory networks with distinct structures, built from comparable datasets generated by both humans and LLMs. We then investigate how these distinct semantic memory structures relate to implicit gender bias using network-based evaluation metrics. We find that semantic memory structures are irreducible only in humans, suggesting that LLMs lack certain types of human-like conceptual knowledge. Moreover, semantic memory structure relates consistently to implicit bias only in humans, with lower levels of bias in System~2 structures. These findings suggest that certain types of conceptual knowledge contribute to bias regulation in humans, but not in LLMs, highlighting fundamental differences between human and machine cognition.
Figures & tables
Figure 1: System 1 and System 2 semantic memory structures. Three different structures of semantic memory are investigated, intended to approximate System 1 and System 2 thinking. System 1 semantic memory structures are built from free associations by connecting cue words to their associate responses. To approximate System 2 thinking, we build two different structures of semantic memory. First, we use definitions, connecting words to those words appearing in its definition, and then we use categorical relations, connecting words that share a categorical relationship such as synonyms, antonyms, hyponyms, and hypernyms.
| Traits Female Male affectionate active cheerful adventure compassionate aggression connection ambitious consideration assert cooperative athlete depend boast emotional competition empathy confidence gentle courageous honest determined kind dominant loyalty force modest hostility nag impulsive nurture independent quiet intellectual sensitive leader support logical sympathetic opinion tender outspoken trusting persistent understanding reckless warm stubborn whine superior | Home/Career Female Male home career house work household job kitchen office ironing employee cooking supervisor cleaning business bake company wash professional dishwasher corporate laundry industry sew manager shopping hire grocery profession decorate ceo family promotion parent earn spouse money child invest birth success infant income daycare salary marriage paycheck wedding pension caregiver workplace |
| Art/Science Female Male art math poetry science dance chemistry pottery physic drawing geology painting astronomy sculpture nasa theater engineering cinema computer literature technology music biology photography calculus performance equation design statistic culture mathematics writing economics creativity logic expression number | Professions Female Male artist scientist psychologist carpenter nurse doctor teacher police officer babysitter firefighter secretary electrician receptionist plumber therapist engineer maid pilot librarian astronaut dancer mechanic cashier surgeon designer captain technician dentist writer mayor singer judge clerk contractor journalist farmer |
Table 1: Target words. The target words used for evaluating gender bias are shown in the tables. The targets, half of which are related to female stereotypes, and half of which are related to male stereotypes, fall into four categories: traits, home vs. career, art vs. science, and professions.
Figure 2: Measuring implicit bias with spreading activation. The simplified example shows how the methodology can be used to evaluate gender bias by modeling semantic priming within a word association network. A spreading activation process is implemented in the network by activating the prime nodes woman and man independently. At the end of the process, the final activation level of doctor is 0.2 when woman is the prime node, compared to 0.3 when man is the prime node. This shows that the association between man and doctor is stronger than the association between woman and doctor , indicating gender bias. [ 2 ]
| Network | Layer | Nodes | Edges | Density | Mean degree |
|---|---|---|---|---|---|
| Free Associations | 24,308 | 317,344 | 0.0011 | 26 | |
| Humans | Definitions | 16,200 | 83,971 | 0.0006 | 10 |
| Categorical relations | 34,211 | 94,541 | 0.0002 | 6 | |
| Free Associations | 20,339 | 199,103 | 0.0010 | 20 | |
| Mistral | Definitions | 16,806 | 153,771 | 0.0011 | 18 |
| Categorical relations | 26,451 | 236,931 | 0.0007 | 18 |
Table 2: Network statistics. The table shows network statistics by network and layer, including number of nodes, number of edges, network density, and mean degree.
Figure 3: Structural reducibility of multilayer semantic networks. Relative entropy values are shown for different layer aggregation configurations. For humans (blue), entropy is maximized when all three layers remain separate. For Mistral (orange) and Llama3 (green), entropy is maximized when the associative and categorical layers are merged, indicating partial reducibility.
Figure 4: Effect sizes disaggregated by stereotype topic. Effect sizes measuring stereotype-consistent associations are shown separately for humans, Mistral, and Llama3 across four stereotype topics: traits, professions, home/career, and arts/sciences. Within each network, effect sizes are computed for the associative, definitional, and categorical layers. Positive values indicate stereotype-consistent associations, while negative values indicate stereotype-inconsistent associations. The plots illustrate how bias varies across semantic structures and topics within each network.
Figure 5: Effect sizes aggregated by stereotype topics. Effect sizes are aggregated across the four stereotype topics to provide an overall measure of stereotype bias within each network and semantic layer. Positive values indicate stereotype-consistent associations, whereas negative values indicate stereotype-inconsistent associations. The results summarize how bias varies across associative, definitional, and categorical representations in humans, Mistral, and Llama3.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Normalized activation levels for the free associations layer in humans. Strong associations are observed between certain prime-target pairs like mother – nurture and father – parent . The male-stereotypes heatmap shows that activation levels are notably greater for male-related targets when activated by male-related primes, evidenced by overall darker shades on the right side of the plot.
Figure 7: Normalized activation levels for the definitions layer in humans. A large degree of symmetry is observed in the heatmaps, such as the concepts parent , daycare , and birth being activated similarly by mother and father . There is little visible evidence of differences in activation levels between stereotype-consistent prime-target pairs and stereotype-inconsistent prime-target pairs, reflecting low levels of bias in this layer.
Figure 8: Normalized activation levels for the categorical relations layer in humans. The heatmaps show large variations in the normalized activation levels of prime-target pairs, in contrast to the free associations and definitions layers.
Figure 9: Normalized activation levels for the free associations layer in Mistral. Strong associations are observed between certain prime-target pairs like mother – birth and masculine – persistent . It appears that certain stereotype topics are particularly activated by certain primes (male-related traits are strongly activated by the prime masculine .
Figure 10: Normalized activation levels for the definitions layer in Mistral. Normalized activation levels appear mostly homogeneous, with little variation across topics and layers, with the exception of a few prime-target pairs like boy – adventure and father – parent .
Figure 11: Normalized activation levels for the categorical relations layer in Mistral. Normalized activation levels appear mostly homogeneous, but with several exceptions of prime-target pairs with high or low activation levels.
Figure 12: Normalized activation levels for the free associations layer in Llama3. It appears that certain stereotype topics are particularly activated by certain primes (male-related traits are strongly activated by the prime masculine .
Figure 13: Normalized activation levels for the definitions layer in Llama3. Normalized activation levels appear mostly homogeneous, with little variation across topics and layers, with the exception of a few prime-target pairs like father – parent .
Figure 14: Normalized activation levels for the categorical relations layer in Llama3. The heatmaps show large variations in the normalized activation levels of prime-target pairs, in contrast to the free associations and definitions layers.