Organizations: The University of Tübingen, Tübingen, Germany · The University of Melbourne, Melbourne, Australia · Charles Darwin University, Darwin, Australia
Anthropology and related disciplines which study culture have often found it useful to consider their epistemological and methodological approaches in terms of emic versus etic. This is the distinction between the insider perspective, how the world is viewed by a member of the culture, versus the outsider perspective, the scientific cataloguing of cultural knowledge and practice. We adopt this framework to systematically analyse assumptions in recent research on culture in NLP along the full pipeline of task selection, data collection, system design and evaluation. We draw attention to the `culture of NLP' which shapes the research focus and approaches of our field, and suggest pathways towards a more culturally attuned AI.
Figures & tables
Figure 1: Hall’s Cultural Iceberg: forms and behaviours (etic) can be observed by outsiders, while corresponding meanings and beliefs (emic) are only accessible to insiders ( Hall, 1976 , after) .
Task
Data
System Design
Deployment / Evaluation
Etic
generic solns to external problems
data extraction for LLMs
simulate or improve on humans
efficiency and scalability
▹ generate facts about any culture
▹ translated or synthesized common sense benchmarks
▹ text chat; adapting to user language; flexibility in tone/register
▹ correctness and completeness of responses
Emic
local agency in shaping their world, identifying problems, exploring solutions
elicit cultural concepts and logics
build / augment human capacity in sustaining cultural vitality
local decision making, maintaining culture and locally meaningful livelihoods
▹ understanding western concepts which have no translation (e.g. “risk”)
▹ exegesis of western cultural concepts into local languages
▹ redesign of text interaction to follow local teacher-led pattern
▹ equipped with actionable information, people make better-informed choices
▹ videos of people demonstrating knowledge practices, sharing lore
▹ collection management and navigation, spoken term retrieval
▹ measure user-reported utility and safety
Table 1: Etic vs Emic Perspectives on Four Stages of AI Development
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Category
N
%
Resource
50
0.467
Survey
8
0.075
Analysis
15
0.140
Method
31
0.290
Appendix
Table A.1: Distribution over main contributions in papers in our survey.
Category
N
%
Assess / augment cultural knowledge in LLMs
78
0.729
Appendix
Table A.2: Fraction of paper with a focus on assessing or augmenting cultural knowledge in LLMs.
Category
N
%
culture annotators
19
0.178
broad data
41
0.383
synthetic
33
0.308
translation
3
0.028
– / LLM output analysis
4
0.037
– / survey
8
0.075
Appendix
Table A.3: Distribution over data set creation approaches in the papers in our survey. Papers of the last two categories were not included, as they did not contribute data sets or investigated culture in LLM output (rather than human artifacts).
The paper explores how "culture" can be operationalised in Natural Language Processing (NLP) and what this reveals about the possibilities and limits of considering a plurality of cultural backgrounds in technological design. It proposes that cultural alignment cannot be achieved only by adding more examples of "other cultures", rather it requires plural epistemologies: allowing multiple, locally grounded ways of knowing. To analyze how this plurality of knowing can be addressed in NLP, the paper uses a socio-technical model of language technology (LT) design, the five layers of technological activity model, for collecting and systematizing approaches to culture in NLP. The analysis shows that while NLP research has made progress toward more culturally sensitive systems, many approaches remain partial, addressing "culture" primarily at the level of output or representation while leaving deeper questions of power, governance, and social context unresolved. The paper concludes that operationalising culture requires much more than technical adaptation; it suggests a reflexive and plural socio-technical approach that navigates potentials and limits of computational formalisation for accounting multiple linguistic and socio-cultural backgrounds.
Gertraud Koch, Fausto Giunchiglia
University of Hamburg, Edmund-Siemers-Allee 1 West, Germany · University of Trento, Dipartimento Ingegneria e Scienza dell'Informazione
Language models are increasingly used to quantify cultural phenomena, but what makes such measurement distinctively cultural? This paper argues that NLP work on culture is a material-discursive practice: the apparatus -- model, data, annotation, evaluation -- participates in constituting the cultural reality it measures, rather than passively recording it. Drawing on Karen Barad's concept of the agential cut -- the contingent boundary between phenomenon and instrument -- I show that the apparatus's substantive design choices draw such boundaries, and that the boundary is entangled from the start because language models have already internalized much of the cultural material they measure. I illustrate this through three case studies on television and film dialogue (measuring structure, interaction, and deviation) and three examinations of the apparatus itself (erasure of cultural markers, attunement to historical material, and agency in an agentic workflow). This big picture analysis proposes a research program that is theory-driven, empirically rigorous, and culturally contingent, treating each agential cut as a conscious commitment, at once methodological and ethical.
Kent K. Chang
School of Information University of California, Berkeley
In recent years, there has been a surge of interest in Cultural NLP, with substantial efforts to create globally inclusive NLP systems. The rapid growth of literature in this field makes it difficult to track trends in methods and data resources. To address this, we analyze over 375 papers to answer three complementary questions: (1) What Cultural Capabilities (CCs) are being targeted in NLP systems? (2) How are cultural data resources being created? and (3) What methods are being used to improve the CCs of those systems? We discuss trends observed across the three questions, and identify relevant research gaps. To facilitate further research in this field, we release our full list of analyzed papers in the form of an interactive web interface, which includes a feature to allow researchers to add their work; we hope this facilitates future research and proves to be a valuable resource for the Cultural NLP community.