Bridging the Language Gap in Scholarly Data II: Enhancing Gender Inference for Chinese Names

Lisette Espín-Noboa; Mingrong She; Ana Maria Jaramillo; Liuhuaying Yang

cs.CLOct 16, 2026

Bridging the Language Gap in Scholarly Data II: Enhancing Gender Inference for Chinese Names

Authors: Lisette Espín-Noboa, Mingrong She, Ana Maria Jaramillo, Liuhuaying Yang

Organizations: Graz University of Technology, Graz, Austria · Complexity Science Hub, Vienna, Austria · Maastricht University, Maastricht, The Netherlands

Abstract

Perceived gender inference from names is important for inclusive scientometrics research, yet existing methods perform poorly on non-Western scholars. We address this gap for Chinese names, which appear in two forms, Hanzi and Pinyin, its Romanized form (e.g., Zhang Wei, Li Na). In international bibliographic records, names are often available only in Pinyin. However, a single pinyin name can correspond to multiple distinct hanzi characters with different gender associations, so gender inference from Pinyin alone is intrinsically ambiguous. To address this, we first develop two hanzi models, a regression model that produces a continuous gender score from 0 (male) to 1 (female), and a classification model that outputs one of three labels (\em{male}, \em{neutral}, \em{female}). We train them on a sample of hanzi names from Chinese-language physics publications, rated by 11 native Chinese speakers. On this sample, the models achieve an RMSE of $0.09$ and an R $^2$ of $0.85$ (regression), and macro-AUC of $0.97$ and a Cohen's $κ$ of $0.81$ (classification), demonstrating high performance that is robust to class imbalance. For pinyin names, we propose a pipeline that aggregates hanzi predictions via transliteration and frequency weighting, together with an ambiguity score that flags names whose gender cannot be resolved from Pinyin alone. Benchmarked against three off-the-shelf tools (Namsor, GPT, and Claude), our models achieve the highest agreement with human annotations for both gender scores and gender labels, in both scripts. Applied to an independent sample of Chinese-named authors in APS journals, our models assign a gender to every name, whereas existing Pinyin-based methods leave between 20% and 100% of names unclassified. These findings demonstrate the importance of hanzi-level information for reliable gender inference of Chinese names. Code and data are publicly available.

Figures & tables

Figure 1: Pinyin transliteration and gender inference pipeline. (a) Transliteration process: A single pinyin syllable (e.g., “Deng”) maps to multiple hanzi characters, each with its own predominant gender association. (b) Gender inference strategy: First, we train CatBoost models to predict gender from hanzi characters: a regressor to produce continuous gender scores and a classifier to produce categorical labels. For pinyin names, we leverage the transliteration process and aggregate the predicted gender from all hanzi characters that map to that syllable.

Panel of Figure 3 — Figure 3: Performance of hanzi-based gender score inference ( $\hat{y}_{h}$ ). Predicted versus true scores on (a) the full ground-truth set ( $N=166$ , train and test combined) and (b) the held-out test set ( $N=34$ ). Points represent hanzi given names. Color indicates whether the name is single-character, and shape indicates whether features from all tools were available. Fit is summarized by $R^{2}$ , root mean squared error (RMSE), and Pearson correlation ( $\rho$ ). (c) CatBoost feature importance and (d) SHAP values both rank the aggregated score mean_ngender_rn and ngender_r as the dominant predictors. The binary indicator single_char and the categorical indicator gender_w contribute minimally.

Panel of Figure 4 — Figure 4: Performance of pinyin-based gender score inference ( $\hat{y}_{p}$ ). True scores (y-axis) versus predicted values (x-axis) on (a) the full ground-truth set (train and test combined) and (b) the held-out test set. Points represent pinyin names transliterated from the hanzi ground-truth dataset. True values are the mean score from the human annotations on the hanzi version of each name. Color indicates whether the name is single-character, and shape indicates whether features from all tools were available. Each column corresponds to one aggregation strategy ( Section 3.2.2 ). Some panels show fewer name pairs because of our single-character augmentation ( Section 3.1.1 ). Fit is summarized by $R^{2}$ , root mean squared error (RMSE), and Pearson correlation ( $\rho$ ).

Figure 5: Ambiguity of pinyin-level gender score aggregation. A sample of pinyin names with varying gender associations, shown across the four aggregation strategies. Circles denote the aggregated pinyin-level score ( $\hat{y}_{p}$ ), with error bars spanning the 10th to 90th percentile ( $q_{10}$ , $q_{90}$ ) of the hanzi-level predictions. Color encodes the ambiguity of the aggregation, defined as the width of this interval ( $q_{90}-q_{10}$ ). Grey crosses mark the individual gender scores of all hanzi variants associated with each pinyin name. The number of hanzi variants ( $n_{h}$ ) is reported next to each name. Names with high ambiguity (warm colors, wide error bars) indicate conflicting gender associations across hanzi variants, warranting caution when interpreting the aggregated score.

Panel of Figure 6 — Figure 6: Comparison of gender score predictions across methods. Mean annotator scores (y-axis) versus predicted gender scores (x-axis) for our model and three off-the-shelf methods ( Namsor , GPT 5.4 Nano , Claude Opus 4.8 ), using (a) hanzi-based and (b) pinyin-based inference. Each point represents one of 34 held-out names; the dashed line indicates perfect agreement. Our model achieves the highest $R^{2}$ and lowest RMSE in both settings.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Journal	hz	en	Papers
Acta Physica Sinica	✓		19,730
Chinese Physics B		✓	8,813
College Physics	✓		6,081
Chinese Journal of Atomic and Molecular Physics	✓	✓	5,743
Wuli (Physics)	✓		5,379
Chinese Journal of Luminescence	✓		2,719

Appendix

Table A.1 : Journals published by the Chinese Physical Society included in the CNKI corpus. For each journal we report the language of its articles ( hz = hanzi, en = English) and the number of papers indexed up to 2024-02-14. Journals marked in both columns publish in both languages, and counts are pooled across former titles of the same journal. The column totals

65{,}243

records, of which

65{,}241

carry a parseable Chinese personal name and enter the analysis.

Figure A.1 : Author name disambiguation pipeline. A name associated with more than one paper is treated as ambiguous. Each pair of papers sharing that name is tested against co-authorship, affiliation, content, and citation criteria; a match on any criterion merges the pair into one identity, and pairs matching none are separated into distinct identities. The disambiguation stage is shaded in dark gray. Adapted from [ 69 ] .

#	Title	Year	Comp. appr.	DB	Code	New meth.	Meth. comp.	Ch. names
1	Comparison and benchmark of name-to-gender inference services [ 64 ]	2018	✓	✓	✓	✕	✓	✕
2	Augmenting Data Quality through High-Precision Gender Categorization [ 55 ]	2019	✓	✕	✕	✓	✕	✕
3	The sound of gender: inferring the gender of names in a foreign language [ 15 ]	2019	✕	✕	✕	✕	✕	✓
4	The impact of gender orientation of names on individuals’ evaluation of impressions and interpersonal interaction [ 12 ]	2021	✕	✕	✕	✕	✕	✓
5	Performance of gender detection tools: a comparative study of name-to-gender inference services [ 65 ]	2021	✓	✓	✕	✕	✓	✕
6	Using genderize.io to infer the gender of first names: how to improve the accuracy of the inference [ 66 ]	2021	✓	✓	✕	✓	✕	✕

Appendix

Table A.2 : Chinese name–to–gender inference literature identified via Web of Science. A structured search in Clarivate Web of Science (WoS) yielded 14 records, of which 10 were retained after excluding studies that did not propose or evaluate methods for Chinese name gender inference. Columns indicate publication year and whether the study includes a computational approach (Comp. appr.), uses an external database (DB), provides code (Code), proposes a new method (New meth.), compares multiple methods (Meth. comp.), or explicitly focuses on Chinese names (Ch. names).

Panel of Figure 9 — Figure A.2 : Agreement between tool-based gender inference and annotator-perceived gender. Comparison between gender scores produced by inference tools and the mean perceived gender assigned by human annotators (gold standard). Each point represents a name; the x-axis shows the average annotator score, and the y-axis the score produced by the tool or tool aggregation. Panels show individual tools (a) N and (b) R, and aggregated estimates obtained by averaging tools: (c) RNW, (d) RN, (e) RW, and (f) NW. Solid lines indicate linear fits; Pearson correlation ( $\rho$ ) and RMSE quantify agreement with the gold standard. Aggregating tools generally improves alignment with annotator judgments, with the RN combination showing the strongest correlation and lowest error. Panels involving tool W (c, e, f) show fewer names ( $N=116$ ) because rare single-character names were excluded, as W defaults to male for all such names.

				$\hat{y}_{h}$					$\hat{y}_{p}$
Pinyin	$n_{H}$	$y_{h}$	sc	min	max	std	$\Delta$	Amb.	M CNKI	M CNKI+	W CNKI	W Pop.
fang	19	0.40	✓	0.28	0.65	0.11	0.37	0.21	0.42	0.41	0.56	0.61
jie	52	0.46	✓	0.27	0.67	0.09	0.40	0.23	0.43	0.38	0.40	0.36
jin	43	0.46	✓	0.26	0.62	0.10	0.36	0.27	0.37	0.40	0.36	0.40
jun	22	0.27	✓	0.25	0.60	0.09	0.35	0.22	0.35	0.36	0.33	0.34
meng	25	0.79	✓	0.28	0.62	0.10	0.34	0.25	0.47	0.40	0.55	0.56

Appendix

Table A.3 : Pinyin-based gender score inference on the held-out set. For each of the 34 pinyin names in the test set, we report the number of associated hanzi names in our corpus (

n_{H}

), the mean gold-standard gender score (

y_{h}

), and whether the name is single-character ( sc ). For the hanzi-level predictions (

\hat{y}_{h}

), we include the range, standard deviation, spread (

\Delta

), and ambiguity (Amb.). We also report our four pinyin-level aggregation strategies (

\hat{y}_{p}

): unweighted averages over CNKI and CNKI+, and weighted averages using CNKI and Population frequencies ( Section 3.2.2 ).

Panel of Figure 11 — Figure A.3 : Hanzi- vs. pinyin-based gender score inference. Hanzi-based predictions $\hat{y}_{h}$ (x-axis) versus pinyin-based predictions $\hat{y}_{p}$ (y-axis) for each aggregation strategy: (a) Mean CNKI, (b) Mean CNKI+, (c) Weighted CNKI by author counts, and (d) Weighted by population counts. Each point represents one unique hanzi given name in the CNKI corpus. Color encodes the ambiguity of the aggregated pinyin-level gender score, defined as the difference between the 90th and 10th percetiles ( $q_{90}-q_{10}$ ) of the hanzi-level predictions for a given pinyin name, binned into intervals ( ${<}0.1$ , $[0.1,0.2)$ , $[0.2,0.3)$ , $[0.3,0.4)$ , $[0.4,0.5)$ , $[0.5,0.6)$ , ${\geq}0.6$ ). The dashed line indicates perfect agreement. Each panel reports goodness of fit ( $R^{2}$ ) and error metrics (MAE, MSE, RMSE). Mean CNKI+ (b) yields the highest agreement among all strategies (lowest RMSE). Panels (a) and (c) show fewer name pairs because of the single-character augmentation ( Section 3.1.1 ).

Figure A.4 : Discretization of gender scores. Aggregated human annotations were mapped into the classes male , neutral , and female using different discretization strategies. (a–c) Equal-width, equal-size, and visually derived cuts based on the mean. (d–e) Visual cuts using the median and mode, respectively. (f) Fuzzy cut following annotation instructions (

0

= definitely male,

0.25

= more likely male,

0.5

= neutral,

0.75

= more likely female,

1

= definitely female). (g) Conservative fuzzy cut assigning 20% to male and female extremes. (h) Conservative fuzzy cut assigning 20% to the neutral middle. These discretizations were later used as target variables in model training.

Panel of Figure 13 — Figure A.5 : Performance of hanzi-based gender label classification ( $\hat{l}_{h}$ ). The best results were obtained with Cut 1 (mean-based equal-width discretization; see Figure A.4 a). (a,b) Confusion matrices for the full ground-truth set (train + test) and the test set, respectively. (c) CatBoost feature importance, showing the aggregated mean predictions from R and N ( mean_ngender_rn ) is the strongest predictor. (d,e) One-vs-rest ROC curves for the entire ground-truth set and the test set, respectively. (f) SHAP summary plot indicating consistent class-discriminative effects of the main aggregated predictors. Overall, the classifier achieves high and stable performance across classes, with predictive power primarily driven by features from tools R and N .

Panel of Figure 14 — Figure A.6 : Pairwise agreement between hanzi-based gender classification methods. a) Accuracy and (b) Cohen’s $\kappa$ for all method pairs, including ground truth (GT). Our method achieves the highest agreement with GT (accuracy $=0.88$ , $\kappa=0.81$ ), followed by Claude ( $0.75$ , $0.64$ ), while GPT ( $0.44$ , $0.26$ ) and Namsor ( $0.44$ , $0.22$ ) show the lowest agreement across all comparisons.

Panel of Figure 15 — Figure A.7 : Per-class precision and recall of hanzi-based gender classification ( $N{=}32$ ). Confusion matrices comparing our method (Ours) against two LLMs ( Claude , GPT ) and Namsor on the held-out test set. (a) Precision: our approach leads for male ( $0.90$ ) and female ( $1.00$ ), and ranks second for neutral ( $0.80$ , behind Claude ’s $0.82$ ). (b) Recall: our method achieves the highest male recall ( $0.82$ ) and neutral recall ( $0.92$ ), and second highest for female ( $0.88$ ), matching Claude ; Namsor reaches $1.00$ only by over-predicting female (precision $0.36$ ).

Panel of Figure 16 — Figure A.8 : Performance of pinyin-based gender classification ( $\hat{l}_{p}$ ). (a,b) Confusion matrices on the full ground-truth set (train + test) and the test set, evaluated against the ground-truth labels of each pinyin’s associated hanzi. (c,d) One-vs-rest ROC curves on the same splits, using the hanzi-label proportions $r_{l}(p)$ as class probabilities. Our supermajority classifier achieves high macro-AUC ( $0.890$ / $0.864$ ) but low Cohen’s $\kappa$ ( $0.328$ / $0.259$ ), reflecting poor performance on the minority class.

Figure A.9 : Disagreement between pinyin- and hanzi-based gender labels. Left: each dot represents a pinyin name, colored by the inferred majority label

\hat{l}_{p}

and with transparency inversely proportional to the entropy of its underlying hanzi-label distribution—low entropy (opaque) indicates strong consensus among hanzi mappings, while high entropy (transparent) indicates mixed labels. The

x

-axis measures the fraction of hanzi labels that disagree with the final pinyin label. Right: hanzi-label composition for each name, showing the proportion of male , neutral , and female labels across all hanzi mappings. Names shown are the same as in Figure 5 for direct comparison.

			Hanzi-based labels $\hat{l}_{h}$				Ambiguity
Pinyin	$n_{h}$	SC	Male	Neutral	Female	$\hat{l}_{p}$	Entropy	Disagreement
shen	35	✓	31	4	0	male	0.32	0.11
jun	22	✓	18	4	0	male	0.43	0.18
qi	84	✓	69	14	1	male	0.47	0.18
jie	52	✓	41	8	3	male	0.58	0.21
zhen	42	✓	29	12	1	male	0.64	0.31

Appendix

Table A.4 : Pinyin-based gender label inference on the held-out set. For each pinyin given name, we report the number of associated hanzi names (

n_{h}

), whether the name is single-character (SC), the hanzi-based gender label counts, the pinyin-based supermajority gender label (

\hat{y}_{p}

), normalized entropy, and disagreement. Entropy measures how mixed the hanzi-based labels are within a pinyin name. Disagreement measures the share of hanzi-based labels that differ from

\hat{y}_{p}

Panel of Figure 19 — Figure A.10 : Per-class precision and recall of pinyin-based gender classification ( $N{=}32$ ). Confusion matrices comparing our supermajority method against two LLMs ( Claude , GPT ) and Namsor on the held-out test set. (a) Precision: our approach leads for male ( $0.50$ ) and female ( $1.00$ ), and ranks second for neutral ( $0.46$ , behind GPT ’s $1.00$ ). (b) Recall: our method achieves the highest male recall ( $0.73$ ) and neutral recall ( $0.46$ ), though all methods show substantially weaker performance than their hanzi-based counterparts ( Figure A.7 ).

Panel of Figure 20 — Figure A.11 : Pairwise agreement between pinyin-based gender classification methods ( $N{=}32$ ). (a) Accuracy and (b) Cohen’s $\kappa$ for all method pairs, evaluated against the ground-truth labels of each pinyin’s associated hanzi on the held-out test set. Our method achieves the highest agreement with GT (accuracy $=0.53$ , $\kappa=0.27$ ), followed by Claude ( $0.47$ , $0.20$ ), with GPT ( $0.38$ , $0.18$ ) and Namsor lowest ( $0.38$ , $0.10$ ). All values are lower than their hanzi-based counterparts ( Figure A.6 ), reflecting the information loss inherent in pinyin-based inference.

Panel of Figure 21 — Figure A.12 : Gender label distribution and agreement across eleven inference methods ( $\hat{l}_{p}$ , $N=666$ ). Comparison of gender label inference for $666$ unique Chinese author given names in APS across three baseline methods (Karimi et al. [ 41 ] , Saeta [ 63 ] , and van Buskirk et al. [ 76 ] ), three commercial tools ( Claude , GPT , and Namsor ), and our pinyin supermajority (PY) and hanzi-based (HZ) classification models. The baselines infer gender from available pinyin, while the commercial tools and our models each independently classify from both scripts. (a) Distribution of predicted gender labels by method. (b) Pairwise classification agreement. (c) Pairwise Cohen’s $\kappa$ , measuring agreement beyond chance. Claude (HZ) achieves the highest agreement with our HZ model in both raw accuracy (b) and chance-corrected $\kappa$ (c), followed by GPT (HZ) and our PY method.

Panel of Figure 22 — Figure A.13: Prompt preamble. Role assignment and script-specific domain knowledge. Instructions unique to the Pinyin ( py ) and Hanzi ( hz ) variants are marked inline; all other text is shared.

Panel of Figure 23 — Figure A.14: Numeric scoring block. Continuous gender score definition appended to the preamble (Figure A.13 ) in the numeric inference mode.

Panel of Figure 24 — Figure A.15: Categorical labeling block. Discrete gender labels appended to the preamble in the categorical inference mode.

Panel of Figure 25 — Figure A.16: Uncertainty and output schema. Uncertainty is defined as intrinsic name ambiguity, explicitly distinguished from model confidence. The JSON schema ensures lossless round-tripping across batches of up to 100 names. This block is shared by both inference modes.

Bibliometric Analysis

Bibliometric analysis uses publication, citation, collaboration, and text data to map and measure the structure and evolution of research.

See all 35 papers

Oct 13, 2026

Pre-Trained Language Models as Zero-Shot Tools for Social Science Research

Record linkage and demographic inference from names are routine demands in social science. Most approaches to these problems require labeled training data that are difficult or costly to acquire. I show that pre-trained language models perform both tasks accurately without requiring task-specific training. For record linkage, I introduce EnsembleLink, an algorithm that uses pre-trained embeddings alongside string-based scorers to retrieve a short list of candidates and then identify the likeliest matches. EnsembleLink matches or exceeds the performance of most prior approaches while generalizing across scripts and formats that those methods are unable to handle. For ethnicity inference from names, I show that large language models (LLMs) achieve up to 83.8% accuracy on U.S. voter files, compared to 78.0% for name-supplemented BISG. Open-weight models offer reproducible implementations that approach these models in accuracy. Moreover, LLMs are able to infer group membership for religious, caste-based, and other ethnic group categories outside of the United States, and reduce bias in downstream estimates of geographic racial gaps. Where pre-training encodes the world knowledge required by each task, task-specific labels add less information.

Noah Dasanaike

PhD Candidate, Department of Government, Harvard University.

Entity Resolution Zero-Shot Learning

Ask PDF

Apr 12, 2026

NameBERT: Scaling Name-Based Nationality Classification with LLM-Augmented Open Academic Data

Inferring nationality from personal names is a critical capability for equity and bias monitoring, personalization, and a valuable tool in biomedical and sociological research. However, existing name-based nationality classifiers are typically trained on relatively small or source-specific labeled datasets, which can introduce coverage gaps and limit performance for underrepresented countries. While large language models (LLMs) demonstrate strong zero-shot performance for name-based nationality prediction, their computational cost and latency make them impractical for real-time, large-scale deployment. In this work, we created a large-scale name-nationality dataset from the Open Academic Graph (OAG) and introduce a framework that leverages LLMs as dataset enrichers rather than inference engines. We augment low-resource countries with LLM-generated names and evaluate on real and synthetic-tail test sets. We find that augmentation produces large gains when evaluation includes synthetic tail names and still offers a modest lift on tail-country metrics otherwise. Overall, NameBERT models achieve significantly higher accuracy than state-of-the-art baselines across both in- and out-of-domain tasks, while remaining efficient for large-scale inference compared to LLMs.

Cong Ming, Ruixin Shi, Yifan Hu

Northeastern University, United States

Bidirectional Encoder Representations from Transformers Large Language Model-Based Data Augmentation Data Augmentation

Ask PDF

Sep 14, 2026

How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions

When foundation models describe people, recent work in AI fairness, accessibility, and ethics recommends avoiding inferred identity labels (e.g., "she", "his") in favor of seemingly "objective" physical descriptions (e.g., "short hair", "a defined jawline"). Yet whether such descriptive language achieves gender-neutral communication remains an open empirical question. To study this, we introduce GAPA (Gender Associations of Physical Attributes), a dataset of 316 common physical attributes drawn from diverse sources, paired with 14,706 gender-association ratings from 304 US-based annotators. Results show that physical descriptions carry structured and graded gender associations among readers, with more consistent and distinctive associations for women and men than for non-binary identities. Next, we evaluate 16 LLMs across model families, sizes, and post-training variants against human ratings. The models partially recover human associations but exhibit systematic alignment biases, including compressed rating distributions, weaker alignment for associations with men, and asymmetric abstention that disproportionately targets the non-binary category. Finally, we release the best-performing proxy model trained to predict humans' gender associations of descriptive language and demonstrate its utility through a sociolinguistic analysis of character descriptions in LitBank. Together, our findings provide the first empirical evidence that seemingly "objective" physical descriptions can retain systematic gender associations in human interpretation, and uncover systematic patterns of model-human misalignment. This challenges the assumption that replacing explicit gender labels with physical descriptions necessarily yields gender-neutral communication, and highlights downstream challenges in using such descriptions to communicate subjective identity categories in human-AI interaction.

Yingjia Wan, Lin Lin, Elisa Kreiss

Department of Communication University of California, Los Angeles (UCLA) · Department of Computer Science University of California, Los Angeles (UCLA)

Gender Bias in Language Models Language Model Bias Evaluation Linguistic Bias in Language Models

Ask PDF

Abstract

Bibliometric Analysis

Explore similar work

Pre-Trained Language Models as Zero-Shot Tools for Social Science Research

NameBERT: Scaling Name-Based Nationality Classification with LLM-Augmented Open Academic Data

How Humans and LLMs Read Gender into "Gender-Neutral" Physical Descriptions