cs.CLOct 16, 2026
SaveBridging the Language Gap in Scholarly Data II: Enhancing Gender Inference for Chinese Names
Organizations: Graz University of Technology, Graz, Austria · Complexity Science Hub, Vienna, Austria · Maastricht University, Maastricht, The Netherlands
Abstract
Perceived gender inference from names is important for inclusive scientometrics research, yet existing methods perform poorly on non-Western scholars. We address this gap for Chinese names, which appear in two forms, Hanzi and Pinyin, its Romanized form (e.g., Zhang Wei, Li Na). In international bibliographic records, names are often available only in Pinyin. However, a single pinyin name can correspond to multiple distinct hanzi characters with different gender associations, so gender inference from Pinyin alone is intrinsically ambiguous. To address this, we first develop two hanzi models, a regression model that produces a continuous gender score from 0 (male) to 1 (female), and a classification model that outputs one of three labels (\em{male}, \em{neutral}, \em{female}). We train them on a sample of hanzi names from Chinese-language physics publications, rated by 11 native Chinese speakers. On this sample, the models achieve an RMSE of and an R of (regression), and macro-AUC of and a Cohen's of (classification), demonstrating high performance that is robust to class imbalance. For pinyin names, we propose a pipeline that aggregates hanzi predictions via transliteration and frequency weighting, together with an ambiguity score that flags names whose gender cannot be resolved from Pinyin alone. Benchmarked against three off-the-shelf tools (Namsor, GPT, and Claude), our models achieve the highest agreement with human annotations for both gender scores and gender labels, in both scripts. Applied to an independent sample of Chinese-named authors in APS journals, our models assign a gender to every name, whereas existing Pinyin-based methods leave between 20% and 100% of names unclassified. These findings demonstrate the importance of hanzi-level information for reliable gender inference of Chinese names. Code and data are publicly available.
Figures & tables
Figure 1: Pinyin transliteration and gender inference pipeline. (a) Transliteration process: A single pinyin syllable (e.g., “Deng”) maps to multiple hanzi characters, each with its own predominant gender association. (b) Gender inference strategy: First, we train CatBoost models to predict gender from hanzi characters: a regressor to produce continuous gender scores and a classifier to produce categorical labels. For pinyin names, we leverage the transliteration process and aggregate the predicted gender from all hanzi characters that map to that syllable.
Figure 3: Performance of hanzi-based gender score inference ( ). Predicted versus true scores on (a) the full ground-truth set ( , train and test combined) and (b) the held-out test set ( ). Points represent hanzi given names. Color indicates whether the name is single-character, and shape indicates whether features from all tools were available. Fit is summarized by , root mean squared error (RMSE), and Pearson correlation ( ). (c) CatBoost feature importance and (d) SHAP values both rank the aggregated score mean_ngender_rn and ngender_r as the dominant predictors. The binary indicator single_char and the categorical indicator gender_w contribute minimally.
Figure 4: Performance of pinyin-based gender score inference ( ). True scores (y-axis) versus predicted values (x-axis) on (a) the full ground-truth set (train and test combined) and (b) the held-out test set. Points represent pinyin names transliterated from the hanzi ground-truth dataset. True values are the mean score from the human annotations on the hanzi version of each name. Color indicates whether the name is single-character, and shape indicates whether features from all tools were available. Each column corresponds to one aggregation strategy ( Section 3.2.2 ). Some panels show fewer name pairs because of our single-character augmentation ( Section 3.1.1 ). Fit is summarized by , root mean squared error (RMSE), and Pearson correlation ( ).
Figure 5: Ambiguity of pinyin-level gender score aggregation. A sample of pinyin names with varying gender associations, shown across the four aggregation strategies. Circles denote the aggregated pinyin-level score ( ), with error bars spanning the 10th to 90th percentile ( , ) of the hanzi-level predictions. Color encodes the ambiguity of the aggregation, defined as the width of this interval ( ). Grey crosses mark the individual gender scores of all hanzi variants associated with each pinyin name. The number of hanzi variants ( ) is reported next to each name. Names with high ambiguity (warm colors, wide error bars) indicate conflicting gender associations across hanzi variants, warranting caution when interpreting the aggregated score.
Figure 6: Comparison of gender score predictions across methods. Mean annotator scores (y-axis) versus predicted gender scores (x-axis) for our model and three off-the-shelf methods ( Namsor , GPT 5.4 Nano , Claude Opus 4.8 ), using (a) hanzi-based and (b) pinyin-based inference. Each point represents one of 34 held-out names; the dashed line indicates perfect agreement. Our model achieves the highest and lowest RMSE in both settings.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Journal | hz | en | Papers |
| Acta Physica Sinica | ✓ | 19,730 | |
| Chinese Physics B | ✓ | 8,813 | |
| College Physics | ✓ | 6,081 | |
| Chinese Journal of Atomic and Molecular Physics | ✓ | ✓ | 5,743 |
| Wuli (Physics) | ✓ | 5,379 | |
| Chinese Journal of Luminescence | ✓ | 2,719 |
Table A.1 : Journals published by the Chinese Physical Society included in the CNKI corpus. For each journal we report the language of its articles ( hz = hanzi, en = English) and the number of papers indexed up to 2024-02-14. Journals marked in both columns publish in both languages, and counts are pooled across former titles of the same journal. The column totals records, of which carry a parseable Chinese personal name and enter the analysis.
Figure A.1 : Author name disambiguation pipeline. A name associated with more than one paper is treated as ambiguous. Each pair of papers sharing that name is tested against co-authorship, affiliation, content, and citation criteria; a match on any criterion merges the pair into one identity, and pairs matching none are separated into distinct identities. The disambiguation stage is shaded in dark gray. Adapted from [ 69 ] .
| # | Title | Year | Comp. appr. | DB | Code | New meth. | Meth. comp. | Ch. names |
| 1 | Comparison and benchmark of name-to-gender inference services [ 64 ] | 2018 | ✓ | ✓ | ✓ | ✕ | ✓ | ✕ |
| 2 | Augmenting Data Quality through High-Precision Gender Categorization [ 55 ] | 2019 | ✓ | ✕ | ✕ | ✓ | ✕ | ✕ |
| 3 | The sound of gender: inferring the gender of names in a foreign language [ 15 ] | 2019 | ✕ | ✕ | ✕ | ✕ | ✕ | ✓ |
| 4 | The impact of gender orientation of names on individuals’ evaluation of impressions and interpersonal interaction [ 12 ] | 2021 | ✕ | ✕ | ✕ | ✕ | ✕ | ✓ |
| 5 | Performance of gender detection tools: a comparative study of name-to-gender inference services [ 65 ] | 2021 | ✓ | ✓ | ✕ | ✕ | ✓ | ✕ |
| 6 | Using genderize.io to infer the gender of first names: how to improve the accuracy of the inference [ 66 ] | 2021 | ✓ | ✓ | ✕ | ✓ | ✕ | ✕ |
Table A.2 : Chinese name–to–gender inference literature identified via Web of Science. A structured search in Clarivate Web of Science (WoS) yielded 14 records, of which 10 were retained after excluding studies that did not propose or evaluate methods for Chinese name gender inference. Columns indicate publication year and whether the study includes a computational approach (Comp. appr.), uses an external database (DB), provides code (Code), proposes a new method (New meth.), compares multiple methods (Meth. comp.), or explicitly focuses on Chinese names (Ch. names).
Figure A.2 : Agreement between tool-based gender inference and annotator-perceived gender. Comparison between gender scores produced by inference tools and the mean perceived gender assigned by human annotators (gold standard). Each point represents a name; the x-axis shows the average annotator score, and the y-axis the score produced by the tool or tool aggregation. Panels show individual tools (a) N and (b) R, and aggregated estimates obtained by averaging tools: (c) RNW, (d) RN, (e) RW, and (f) NW. Solid lines indicate linear fits; Pearson correlation ( ) and RMSE quantify agreement with the gold standard. Aggregating tools generally improves alignment with annotator judgments, with the RN combination showing the strongest correlation and lowest error. Panels involving tool W (c, e, f) show fewer names ( ) because rare single-character names were excluded, as W defaults to male for all such names.
| Pinyin | sc | min | max | std | Amb. | M CNKI | M CNKI+ | W CNKI | W Pop. | |||
| fang | 19 | 0.40 | ✓ | 0.28 | 0.65 | 0.11 | 0.37 | 0.21 | 0.42 | 0.41 | 0.56 | 0.61 |
| jie | 52 | 0.46 | ✓ | 0.27 | 0.67 | 0.09 | 0.40 | 0.23 | 0.43 | 0.38 | 0.40 | 0.36 |
| jin | 43 | 0.46 | ✓ | 0.26 | 0.62 | 0.10 | 0.36 | 0.27 | 0.37 | 0.40 | 0.36 | 0.40 |
| jun | 22 | 0.27 | ✓ | 0.25 | 0.60 | 0.09 | 0.35 | 0.22 | 0.35 | 0.36 | 0.33 | 0.34 |
| meng | 25 | 0.79 | ✓ | 0.28 | 0.62 | 0.10 | 0.34 | 0.25 | 0.47 | 0.40 | 0.55 | 0.56 |
Table A.3 : Pinyin-based gender score inference on the held-out set. For each of the 34 pinyin names in the test set, we report the number of associated hanzi names in our corpus ( ), the mean gold-standard gender score ( ), and whether the name is single-character ( sc ). For the hanzi-level predictions ( ), we include the range, standard deviation, spread ( ), and ambiguity (Amb.). We also report our four pinyin-level aggregation strategies ( ): unweighted averages over CNKI and CNKI+, and weighted averages using CNKI and Population frequencies ( Section 3.2.2 ).
Figure A.3 : Hanzi- vs. pinyin-based gender score inference. Hanzi-based predictions (x-axis) versus pinyin-based predictions (y-axis) for each aggregation strategy: (a) Mean CNKI, (b) Mean CNKI+, (c) Weighted CNKI by author counts, and (d) Weighted by population counts. Each point represents one unique hanzi given name in the CNKI corpus. Color encodes the ambiguity of the aggregated pinyin-level gender score, defined as the difference between the 90th and 10th percetiles ( ) of the hanzi-level predictions for a given pinyin name, binned into intervals ( , , , , , , ). The dashed line indicates perfect agreement. Each panel reports goodness of fit ( ) and error metrics (MAE, MSE, RMSE). Mean CNKI+ (b) yields the highest agreement among all strategies (lowest RMSE). Panels (a) and (c) show fewer name pairs because of the single-character augmentation ( Section 3.1.1 ).
Figure A.4 : Discretization of gender scores. Aggregated human annotations were mapped into the classes male , neutral , and female using different discretization strategies. (a–c) Equal-width, equal-size, and visually derived cuts based on the mean. (d–e) Visual cuts using the median and mode, respectively. (f) Fuzzy cut following annotation instructions ( = definitely male, = more likely male, = neutral, = more likely female, = definitely female). (g) Conservative fuzzy cut assigning 20% to male and female extremes. (h) Conservative fuzzy cut assigning 20% to the neutral middle. These discretizations were later used as target variables in model training.
Figure A.5 : Performance of hanzi-based gender label classification ( ). The best results were obtained with Cut 1 (mean-based equal-width discretization; see Figure A.4 a). (a,b) Confusion matrices for the full ground-truth set (train + test) and the test set, respectively. (c) CatBoost feature importance, showing the aggregated mean predictions from R and N ( mean_ngender_rn ) is the strongest predictor. (d,e) One-vs-rest ROC curves for the entire ground-truth set and the test set, respectively. (f) SHAP summary plot indicating consistent class-discriminative effects of the main aggregated predictors. Overall, the classifier achieves high and stable performance across classes, with predictive power primarily driven by features from tools R and N .
Figure A.6 : Pairwise agreement between hanzi-based gender classification methods. a) Accuracy and (b) Cohen’s for all method pairs, including ground truth (GT). Our method achieves the highest agreement with GT (accuracy , ), followed by Claude ( , ), while GPT ( , ) and Namsor ( , ) show the lowest agreement across all comparisons.
Figure A.7 : Per-class precision and recall of hanzi-based gender classification ( ). Confusion matrices comparing our method (Ours) against two LLMs ( Claude , GPT ) and Namsor on the held-out test set. (a) Precision: our approach leads for male ( ) and female ( ), and ranks second for neutral ( , behind Claude ’s ). (b) Recall: our method achieves the highest male recall ( ) and neutral recall ( ), and second highest for female ( ), matching Claude ; Namsor reaches only by over-predicting female (precision ).
Figure A.8 : Performance of pinyin-based gender classification ( ). (a,b) Confusion matrices on the full ground-truth set (train + test) and the test set, evaluated against the ground-truth labels of each pinyin’s associated hanzi. (c,d) One-vs-rest ROC curves on the same splits, using the hanzi-label proportions as class probabilities. Our supermajority classifier achieves high macro-AUC ( / ) but low Cohen’s ( / ), reflecting poor performance on the minority class.
Figure A.9 : Disagreement between pinyin- and hanzi-based gender labels. Left: each dot represents a pinyin name, colored by the inferred majority label and with transparency inversely proportional to the entropy of its underlying hanzi-label distribution—low entropy (opaque) indicates strong consensus among hanzi mappings, while high entropy (transparent) indicates mixed labels. The -axis measures the fraction of hanzi labels that disagree with the final pinyin label. Right: hanzi-label composition for each name, showing the proportion of male , neutral , and female labels across all hanzi mappings. Names shown are the same as in Figure 5 for direct comparison.
| Hanzi-based labels | Ambiguity | |||||||
| Pinyin | SC | Male | Neutral | Female | Entropy | Disagreement | ||
| shen | 35 | ✓ | 31 | 4 | 0 | male | 0.32 | 0.11 |
| jun | 22 | ✓ | 18 | 4 | 0 | male | 0.43 | 0.18 |
| qi | 84 | ✓ | 69 | 14 | 1 | male | 0.47 | 0.18 |
| jie | 52 | ✓ | 41 | 8 | 3 | male | 0.58 | 0.21 |
| zhen | 42 | ✓ | 29 | 12 | 1 | male | 0.64 | 0.31 |
Table A.4 : Pinyin-based gender label inference on the held-out set. For each pinyin given name, we report the number of associated hanzi names ( ), whether the name is single-character (SC), the hanzi-based gender label counts, the pinyin-based supermajority gender label ( ), normalized entropy, and disagreement. Entropy measures how mixed the hanzi-based labels are within a pinyin name. Disagreement measures the share of hanzi-based labels that differ from .
Figure A.10 : Per-class precision and recall of pinyin-based gender classification ( ). Confusion matrices comparing our supermajority method against two LLMs ( Claude , GPT ) and Namsor on the held-out test set. (a) Precision: our approach leads for male ( ) and female ( ), and ranks second for neutral ( , behind GPT ’s ). (b) Recall: our method achieves the highest male recall ( ) and neutral recall ( ), though all methods show substantially weaker performance than their hanzi-based counterparts ( Figure A.7 ).
Figure A.11 : Pairwise agreement between pinyin-based gender classification methods ( ). (a) Accuracy and (b) Cohen’s for all method pairs, evaluated against the ground-truth labels of each pinyin’s associated hanzi on the held-out test set. Our method achieves the highest agreement with GT (accuracy , ), followed by Claude ( , ), with GPT ( , ) and Namsor lowest ( , ). All values are lower than their hanzi-based counterparts ( Figure A.6 ), reflecting the information loss inherent in pinyin-based inference.
Figure A.12 : Gender label distribution and agreement across eleven inference methods ( , ). Comparison of gender label inference for unique Chinese author given names in APS across three baseline methods (Karimi et al. [ 41 ] , Saeta [ 63 ] , and van Buskirk et al. [ 76 ] ), three commercial tools ( Claude , GPT , and Namsor ), and our pinyin supermajority (PY) and hanzi-based (HZ) classification models. The baselines infer gender from available pinyin, while the commercial tools and our models each independently classify from both scripts. (a) Distribution of predicted gender labels by method. (b) Pairwise classification agreement. (c) Pairwise Cohen’s , measuring agreement beyond chance. Claude (HZ) achieves the highest agreement with our HZ model in both raw accuracy (b) and chance-corrected (c), followed by GPT (HZ) and our PY method.
Figure A.13: Prompt preamble. Role assignment and script-specific domain knowledge. Instructions unique to the Pinyin ( py ) and Hanzi ( hz ) variants are marked inline; all other text is shared.
Figure A.14: Numeric scoring block. Continuous gender score definition appended to the preamble (Figure A.13 ) in the numeric inference mode.
Figure A.15: Categorical labeling block. Discrete gender labels appended to the preamble in the categorical inference mode.
Figure A.16: Uncertainty and output schema. Uncertainty is defined as intrinsic name ambiguity, explicitly distinguished from model confidence. The JSON schema ensures lossless round-tripping across batches of up to 100 names. This block is shared by both inference modes.
More in this topic
Bibliometric Analysis
Bibliometric analysis uses publication, citation, collaboration, and text data to map and measure the structure and evolution of research.
See all 35 papers