Profile-Based Auditing of Domestic Violence Survey Selection Across South Asia
Organizations: Institute of Advanced Research, United International University, Dhaka, Bangladesh · Dept of Computer Science & Engineering, United International University, Dhaka, Bangladesh
Abstract
Domestic violence (DV) data in the Demographic and Health Surveys (DHS) are collected through a specialized module whose household-level sampling and privacy requirements create a distinct observation process. We ask whether demographic, socioeconomic, and household characteristics form reproducible population profiles and whether these profiles contain information about selection into the DV module across South Asia. Using DHS Individual Recode data for 796,374 women aged 15-49 across Afghanistan, India, Myanmar, Nepal, and Pakistan, we harmonize 61 numeric and 8 categorical features, reduce the profile space with PCA, and identify six interpretable profiles. We then evaluate 22 machine-learning models under primary-sampling-unit-aware holdout validation. The best model, HistGradientBoosting, achieves ROC-AUC 0.9167 and PR-AUC 0.9612, versus 0.8475 and 0.9203 for a design-only baseline. Adding woman-level characteristics yields the largest incremental gain, while explicit interaction terms add little out-of-sample performance. Country-specific analyses reveal substantial heterogeneity, and leave-one-country-out evaluation shows that transferability of the incremental profile signal is limited. These results demonstrate that interpretable ML can audit differential coverage in sensitive survey modules beyond the formal sampling design.
Figures & tables
| Country | Women | Selected | Selected (%) | Interviewed | Interviewed/Selected (%) |
|---|---|---|---|---|---|
| Afghanistan | 29,461 | 22,132 | 75.12 | 21,324 | 96.35 |
| India | 724,115 | 494,830 | 68.34 | 72,320 | 14.62 |
| Myanmar | 12,885 | 4,563 | 35.41 | 4,530 | 99.28 |
| Nepal | 14,845 | 5,192 | 34.97 | 5,178 | 99.73 |
| Pakistan | 15,068 | 4,216 | 27.98 | 4,085 | 96.89 |
| Total | 796,374 | 530,933 | 66.67 | 107,437 | 20.24 |
| Feature representation | HistGradientBoosting | Logistic | LightGBM |
|---|---|---|---|
| Design | 0.8287 | 0.8264 | 0.8285 |
| Design + country | 0.8516 | 0.8479 | 0.8511 |
| woman characteristics | 0.9161 | 0.9013 | 0.9156 |
| household characteristics | 0.9166 | 0.9020 | 0.9144 |
| agency | 0.9167 | 0.9021 | 0.9139 |
| norms | 0.9167 | 0.9021 | 0.9139 |
Appendix figures & tables40 assets
Supplementary material from the paper’s appendix.
Appendix
| Country | Survey | Recode | Primary unit | Analysis role |
|---|---|---|---|---|
| Afghanistan | 2015 | HR/PR/IR | Household / woman | Included |
| Bangladesh | 2022 | HR/IR | Household / woman | IR audit; excluded from models |
| India | 2019–21 | PR/IR | Member / woman | Included |
| Myanmar | 2015–16 | PR/IR | Member / woman | Included |
| Nepal | 2022 | PR/IR | Member / woman | Included |
| Pakistan | 2017–18 | PR/IR | Member / woman | Included |
| Block | Principal variables and derived measures |
|---|---|
| Design | Eligible women, reciprocal eligible-women count, single-eligible-woman indicator, household size, children under five, crowding, capped eligibility categories. |
| Country | Country and survey context, including survey year representation. |
| Woman | Age, education, marital and union status, age at cohabitation, fertility, parity, pregnancy, child-marriage indicators, partner age and education, and related gaps. |
| Household | Urban residence, wealth, assets, water, sanitation, fuel, dwelling quality, household-head characteristics, and relationship to household head. |
| Agency | Employment, occupation, earnings control, household decision-making, asset ownership, healthcare barriers, insurance, mobile phone, financial accounts, internet, and media exposure. |
| Norms | Indicators and derived indices representing attitudes toward wife beating. |
| Meaning | |
|---|---|
| 0 | Not selected for the DV module |
| 1 | Selected and interviewed |
| 2 | Selected but privacy was not possible |
| 3 | Selected but not interviewed for another reason |
| Country | Women | Selected | Selected (%) | Interviewed among selected (%) |
|---|---|---|---|---|
| Afghanistan | 29,461 | 21,323–21,325 | 75.12% | 96.35% |
| India | 724,115 | 494,824–494,896 | 68.34% | 14.6% |
| Myanmar | 12,885 | 4,562–4,564 | 35.41% | 99.3% |
| Nepal | 14,845 | 5,191–5,192 | 34.97% | 99.7% |
| Pakistan | 15,068 | 4,216–4,217 | 27.98% | 96.9% |
| Total | 796,374 | 530,933 | 66.67% | – |
| Target | Model | ROC-AUC | PR-AUC |
|---|---|---|---|
| Completed, all women | Logistic regression | 0.7350 | 0.4340 |
| Completed, all women | Random forest | 0.7377 | 0.4367 |
| Completed, all women | LightGBM | 0.7332 | 0.4317 |
| Interviewed selected | Logistic regression | 0.6607 | 0.5307 |
| Interviewed selected | Random forest | 0.6600 | 0.5272 |
| Interviewed selected | LightGBM | 0.6638 | 0.5330 |
| Profile | Interpretation | Key cross-country pattern |
|---|---|---|
| P0 | Middle rural, no/low education, high parity | Most prevalent in Nepal; also substantial in Pakistan and India. |
| P1 | Richer urban, higher education | Common in India, Nepal, and Pakistan; uncommon in Afghanistan. |
| P2 | Middle rural, secondary education | Broadly distributed and particularly prevalent in India and Nepal. |
| P3 | Poorer rural, secondary education, never-married | Especially represented in Myanmar and India; nearly absent in Afghanistan and Pakistan. |
| P4 | Poorer rural, no/low education, high parity | Dominant in Afghanistan and strongly represented in Pakistan and Myanmar. |
| P5 | Richer urban, higher education, never-married | More represented in Myanmar, Nepal, and India; nearly absent in Afghanistan and Pakistan. |
| Country | P0 | P1 | P2 | P3 | P4 | P5 |
|---|---|---|---|---|---|---|
| Afghanistan | 11.7% | 4.3% | 5.4% | 0.1% | 78.1% | 0.3% |
| India | 21.0% | 20.9% | 22.5% | 12.5% | 11.4% | 11.8% |
| Myanmar | 9.9% | 12.9% | 17.9% | 19.5% | 25.4% | 14.3% |
| Nepal | 28.7% | 19.0% | 21.6% | 10.1% | 8.5% | 12.2% |
| Pakistan | 22.6% | 21.5% | 21.5% | 0.3% | 33.7% | 0.4% |
| Model | ROC-AUC | PR-AUC | F1 |
|---|---|---|---|
| Majority class | 0.5000 | 0.6670 | 0.800 |
| Stratified random | 0.5016 | 0.6677 | 0.668 |
| Design-only baseline | 0.8475 | 0.9203 | 0.784 |
| Logistic regression | 0.9022 | 0.9547 | 0.840 |
| Linear SVM | 0.9024 | 0.9547 | 0.835 |
| Linear discriminant analysis | 0.9031 | 0.9550 | 0.860 |
| Specification | Added block | ROC-AUC | PR-AUC | ROC-AUC | PR-AUC | Interpretation |
|---|---|---|---|---|---|---|
| Design-only | Design | 0.8287 | 0.8986 | – | – | Sampling mechanism |
| Design + country | Country | 0.8516 | 0.9250 | +0.0229 | +0.0264 | Country variation |
| Design + demographics | Woman | 0.9161 | 0.9609 | +0.0874 | +0.0623 | Large additional signal |
| * household | Household | 0.9166 | 0.9611 | +0.0879 | +0.0625 | Marginal additional signal |
| agency + norms | Agency + norms | 0.9167 | 0.9612 | +0.0880 | +0.0626 | Performance plateau |
| interactions | Explicit interactions | 0.9165 | 0.9611 | +0.0878 | +0.0625 | Negligible incremental gain |
| Interaction | OR | 95% CI lower | 95% CI upper |
|---|---|---|---|
| Age union status | 0.241 | 0.226 | 0.258 |
| Eligible women education | 1.244 | 1.188 | 1.303 |
| Eligible women wealth | 0.806 | 0.757 | 0.859 |
| Wealth education | 1.235 | 1.170 | 1.304 |
| Urban wealth | 0.900 | 0.850 | 0.954 |
| Eligible women urban | 1.084 | 1.041 | 1.128 |
| Country | Profile model ROC-AUC | Design ROC-AUC | Gain |
|---|---|---|---|
| India | 0.9196 | 0.8486 | +0.0710 |
| Afghanistan | 0.8713 | 0.8746 | -0.0033 |
| Pakistan | 0.6102 | 0.6268 | -0.0166 |
| Nepal | 0.6331 | 0.6642 | -0.0311 |
| Myanmar | 0.6427 | 0.6771 | -0.0344 |
| Held-out country | Transfer ROC-AUC |
|---|---|
| Afghanistan | 0.8719 |
| India | 0.8374 |
| Myanmar | 0.6668 |
| Nepal | 0.6484 |
| Pakistan | 0.6219 |
| Mean transfer penalty | -0.0061 |
Selection Bias
Selection bias studies how nonrandom sampling or observation distorts statistical inference, model training, and evaluation relative to a target population.
See all 48 papers