Anthropogenic Regional Adaptation in Multimodal Vision-Language Model

Samuel Cahyawijaya; Peerat Limkonchotiwat; Tack Hwa Wong; Hitesh Laxmichand Patel; Amit Agarwal; Manuel Antonio Rufino; Carlos Rafael Catalan; Muhammad Reza Qorib

cs.AIApr 13, 2026

Anthropogenic Regional Adaptation in Multimodal Vision-Language Model

Authors: Samuel Cahyawijaya, Peerat Limkonchotiwat, Tack Hwa Wong, Hitesh Laxmichand Patel, Amit Agarwal, Manuel Antonio Rufino, Carlos Rafael Catalan, Muhammad Reza Qorib, Vicky Feliren, Holy Lovenia, Aye Hninn Khine, Frederikus Hudi, David Anugraha, Alham Fikri Aji, Romrawin Chumpu, Viet-Thanh Pham, Minghan Wang, Mohamed Fazli Imam, Ruochen Zhang, Joseph Marvin Imperial, Khumaisa Nur'aini, Do Xuan Long, Musa Izzanardi Wijanarko, Joel Ruben Antony Moniz, Patrick Amadeus Irawan, Hanif Muhammad Zhafran, Isaiah Flores, Salsabila Zahirah Pranida, Jun Kevin, Jostin Jerico Rosal, Patricia Nicole Monderin, Kun Kerdthaisong, Ahmad Mustafid, My Chiffon Nguyen, Natchapon Jongwiriyanurak, Siva Worajitwannakul, Haochen Li, Adrian Xuan Wei Lim, Bin Wang, Muhammad Ravi Shulthan Habibi, Lynnette Hui Xian Ng, Mithil Bangera, Yeshil Bangera, Priyaranjan Pattnayak, Dun Li Chan, Sherissa Caren Djuniwar, Cho Chan Myei Oo, Hee Ming Shan

Abstract

While the field of vision-language (VL) has achieved remarkable success in integrating visual and textual information across multiple languages and domains, there is still no dedicated framework for assessing human-centric alignment in vision-language systems. We offer two contributions to address this gap. First, we introduce Anthropogenic Regional Adaptation: a novel paradigm that aims to optimize model relevance to specific regional contexts while ensuring the retention of global generalization capabilities. Second, we present a simple, but effective adaptation method named Geographical-generalization-made-easy (GG-EZ), which utilizes regional data filtering and model merging. Through comprehensive experiments on 3 VL architectures: large vision-language models, text-to-image diffusion models, and vision-language embedding models, and a case study in Southeast Asia (SEA) regional adaptation, we demonstrate the importance of Anthropogenic Regional Adaptation and the effectiveness of GG-EZ, showing 5-15% gains in cultural relevance metrics across SEA while maintaining over 98% of global performance and even occasionally surpassing it. Our findings establish Anthropogenic Regional Alignment as a foundational paradigm towards applicability of multimodal vision-language models in diverse regions and demonstrate a simple-yet-effective baseline method that optimizes regional value alignment while preserving global generalization.

Figures & tables

Panel of Figure 1 — Figure 1: Through anthropogenic regional adaptation, we identify two primary model archetypes: (left) Global model with strong overall global performance, but struggle to represent certain regions appropriately, and (right) Regional-specific model that has a strong representation towards certain regions, but fall short on the global context.

Figure 2: Overview of our Geographical-generalization-made-easy (GG-EZ) framework. Our framework consist of 3 constituents: (1) High-quality regional data filtering pipeline; (2) supervised fine-tuning to create a high quality regional-specific model; and (3) model merging to capture the best combination between regional-specific and global represention while also maintaining the generalization capabilities of the model.

Model	GRP	Global			SEA-Specific
Model	GRP	Avg.	WC	CVQA	Avg.	SEAVQA	WC	CVQA
Google Gemma-3	59.4	63.5	59.8	67.2	56.3	41.0	60.1	67.8
SEA-Gemma-3 5%	64.0	64.3	60.0	68.7	63.7	61.2	60.3	69.5
SEA-Gemma-3 10%	64.1	64.4	60.0	68.8	63.8	61.7	60.2	69.5
SEA-Gemma-3 50%	57.3	56.7	51.6	61.8	57.8	59.5	51.4	62.6
SEA-Gemma-3 70%	56.1	56.3	51.9	60.6	56.0	54.0	52.6	61.3

Table 1: SEA-VLM evaluation results on CVQA, SEA-VQA, and WorldCuisine (WC). <MODEL> (X%) denotes a merged model where X% is the weight

\beta

use in the linear merging process of the region-specific model.

Model	Overall			Language Breakdown
Model	GRP	Global	SEA	fil	ind	tha	vie	zsm
Google Gemma-3	2.29	2.54	2.09	1.69	2.15	2.17	2.37	2.07
SEA-Gemma-3 10%	2.31	2.42	2.22	1.88	2.07	2.29	2.61	2.25
SEA-Gemma-3 (w/o merge)	1.74	1.18	2.23	2.75	2.29	2.33	1.76	2.00

Table 2: SEA-VLM human evaluation on SEA-AYA and SEA-VL VQA datasets. We report the average rank across different models (Higher is better). <MODEL> (X%) denotes a linear merge where X% is the weight

\beta

of the region-specific model.

Model	Overall	Aspects
Model	Score	Attribute	Relation	Entity	Other	Global
StabilityAI SDXL	73.75	79.18	86.38	81.07	60.00	83.89
SEA-SDXL 25%	74.75	80.43	86.34	81.86	61.60	85.41
SEA-SDXL 50%	74.61	80.93	87.12	81.15	60.00	86.02
SEA-SDXL 75%	74.39	79.98	85.96	81.36	63.20	82.67
SEA-SDXL (w/o merge)	74.32	80.79	86.92	81.57	65.20	81.16

Table 3: SEA-SDXL Performance on Image Generation Benchmark DPGBench (Higher is better). <MODEL> (X%) denotes a linear merge where X% is the weight

\beta

of the region-specific model.

Model	Correctness				Naturalness
Model	Overall	T	L	C	Overall	T	L	C
StabilityAI SDXL	1.491	1.470	1.636	1.387	1.675	1.436	2.023	1.613
SEA-SDXL 25%	1.569	1.587	1.729	1.413	1.767	1.473	2.124	1.753
SEA-SDXL (w/o merge)	1.431	1.473	1.527	1.307	1.557	1.340	1.806	1.560

Table 4: SEA-SDXL human evaluation results on 3 distinct cultural aspects. “T” denotes Tradition, “L” denotes Landmark, and “C” denotes Cuisine

Model	GRP	CVQA			SEAVQA
Model	GRP	Global	SEA	Others	SEA	Khm	Ind	Vie
Google SigLIP2	25.17	25.51	24.02	25.84	25.81	28.95	25.00	25.56
SEA-SigLIP2 25%	24.96	24.38	24.44	24.36	26.36	27.63	25.40	23.96
SEA-SigLIP2 50%	27.10	27.52	25.50	27.97	28.06	31.25	26.99	27.16
SEA-SigLIP2 75%	27.96	27.12	27.51	27.03	29.66	28.62	30.05	28.75
SEA-SigLIP2 (w/o merge)	26.96	26.75	25.29	27.07	28.96	28.29	28.59	29.07

Table 5: SEA-VL Embedding Zero-Shot Performance on Cultural VQA Benchmarks. <MODEL> (X%) denotes a linear merge where X% is the weight

\beta

of the region-specific model. For SEA-VQA, we provide additional breakdown of 3 of the biggest country data in the dataset with “Khm”=Cambodia, “Ind”=Indonesia, “Vie”=Vietnam.

Figure 3: Impact of regional-specific data curation strategy on SEA-Gemma-3.

Panel of Figure 4 — Figure 4: (left) Impact of globalization factor $alpha$ to GRP across different models. Optimizing on a misaligned $\alpha$ can lead to suboptimal performance. (right) We derive $\alpha$ from the KOF globalization index [ 26 , 25 ] to better reflect the degree of globalization across regions. The globalization index is distinct across regions and evolves over time.

Language	Model	Grammar	Naturalness
Burmese (mya)	NLLB	4.6 $\pm$ 0.8	4.1 $\pm$ 1.1
	Gemma-3-27b	4.1 $\pm$ 1.2	3.6 $\pm$ 1.2
	Gemini-2.5-flash	4.8 $\pm$ 0.6	4.6 $\pm$ 0.8
	Gemini-2.5-pro	4.9 $\pm$ 0.4	4.9 $\pm$ 0.5
Thai (tha)	NLLB	3.3 $\pm$ 1.1	3.0 $\pm$ 1.2
	Gemma-3-27b	4.2 $\pm$ 0.9	4.1 $\pm$ 0.9

Table 6: Human evaluation of English→5 SEA languages translation quality using a five-point scale (mean ± standard deviation)

Reward Model	Rate where $s^{a}_{rm}>s^{b}_{rm}$
ImageReward	0.394
HPSv2	0.384
VisionReward-Image	0.466
UnifiedReward	0.442

Table 7: Rate at which

s^{a}_{rm}>s^{b}_{rm}

for each reward model under consideration.

	Llama-3.2	Pixtral	Gemma-3	SEA-Gemma-3	SEA-Gemma-3	Aya-Vision	Qwen-3 VL	Qwen-2.5-VL	Molmo
Language	Vision (11B)	(12B)	(27B)	(27B)	10% (27B)	(32B)	(35B)	(72B)	(72B)
Indonesian	56.31	62.86	66.50	21.60	66.26	67.88	66.26	66.50	63.83
Filipino	51.72	64.53	66.01	24.14	70.94	66.34	64.04	65.02	64.53
Malay	56.19	61.90	67.62	29.84	71.43	72.38	70.16	62.50	69.84
Chinese	62.80	68.93	72.66	40.73	75.14	77.48	78.28	80.98	79.29
Tamil	58.41	51.87	73.36	40.65	74.30	61.68	61.68	58.88	58.41

Table 8: Comparison of our best model (SEA-Gemma-3 10%) againsts other competitor models and the regionally-adapted Gemma-3 (SEA-Gemma-3) model on CVQA.

	Gemma-3	SEA-Gemma-3	SEA-Gemma-3	Aya-Vision	Qwen-3 VL
Region	(27B)	(27B)	10% (27B)	(32B)	(35B)
Cambodia	50.7	40.5	52.3	56.25	50.99
Indonesia	43.5	49.3	69.0	66.09	66.09
Laos	0.0	50.0	68.1	45.83	63.89
Malaysia	37.5	44.4	62.4	69.31	60.32
Philippines	63.4	46.6	58.8	63.40	56.21

Table 9: Comparison of our best model (SEA-Gemma-3 10%) againsts other competitor models and the regionally-adapted Gemma-3 (SEA-Gemma-3) model on SEA-VQA.

	Avg.	Country
Model	Score	Camb.	Indo.	Laos	Mala.	Phil.	Sing.	Thai.	Viet.
Google Gemma-3	41.0	50.7	43.5	0.0	37.5	63.4	56.3	51.6	18.9
SEA-Gemma-3 5%	61.2	50.3	68.8	66.7	62.4	59.5	65.6	51.1	57.8
SEA-Gemma-3 10%	61.7	52.3	69.0	68.1	62.4	58.8	68.8	52.2	57.5
SEA-Gemma-3 50%	59.5	52.0	67.3	62.5	57.1	56.9	53.1	52.7	55.0
SEA-Gemma-3 70%	54.0	51.3	67.3	63.9	56.6	58.2	53.1	51.6	20.5

Table 10: SEA-VLM Zero-shot performance on SEA-VQA. The result is broken down per country. “Camb.” = Cambodia, “Indo.” = Indonesia, “Mala.” = Malaysia, ‘Phil.’ = Phillipines, “Sing.”=Singapore, “Thai.” = Thailand, “Viet.” = Vietname

	Overall	SEA	By Prompt Type
Model	Score	Avg.	Task 1 (NC)	Task 2	Task 1 (C)	Task 1 (Adv.)
Google Gemma-3	59.8	60.1	64.8	48.3	78.5	47.5
SEA-Gemma-3 5%	60.0	60.3	64.5	48.1	79.2	48.1
SEA-Gemma-3 10%	60.0	60.2	64.5	48.0	79.1	48.1
SEA-Gemma-3 50%	51.6	51.4	53.7	42.0	67.0	43.6
SEA-Gemma-3 70%	51.9	52.6	53.5	43.5	67.1	43.4

Table 11: SEA-VLM Zero-shot performance on World Cuisine. (NC) = No Context, (C) = Context, (Adv.) = Adversarial

Model	Overall SEA		Prompt Type
Model	Score	Avg.	English	Local Lang.
Gemma-3	67.2	67.83	67.96	66.44
SEA-Gemma-3 5%	68.72	69.45	69.39	68.05
SEA-Gemma-3 10%	68.76	69.45	69.42	68.09
SEA-Gemma-3 50%	61.84	62.62	62.9	60.79
SEA-Gemma-3 70%	60.58	61.27	61.51	59.66

Table 12: SEA-VLM Zero-shot performance on CVQA.

	Attribute					Entity			Other		Relation
Model	color	other	shape	size	texture	part	state	whole	count	text	non-spatial	spatial
StabilityAI SDXL	80.88	79.49	69.87	67.36	79.98	78.52	74.23	82.71	56.00	76.00	81.13	86.73
SEA-SDXL 25%	82.34	81.49	75.11	63.22	80.82	82.81	73.39	83.45	59.50	70.00	77.36	86.93
SEA-SDXL 50%	82.39	80.49	73.80	66.53	82.52	80.08	74.13	82.67	58.50	66.00	76.73	87.80
SEA-SDXL 75%	81.59	79.49	72.49	67.36	81.19	79.49	73.71	83.09	61.50	70.00	81.76	86.23
SEA-SDXL 100%	82.14	81.26	77.73	64.46	81.73	79.69	74.45	83.20	62.00	78.00	84.91	87.06

Table 13: Detailed per aspect breakdown (Attribute, Entity, Other, and Relation) of the DPGBench evaluation.

Model	SEAVQA Zero-Shot Performance
	Overall	Cambodia	Indonesia	Laos	Malaysia	Phillipines	Singapore	Thailand	Vietnam
Google SigLIP2	25.81	28.9	25.0	22.2	24.3	28.8	21.9	25.5	25.6
SEA-SigLIP2 25%	26.36	27.6	25.4	29.2	29.1	28.8	28.1	26.1	24.0
SEA-SigLIP2 50%	28.06	31.3	27.0	22.2	29.6	27.5	21.9	31.0	27.2
SEA-SigLIP2 75%	29.66	28.6	30.1	19.4	32.3	34.0	25.0	29.9	28.8
SEA-SigLIP2 100%	28.96	28.3	28.6	29.2	26.5	32.7	31.3	30.4	29.1

Table 14: Detailed per country zero-shot performance of the SEAVQA evaluation.

	Avg.	Country
Model	Score	Khm	Ind	Lao	Mys	Phl	Sgp	Tha	Vie
SEA-Mammoth 50k	13.0	9.9	18.8	11.1	11.1	11.8	18.8	14.67	2.88
SEA-Mammoth 250k	39.6	13.8	52.9	52.8	42.9	47.1	46.9	44.0	20.8
SEA-Mammoth 250k + CG OE	41.9	40.5	49.3	50.0	44.4	46.6	46.9	39.1	18.5
SEA-Mammoth 250k + CG MC	21.6	26.6	20.9	26.4	16.4	24.8	28.1	28.3	14.1
SEA-Mammoth 250k + WC	30.4	10.2	39.9	34.7	37.6	34.0	50.0	32.6	16.9

Table 15: SEA-VLM Zero-shot performance on SEA-VQA. The result is broken down per country. “Khm” = Cambodia, “Lao” = Laos, “Ind” = Indonesia, “Mys” = Malaysia, ‘Phl’ = Phillipines, “Sgp”=Singapore, “Th.” = Thailand, “Vie” = Vietnam. “CG” denotes CulturalGround, “WC” denotes WorldCuisine, while “OE” and “MC” denote open-ended and multiple-choice, respectively.

Panel of Figure 5 — Figure 5: Generated responses using different model architypes. From left-to-right: global model (Gemma-3), our regional model (SEA-Gemma-3), our merged model (SEA-Gemma-3 10%) along with the prompts. Our model produces the most correct image among others, while retaining the image naturalness and overall quality of the original Gemma-3.

Figure 6: Generated image using different model architypes. From left-to-right: global model (SDXL), our regional model (SEA-SDXL), our merged model (SEA-SDXL 25%), and reference of natural images.

Year	EAP	ECA	LAC	MENA	NA	SA	SEA	SSA	World
1993	49.32	58.82	51.20	57.00	68.59	28.05	36.04	34.37	48.86
1994	51.07	59.15	51.00	56.85	68.58	28.25	36.40	34.26	49.15
1995	51.40	59.72	51.67	56.62	68.66	28.39	37.34	34.13	49.42
1996	52.02	60.23	51.67	56.74	69.43	28.37	37.89	33.99	49.64
1997	51.68	60.43	51.75	56.79	69.48	28.35	37.76	33.43	49.53
1998	51.63	60.71	51.86	56.89	69.83	28.33	37.70	33.55	49.66

Table 16: Globalization index by region by yea from 1993 to 2023. “EAP” = East Asia and Pacific , ‘ECA” = Europe and Central Asia,“LAC” = Latin America and Caribbean, “MENA” Middle East and North Africa, “NA” = North America, “SA” = South Asia, “SEA” = Southeast Asian, “SSA” = Sub-Saharan Africa.

VLM Adaptation

Vision-language model adaptation tailors pretrained image-text and multimodal models to new tasks and domains while retaining transferable capabilities.

See all 545 papers

Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models
Sep 26, 2026Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan +1VLM Adaptation Vision-Language Models
University of Florida Gainesville, FL, USA
Chain-of-Adaptation: Surgical Vision-Language Adaptation with Reinforcement Learning
Mar 20, 2026Jiajie Li, Chenhui Xu, Meihuan Liu +1VLM Adaptation Medical VLMs
University at Buffalo Buffalo, NY, USA
Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability
Apr 19, 2026Lijie ZhouCross-Modal Learning VLM Robustness
School of Computer Science University of Nottingham Ningbo China Ningbo, China

Sep 26, 2026J/K move · Enter open · S save

Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models

Dongdong Wang, Deepak Balakrishnan, Ravi Srinivasan, Shenhao Wang

University of Florida Gainesville, FL, USA

VLM Adaptation Vision-Language Models Multi-Task Learning

Open paper AskPDF arXiv

Existing geospatial vision-language models (Geo-VLMs) typically optimize diverse geospatial tasks through a unified multi-task adaptation paradigm without explicitly accounting for the heterogeneous optimization characteristics. Our empirical observations reveal heterogeneous gradient characteristics across tasks, including vision-language differences, intra-branch gradient relationships, and task interference, which hinder effective multi-task optimization. Motivated by these observations, we propose Gradient-Guided Decoupled Adaptation (G2DA), a gradient-aware optimization framework for multi-task Geo-VLM learning. G2DA first partitions tasks into vision- and language-centric groups through gradient-guided cross-modal decoupling. It then constructs modality-specific curricula based on task gradient similarity and employs bidirectional rehearsal to mitigate the recency effects introduced by sequential optimization. We evaluate G2DA on three Geo-VLM benchmarks using six InternVL3 and Qwen3.5-VL variants, along with GeoChat and GeoLLaVA. Across all 24 benchmark-model combinations, G2DA consistently outperforms representative baselines, improving over the strongest competitor by 3.08, 4.30, and 2.81 percentage points on UrBench-MCQ, XLRS-Bench-Lite, and VRS-Bench-VQA, respectively. These results demonstrate the effectiveness of gradient-guided task organization for Geo-VLM adaptation.

Abstract

VLM Adaptation

Explore similar work

Gradient-Guided Decoupled Adaptation for Geospatial Vision-Language Models

Chain-of-Adaptation: Surgical Vision-Language Adaptation with Reinforcement Learning

Cross-Modal Attention Analysis and Optimization in Vision-Language Models: A Study on Visual Reliability