RL-Driven Sustainable Land-Use Allocation for the Lake Malawi Basin

Ying Yao

cs.AIApr 4, 2026

RL-Driven Sustainable Land-Use Allocation for the Lake Malawi Basin

Authors: Ying Yao

Organizations: Georgia Institute of Technology

Abstract

Unsustainable land-use practices in ecologically sensitive regions threaten biodiversity, water resources, and the livelihoods of millions. This paper presents a deep reinforcement learning (RL) framework for optimizing land-use allocation in the Lake Malawi Basin to maximize total ecosystem service value (ESV). Drawing on the benefit transfer methodology of Costanza et al., we assign biome-specific ESV coefficients -- locally anchored to a Malawi wetland valuation -- to nine land-cover classes derived from Sentinel-2 imagery. The RL environment models a 50x50 cell grid at 500m resolution, where a Proximal Policy Optimization (PPO) agent with action masking iteratively transfers land-use pixels between modifiable classes. The reward function combines per-cell ecological value with spatial coherence objectives: contiguity bonuses for ecologically connected land-use patches (forest, cropland, built area etc.) and buffer zone penalties for high-impact development adjacent to water bodies. We evaluate the framework across three scenarios: (i) pure ESV maximization, (ii) ESV with spatial reward shaping, and (iii) a regenerative agriculture policy scenario. Results demonstrate that the agent effectively learns to increase total ESV; that spatial reward shaping successfully steers allocations toward ecologically sound patterns, including homogeneous land-use clustering and slight forest consolidation near water bodies; and that the framework responds meaningfully to policy parameter changes, establishing its utility as a scenario-analysis tool for environmental planning.

Figures & tables

Class	Type	ET (kg/m 2 /yr)
Water	Protected	616.93
Trees	Modifiable	933.57
Flooded	Protected	767.62
Crops	Modifiable	675.66
Built Area	Modifiable	648.95
Bare Ground	Modifiable	591.53

TABLE I: Per-Class Evapotranspiration (2024, MOD16A3GF)

Class	Type	ESV	Ratio
		(USD/ha/yr)
Water	Protected	554	1.00
Trees	Modifiable	238	0.43
Flooded	Protected	1,136	2.05
Crops	Modifiable	246 * 1.35 = 332	0.60
Built Area	Modifiable	295	0.53

TABLE II: Land-Cover Classes and Ecosystem Service Values. Crops carries the

1.35\times

regenerative-agriculture uplift used in the headline reward; the base value

246

is used only in the Appendix A ablation.

Fig. 2: Overview of the proposed RL framework. The agent observes a

10\!\times\!10\!\times\!5

land-use fraction grid, extracts spatial features via a shared GridCNN, and produces both a masked policy (Actor) and state value estimate (Critic). The environment executes the selected action and returns a reward combining ESV change and spatial coherence metrics.

Component	Symbol	Weight
Tree contiguity bonus	$w_{T}$	1.0
Crop contiguity bonus	$w_{C}$	4.0
Built-area contiguity bonus	$w_{B}$	2.0
Water buffer penalty	$w_{W}$	6.0 (anneal $1{\rightarrow}6$ )
Riparian-tree bonus	$w_{R}$	5.0

TABLE III: Spatial Reward Weights.

w_{W}

is annealed linearly from

1

6

over the first

60\%

of training; the other weights are held fixed.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

	ExpI	ExpII	ExpIII
	Eco-only	Spatial-without-regen	Headline (Spatial+Regen)
Dominant conversion	$\rightarrow$ Built	Mixed	$\rightarrow$ Crops
Forest near water	Not obvious	Increased	Increased
Built expansion	Aggressive	Clustered	Clustered, restrained
Crop allocation	Moderate	Moderate	High
Ecological realism	Low	High	High

Appendix

TABLE IV: Qualitative Behaviour Across Reward-Design Ablations

Parameter	Value
Learning rate	$5{\times}10^{-5}\rightarrow 5{\times}10^{-6}$ (last 33%)
Rollout length ( $n_{\text{steps}}$ )	2,048
Mini-batch size	128
PPO epochs	10
Discount ( $\gamma$ )	0.99
GAE $\lambda$	0.95

Appendix

TABLE V: PPO Hyperparameters (spatially-aware runs).

Aug 17, 2026

KC-BFPRL: Knowledge-Guided Multi-UAV Collaboration for Grassland Restoration via Bilevel Formerpointer-Based Reinforcement Learning

Multi-unmanned aerial vehicle (UAV) systems provide scalable service platforms for large-scale environmental tasks, such as grassland ecosystem restoration. However, coordinating fleet operations requires solving the restoration area maximization problem (RAMP). This non-linear combinatorial optimization challenge is complicated by payload-dependent energy dynamics and heterogeneous ecological degradation. We propose a novel knowledge-guided collaborative bilevel formerpointer reinforcement learning framework (KC-BFPRL) to address this complexity. Using a hierarchical paradigm, KC-BFPRL decomposes RAMP into global task allocation and local restoration planning, with the latter further divided into upper-level trajectory planning and lower-level restoration area allocation. Our specialized architecture pairs featuring a Transformer-based encoder that fuses static environmental features with dynamic UAV states, and a Pointer Network decoder trained via a robust actor-critic framework. By embedding ecological priority rules and heuristic logic, KC-BFPRL achieves a structured warm-start, solving the RL cold-start problem while ensuring strict constraint satisfaction. Extensive experiments demonstrate that KC-BFPRL consistently outperforms state-of-the-art baselines, achieving superior objective values and efficiency. It maintains a

0.00\%

optimality gap in the most complex scenarios U8-R160 and operates nearly three times faster than MAPDP, validating its robustness, scalability, and real-time applicability for large-scale automated ecological restoration.

Dongbin Jiao, Xianyi Wang, Yuchen Yuan +5

School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, P. R. China · Key Laboratory of Tourism Information Fusion Processing and Data Ownership Protection, Ministry of Culture and Tourism, Lanzhou University, Lanzhou, 730000, P. R. China · School of Automobile, Chang’an University, Xi’an, 710064, P. R. China +2

Reinforcement Learning Multi-Agent Coordination UAV Trajectory Optimization Multi-Robot Task Allocation Hierarchical RL

Ask PDF

Mar 15, 2026

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees

Global decarbonisation targets and tightening market pressures demand maritime logistics solutions that are simultaneously efficient, sustainable, and equitable. We introduce EcoFair-CH-MARL, a constrained hierarchical multi-agent reinforcement learning framework that unifies three innovations: (i) a primal-dual budget layer that provably bounds cumulative emissions under stochastic weather and demand; (ii) a fairness-aware reward transformer with dynamically scheduled penalties that enforces max-min cost equity across heterogeneous fleets; and (iii) a two-tier policy architecture that decouples strategic routing from real-time vessel control, enabling linear scaling in agent count. New theoretical results establish O(\sqrt{T}) regret for both constraint violations and fairness loss. Experiments on a high-fidelity maritime digital twin (16 ports, 50 vessels) driven by automatic identification system traces, plus an energy-grid case study, show up to 15% lower emissions, 12% higher through-put, and a 45% fair-cost improvement over state-of-the-art hierarchical and constrained MARL baselines. In addition, EcoFair-CH-MARL achieves stronger equity (lower Gini and higher min-max welfare) than fairness-specific MARL baselines (e.g., SOTO, FEN), and its modular design is compatible with both policy- and value-based learners. EcoFair-CH-MARL therefore advances the feasibility of large-scale, regulation-compliant, and socially responsible multi-agent coordination in safety-critical domains.

Saad Alqithami

Computer Science Department, Al-Baha University, Albaha 65779, Saudi Arabia

Constrained MARL Multi-Agent Reinforcement Learning Multi-Agent Coordination Hierarchical RL Heterogeneous MARL

Ask PDF

Feb 18, 2026

Auction-Based RIS Allocation With DRL: Controlling the Cost-Performance Trade-Off

We study the allocation of reconfigurable intelligent surfaces (RISs) in a multi-cell wireless network, where base stations compete for control of shared RIS units deployed at the cell edges. These RISs, provided by an independent operator, are dynamically leased to the highest bidder using a simultaneously ascending auction format. Each base station estimates the utility of acquiring additional RISs based on macroscopic channel parameters, enabling a scalable and low-overhead allocation mechanism. To optimize the bidding behavior, we integrate deep reinforcement learning (DRL) agents that learn to maximize performance while adhering to budget constraints. Through simulations in clustered cell-edge environments, we demonstrate that reinforcement learning (RL)-based bidding significantly outperforms heuristic strategies, achieving optimal trade-offs between cost and spectral efficiency. Furthermore, we introduce a tunable parameter that governs the bidding aggressiveness of RL agents, enabling a flexible control of the trade-off between network performance and expenditure. Our results highlight the potential of combining auction-based allocation with adaptive RL mechanisms for efficient and fair utilization of RISs in next-generation wireless networks.

Martin Mark Zan, Stefan Schwarz

Institute of Telecommunications, TU Wien, Vienna, Austria

Wireless Resource Allocation Reconfigurable Intelligent Surfaces Automated Bidding

Ask PDF

Abstract

Explore similar work

KC-BFPRL: Knowledge-Guided Multi-UAV Collaboration for Grassland Restoration via Bilevel Formerpointer-Based Reinforcement Learning

EcoFair-CH-MARL: Scalable Constrained Hierarchical Multi-Agent RL with Real-Time Emission Budgets and Fairness Guarantees

Auction-Based RIS Allocation With DRL: Controlling the Cost-Performance Trade-Off