World-Grounded Reinforcement Learning for Pollution-Aware Bird Movement Modeling Using a Multi-Instance DQN Framework
Organizations: Vellore Institute of Technology (VIT), Chennai, India · MBZUAI, Abu Dhabi, UAE
Abstract
Birds are modeled as environmentally sensitive agents whose movement reflects real-world air quality conditions, where AQI serves as a measurable world feedback signal representing pollution-driven consequences. Instead of relying on human preference or synthetic rewards, the system uses environmental outcomes as the primary training signal, enabling policy learning grounded in ecological impact. We propose a multi-instance reinforcement learning simulation in which agents navigate a 2D grid representation of India integrated with a precomputed AQI matrix and geographic state boundaries. The framework combines bird call metadata from the Xeno-canto dataset, spatial discretization, and real environmental data to learn adaptive navigation strategies under pollution constraints. To ensure scalability and efficiency, repeated real-time API queries are replaced by a structured AQI mapping that enables fast lookup during training and execution. Multiple baselines, including Monte Carlo, Q-Learning, and Value Iteration, are evaluated for comparison. Deep Q-Network demonstrates stronger stability, adaptability, and cumulative performance across varying environmental conditions, establishing a scalable foundation for world-feedback driven ecological decision modeling.
Figures & tables
| Ref. | Method | Advantages | Disadvantages |
|---|---|---|---|
| [ 23 ] | Biomonitoring for Metal Pollution | Evaluates metal effects via biological specimens. Identifies high-risk areas. | Limited by sample availability. Costly atomic absorption spectrometry. |
| [ 24 ] | Drone-Enabled IoRT | Real-time monitoring in remote areas. Reduces human risk. | High cost. Impacted by weather and regulations. |
| [ 29 ] | eDNA for Bird Pollination | Non-invasive. Works across ecosystems. | DNA degradation affects accuracy. Needs extensive databases. |
| [ 31 ] | GPS-Based Bird Tracking | High-resolution migration insights. Supports conservation. | Expensive. Data gaps due to environmental factors. |
| [ 33 ] | GPS Tracking of Free-Roaming Dogs | Reveals habitat reliance. Aids rabies control. | Assumes data represents all dogs. Region-specific. |
| [ 34 ] | Vulture Roost-Site Analysis | Identifies nocturnal vs. diurnal patterns. Helps urban planning. | Site-specific findings. May not reflect natural behaviors. |
| Study | Context | Specified Indicators | Data Collection Method | Challenges |
| Singh et al. | Environmental Monitoring | Data Openness, ML Algorithms | Aggregated multi-source data | Data availability, quality issues |
| Zhang et al. | Ecosystem Monitoring | Drone Relay, IoRT | Drones, IoRT devices | Connectivity issues, high energy use |
| Iverson et al. | Migration Ecology | Land Cover, NDVI, GPS Tracking | GPS tracking, NDVI satellite data | Small bird tracking, data accuracy |
| Cunha Silva et al. | Disease Ecology | Habitat Selection, Rabies Risk | Field observations, GPS, surveys | Dog behavior variability, rural data gaps |
| Hill et al. | Urban Ecology | Urbanization, Roost Sites | GPS tracking, field observations | Urban accessibility, observer bias |
| Proposed Work | Environmental Health Monitoring | AQI, Geographic Data, Bird Calls | API-based bird data, measured AQI, geolocation | Computational cost, static AQI snapshot |
| Paper | Proposed Methodology | Metrics Used | Results | Challenges |
| Saulnier et al. | Statistical Analysis, Climate Models | Reproductive Success Rates, Environmental Correlations | Significant differences in reproductive success | Data variability, modeling diverse conditions |
| Antelmi et al. | Agent-Based Models (ABM) | Simulation Accuracy, Agent Behavior Patterns | Captures complex interactions at a micro level | Computational intensity, scalability issues |
| Perry et al. | Deep Learning, Neural Networks | Model Accuracy, Predictive Power | Improved ecosystem modeling | High data requirements, model interpretability |
| Tovey et al. | Multi-Agent Reinforcement Learning (MARL) | Strategy Effectiveness, Adaptability | Models complex adaptive behaviors | High training data demands, computational complexity |
| Tappler et al. | Continuous Stochastic Dynamics, RL | Predictive Accuracy, Adaptability | Captures dynamic environmental processes | High computational cost, complex implementation |
| Ellis et al. | Data Integration, Machine Learning | Habitat Use Patterns, Predictive Accuracy | Insights into habitat use patterns | Data integration complexity, data variability |
| Paper | Proposed RL Methodology | Metrics Used | Results | Inferences | Cons |
| Haensel et al. | Agent-Based Modeling (ABM) | Student Outcomes in Complexity | ABM aids teaching of human-environment interactions | Suitable for complex system simulations in education | Simplistic model, limited real-time adaptability |
| Martínez et al. | DRL for Autonomous Flocking | Convergence, Stability, Flocking Behavior | DRL optimizes coordination and navigation in flocking | Adaptable to dynamic environments, real-time effectiveness | Computationally expensive, lengthy training, complex tuning |
| Tovey et al. | Micro-scale Multi-Agent RL (MARL) | Strategy Emergence, Adaptability | MARL models adaptive behavior under diverse conditions | Suitable for emergent behavior modeling in dynamic systems | High training data needs, high computational cost |
| Tappler et al. | Continuous Stochastic Dynamics in RL | Model Accuracy, Predictive Performance | Captures dynamic environmental processes effectively | Supports continuous learning in environmental systems | High computational cost, complex implementation |
| Li et al. | Ensemble RL Gated Unit Model | PM2.5 Forecasting Accuracy | Improved air quality forecasting with ensemble learning | Effective for environmental predictions | Complex model, high data requirements |
| Proposed Work | Multi-instance DQN in a world-grounded AQI environment | Reward, diversity, efficiency, distance, MAE | Peak per-step reward of 722; highest cumulative reward among compared methods | Pollution-aware movement simulation grounded in measured AQI | High computational needs; static AQI; validation pending |