HeatPrompt: Zero-Shot Vision-Language Modeling of Urban Heat Demand from Satellite Images

Urban Building Energy Modeling plays a critical role in achieving the United Nations' Sustainable Development Goals 7 and 11. Although existing studies based on satellite imagery and deep learning have achieved remarkable progress, many challenges exist: most existing studies are inherently predictive, failing to reflect the generative nature of urban planning; although generative AI and diffusion models have seen explosive growth in satellite imagery, they lack the urban functional generation (e.g., energy layer); third, aligned high-quality high-resolution building energy data with satellite imagery is limited and scarce. Here we propose SENSE (Satellite-based ENergy Synthesis for Sustainable Environment), a unified generative UBEM framework that jointly synthesizes realistic urban satellite imagery and aligned high-quality building energy consumption and height maps. By conditioning on road networks and urban density metrics, SENSE, based on a controllable diffusion model, leverages the knowledge learned by large vision models to generate urban building energy consumption and height information (annotations) in the latent space. Experiments across four cities (New York City, Boston, Lyon, Busan) demonstrate that SENSE achieves high visual fidelity and strong physical consistency, satisfying the ASHRAE standard metric. Experiments demonstrate that SENSE can generate enough annotated synthetic data using less than 20% labeled energy data, boosting downstream prediction performance by 10% IoU. Compared to SOTA urban energy prediction methods, SENSE significantly reduced prediction error (reduced 3%-11% NMBE and 1%-9% CVRMSE). This study offers an energy-efficiency urban planning and physical generation solution for urban science, energy science and building science. The dataset and code: https://huggingface.co/datasets/skl24/MUSE and https://github.com/kailaisun/GenAI4Urban-Energy/.

Land Surface Temperature (LST) is a widely used satellite-derived measure of urban surface heat, but there is no shared benchmark for forecasting it at 30 m. Prior studies usually cover one to three cities, use kilometer-scale products, or do not release data and code. We introduce HeatCast, a Landsat-based benchmark for monthly LST forecasting across 124 U.S. cities from 2013 through June 2025. HeatCast contains 30 m monthly tiles with LST, elevation, surfacereflectance RGB, three spectral indices, broadband albedo, quality masks, and Local Climate Zone (LCZ) labels, together with a fixed temporal split, LCZ-stratified metrics, and a reference evaluation harness. We evaluate a CNN+LSTM and Earthformer on next-month forecasting, where Earthformer reaches 7.74 K RMSE against 10.42 K for the CNN+LSTM. Forecasting from the eight nonLST channels alone reaches 7.72 K, against 8.15 K from LST history and 8.68 K from RGB. The data, code, and weights are released under MIT at https://doi.org/10.57967/hf/9889.

Rapid estimation of impacted structures - critical for conflict-zone humanitarian response - is frequently hindered by post-strike satellite data embargoes and imagery blackouts. We bypass this operational bottleneck by reframing impacted building mapping as a zero-shot geometric projection task on archival, pre-strike maps. Using coordinate and incident text from LiveUAMap and ArcGIS, Large Language Models extract weapon payloads (W) to project kinetic blast perimeters via Hopkinson-Cranz scaling (R_base = Z * W^(1/3)). To count exposed structures within these zones without post-strike imagery, we introduce two technical innovations: Adaptive Field-of-View to eliminate resolution (zoom) bias in 2D segmentation (SAMGeo), and 2.5D pseudo-height depth maps combined with segmentation masks to help Large Vision-Language Models (LVLMs) resolve overlapping, dense rooftops. Evaluated on 2026 Middle East conflict data, depth-augmented LVLMs dramatically outperform traditional segmentation in congested urban centers. This establishes a powerful hybrid paradigm for zero-shot crisis mapping: ultra-fast 2D segmentation for sparse rural zones, and depth-augmented LVLMs for dense urban environments.