Organizations: SMART, Singapore · Massachusetts Institute of Technology, Cambridge, MA, USA · National University of Singapore, Singapore · University of Florida, Gainesville, FL, USA
Cities account for over two-thirds of the world's energy and more than 70% of global carbon dioxide emissions. City energy is essential for achieving the 2050 net-zero target and the UN's Sustainable Development Goals 7 and 11. Although existing studies have made progress, they are limited by data and methodological gaps. Existing studies are largely predictive for existing cities rather than generative for future cities; fine-grained, aligned monthly dynamic energy data are scarce; and existing GenAI models struggle to generate dynamic urban functional monthly energy. We present DySENSE, a joint multimodal GenAI framework that synthesizes static urban satellite imagery together with spatially aligned, dynamic monthly building energy consumption maps. We decouple long-term static urban morphology from short-term dynamic energy consumption. By conditioning on static urban land use, DySENSE can generate realistic, diverse and new static urban imagery with aligned energy maps. By conditioning on dynamic monthly and climate prompts, DySENSE can jointly generate dynamic, monthly energy consumption maps while keeping the urban morphology static. Experiments across four cities (New York, Washington, Norfolk, Busan) demonstrated that DySENSE can generate annotated high-fidelity synthetic energy data using less than 25% labelled data, boosting downstream prediction performance by 8% mIoU, and achieving strong energy dynamic consistency. Compared to existing methods, DySENSE significantly improved prediction performance (8.7% accuracy) and reduced prediction error (9%-18% CVRMSE, ASHRAE). DySENSE offers an effective foundation for energy-efficient cities, peak-load analysis, building performance simulation and dynamic demand response for city, energy, and building science. The data and code: https://huggingface.co/datasets/skl24/MUSE2 and https://github.com/kailaisun/DySENSE
Figures & tables
Figure 1. The proposed framework includes four modules: (1) Energy modality-specific variational autoencoders that compress the energy map into compact latent tokens; (2) Two text encoders (CLIP and RemoteCLIP ( Liu et al., 2024 ) ) for embedding the static and dynamic prompts, respectively; (3) A U-ViT joint denoising network that models the joint distribution of all modalities via the self-attention mechanism; (4) Dynamic monthly energy consumption generation under static urban form.
Figure 2. Qualitative visualization of ground-truth and generated urban imagery and monthly energy consumption. The Energy changes are computed by the difference between monthly energy consumption values and the annual minimum values. Our generated monthly energy consumption and its trend are very similar to the ground truth.
Figure 3. Qualitative visualization of generated results across 4 cities. We sample some monthly generated energy results across seasons. (a) The comparison between GT and generated urban imagery and monthly building energy consumption maps in NYC. (b,c,d) Other cases in Washington, D.C., Busan, and Norfolk. The dynamic energy consumption trends are similar.
Method
Training Data
Acc ↑
mIoU ↑
NMBE
CVRMSE ↓
Yap et al. (2025)
Strategy A1: Real
0.5343
0.3687
6.02%
33.58%
Yap et al. (2025)
Strategy A2: Real duplication
0.5340
0.3686
7.73%
32.56%
Yap et al. (2025)
Strategy A3: Real augmentation
0.5462
0.3712
5.57%
30.70%
Yap et al. (2025) + Ours
Strategy B1: Static Mix
0.6034
0.4364
3.37%
16.91%
Yap et al. (2025) + Ours
Strategy B2: Dynamic Mix
0.6217
0.4555
2.82%
15.31%
Diff ( | Ours | - | Baseline | )
+0.0874
+0.0868
-3.20%
-18.27%
Table 1. 12-month energy prediction performance comparison on the fixed real test set (20%) using ASHRAE metrics ( American Society of Heating and Air Conditioning Engineers (Atlanta, 2014 ) . Strategy B uses real + synthetic training data; Strategy A1 uses real data only; Strategy A2 uses real data with equal-size simple duplication; Strategy A3 uses real data with equal-size simple augmentation by Gaussian noise. NMBE is reported with their original signs; values closer to 0 indicate better performance. More results can be found in Appendix Tab. 7 .
Figure 4. Generative augmentation effect on downstream building energy prediction metrics across different training set ratios.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Hyperparameter
4 cities, 10-class
in channels
4
intermediate channels
512
out channels
10
block out channels
(128, 256, 512, 512)
resolution
512
training augmentation
Resize, RandomCrop, RandomHorizontalFlip
Appendix
Table 2. Training hyperparameters of the Energy VAE.
Hyperparameter
4 cities, 10-class
resolution
512
sequence length
2206
batch size
12
epoch
80
learning rate
5e-5
lr scheduler
cosine (500 warmup steps)
Appendix
Table 3. Training hyperparameters of the joint diffusion.
Figure 5. Monthly energy consumption patterns generated by DySENSE across different cities. The curves compare the monthly mean energy consumption of generated maps and ground-truth maps. DySENSE captures seasonal and monthly dynamics well, with peak months exactly aligned with the ground truth in different cities.
Metric
RemoteCLIP
CLIP
FID ↓
36.7362
37.4800
PSNR ↑
11.5177
10.9076
LPIPS ↓
0.5604
0.5695
Appendix
Table 4. Ablation study of the climate text encoder in the proposed generative framework.
Ratio
mIoU ↑
∣ NMBE ∣ (%) ↓
CVRMSE (%) ↓
RemoteCLIP
CLIP
RemoteCLIP
CLIP
RemoteCLIP
CLIP
Strategy A: Real Only
1/16
18.50
18.92
17.96
28.93
82.30
103.12
1/8
29.30
29.36
13.68
16.99
53.70
61.96
1/4
36.90
35.68
6.02
7.32
33.60
33.43
1/2
49.50
48.59
2.63
3.18
14.70
13.13
Appendix
Table 5. Ablation on the dynamic climate text encoder of the downstream prediction model: RemoteCLIP vs. CLIP.
City
Satellite imagery
Urban density
Original energy map
Filtered energy map
Used energy map
New York City
1589
1589
1589
579
500
Busan
1438
1438
1438
996
500
Washington, D.C.
1914
1914
1914
500
500
Norfolk
1328
1328
1328
490
490
Appendix
Table 6. City-level statistics data in MUSE2. Each tile has 12-month energy records.
Method
Training Data
IoU (per class)
Acc (per class)
C1
C2
C3
C4
C5
C6
C7
C8
C9
C1
C2
C3
C4
C5
C6
C7
C8
C9
Yap et al. (2025)
Strategy A: real
0.5106
0.3666
0.3188
0.2930
0.3059
0.2938
0.3121
0.3684
0.5488
0.6731
0.5189
0.4780
0.4395
0.5033
0.4353
0.4495
0.5643
0.7470
Yap et al. (2025) + Ours
Strategy B1: Static Mix
0.5790
0.4304
0.3857
0.3706
0.3609
0.3637
0.3873
0.4319
0.6179
0.7350
0.5899
0.5375
0.5676
0.5165
0.5195
0.5596
0.6141
0.7903
Yap et al. (2025) + Ours
Strategy B2: Dynamic Mix
0.5911
0.4544
0.3997
0.3821
0.3857
0.3927
0.4104
0.4545
0.6285
0.7278
0.6303
0.5589
0.5485
0.5481
0.5759
0.5844
0.6229
0.7983
Mayer et al. (2023)
Strategy A: real
0.5592
0.4261
0.3735
0.3463
0.3541
0.3576
0.3710
0.4164
0.5866
0.6988
0.6127
0.5509
0.5066
0.5192
0.5169
0.5236
0.6041
0.7649
Mayer et al. (2023) + Ours
Strategy B1: Static Mix
0.5925
0.4496
0.4016
0.3773
0.3791
0.3824
0.3976
0.4447
0.6225
0.7323
0.6179
0.5737
0.5403
0.5456
0.5608
0.5543
0.6184
0.8036
Appendix
Table 7. Per-class IoU and accuracy results under different training strategies.
Figure 6. Performance comparison on the downstream building energy consumption prediction task under different training-data strategies.
Class
Busan
New York City
Norfolk
Washington
kBtu
log1p
kBtu
log1p
kBtu
log1p
kBtu
log1p
0
0.00
0.000
0.00
0.000
0.00
0.000
0.00
0.000
1
1693.45
7.435
42538.34
10.658
229.11
5.439
63285.79
11.055
2
7777.19
8.959
111640.48
11.623
855.97
6.753
140097.29
11.850
3
19192.97
9.862
185658.64
12.132
2370.11
7.771
228871.67
12.341
4
40711.96
10.614
276446.47
12.530
5495.96
8.612
349862.14
12.765
Appendix
Table 8. Representative continuous energy-consumption values for each class.
Urban Building Energy Modeling plays a critical role in achieving the United Nations' Sustainable Development Goals 7 and 11. Although existing studies based on satellite imagery and deep learning have achieved remarkable progress, many challenges exist: most existing studies are inherently predictive, failing to reflect the generative nature of urban planning; although generative AI and diffusion models have seen explosive growth in satellite imagery, they lack the urban functional generation (e.g., energy layer); third, aligned high-quality high-resolution building energy data with satellite imagery is limited and scarce. Here we propose SENSE (Satellite-based ENergy Synthesis for Sustainable Environment), a unified generative UBEM framework that jointly synthesizes realistic urban satellite imagery and aligned high-quality building energy consumption and height maps. By conditioning on road networks and urban density metrics, SENSE, based on a controllable diffusion model, leverages the knowledge learned by large vision models to generate urban building energy consumption and height information (annotations) in the latent space. Experiments across four cities (New York City, Boston, Lyon, Busan) demonstrate that SENSE achieves high visual fidelity and strong physical consistency, satisfying the ASHRAE standard metric. Experiments demonstrate that SENSE can generate enough annotated synthetic data using less than 20% labeled energy data, boosting downstream prediction performance by 10% IoU. Compared to SOTA urban energy prediction methods, SENSE significantly reduced prediction error (reduced 3%-11% NMBE and 1%-9% CVRMSE). This study offers an energy-efficiency urban planning and physical generation solution for urban science, energy science and building science. The dataset and code: https://huggingface.co/datasets/skl24/MUSE and https://github.com/kailaisun/GenAI4Urban-Energy/.
Kailai Sun, Mingyi He, Heye Huang +5
SMART, Singapore · Massachusetts Institute of Technology Cambridge, MA, USA · University of Florida Gainesville, Florida, USA
Fine-grained energy consumption data are essential for applications such as demand forecasting, demand response planning, and grid reliability assessment. However, access to such data is often restricted by privacy concerns and data-sharing constraints, motivating growing interest in synthetic energy data generation. Although existing methods can reproduce overall consumption distributions and recurring temporal patterns, they often smooth out or underrepresent anomalous events caused by extreme weather, infrastructure failures, and behavioral shifts. Preserving these events is challenging because they are sparse, localized in time and space, and shaped by heterogeneous dependencies across geographical proximity and regional attributes. To address these challenges, we propose SynEnergy, a two-stage diffusion-based framework for anomaly-preserving energy consumption data generation. The first stage, Heterogeneous Graph-based Anomaly Semantic Learning (HG-ASL), extracts region-specific anomaly semantics from sparse residual structures by jointly modeling spatial and attribute dependencies across urban regions. The second stage, Anomaly Semantic-guided Diffusion (AS-Diff), injects the learned anomaly semantics into the denoising process to generate realistic consumption sequences while preserving anomalous patterns. This design enables controllable generation for individual regions and scales naturally to city-wide settings. We evaluate SynEnergy on four real-world energy consumption datasets against 11 general-purpose and energy-specific generation baselines. Experimental results show that SynEnergy improves anomaly preservation fidelity by an average of 12.21% and downstream quality by 2.96%, while maintaining competitive overall generation fidelity compared to baselines.
Lin Jiang, Dahai Yu, Ravikumar Gelli +1
Department of Computer Science Florida State University Tallahassee, Florida, USA · Department of Electrical and Computer Engineering Florida State University Tallahassee, Florida, USA
Accurate heat-demand maps play a crucial role in decarbonizing space heating, yet most municipalities lack detailed building-level data needed to calculate them. We introduce HeatPrompt, a zero-shot vision-language energy modeling framework that estimates annual heat demand using semantic features extracted from satellite images, basic Geographic Information System (GIS), and building-level features. We feed pretrained Large Vision Language Models (VLMs) with a domain-specific prompt to act as an energy planner and extract the visual attributes such as roof age, building density, etc, from the RGB satellite image that correspond to the thermal load. A Multi-Layer Perceptron (MLP) regressor trained on these captions shows an R2 uplift of 93.7% and shrinks the mean absolute error (MAE) by 30% compared to the baseline model. Qualitative analysis shows that high-impact tokens align with high-demand zones, offering lightweight support for heat planning in data-scarce regions.
Kundan Thota, Xuanhao Mu, Thorsten Schlachter +1
Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology (KIT), Germany