TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translation

Dong-Guw Lee; Tai Hyoung Rhee; Hyunsoo Jang; Young-Sik Shin; Ukcheol Shin; Ayoung Kim

cs.CVFeb 23, 2026

TherA: Thermal-Aware Visual-Language Prompting for Controllable RGB-to-Thermal Infrared Translation

Authors: Dong-Guw Lee, Tai Hyoung Rhee, Hyunsoo Jang, Young-Sik Shin, Ukcheol Shin, Ayoung Kim

Organizations: KENTECH

Abstract

Despite the inherent advantages of thermal infrared(TIR) imaging, large-scale data collection and annotation remain a major bottleneck for TIR-based perception. A practical alternative is to synthesize pseudo TIR data via image translation; however, most RGB-to-TIR approaches heavily rely on RGB-centric priors that overlook thermal physics, yielding implausible heat distributions. In this paper, we introduce TherA, a controllable RGB-to-TIR translation framework that produces diverse and thermally plausible images at both scene and object level. TherA couples TherA-VLM with a latent-diffusion-based translator. Given a single RGB image and a user-prompted condition pair, TherA-VLM yields a thermal-aware embedding that encodes scene, object, material, and heat-emission context reflecting the input scene-condition pair. Conditioning the diffusion model on this embedding enables realistic TIR synthesis and fine-grained control across time of day, weather, and object state. Compared to other baselines, TherA achieves state-of-the-art translation performance, demonstrating improved zero-shot translation performance up to 33% increase averaged across all metrics.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of TherA. (a) Compared to InstructPix2Pix [ 2 ] , our thermal-aware VLM distinguishes between active (heat-emitting car) and passive (parked car) objects as shown in the translated image from WayMo [ 53 ] open dataset. (b) TherA is the first RGB-to- TIR translation model offering both text-guided and reference image-guided translations. RGB images from M3FD [ 38 ] are used for reference

Figure 2: Overview of the TherA framework. Our model consists of two stages. (a) TherA-VLM analyzes the input/reference RGB and the input prompt to produce a physically grounded thermal embedding ( $\textbf{h}_{N}$ ). (b) This thermal embedding is injected via TE Adapter ( $\phi$ ) into the cross-attention layers of a UNet. The UNet, also conditioned on the RGB latent ( $z_{\mathrm{rgb}}$ ), guides the denoising of the noisy TIR latent ( $z_{t}$ ) to generate a physically plausible TIR image. The input prompt and reference RGB supports controllability.

Panel of Figure 3 — Figure 3: Comparison of textual conditioning. Standard LLaVA produces appearance-based, RGB-centric descriptions (c). Our TherA-VLM outputs concise, structured schemas encoding scene type, materials, and heat emissionstates (d).

Method		M3FD [ 38 ]				FLIR [ 13 ]
Method		PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
General I2I	InstructPix2Pix [ 2 ]	13.94	0.40	138.94	0.37	14.44	0.38	178.03	0.41
	StegoGAN [ 58 ]	13.14	0.30	247.03	0.47	12.58	0.29	152.57	0.46
	Pix2Pix-Turbo [ 44 ]	16.69	0.60	142.26	0.32	17.03	0.40	91.17	0.32
	UNSB [ 27 ]	12.86	0.39	160.72	0.47	16.91	0.42	139.46	0.40
	CSGO [ 61 ]	9.61	0.36	204.53	0.56	10.12	0.28	116.85	0.55

Table 1: RGB-to-TIR image translation evaluation on M3FD and FLIR. Best results are highlighted in bold and second best are underlined .

Method	M3FD [ 38 ]				FLIR [ 13 ]				CART [ 31 ]
Method	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
F-ViTA [ 43 ]	13.86	0.46	145.83	0.42	13.86	0.45	145.83	0.42	11.25	0.44	221.83	0.65
DiffV2IR [ 45 ]	11.77	0.49	132.49	0.46	11.41	0.18	253.82	0.66	10.92	0.43	184.93	0.52
ThermalGen [ 59 ]	12.84	0.35	177.85	0.54	14.06	0.39	127.14	0.55	12.17	0.49	215.73	0.61
TherA	18.24	0.65	105.52	0.29	16.56	0.49	112.93	0.38	15.38	0.60	169.33	0.36

Table 2: Zero-shot evaluation of RGB-to-TIR translation on M3FD, FLIR and CART.

Panel of Figure 4 — Figure 4: Qualitative results on zero-shot image translation. No TIR ground truth exists for RGB-only datasets.

Panel of Figure 5 — Figure 5: Qualitative results for scene-level and object-level controllability on M3FD and FLIR. Yellow boxes show zoomed views.

Figure 6: Day-Night Controllability (Thermal Inversion) Input RGB and ground truth TIR images are taken from STheReO [ 64 ] . TherA simulates the diurnal cycle from a single RGB image, matching the ground truth thermal characteristics.

Method	M3FD [ 38 ]				FLIR [ 13 ]
Method	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
InstructPix2Pix [ 2 ]	12.40	0.32	203.01	0.45	13.38	0.35	133.51	0.46
LLaVA	11.85	0.47	135.12	0.37	14.25	0.40	101.82	0.39
Lavi-LLama	13.23	0.51	142.37	0.41	14.65	0.42	113.75	0.40
TherA	18.24	0.65	105.52	0.29	16.56	0.49	112.93	0.38

Table 3: Ablation results. Best results are highlighted in bold and second best are underlined .

Dataset	Sample Number (Translation)	Average Resolution	Viewpoint	Split
AVIID [ 20 ]	683	480×480	Aerial	Train
CAMEL [ 15 ]	3481	Diverse	CCTV	Train
FLIR [ 13 ]	5141	640×512	Driving	Finetune/Test
KAIST [ 8 ]	6457	640×512	Driving	Train
LLVIP [ 24 ]	15488	640×512	CCTV	Train
M3FD [ 38 ]	4200	512×384	Driving	Finetune/Test

Table 4: Constituent datasets of R2T2 and their statistics. “Sample Number (Translation)” denotes the number of RGB–TIR pairs used for training the translation model. The total across all sources is 112,970 pairs.

Panel of Figure 7 — Figure 7: Examples of RGB–TIR pairs in R2T2 dataset.

Panel of Figure 8 — Figure 8: Examples of pseudo-aligned RGB–TIR pairs. We use alpha-blended overlays (right) of RGB (left) and TIR (middle) to visually screen for residual misalignment, and keep only pairs with minimal visible parallax.

Panel of Figure 10 — Figure 10: TherA-VLM Examples. Example output results for TherA-VLM across different benchmarks

Figure 11: Text-guided controllability Qualitative results of cloudy, sunny, rainy weather on different datasets: CART [ 31 ] , custom dataset, FMB [ 39 ] , MegaDepth [ 36 ] , MS2 [ 48 ] , STheReO [ 64 ] . MS2 results were horizontally split due to its long aspect ratio.

Figure 12: Reference-guided controllability Qualitative results of reference-guided control with reference RGB images from LLVIP [ 24 ] , M3FD [ 38 ] , NSAVP [ 5 ] on translating images from KITTI [ 16 ] , NSAVP [ 5 ] , SMOD [ 7 ] , Waymo [ 53 ] . KITTI results were horizontally split due to its long aspect ratio.

Figure 13: Zero-shot Results Qualitative results on RGB-TIR datasets: CART [ 31 ] , custom dataset, Trimodal [ 51 ] , Multispectral Motion Dataset [ 11 ] .

Figure 14: Zero-shot Results Qualitative results on RGB-only datasets: Cityscapes [ 10 ] , NuScenes [ 3 ] , MegaDepth [ 36 ] , KITTI [ 16 ] , Waymo [ 53 ] . KITTI results were horizontally split due to its long aspect ratio.

CFG Scale		PSNR(↑)	SSIM(↑)	FID(↓)	LPIPS(↓)
Thermal VLM	Image control	PSNR(↑)	SSIM(↑)	FID(↓)	LPIPS(↓)
0.5	0.5	19.46	0.66	87.17	0.21
	1.5	18.67	0.64	97.60	0.23
	2.5	18.29	0.62	102.69	0.24
1.5	0.5	19.54	0.67	87.08	0.21
	1.5	18.78	0.64	96.55	0.22

Table 5: Ablation results for CFG scales. Best results are highlighted in bold and second best are underlined .

Method	FMB [ 39 ]		MFNet [ 18 ]
Method	Zero-shot	Fine-tuned	Zero-shot	Fine-tuned
Real TIR	N/A	42.04	N/A	57.09
F-ViTA [ 43 ]	26.25	50.81	21.98	64.80
ThermalMGen [ 59 ]	23.49	50.19	25.38	66.24
DiffV2IR [ 45 ]	26.17	50.88	23.75	64.38
TherA	27.14	52.05	30.43	68.60

Table 6: Quantitative comparison of class-free TIR semantic segmentation mIoU results [%] on FMB [ 39 ] and MFNet [ 18 ] datasets. Zero-shot : SegFormer trained on the Cityscapes dataset [ 10 ] translated using each model. Fine-tuned : The models further fine-tuned on the FMB or MFNet datasets. Best results are highlighted in bold and second best are underlined .

Method	LoFTR
Method	AUC@ $5^{\circ}$	AUC@ $10^{\circ}$	AUC@ $20^{\circ}$
Baseline (RGB)	5.44	12.58	24.28
F-ViTA [ 43 ]	4.93	12.73	25.32
ThermalGen [ 59 ]	5.81	14.24	27.65
DiffV2IR [ 45 ]	11.83	26.03	43.17
TherA	14.98	28.98	45.24

Table 7: Quantitative comparison of RGB-TIR image matching AUC results [%] on METU_VisTIR [ 56 ] dataset. Best results are highlighted in bold and second best are underlined .

Figure 15: Reference-guided translation comparison. Qualitative comparison of existing reference-guided translation models and TherA. Source images taken from popular RGB benchmarks (from the top: Visdrone [ 70 ] , Waymo [ 53 ] , NuScenes [ 3 ] , Waymo [ 53 ] , and MegaDepth [ 36 ] )

Figure 16: Limitation of controllable active/passive editing for indoor electronics. In the RGB image (left), a powered-on monitor (green box) and a powered-off monitor (red box) are clearly distinguishable. In the corresponding real TIR image (right), however, the temperature contrast between the two displays is subtle, making it difficult for TherA to produce a visually striking on/off effect for such objects.

Image-to-Image Translation

Image-to-image translation maps images between visual domains or modalities while preserving relevant content, structure, and semantics.

See all 115 papers

Text2Thermal: Physics-Aware Thermal Image Synthesis from Textual Priors
Sep 3, 2026Tayeba Qazi, Brejesh Lall, Prerana MukherjeeThermal Imaging Conditional Image Generation
Bharti School of Telecommunications Technology and Management, Indian Institute of Technology Delhi, Delhi, India · School of Engineering, Jawaharlal Nehru University, Delhi, India
T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining
May 30, 2026Tayeba Qazi, Ayush Maheshwari, Prerana Mukherjee +1Thermal Imaging Image Captioning
Indian Institute of Technology Delhi, India · NVIDIA AI Technology Center, India · Jawaharlal Nehru University, India
Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception
May 21, 2026Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape +1Thermal Imaging Visual Question Answering
Department of Electrical & Computer Engineering, Johns Hopkins University, Baltimore, MD 21218 USA

Sep 3, 2026J/K move · Enter open · S save

Text2Thermal: Physics-Aware Thermal Image Synthesis from Textual Priors

Tayeba Qazi, Brejesh Lall, Prerana Mukherjee

Bharti School of Telecommunications Technology and Management, Indian Institute of Technology Delhi, Delhi, India · School of Engineering, Jawaharlal Nehru University, Delhi, India

Thermal Imaging Conditional Image Generation T2I Diffusion Models

Open paper AskPDF arXiv

Thermal infrared imaging offers reliable perception in darkness and adverse weather, but thermal datasets remain scarce, motivating extensive work on translating abundant RGB images into thermal. Such translation is fundamentally ill-posed as thermal appearance is governed by surface emissivity and object temperature, neither of which is observable in the visible spectrum, so a single RGB image is consistent with many valid thermal outputs. We argue that language offers a natural means of resolving this ambiguity, and propose Text2Thermal, a framework for physics-aware thermal image synthesis from textual priors. Rather than inferring the unobservable radiometric factors from RGB, we supply them explicitly through thermally grounded captions encoding material, weather, time-of-day, and heat-emission state, and adapt a pre-trained Stable Diffusion backbone to the thermal domain. Because the radiometric content is determined entirely by the prompt, Text2Thermal synthesizes thermal imagery without requiring a registered RGB image at inference. Where spatial guidance is desired, an optional control signal imparts scene geometry without disturbing the prompt-specified radiometry. On M3FD and FLIR, Text2Thermal achieves state-of-the-art FID among thermal image synthesis methods, and we additionally report results on the FMB dataset, while offering text-level control that translation-based approaches cannot provide.

Abstract

Image-to-Image Translation

Explore similar work

Text2Thermal: Physics-Aware Thermal Image Synthesis from Textual Priors

T-CLIP: Enabling Thermal Perception for Contrastive Language-Image Pretraining

Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception