Physically Guided TeX Generation for Low-Light Visual Tasks

Daoheng Li; Jianjun Qian; Mengkai Yan; Jian Yang

cs.CVNov 7, 2026

Physically Guided TeX Generation for Low-Light Visual Tasks

Authors: Daoheng Li, Jianjun Qian, Mengkai Yan, Jian Yang

Organizations: PCA Lab, School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China · College of Computer and Software, Hohai University, Nanjing, China · PCA Lab, School of Intelligence Science and Technology, Nanjing University, Nanjing, China

Abstract

Low-light conditions reduce visible contrast and obscure details, while infrared images lack color and texture, motivating the fusion of infrared and visible information for visual tasks. Existing fusion methods primarily optimize image appearance or task-specific features without explicitly disentangling the coupled factors underlying thermal observations, limiting their physical interpretation under thermal degeneracy. Hyperspectral thermal measurements enable physical decomposition, but they require specialized imaging hardware, while pixel-level physical annotations for supervision or evaluation incur additional costs. We propose TeX-Gen, a physically guided framework for TeX generation from paired visible and infrared images in low-light visual tasks. TeX Generation constructs complementary multi-view features and employs independent branch-specific gates to route cues into the Temperature, Emissivity, and Texture maps. TeXMetric integrates radiometric reconstruction, channel structure, and material consistency to assess the generated maps without requiring pixel-level physical labels. TeX Tune improves performance through task-guided adjustments, jointly optimizing the adjustment modules and task network while keeping the generator frozen. Experiments show that TeX-Gen achieves competitive task performance in low-light environments on the FLIR, LLVIP, M3FD, and MSRS datasets. Experiments on HADAR also examine the association between TeXMetric scores and downstream detection performance.

Figures & tables

Fig. 2: Overview of TeX-Gen. TeX Generation constructs multi-view features online and selects complementary evidence through channel-specific gates to form the baseline TeX $Z^{b}$ under TeXMetric constraints. With the generator frozen, TeX Tune applies task-specific corrections to $Z^{b}$ through Multi-scale Guide, Local Activity Gate, and Region-Aware processing before passing the tuned TeX $Z^{a}$ to the complete downstream task network. Task supervision jointly optimizes the channel-tuning modules and task network, whereas the task penalty updates only the channel-tuning modules.

Fig. 3: Radiometric reconstruction in TeXMetric. Emitted and reflected environmental contributions form a normalized infrared reconstruction for comparison with the observed image.

Fig. 4: Channel-structure constraints involving all three channels. Shared source structure guides $T$ , $e$ , and $X$ . Candidate material edges are attenuated in $z_{X}$ , while eligible pairs with similar $z_{e}$ guide temperature ordering in $z_{T}$ .

Fig. 5: Material consistency through fixed-interval emissivity decoding, edge-aware local smoothness, and local ordering. The ordering schematic uses visible contrast to guide relative $z_{e}$ responses for infrared-similar pixel pairs.

Method	Year	Modality	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$
DDQ-DETR [ 55 ]	2023	RGB	73.9	32.2	37.1
YOLOv5 [ 56 ]	2020	RGB	67.8	25.9	31.8
DDQ-DETR [ 55 ]	2023	IR	64.9	24.5	30.9
YOLOv5 [ 56 ]	2020	IR	73.9	35.7	39.5
GAFF [ 5 ]	2021	IR+RGB	74.6	31.3	37.4
ProbEn [ 9 ]	2022	IR+RGB	75.5	31.8	37.9

TABLE I: Detection results on FLIR (%). N/A indicates a value not reported in the corresponding publication.

Method	Year	Modality	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$
DDQ-DETR [ 55 ]	2023	RGB	86.1	45.8	46.7
YOLOv5 [ 56 ]	2020	RGB	90.8	51.9	50.0
DDQ-DETR [ 55 ]	2023	IR	93.9	64.6	58.6
YOLOv5 [ 56 ]	2020	IR	94.6	72.2	61.9
GAFF [ 5 ]	2021	IR+RGB	94.0	60.2	55.8
ProbEn [ 9 ]	2022	IR+RGB	93.4	50.2	51.5

TABLE II: Detection results on LLVIP (%). N/A indicates a value not reported in the corresponding publication.

Method	Year	Modality	$mAP_{50}$	$mAP_{50:95}$
CFT [ 6 ]	2022	IR+RGB	89.8	60.7
TarDAL [ 7 ]	2022	IR+RGB	86.0	56.0
MetaFusion [ 18 ]	2023	IR+RGB	86.7	56.8
CDDFuse [ 58 ]	2023	IR+RGB	87.0	56.5
ICAFusion [ 27 ]	2024	IR+RGB	87.4	57.2
Text-IF [ 59 ]	2024	IR+RGB	N/A	40.4

TABLE III: Aggregate detection results on M3FD (%). N/A indicates a value not reported in the corresponding publication.

Method	Year	Modality	Unlabelled	Car	Person	Bike	Curve	Car Stop	Guardrail	Color Cone	Bump	$mIoU$
EMMA [ 64 ]	2024	IR+RGB	98.37	89.32	71.15	70.58	60.71	70.37	69.59	64.75	75.50	74.48
DCINN [ 65 ]	2024	IR+RGB	98.34	89.08	70.94	69.19	59.31	73.27	72.21	64.02	72.76	74.35
MRFS [ 66 ]	2024	IR+RGB	98.32	88.80	70.55	70.41	58.76	73.85	74.66	62.80	72.37	74.50
TIMFusion [ 67 ]	2024	IR+RGB	98.30	89.19	68.68	69.64	59.55	71.94	69.48	63.69	71.75	73.58
TDFusion [ 68 ]	2025	IR+RGB	98.38	89.75	71.76	70.36	59.77	72.40	75.06	64.61	73.73	75.09
KDFuse [ 69 ]	2025	IR+RGB	98.20	87.50	69.70	60.50	64.70	60.60	85.50	46.90	82.90	73.00

TABLE IV: Nine-class MSRS semantic-segmentation results (%). Unlabelled denotes Background, and

mIoU

includes all nine classes.

Fig. 6: Detection comparison summarized from Tables I through III . Solid blue and hatched orange bars denote Ours and MVR-MoE, respectively, and

\Delta

is Ours minus MVR-MoE in AP percentage points.

Variant	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$	$mAP_{s}$	$mAP_{m}$	$mAP_{l}$
Full TeX Tune	87.5	54.2	52.9	39.1	59.0	73.3

w/o penalty term	86.0	49.0	50.0	36.4	56.0	71.6

w/o multi-scale guide	86.2	49.0	49.8	36.5	55.5	70.9

TABLE V: Ablation of TeX Tune components on FLIR.

Generation criterion	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$	$mAP_{s}$	$mAP_{m}$	$mAP_{l}$
Radiometric reconstruction	80.5	45.4	46.2	32.1	52.5	70.9

Reconstruction + $X$ structure	81.8	46.0	47.3	33.5	53.9	68.3

Reconstruction + material consistency	81.9	45.1	47.0	33.3	53.2	69.4

TABLE VI: Ablation of TeXMetric constraint groups on FLIR.

Fig. 7: Downstream responses to TeXMetric constraints on FLIR, summarized from Table VI .

Direct-input control and task-adjusted TeX
Condition	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$
Generated TeX (without adjustment)	85.5	47.8	49.3
Task-adjusted TeX	87.5	54.2	52.9
Joint tuning ( $mAP_{50:95}$ )
Trainable components	Best epoch	Best before end Best epoch before final epoch
Channel adjustment only (detector frozen)	100	No

TABLE VII: FLIR input-configuration comparison and joint task tuning.

Fig. 8: Joint task tuning on FLIR dataset during TeX Tune. Updating channel corrections alone with the inherited detector frozen makes limited progress, whereas jointly updating the corrections and task branch enables the task network to learn from the tuned TeX.

Fig. 9: Local TeX changes after task tuning on LLVIP. Brighter pixels in the cropped absolute-difference maps indicate larger changes.

Fig. 10: TeXMetric improvement and downstream detection on HADAR. (a) Separate plots of TeXMetric score (lower is better) and AP under YOLOv8-S detector (higher is better), evaluated at the same generator checkpoints. (b) Ground-truth labels and generated maps at epochs 50, 100, 150, and 200 for street, crowded-street, and highway views.

Generation objective	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$
Radiometric reconstruction	80.5	45.4	46.2
Radiometric reconstruction + Entropy	81.4	43.8	45.7
Radiometric reconstruction + Gradient energy	80.9	43.3	45.4
Radiometric reconstruction + MI	81.9	44.6	46.2
Radiometric reconstruction + SSIM	83.9	44.8	47.1
Radiometric reconstruction + PSNR	82.5	43.5	45.9

TABLE VIII: FLIR detection results for radiometric reconstruction with conventional image-quality constraints.

TeX generation objective	MSE $\downarrow$	RMSE $\downarrow$	$mAP_{50}$	$mAP_{75}$	$mAP_{50:95}$	$AR_{100}$
Radiometric reconstruction	0.1357	0.3684	50.4	26.5	28.7	40.2
Radiometric reconstruction + pixel MSE	0.0555	0.2355	54.0	25.0	27.4	41.8
Radiometric reconstruction + pixel RMSE	0.0535	0.2313	42.4	16.9	21.5	36.4
TeXMetric	0.1319	0.3632	54.1	38.1	34.4	43.4

TABLE IX: Pixel-level supervision versus TeXMetric on HADAR under matched training budgets.

Fig. 11: Downstream predictions and map results on HADAR for TeX generated under TeXMetric constraints versus pixel-level supervision (MSE and RMSE).

Physics-Informed Generative Modeling

Physics-informed generative modeling uses physical laws and data-driven generators to create plausible samples and solve scientific inverse problems.

See all 115 papers

Multi-View Structural Representations for Physically-Consistent TeX Generation
Oct 19, 2026Daoheng Li, Jianjun Qian*, Mengkai Yan +1Image Generation Physics-Informed Generative Modeling
Nanjing University of Science and Technology, Nanjing, Jiangsu, China · Hohai University, Nanjing, Jiangsu, China
TeX-1500: A Paired Real-World LWIR Hyperspectral Dataset and Benchmark for Temperature-Emissivity-Texture Decomposition
Jun 2, 2026Cheng Dai, Jiale Lin, Hongyi Xu +3Thermal Imaging Remote Sensing
School of Science, Westlake University, Hangzhou 310030, China · School of Engineering, Westlake University, Hangzhou 310030, China
Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception
May 21, 2026Rusiru Thushara, Yasiru Ranasinghe, Jay Paranjape +1Thermal Imaging Visual Question Answering
Department of Electrical & Computer Engineering, Johns Hopkins University, Baltimore, MD 21218 USA

Oct 19, 2026J/K move · Enter open · S save

Multi-View Structural Representations for Physically-Consistent TeX Generation

Daoheng Li, Jianjun Qian*, Mengkai Yan, Jian Yang

Nanjing University of Science and Technology, Nanjing, Jiangsu, China · Hohai University, Nanjing, Jiangsu, China

Image Generation Physics-Informed Generative Modeling Multi-View Feature Fusion Infrared-Visible Image Fusion Multi-View Image Synthesis

Open paper AskPDF arXiv

Infrared and visible images provide complementary cues for low-light image generation, and physics-based methods can better exploit this complementarity. However, existing physics-based methods still rely heavily on paired data. To address these limitations, we propose Multi-View Routing Mixture-of-Experts (MVR-MoE), a framework for physically consistent TeX generation that learns multi-view structural representations. MVR-MoE mines intrinsic structural cues from infrared and visible images to organize them into multi-view representations, while the proposed multi-view hyper connections expand the feature space and preserve both shared evidence and modality-specific details. A hierarchical routing mechanism further regulates feature utilization at both the structural-view level and the expert level, enabling adaptive exploitation of heterogeneous cues. The predicted temperature, emissivity, and texture maps are constrained by the Stefan-Boltzmann law to ensure physically consistent. Extensive experiments on challenging datasets demonstrate that MVR-MoE achieves superior robustness and discriminability while remaining lightweight.

Abstract

Physics-Informed Generative Modeling

Explore similar work

Multi-View Structural Representations for Physically-Consistent TeX Generation

TeX-1500: A Paired Real-World LWIR Hyperspectral Dataset and Benchmark for Temperature-Emissivity-Texture Decomposition

Thermo-VL: Extending Vision-Language Models to Thermal Infrared Perception