Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier

Jingtao Lei; Hongji Li; Dexiang Shu

cs.LGSep 6, 2026

Layer-Wise Gate-Controlled Prompt Truncation in a Multimodal Chest X-Ray Classifier

Authors: Jingtao Lei, Hongji Li, Dexiang Shu

Organizations: Central South University, Changsha 410083, China · Hong Kong Baptist University, Hong Kong, China · University of Minnesota, Minneapolis, MN 55414, USA

Abstract

Mixture of Prompt Experts (MoPE) adapts multimodal transformers through input-dependent prompt composition, while retaining a fixed prompt length. We investigate a layer-wise gating extension in a binary chest X-ray classification pilot study. The controller predicts a retention ratio for each sample, averages these ratios within a mini-batch, and uses the resulting integer length to truncate the static and mixed visual prompts. Retained mixed prompts are also scaled by the individual ratios. In one recorded run per configuration, the gated model reached a best validation accuracy of 0.8996, compared with 0.8969 for the fixed-length baseline; the corresponding final values were 0.8963 and 0.8802. The exported gate statistics imply a retained length of one at all recorded training points, relative to a configured maximum of six. This reduces the complete visual sequence from 210 to 200 tokens, but no direct runtime measurements establish an acceleration benefit. Report-derived labels, report text as input, sequential data partitioning, and the absence of repeated controlled experiments limit interpretation. The findings document prompt shortening under the configured gate penalty; they do not establish sample-specific length allocation, superiority over fixed short prompts, or clinical utility. Code is available at: https://github.com/jingtaolei/mope-dynamic-prompt-truncation.

Figures & tables

Figure 1: Implemented gate-controlled prompt computation at a visual transformer layer. The gate produces individual retention ratios, but their mini-batch mean determines one shared prefix length. Static and mixed prompts are both truncated; only the mixed prompt is multiplied by the individual ratios. Class and image-patch tokens are retained after the block. Dashed arrows identify gradient paths to the gate, rather than gradients through the discrete length decision.

Token component	$L=6$	$L=1$	Reduction (%)
Mixed prompt	6	1	83.33
Static prompt	6	1	83.33
All inserted tokens, including instruction	13	3	76.92
Complete visual sequence	210	200	4.76

Table 1: Analytical token counts per visual layer. Relative reductions use the fixed-length configuration as the denominator.

Setting	Value
Visual encoder	ViT-B/16; 12 layers; dimension 768
Text encoder	BERT-base-uncased; entire branch frozen
Visual / text prompt length	6 maximum / 4
Prompt experts	4
Gate hidden dimension	192
Batch size / random seed	8 / 42

Table 2: Documented gated-run configuration. Values reflect the launch file and archived code; an independently captured run environment is not available.

Measure	Fixed length 6	Gated, maximum length 6
Best validation accuracy	0.8969	0.8996
Step of best validation record	6959	5713
Final validation accuracy	0.8802	0.8963
Final recorded mini-batch accuracy	0.7500	1.0000
Final recorded total training loss	0.6998	0.1553
Final recorded importance loss	3.1456	1.4875

Table 3: Scalar summaries from one recorded run per configuration. Best and final validation values summarize the same repeatedly evaluated validation subset. Training statistics are final exported mini-batch values.

Figure 2: Unsmoothed exported scalar trajectories for the fixed-length baseline and gated run. (a) Validation accuracy over all 31 recorded evaluations. (b) Training mini-batch accuracy. (c) Total training loss, including the configuration-specific auxiliary terms. (d) Importance loss. Panels (b–d) each contain 149 recorded training points; these are neither epoch averages nor repeated independent runs. The vertical axis in panel (a) is restricted to 0.70–0.92 for readability.

Figure 3: Gate behavior at the 149 exported training points. (a) Logged mean gate ratio, averaged over the mini-batch and 12 visual layers. Values equal to zero are shown as stored, without introducing a logarithmic offset. (b) One-token retained length inferred for all visual layers at those recorded points using the non-negativity bound in Equation ( 7 ). The dashed line denotes the configured maximum of six. Panel (b) is an inference from the gate summary and implementation, not a direct length log or a record of unobserved batches.

Prompt Tuning

Prompt tuning adapts pretrained models by learning compact prompts or prefixes instead of updating all model parameters.

See all 76 papers

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal Inference Multimodal Large Language Models
Huawei Technologies Ltd. Shanghai, China
Token-Efficient Multimodal Reasoning via Image Prompt Packaging
Apr 2, 2026Joong Ho Choi, Jiayang Zhao, Avani Appalla +3Efficient Multimodal Inference Multimodal Token Compression
BNY, Pittsburgh, US
ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation
Oct 22, 2026Vu Minh Tran, Thuc Huynh, Son Thanh Luu +1Gated Multimodal Fusion Radiology Report Generation
University of Information Technology · Viet Nam National University Ho Chi Minh City

Sep 4, 2026J/K move · Enter open · S save

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

Guangheng Yang, Zhenliang Ni, Zhenkai Wu, Han Shu+3

Huawei Technologies Ltd. Shanghai, China

Efficient Multimodal Inference Multimodal Large Language Models Preference Optimization Multimodal CoT Reasoning

Open paper AskPDF arXiv

Recently, multimodal large-scale reasoning models have demonstrated remarkable capabilities in solving complex tasks through long Chains-of-Thought (M-CoT). However, excessively long reasoning trajectories incur substantial computational costs and significant KV-cache pressure. Existing CoT compression and alignment paradigms mainly rely on static rules or single-dimensional preferences, lacking fine-grained cross-modal constraints; as a result, they are prone to inducing visual laziness and hallucinatory reasoning. To address these issues, we propose Modality-Contrastive Preference Optimization (MCPO), a highly sample-efficient two-stage length-compression method that requires fewer than 900 training samples. In the compression stage, we introduce a step-level Normalized Cross-Modal Mutual Information (NCMI) pruning algorithm, which automatically identifies and removes visual-independent reasoning steps by comparing the reasoning discrepancies between with-image and no-image contexts. This significantly reduces redundancy and hallucinatory content in the reasoning chains. In the alignment stage, the model first undergoes supervised fine-tuning to achieve domain-adaptive initialization, followed by optimization using an asymmetric multimodal length-controlled preference loss. This objective adopts a highly nonlinear odds-ratio formulation that provides steep gradients in the with-image context to reinforce length constraints for preferred trajectories, while applying a scaled, flat-gradient linear difference in the no-image context to maintain modality consistency, thereby achieving stable cross-modal preference alignment. Extensive experiments on mainstream base models such as Qwen3-VL-Thinking show that our method can reduce CoT length by up to 69.5% and achieve up to 3.34x end-to-end inference speedup while preserving original accuracy.

Abstract

Prompt Tuning

Explore similar work

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

Token-Efficient Multimodal Reasoning via Image Prompt Packaging

ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation