TF-CADE: Foreground-Concentrated Text-Video Alignment for Zero-Shot Temporal Action Detection

Yearang Lee; Ho-Joong Kim; Seong-Whan Lee

cs.CVAug 18, 2026

TF-CADE: Foreground-Concentrated Text-Video Alignment for Zero-Shot Temporal Action Detection

Authors: Yearang Lee, Ho-Joong Kim, Seong-Whan Lee

Organizations: Dept. of Artificial Intelligence, Korea University, Seoul, Korea

Abstract

Zero-Shot Temporal Action Detection (ZSTAD) aims to lo- calize and recognize action instances from unseen action categories in untrimmed videos. Although existing meth- ods have shown effectiveness by advancing architectural text-video alignment, they still struggle with capturing se- mantic distinctions between action classes, resulting in text- irrelevant predictions. To address this issue, we propose a Text-Foreground Concentrated Alignment for zero-shot temporal action DEtector (TF-CADE) that explicitly aligns textual information with action-relevant foreground regions. Specifically, we introduce Action Concentrate Aggregation (ACA), which extracts action concentrate scores to aggregate temporally informative video segments into a foreground- weighted video embedding. This foreground concentrated alignment enhances the semantic consistency between text and video features and improves inter-class discriminabil- ity. In addition, a Certainty-based Confidence Re-weighting (CCR) strategy refines per-snippet confidence scores by lever- aging foreground-aware similarity, effectively suppressing irrelevant action classes during inference. Extensive evalua- tions show that our TF-CADE not only achieves state-of-the- art performance under in-distribution settings but also excels in cross-dataset generalization to unseen action classes.

Figures & tables

Panel of Figure 1 — Figure 1: Class confidence scores under different textual inputs. (a) Ti-FAD [ 13 ] produces nearly identical confidence distributions whether the input text is the correct action label (“ThrowDiscus”) or an unrelated word (“XYZ”). In contrast, our (b) TF-CADE generates clearly differentiated confidence scores.

Panel of Figure 2 — Figure 2: Cosine similarity heatmap of the visual-aware text features produced by the cross-modal adaptation in Ti-FAD [ 13 ] across various action classes on THUMOS14 [ 8 ] . (a) With video features including both foreground and background regions, and (b) with only foreground regions used for cross-modal adaptation.

Panel of Figure 3 — Figure 3: Overview of TF-CADE. (a) Overall framework of the proposed TF-CADE, which builds upon the cross-modal adaptation baseline and introduces text-foreground concentrated alignment to explicitly associate textual information with action-relevant visual regions. (b) Illustration of the text-foreground concentrated alignment method at the $l$ -th layer, which consists of ACA and CCR. ACA computes temporal action certainty scores and generates a foreground-weighted video embedding for foreground-text alignment during training, while CCR refines per-snippet classification scores using the foreground-based video-level similarity score during inference.

Train Split	Model	Feature		THUMOS14						ActivityNet v1.3
Train Split	Model	Video	Text	0.3	0.4	0.5	0.6	0.7	Avg.	0.5	0.75	0.95	Avg.
50% Seen 50% Unseen	EffPrompt [ 10 ]	I3D [ 4 ]	CLIP-B [ 23 ]	37.2	29.6	21.6	14.0	7.2	21.9	32.0	19.3	2.9	19.6
	STALE [ 20 ]	I3D [ 4 ]	CLIP-B [ 23 ]	38.3	30.7	21.2	13.8	7.0	22.2	32.1	20.7	5.9	20.5
	DeTAL [ 15 ]	I3D [ 4 ] / R(2+1)D [ 27 ]	CLIP-B [ 23 ]	38.3	32.3	24.4	16.3	9.0	24.1	34.4	23.0	4.0	22.4
	STOV-TAL [ 7 ]	CLIP-B [ 23 ]	CLIP-B [ 23 ]	44.2	35.7	25.7	16.5	8.0	26.0	42.1	25.0	1.3	24.8
	ZEETAD [ 22 ]	I3D [ 4 ]	CLIP-B [ 23 ]	45.2	38.8	30.8	22.5	13.7	30.2	39.2	25.7	3.1	24.9

Table 1: Performance comparison with the state-of-the-art methods on THUMOS14 and ActivityNet v1.3 under in-distribution setting. The average mAP is calculated to evaluate performance at different tIoU thresholds: [0.3:0.1:0.7] for THUMOS14 and [0.5:0.05:0.95] for ActivityNet v1.3. † indicates that the results are produced utilizing additional text descriptions (e.g., sentence-level captions by CoCa [ 31 ] ).

Evaluation Setting	Model	Feature		THUMOS14
Evaluation Setting	Model	Video	Text	0.3	0.4	0.5	0.6	0.7	Avg.
50%-50%	STALE [ 20 ]	I3D [ 4 ]	CLIP-B [ 23 ]	1.3	0.7	0.6	0.6	0.4	0.7
	EffPrompt [ 10 ]	I3D [ 4 ]	CLIP-B [ 23 ]	5.4	4.4	3.5	2.7	1.9	3.6
	T3AL [ 16 ]	CoCA [ 31 ]	CoCA [ 31 ]	20.7	14.3	8.9	5.3	2.7	10.4
	Ti-FAD [ 13 ]	I3D [ 4 ]	CLIP-B [ 23 ]	19.3	15.8	11.7	7.7	4.1	11.7
	TF-CADE (Ours)	I3D [ 4 ]	CLIP-B [ 23 ]	40.5	34.1	26.5	17.2	8.9	28.2

Table 2: Performance comparison with the state-of-the-art methods on THUMOS14 under zero-shot cross-dataset generalization setting. All methods are trained only on ActivityNet v1.3 and directly evaluated on THUMOS14 without any fine-tuning.

Table 3: Performance comparison with the state-of-the-art methods under zero-shot cross-dataset generalization setting. All models are trained on only the training dataset and directly evaluated on the target datasets without any fine-tuning.

Method	Text	Video	mAP@tIoU (%)
Method	Text	Video	0.3	0.5	0.7	Avg.
Baseline	✓		20.1	15.8	8.0	14.9
		✓	21.4	17.0	8.7	16.0
	✓	✓	22.3	17.4	8.4	16.3
Ours	✓		27.3	21.2	9.8	19.9
		✓	28.6	22.6	10.7	21.1

Table 4: Comparison of cross-attention configurations. The table shows how different modality update choices affect performance and support our simplified video-only update design.

Method	mAP@tIoU(%)
Method	0.3	0.4	0.5	0.6	0.7	Avg.
Baseline	21.4	19.6	17.0	13.4	8.7	16.0
$+\mathcal{L}_{video}$	22.0	20.2	17.4	13.6	8.6	16.4
$+\text{CCR}$	27.1	24.9	21.0	16.0	9.8	19.7
$+\mathcal{L}_{video}$ & CCR	28.6	26.2	22.6	17.1	10.7	21.1

Table 5: Component analysis on THUMOS14. The average mAP is calculated to evaluate performance at all tIoU thresholds.

Method	mAP@tIoU(%)
Method	0.3	0.4	0.5	0.6	0.7	Avg.
w/o filter	35.7	28.6	21.7	13.5	6.7	21.3
w/ filter	42.8	36.5	28.6	19.0	10.1	27.4

Table 6: Effectiveness of smoothing filter under zero-shot cross-dataset generalization setting. The models are trained on ActivityNet v1.3 and evaluated on THUMOS14.

Table 7: Ablation studies of ACA on THUMOS14 in the 50%-50% setting.

Panel of Figure 4 — Figure 4: False positive analysis on THUMOS14 using DETAD [ 2 ] . Compared to Ti-FAD, our TF-CADE reduces the dependency on the Wrong Label Error.

Figure 5: False negative analysis on THUMOS14 using DETAD [ 2 ] . CCR reduces false negatives consistently across actions of different temporal durations, especially for extremely short (XS) and long (XL) actions.

Figure 6: Visualization of Temporal Action Certainty Maps on THUMOS14. (a) Certainty map $m_{\text{max}}^{(l)}$ without smoothing. (b) Filtered Certainty map $m_{\text{filter}}^{(l)}$ by Gaussian kernel. The bar below plot indicates the ground-truth action segments.

Temporal Action Detection

Temporal action detection identifies action categories and their start and end times in untrimmed video.

See all 41 papers

Zero-Shot Temporal Action Localization Through Textual Guidance
May 21, 2026Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero +3Zero-Shot Learning Temporal Action Detection
University of Trento, Trento, Italy · Fondazione Bruno Kessler, Trento, Italy
Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection
Mar 25, 2026Sa Zhu, Wanqian Zhang, Lin Wang +4Temporal Action Detection Temporal Video Understanding
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences · State Key Laboratory of Cyberspace Security Defense +2
Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detection
Apr 20, 2026Sa Zhu, Wanqian Zhang, Lin Wang +3Cross-Modal Alignment Temporal Action Detection
Institute of Information Engineering, Chinese Academy of Sciences, School of Cyber Security, University of Chinese Academy of Sciences, State Key Laboratory of Cyberspace Security Defense, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Hangzhou Dianzi University, Hangzhou, China +1

May 21, 2026J/K move · Enter open · S save

Zero-Shot Temporal Action Localization Through Textual Guidance

Benedetta Liberatori, Alessandro Conti, Lorenzo Vaquero, Paolo Rota+2

University of Trento, Trento, Italy · Fondazione Bruno Kessler, Trento, Italy

Zero-Shot Learning Temporal Action Detection Temporal Video Understanding Temporal Action Localization Zero-Shot Action Recognition

Open paper AskPDF arXiv

Zero-shot temporal action localization (ZS-TAL) consists of classifying and localizing actions in untrimmed videos, where action classes are unseen at training time. Existing work uses Vision and Language Models (VLMs), taking advantage of their strong zero-shot transfer capabilities. Yet, these models face evident challenges with fine-grained action classification, making it difficult to directly use them to distinguish between the presence and absence of an action. Most current methods for ZS-TAL address these challenges by training models on large-scale video datasets, which require annotated data and often result in limited generalization performance. Recently, approaches discarding the use of labeled data have emerged as an alternative. Following this direction, we propose a novel approach, ``Textual Guidance for finer localization of actions in videos'' (TEGU), that compensates for the lack of supervision from training data by exploiting rich textual information derived from large language models and structured text extracted from captions. This additional linguistic context can improve fine-grained discrimination by providing richer cues about fine-grained action differences within videos. We validate the effectiveness of the proposed method by conducting experiments on the THUMOS14 and the ActivityNet-v1.3 datasets. Our results show that, by exploiting rich textual information for improved action localization, TEGU outperforms state-of-the-art ZS-TAL approaches that do not involve training

Abstract

Temporal Action Detection

Explore similar work

Zero-Shot Temporal Action Localization Through Textual Guidance

Decompose and Transfer: CoT-Prompting Enhanced Alignment for Open-Vocabulary Temporal Action Detection

Denoise and Align: Diffusion-Driven Foreground Knowledge Prompting for Open-Vocabulary Temporal Action Detection