From Imitation to Intuition: Intrinsic Reasoning for Open-Instance Video Classification

Ke Zhang; Xiangchen Zhao; Yunjie Tian; Jiayu Zheng; Vishal M. Patel; Di Fu

cs.CVMar 11, 2026

From Imitation to Intuition: Intrinsic Reasoning for Open-Instance Video Classification

Authors: Ke Zhang, Xiangchen Zhao, Yunjie Tian, Jiayu Zheng, Vishal M. Patel, Di Fu

Organizations: Johns Hopkins University · ByteDance Inc

Abstract

Conventional video classification models, acting as effective imitators, excel in scenarios with homogeneous data distributions. However, real-world applications often present an open-instance challenge, where intra-class variations are vast and complex, beyond existing benchmarks. While traditional video encoder models struggle to fit these diverse distributions, vision-language models (VLMs) offer superior generalization but have not fully leveraged their reasoning capabilities (intuition) for such tasks. In this paper, we bridge this gap with an intrinsic reasoning framework that evolves open-instance video classification from imitation to intuition. Our approach, namely DeepIntuit, begins with a cold-start supervised alignment to initialize reasoning capability, followed by refinement using Group Relative Policy Optimization (GRPO) to enhance reasoning coherence through reinforcement learning. Crucially, to translate this reasoning into accurate classification, DeepIntuit then introduces an intuitive calibration stage. In this stage, a classifier is trained on this intrinsic reasoning traces generated by the refined VLM, ensuring stable knowledge transfer without distribution mismatch. Extensive experiments demonstrate that for open-instance video classification, DeepIntuit benefits significantly from transcending simple feature imitation and evolving toward intrinsic reasoning. Our project is available at https://bwgzk-keke.github.io/DeepIntuit/.

Figures & tables

Figure 1: Overview of DeepIntuit. Unlike conventional classifiers that rely on direct input-to-label mapping, DeepIntuit evolves open-instance video classification from imitation to intuition. Through staged training, it develops intrinsic reasoning that enables stable and calibrated decisions.

Figure 2: Close-instance vs. open-instance video classification. (a) Close-instance benchmarks have relatively homogeneous intra-class distributions. (b) Open-instance settings exhibit broader, open-ended intra-class variation that better reflects real-world data. (c) Consequently, conventional video encoders fit close-instance data well but struggle to generalize, whereas VLMs with stronger semantic priors are more robust in the open-instance regime.

Figure 3: Pipeline of DeepIntuit. The framework follows three stages: (1) cold-start supervised alignment for initializing reasoning capability, (2) GRPO-based reinforcement learning to refine intrinsic reasoning, and (3) intuitive calibration that translates intrinsic reasoning into stable and calibrated final decisions.

Figure 4: Effect of calibration and reasoning model choice. Left: Initializing Stage-3 from the Stage-2 model yields a $>10\%$ F1 improvement compared with using an external VLM model. Right: DeepIntuit-S 2 , trained with cold-start supervised alignment and GRPO, consistently outperforms the baseline reasoning model ( e.g. , Qwen2.5-VL) across categories.

Figure 5: Qualitative examples on open-instance videos. The refined model generates structured intrinsic reasoning ( e.g. , observations and context) before predicting the final label. The examples show accurate classification of both normal and abnormal events, illustrating robust open-instance generalization in real-world scenarios.

Model	Method	Open	Acc. ( $\uparrow$ )	F1 Score ( $\%,\uparrow$ )
Model	Method	Open	Acc. ( $\uparrow$ )	Nor.	Hat.	Off.	Avg.
GPT	GPT-4o [ 12 ]	$\checkmark$	72.19	84.16	55.45	12.50	50.70
GPT	GPT-4.1 [ 1 ]	$\checkmark$	70.46	83.58	60.38	23.53	55.83
Gemini	Gemini-2.5-Flash [ 8 ]	$\checkmark$	70.94	83.98	18.18	40.00	47.39
Gemini	Gemini-2.5-Pro [ 8 ]	$\checkmark$	75.81	84.71	61.11	0.00	48.61
Video Encoder	UniFormerV2 [ 19 ]	$\times$	59.17	72.10	8.33	35.42	38.62

Table 1: Quantitative results on the MultiHateClip benchmark. We compare DeepIntuit with close-sourced VLMs, conventional video encoders, and open-sourced Qwen2.5-VL-7B variants under different post-training strategies. Here, Nor., Hat., Off., and Avg. denote Normal, Hateful, Offensive, and Average, respectively.

Model	Method	Open	Acc. ( $\uparrow$ )	F1 Score ( $\%,\uparrow$ )
Model	Method	Open	Acc. ( $\uparrow$ )	Normal	Abnormal	Avg.
GPT	GPT-4o [ 12 ]	$\checkmark$	62.76	63.32	62.18	62.75
GPT	GPT-4.1 [ 1 ]	$\checkmark$	70.53	69.23	71.72	70.48
Gemini	Gemini-2.5-Flash [ 8 ]	$\checkmark$	73.33	70.79	75.47	73.13
Gemini	Gemini-2.5-Pro [ 8 ]	$\checkmark$	73.47	70.11	76.15	73.13
Video Encoder	UniFormerV2 [ 19 ]	$\times$	70.12	38.02	78.95	58.49

Table 2: Quantitative results on the SmartHome-LLM benchmark. DeepIntuit achieves the best results on all reported metrics, including overall accuracy, class-wise F1, and average F1, demonstrating stronger and more balanced performance on both Normal and Abnormal events.

Figure 7: Effect of reasoning token length on performance. Increasing intrinsic reasoning improves performance from short to moderate lengths, while very long reasoning yields little additional benefit.

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning
Aug 24, 2026Jiapeng Li, Ping Wei, Wenjuan Han +2Video Reasoning Video QA
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, Xi’an, China · School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China · National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China
A Very Big Video Reasoning Suite
Feb 23, 2026Maijunxian Wang, Ruisi Wang, Juyi Lin +53Reasoning Evaluation Video Reasoning
VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation
May 15, 2026Yiming Zhao, Yu Zeng, Wenxuan Huang +11Temporal Video Grounding Tool-Using Vision-Language Agents
University of Science and Technology of China · Xiaohongshu Inc. · East China Normal University +1

Aug 24, 2026J/K move · Enter open · S save

IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning

Jiapeng Li, Ping Wei, Wenjuan Han, Song-Chun Zhu+1

National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, Xi’an, China · School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China · National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China

Video Reasoning Video QA

Open paper AskPDF arXiv

Video understanding requires intelligent agents to transcend mere recognition of visual facts and comprehend the underlying intents behind human actions (often termed the "dark matter" of social intelligence). To bridge the gap between visual observation and intent reasoning, we introduce a novel task, IntentQA, and contribute a large-scale VideoQA dataset specifically tailored for this purpose. However, recognizing that standard metrics may overestimate capabilities due to dataset biases, we go beyond simple accuracy to rigorously evaluate model robustness. We augment the benchmark by generating five distinct contrast sets via Large Language Models (LLMs) and introducing a "Contrast Performance Decline" metric. We propose the X-CaVIR (eXplainable Context-aware Video Intent Reasoning) framework, which leverages three types of "Cognitive Context" to enhance video analysis: i) Situational Context via a cross-modal Video Query Language (VQL) module, ii) Contrastive Context via a Contrastive Learning module, and iii) Commonsense Context via a Commonsense Reasoning module. Crucially, to overcome the opacity of traditional black-box models, we refine the integration of LLMs within X-CaVIR by employing a transparent pipeline that synergizes video captions with VQA model outputs. This approach not only improves performance by effectively utilizing rich commonsense knowledge but also renders the reasoning process explicitly interpretable. Extensive experiments demonstrate the effectiveness of our components, the superiority of X-CaVIR over state-of-the-art baselines, and its stability against perturbations on the contrast sets.

Abstract

VLM Reasoning

Explore similar work

IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning

A Very Big Video Reasoning Suite

VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocation