cs.CVOct 1, 2026

Concept Driven Domain Adaptation: Finding an Abstract Needle in a Haystack

Authors: Haiming Zhao, Tai Wang, Kun Zhang, Xicheng Peng, Zhiyang Li

Organizations: Laboratory for Artificial Intelligence and New Forms of Education Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China · Wuhan United Imaging Healthcare Co., Ltd., Wuhan, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Science teachers frequently search for documentary excerpts not by describing what appears on screen, but by querying the abstract concepts they intend to teach. This use case exposes a limitation of existing language-based video moment retrieval methods, which typically assume that queries describe observable events, whereas instructional search requires retrieving concrete visual phenomena that instantiate an underlying scientific principle. We study this setting as concept-to-example video retrieval, an abstract-needle-in-a-haystack problem where compact curriculum concepts must be grounded in temporally sparse documentary evidence. To bridge this abstraction gap, we propose Concept-Driven Domain Adaptation (CDDA), a three-stage framework for adapting two-tower vision-language models to concept-level retrieval. CDDA treats concepts as intermediate semantic anchors: it first structures the textual embedding space with textbook and teacher-handbook example-concept pairs, then transfers this concept-aware geometry to documentary visuals under a frozen visual encoder, and finally jointly adapts both encoders with sparse visual concept supervision. From a geometric perspective, this staged alignment reduces text-concept and vision-concept angular gaps, thereby encouraging concept-level adaptation while preserving the pretrained model's concrete image description alignment. On a curated middle-school physics retrieval benchmark, CDDA achieves stronger pedagogically oriented concept retrieval than several competitive multimodal baselines, including Qwen3-VL-Embedding-2B, while maintaining concrete image-text matching after adaptation.

Figures & tables

Figure 1 : Illustration of concept-to-example video retrieval and its geometric motivation. Left: an abstract curriculum concept is used as a query to retrieve concrete documentary clips that instantiate the underlying scientific principle. Right: the concept embedding serves as an intermediate semantic anchor between concrete textual examples and visual instances.
Figure 2 : Overview of CDDA. Stage 1 structures the text space using example–concept pairs from educational texts. Stage 2 aligns documentary clips with the curriculum-aware concept space while keeping the visual encoder frozen. Stage 3 jointly adapts both encoders to the documentary domain.
CollectionEpisodesDurationProduction team
When We Left Earth: The NASA Missions445 min.Discovery Channel
Earth: The Power of the Planet160 min.BBC Two
The Fabric of the Cosmos255 min.PBS NOVA
100 Greatest Discoveries144 min.Discovery Channel
Cosmos: A Spacetime Odyssey145 min.National Geographic
What on Earth Is Wrong with Gravity?144 min.BBC Horizon
Table 1: Documentary sources used to construct the image-concept supervision set. “Episodes” denotes the number of episodes used in our dataset, not the total number of episodes in the original series.
Figure 3 : Examples of visual concept supervision. Each image is paired with an abstract curriculum concept cc ; concrete descriptions uu are shown only for readability.
 ForceGravityNewtonNewton’sUniversalpedagogical
 First LawGravityscores
# of segments(6)(20)(9)(1)(25) 
CDDA565almost5+1NtM24
CDDA(II&III)364almost4+1NtM20
CDDA(II&III) d+c153almost413
CDDA(II&III) d04+11almost3+1NtM12
Table 2 : Total hits of multiple large models on each abstract concept in top 10 results
MethodMAP@10NDCG@10MAP@5NDCG@5
Gemini Embed. 20.49240.57100.53000.5968
CDDA0.38480.54020.38070.5094
Qwen3-VL-2B0.37230.53740.37930.4814
InternVideo-Next0.32450.44480.41670.4941
CDDA(II&III)0.29740.43090.37530.4601
ImageBind0.27890.41610.23400.3470
Table 3: Standard retrieval metrics averaged over the five concept queries. Relevance is binary; the NtM segment is treated as an ordinary relevant segment for MAP and NDCG.
Table 4 : Preservation of image–description matching after concept adaptation. Each entry reports image–description correctness, followed by image–concept correctness in parentheses.
Figure 4 : Commercially highlighted moments for Newton’s Dark Secrets collected in April 2026. These screenshots are used only as qualitative context, not as a reproducible baseline.
Figure 5 : Original Newton’s cannonball thought experiment.
Figure 6 : Non-trivial match (NtM): a variant of Newton’s cannonball thought experiment that bridges Gravity and Universal Gravity.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1 : Diverse forces in Newton’s Dark Secrets besides gravity or universal gravity
Model Name and VersionParameter ScalesCheckpoint File Size
Gemini-Embedding-2Not Officially DeclaredNot Officially Declared
Qwen3-VL-Embedding-2B2B4.3 GB
ImageBind-hugeNot Officially Declared4.8 GB
InternVideo-Next large_p14_res224_f16)0.3B 1622.1 MB
CN-CLIP ViT-B/16188M [ 35 ]753.2 MB
Appendix
Table A1 : Parameter scales of models
ConceptExample / fact
ForceThe tires, pedals, or handlebars of a bicycle are engraved with uneven patterns to increase friction by increasing the roughness of the contact surface.
 A person kicks a ball; a horse pulls a cart; a person pushes a table.
GravityThe effect of gravity on an object in free fall is to make its velocity increase.
 An astronaut experiences weightlessness in the Space Station.
NewtonIn 1664, force was defined as the time derivative of momentum.
 Newton’s three laws of motion were formulated as axioms.
Appendix
Table A2: Representative textual concept–example pairs used in Stage 1 of CDDA.
Table A3 : Detailed outputs for the mapping-preservation examination. The text before parentheses is the image description; the term in parentheses is the associated or predicted concept.
Data sourceSizeUsage
Image–concept examples44 / 12 / 6Train / validation / test diagnostic
Textual example–concept pairs118 + 32 = 150Stage 1 text structuring
Evaluation documentary subtitles832Candidate segmentation
Evaluation candidate segments146Retrieval evaluation
Concept queries5Retrieval queries
Appendix
Table A4: Summary of supervision and evaluation data.
Concept queryRelevant segments
Force6
Gravity20
Newton9
Newton’s First Law1
Universal Gravity25
Appendix
Table A5: Number of relevant segments per concept query in the 146-segment evaluation corpus.
Figure A2 : Five representative concept images used as InternVideo-Next query inputs. F. L. is short for First Law, and G. is short for Gravity.
VariantDifference from CDDA
CDDA(II&III)Removes Stage 1 textual concept structuring and trains only with image–concept pairs.
CDDA(II&III) dReplaces abstract concept labels with literal fact descriptions.
CDDA(II&III) d+cAppends a concept sentence to each fact description, e.g., “Planets orbit the Sun in elliptical orbits. This is universal gravity.”
CDDA(II&III) d textText-only retrieval counterpart of the description-based variant.
Appendix
Table A6: Ablation variants used in the retrieval experiments.
ItemValue
Base modelCN-CLIP [ 35 ]
Text adaptation componentChinese RoBERTa [ 36 ]
OptimizerAdamW
Batch size, Stage 2 / Stage 34 / 4
Learning rate, Stage 2 / Stage 33×10−63\times 10^{-6} / 3×10−63\times 10^{-6}
Epochs, Stage 2 / Stage 315 / 15
Appendix
Table A7: Training and inference settings for CDDA.
Figure A3 : Generated depictions of Newton’s cannonball thought experiment. Subfigures (a)–(c) share the prompt: “Newton’s cannonball thought experiment, Earth in space, parabolic trajectories, scientific diagram, vintage engraving style, clear labels, white background”. Subfigure (d) uses the Chinese prompt “Newton’s cannonball thought experiment”.
Figure A4 : Valid loss curve

Explore similar work

CardsList
May 27, 2026cs.CV

Reflective Dialogue between Teacher and Solver Agents for Video Question Answering

Various approaches have been proposed to adapt Vision-Language Models (VLMs) to specialized domains for Video Question Answering, including fine-tuning and in-context learning. However, acquiring task-specific knowledge at the inference phase from only a small labeled support set without fine-tuning remains a challenge. In this paper, we propose a method that achieves adaptation solely through inference-time context injection. Our method first constructs a Reflective Dialogue (RD) -- a multi-turn conversation between two agents, in which Teacher poses each support question and delivers correctness feedback, and Solver answers and provides visual grounding explanations (or reflections) for both correct and incorrect answers. This dialogue history is then used as context at the inference phase. Experiments on the EgoCross benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach that passes support set examples directly, achieving 3rd place in the Open-source Track of the 1st Cross-Domain EgoCross Challenge at the CVPR 2026 EgoVis Workshop, for which this paper also serves as a technical report.

Takuya Murakawa, Toru Tamaki

Nagoya Institute of Technology, Nagoya, Japan

Long Video Question AnsweringIn-Context LearningReflectionsTeacher
AskPDF
Jun 12, 2026cs.CV

S2^2COPE: Self-Supervised Concept Discovery via Preference Learning

Current representation learning paradigms force a fundamental compromise: self-supervised methods scale to massive datasets but yield opaque features, whereas interpretable models remain bottlenecked by the need for dense human annotation. We introduce Self-Supervised Concept discOvery via Preference lEarning (\model), a label-free framework that resolves this dilemma. Instead of treating Vision-Large-Language Models (VLLMs) as static feature extractors, \model leverages them as active participants in a self-supervised preference optimization loop. By autonomously hypothesizing, validating, and reinforcing candidate visual attributes directly from raw imagery, our framework discovers novel, structured concepts without a single label. Extensive experiments across natural, medical, and physics domains demonstrate that \model successfully extracts domain-specific concepts where standard VLLMs often fail to generate. By amortizing concept discovery directly into the VLLM backbone through our self-supervised preference objective -- rather than relying on static generation and disjoint filtering -- we achieve up to a 24-point absolute improvement in downstream top-1 classification accuracy on unseen data. Our work suggest that interpretability can emerge through a model's autonomous interaction with incidental visual structures, without any human supervision.

Shilong Xiang, Zirui Zhang, Chengzhi Mao

Rutgers University

UnsupervisedPreference LearningLabel-Free
AskPDF
Aug 3, 2026cs.CV

AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning

Chain-of-thought (CoT) reasoning can improve performance on difficult video questions but often wastes decoding tokens on simple ones. We study whether a video multimodal large language model can adapt its reasoning effort to each question. We propose AdaThinkV, an adaptive framework for video reasoning that learns whether to reason explicitly without offline difficulty labels, manually tuned confidence thresholds, or an external router. During reinforcement learning, AdaThinkV samples matched rollouts in explicit reasoning and direct answering modes for each prompt. ThinkGain estimates the prompt-level utility of explicit reasoning by balancing its accuracy gain against additional response length, providing supervision for both conditional response generation and autonomous mode selection. For difficult prompts, limited rollout exploration can yield groups in which every response is unsuccessful and accuracy rewards show little variation, providing insufficient signal for learning. We therefore introduce Variance Recovery Policy Optimization (VRPO), which retains and progressively expands these groups to recover informative signals from prompts that are difficult yet solvable. At inference, AdaThinkV selects a response mode and generates the response in a single autoregressive sequence. Across a unified suite of video reasoning evaluations, AdaThinkV achieves a mean accuracy of 40.79 with an average of 257.20 output tokens, outperforming the strongest evaluated adaptive baseline by 2.98 points while using 22.7% fewer tokens. Project page: https://trilarflagz.github.io/AdaThinkV/

Jingqi Tian, Haoji Zhang, Lin Chen +7

Tsinghua Shenzhen International Graduate School, Tsinghua University · Alibaba Group · Peking University

Think
AskPDF