Concept Driven Domain Adaptation: Finding an Abstract Needle in a Haystack

Haiming Zhao; Tai Wang; Kun Zhang; Xicheng Peng; Zhiyang Li

cs.CVOct 1, 2026

Concept Driven Domain Adaptation: Finding an Abstract Needle in a Haystack

Authors: Haiming Zhao, Tai Wang, Kun Zhang, Xicheng Peng, Zhiyang Li

Organizations: Laboratory for Artificial Intelligence and New Forms of Education Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan, China · Wuhan United Imaging Healthcare Co., Ltd., Wuhan, China

Abstract

Science teachers frequently search for documentary excerpts not by describing what appears on screen, but by querying the abstract concepts they intend to teach. This use case exposes a limitation of existing language-based video moment retrieval methods, which typically assume that queries describe observable events, whereas instructional search requires retrieving concrete visual phenomena that instantiate an underlying scientific principle. We study this setting as concept-to-example video retrieval, an abstract-needle-in-a-haystack problem where compact curriculum concepts must be grounded in temporally sparse documentary evidence. To bridge this abstraction gap, we propose Concept-Driven Domain Adaptation (CDDA), a three-stage framework for adapting two-tower vision-language models to concept-level retrieval. CDDA treats concepts as intermediate semantic anchors: it first structures the textual embedding space with textbook and teacher-handbook example-concept pairs, then transfers this concept-aware geometry to documentary visuals under a frozen visual encoder, and finally jointly adapts both encoders with sparse visual concept supervision. From a geometric perspective, this staged alignment reduces text-concept and vision-concept angular gaps, thereby encouraging concept-level adaptation while preserving the pretrained model's concrete image description alignment. On a curated middle-school physics retrieval benchmark, CDDA achieves stronger pedagogically oriented concept retrieval than several competitive multimodal baselines, including Qwen3-VL-Embedding-2B, while maintaining concrete image-text matching after adaptation.

Figures & tables

Panel of Figure 1 — Figure 1 : Illustration of concept-to-example video retrieval and its geometric motivation. Left: an abstract curriculum concept is used as a query to retrieve concrete documentary clips that instantiate the underlying scientific principle. Right: the concept embedding serves as an intermediate semantic anchor between concrete textual examples and visual instances.

Figure 2 : Overview of CDDA. Stage 1 structures the text space using example–concept pairs from educational texts. Stage 2 aligns documentary clips with the curriculum-aware concept space while keeping the visual encoder frozen. Stage 3 jointly adapts both encoders to the documentary domain.

Collection	Episodes	Duration	Production team
When We Left Earth: The NASA Missions	4	45 min.	Discovery Channel
Earth: The Power of the Planet	1	60 min.	BBC Two
The Fabric of the Cosmos	2	55 min.	PBS NOVA
100 Greatest Discoveries	1	44 min.	Discovery Channel
Cosmos: A Spacetime Odyssey	1	45 min.	National Geographic
What on Earth Is Wrong with Gravity?	1	44 min.	BBC Horizon

Table 1: Documentary sources used to construct the image-concept supervision set. “Episodes” denotes the number of episodes used in our dataset, not the total number of episodes in the original series.

Panel of Figure 3 — Figure 3 : Examples of visual concept supervision. Each image is paired with an abstract curriculum concept $c$ ; concrete descriptions $u$ are shown only for readability.

	Force	Gravity	Newton	Newton’s	Universal	pedagogical
	Force	Gravity	Newton	First Law	Gravity	scores
# of segments	(6)	(20)	(9)	(1)	(25)
CDDA	5	6	5	almost	5+1NtM	24
CDDA(II&III)	3	6	4	almost	4+1NtM	20
CDDA(II&III) d+c	1	5	3	almost	4	13
CDDA(II&III) d	0	4+1	1	almost	3+1NtM	12

Table 2 : Total hits of multiple large models on each abstract concept in top 10 results

Method	MAP@10	NDCG@10	MAP@5	NDCG@5
Gemini Embed. 2	0.4924	0.5710	0.5300	0.5968
CDDA	0.3848	0.5402	0.3807	0.5094
Qwen3-VL-2B	0.3723	0.5374	0.3793	0.4814
InternVideo-Next	0.3245	0.4448	0.4167	0.4941
CDDA(II&III)	0.2974	0.4309	0.3753	0.4601
ImageBind	0.2789	0.4161	0.2340	0.3470

Table 3: Standard retrieval metrics averaged over the five concept queries. Relevance is binary; the NtM segment is treated as an ordinary relevant segment for MAP and NDCG.

Panel of Table 4 — Table 4 : Preservation of image–description matching after concept adaptation. Each entry reports image–description correctness, followed by image–concept correctness in parentheses.

Panel of Figure 4 — Figure 4 : Commercially highlighted moments for Newton’s Dark Secrets collected in April 2026. These screenshots are used only as qualitative context, not as a reproducible baseline.

Panel of Figure 5 — Figure 5 : Original Newton’s cannonball thought experiment.

Panel of Figure 6 — Figure 6 : Non-trivial match (NtM): a variant of Newton’s cannonball thought experiment that bridges Gravity and Universal Gravity.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 11 — Figure A1 : Diverse forces in Newton’s Dark Secrets besides gravity or universal gravity

Model Name and Version	Parameter Scales	Checkpoint File Size
Gemini-Embedding-2	Not Officially Declared	Not Officially Declared
Qwen3-VL-Embedding-2B	2B	4.3 GB
ImageBind-huge	Not Officially Declared	4.8 GB
InternVideo-Next large_p14_res224_f16)	0.3B 1	622.1 MB
CN-CLIP ViT-B/16	188M [ 35 ]	753.2 MB

Appendix

Table A1 : Parameter scales of models

Concept	Example / fact
Force	The tires, pedals, or handlebars of a bicycle are engraved with uneven patterns to increase friction by increasing the roughness of the contact surface.
	A person kicks a ball; a horse pulls a cart; a person pushes a table.
Gravity	The effect of gravity on an object in free fall is to make its velocity increase.
	An astronaut experiences weightlessness in the Space Station.
Newton	In 1664, force was defined as the time derivative of momentum.
	Newton’s three laws of motion were formulated as axioms.

Appendix

Table A2: Representative textual concept–example pairs used in Stage 1 of CDDA.

Panel of Table 14 — Table A3 : Detailed outputs for the mapping-preservation examination. The text before parentheses is the image description; the term in parentheses is the associated or predicted concept.

Data source	Size	Usage
Image–concept examples	44 / 12 / 6	Train / validation / test diagnostic
Textual example–concept pairs	118 + 32 = 150	Stage 1 text structuring
Evaluation documentary subtitles	832	Candidate segmentation
Evaluation candidate segments	146	Retrieval evaluation
Concept queries	5	Retrieval queries

Appendix

Table A4: Summary of supervision and evaluation data.

Concept query	Relevant segments
Force	6
Gravity	20
Newton	9
Newton’s First Law	1
Universal Gravity	25

Appendix

Table A5: Number of relevant segments per concept query in the 146-segment evaluation corpus.

Panel of Figure 17 — Figure A2 : Five representative concept images used as InternVideo-Next query inputs. F. L. is short for First Law, and G. is short for Gravity.

Variant	Difference from CDDA
CDDA(II&III)	Removes Stage 1 textual concept structuring and trains only with image–concept pairs.
CDDA(II&III) d	Replaces abstract concept labels with literal fact descriptions.
CDDA(II&III) d+c	Appends a concept sentence to each fact description, e.g., “Planets orbit the Sun in elliptical orbits. This is universal gravity.”
CDDA(II&III) d text	Text-only retrieval counterpart of the description-based variant.

Appendix

Table A6: Ablation variants used in the retrieval experiments.

Item	Value
Base model	CN-CLIP [ 35 ]
Text adaptation component	Chinese RoBERTa [ 36 ]
Optimizer	AdamW
Batch size, Stage 2 / Stage 3	4 / 4
Learning rate, Stage 2 / Stage 3	$3\times 10^{-6}$ / $3\times 10^{-6}$
Epochs, Stage 2 / Stage 3	15 / 15

Appendix

Table A7: Training and inference settings for CDDA.

Panel of Figure 20 — Figure A3 : Generated depictions of Newton’s cannonball thought experiment. Subfigures (a)–(c) share the prompt: “Newton’s cannonball thought experiment, Earth in space, parabolic trajectories, scientific diagram, vintage engraving style, clear labels, white background”. Subfigure (d) uses the Chinese prompt “Newton’s cannonball thought experiment”.

Explore similar work

Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
May 27, 2026Takuya Murakawa, Toru TamakiLong Video Question Answering In-Context Learning
Nagoya Institute of Technology, Nagoya, Japan
S $^2$ COPE: Self-Supervised Concept Discovery via Preference Learning
Jun 12, 2026Shilong Xiang, Zirui Zhang, Chengzhi MaoUnsupervised Preference Learning
Rutgers University
AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning
Aug 3, 2026Jingqi Tian, Haoji Zhang, Lin Chen +7Think
Tsinghua Shenzhen International Graduate School, Tsinghua University · Alibaba Group · Peking University

May 27, 2026, cs.CVJ/K move · Enter open · S save

Reflective Dialogue between Teacher and Solver Agents for Video Question Answering

Takuya Murakawa, Toru Tamaki

Nagoya Institute of Technology, Nagoya, Japan

Long Video Question Answering In-Context Learning Reflections Teacher

Open paper AskPDF arXiv

Various approaches have been proposed to adapt Vision-Language Models (VLMs) to specialized domains for Video Question Answering, including fine-tuning and in-context learning. However, acquiring task-specific knowledge at the inference phase from only a small labeled support set without fine-tuning remains a challenge. In this paper, we propose a method that achieves adaptation solely through inference-time context injection. Our method first constructs a Reflective Dialogue (RD) -- a multi-turn conversation between two agents, in which Teacher poses each support question and delivers correctness feedback, and Solver answers and provides visual grounding explanations (or reflections) for both correct and incorrect answers. This dialogue history is then used as context at the inference phase. Experiments on the EgoCross benchmark demonstrate that our method outperforms both a baseline zero-shot setting and a standard in-context learning approach that passes support set examples directly, achieving 3rd place in the Open-source Track of the 1st Cross-Domain EgoCross Challenge at the CVPR 2026 EgoVis Workshop, for which this paper also serves as a technical report.

Abstract

Explore similar work

Reflective Dialogue between Teacher and Solver Agents for Video Question Answering

S2^22COPE: Self-Supervised Concept Discovery via Preference Learning

AdaThinkV: Adaptive Thinking for Token-Efficient Video Reasoning

S $^2$ COPE: Self-Supervised Concept Discovery via Preference Learning