cs.AIApr 7, 2026
SaveReason Analogically via Cross-domain Prior Knowledge: An Empirical Study of Cross-domain Knowledge Transfer for In-Context Learning
Organizations: Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China
Abstract
Despite its success, existing in-context learning (ICL) relies on in-domain expert demonstrations, limiting its applicability when expert annotations are scarce. We posit that different domains may share underlying reasoning structures, enabling source-domain demonstrations to improve target-domain inference despite semantic mismatch. To test this hypothesis, we conduct a comprehensive empirical study of different retrieval methods to validate the feasibility of achieving cross-domain knowledge transfer under the in-context learning setting. Our results demonstrate conditional positive transfer in cross-domain ICL. We identify a clear example absorption threshold: beyond it, positive transfer becomes more likely, and additional demonstrations yield larger gains. Further analysis suggests that these gains stem from reasoning structure repair by retrieved cross-domain examples, rather than semantic cues. Overall, our study validates the feasibility of leveraging cross-domain knowledge transfer to improve cross-domain ICL performance, motivating the community to explore designing more effective retrieval approaches for this novel direction.\footnote{Our implementation is available at https://github.com/littlelaska/ICL-TF4LR}
Figures & tables
Figure 1: Overview of our cross-domain ICL evaluation workflow. ( A ) A source demonstration pool is encoded and indexed into a retrieval database. ( B ) For a target query, the system retrieves semantically similar source demonstrations. ( C ) Retrieved demonstrations are composed into a prompt and fed to a frozen LLM to produce step-by-step reasoning and the final answer.
| Model | Source | Method | ProntoQA | FOLIO | ProofWriter | Logical Deduction | AR-LSAT | GSM8K |
| Gemma3-4B | ProntoQA | BM25 | — | 66.2_{\pagecolor{cyan!15}{\color[rgb]{1,0,0}+2.7}} | — | — | ||
| Embed | — | 66.2_{\pagecolor{cyan!15}{\color[rgb]{1,0,0}+2.9}} | 67.8_{\pagecolor{cyan!25}{\color[rgb]{1,0,0}+4.3}} | 58.3_{\pagecolor{cyan!15}{\color[rgb]{1,0,0}+2.0}} | ||||
| ConE | — | 63.3_{\pagecolor{cyan!25}{\color[rgb]{1,0,0}+7.0}} | ||||||
| FOLIO | BM25 | 93.2_{\pagecolor{cyan!5}{\color[rgb]{1,0,0}+1.2}} | — | 63.0_{\pagecolor{cyan!5}{\color[rgb]{1,0,0}+1.3}} | ||||
| Embed | 93.0_{\pagecolor{cyan!5}{\color[rgb]{1,0,0}+1.0}} | — | 65.0_{\pagecolor{cyan!25}{\color[rgb]{1,0,0}+3.3}} | 27.0_{\pagecolor{cyan!5}{\color[rgb]{1,0,0}+1.7}} | ||||
| ConE | 94.4_{\pagecolor{cyan!15}{\color[rgb]{1,0,0}+2.4}} | — | 64.3_{\pagecolor{cyan!15}{\color[rgb]{1,0,0}+2.7}} | 62.7_{\pagecolor{cyan!5}{\color[rgb]{1,0,0}+1.7}} | 30.0_{\pagecolor{cyan!25}{\color[rgb]{1,0,0}+4.8}} | — |
Table 1: Relative performance of different retrieval strategies across source–target domain pairs, measured against the 0-shot baseline. Deeper blue indicates larger gains, while deeper red denotes greater degradation.
Figure 2: Comparison between zero-shot and cross-domain ICL. Bottom left: Zero-shot reasoning omits a required intermediate link, leading to an incorrect prediction. Bottom right: Cross-domain ICL restores the missing link via a structurally compatible demonstration, yielding the correct answer.
Figure 3: Scaling behaviour of cross-domain transfer under different retrieval baselines . Average Spearman between model size and few-shot performance is plotted for Embedding and BM25 across model families. Shaded regions denote positive and negative transfer.
Figure 4: Shot–performance scaling across cross-domain ICL settings.
Figure 5: Four types of forward chaining.
Figure 6: Distribution of demonstration topologies among repaired zero-shot errors across retrieval methods and transfer directions.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Setting |
| Temperature | 0.0(Greedy) |
| Max Gen Length | 8192 tokens |
| Random Seed | 1-30 |
Table 2: Decoding setup used throughout all experiments.
Figure 7: Shot–performance scaling across cross-domain ICL settings. Accuracy curves for multiple models and transfer directions show that increasing the number of demonstrations does not yield reliable positive scaling. While structurally aligned transfers exhibit eAR-LSATy saturation, mismatched pairs often experience instability or negative transfer as shot size increases.
| Metric | Value |
| Mean EM (Dense Retrieval) | 0.6658 |
| Mean EM (BM25) | 0.6671 |
| Mean Difference (BM25 Dense) | +0.0013 |
| Wilcoxon statistic | 29845.0 |
| Wilcoxon -value | 0.2803 |
| Paired -test | 1.1458 |
Table 3: Overall significance test comparing BM25 and dense retrieval.
| Model | Mean Diff | Wilcoxon | -test | |
| LLaMA3-8B | 100 | +0.0029 | 0.818 | 0.306 |
| Qwen2.5-14B | 100 | +0.0031 | 0.093 | 0.157 |
| Qwen3-4B | 100 | +0.0053 | 0.135 | 0.024 |
| Qwen2.5-32B | 145 | 0.0009 | 0.692 | 0.626 |
| Qwen2.5-7B | 100 | +0.0005 | 0.850 | 0.790 |
Table 4: Per-model significance tests for BM25 vs. dense retrieval.
| Source Target | Mean Diff | Wilcoxon | -test |
| FOL GSM | 0.0039 | 0.0030 | 0.0035 |
| LD FOL | 0.0108 | 0.0089 | 0.0071 |
| PQA PW | 0.0497 | 0.0679 | 0.0333 |
| PW GSM | 0.0055 | 0.0254 | 0.0179 |
| PW LD | +0.0407 | 0.0625 | 0.0021 |
Table 5: Per-direction significance tests for BM25 vs. dense retrieval.
| Metric | Coefficient | -value |
| Pearson | 0.2070 | |
| Spearman | 0.2079 | |
| Kendall | 0.1510 |
Table 6: Correlation between model size and few-shot monotonicity.
| Correlation Metric | Coefficient | -value |
| Pearson | 0.2336 | |
| Spearman | 0.3864 | |
| Kendall | 0.2808 |
Table 7: Overall correlation between task complexity and few-shot monotonicity.
Figure 8: Shot–performance scaling across cross-domain ICL settings. Accuracy curves for multiple models and transfer directions show that increasing the number of demonstrations does not yield reliable positive scaling. While structurally aligned transfers exhibit eAR-LSATy saturation, mismatched pairs often experience instability or negative transfer as shot size increases.
Figure 9: Shot–performance scaling across cross-domain ICL settings. Accuracy curves for multiple models and transfer directions show that increasing the number of demonstrations does not yield reliable positive scaling. While structurally aligned transfers exhibit eAR-LSATy saturation, mismatched pairs often experience instability or negative transfer as shot size increases.
Figure 10: Shot–performance scaling across cross-domain ICL settings. Accuracy curves for multiple models and transfer directions show that increasing the number of demonstrations does not yield reliable positive scaling. While structurally aligned transfers exhibit eAR-LSATy saturation, mismatched pairs often experience instability or negative transfer as shot size increases.
Figure 11: Shot–performance scaling across cross-domain ICL settings. Accuracy curves for multiple models and transfer directions show that increasing the number of demonstrations does not yield reliable positive scaling. While structurally aligned transfers exhibit eAR-LSATy saturation, mismatched pairs often experience instability or negative transfer as shot size increases.
| Setting | Model Reasoning and Output |
| Context | A Japanese game company created the game The Legend of Zelda.All games on the Top 10 list are made by Japanese game companies.If a game sells more than one million copies, then it will be included in the Top 10 list.The Legend of Zelda sold more than one million copies. |
| Zero-shot | Since the Legend of Zelda sold more than one million copies, it should be included in the Top 10 list. However, it is stated that the Legend of Zelda is not on the Top 10 list. This creates a contradiction with the information that all games on the Top 10 list are made by Japanese game companies and the Legend of Zelda is made by a Japanese game company. Therefore, the statement "The Legend of Zelda is not on the Top 10 list" must be true. Final answer: A |
| Final answer: A (True) | |
| Demostration | Question: In the last 25 years, 60 of Scrabble champions have been women and the rest have been men. There is one champion per year. Of the men, 40 had a beard. How many men with beards have been Scrabble champion? Reasoning: 40 of champions are men because 100 - 60 = <<100-60=40>>40 10 champions have been men because 25 x .4 = <<25*.4=10>>10 4 champions have been men with beards because 10 x .4 = <<10*.4=4>>4 |
| Cross-domain | Since the Legend of Zelda sold more than one million copies and all games on the Top 10 list are made by Japanese game companies, and the Legend of Zelda is made by a Japanese game company, it must be on the Top 10 list. Therefore, the statement "The Legend of Zelda is not on the Top 10 list" is false. Final answer: B |
| Final answer: B (False) |
Table 14: Comparison between zero-shot reasoning and cross-domain reasoning reasoning on a cross-domain example. DIN-guided retrieval corrects the logical inconsistency made by zero-shot prompting.
| Setting | Model Reasoning and Output |
| Context | People in this club who perform in school talent shows often attend and are very engaged with school events.People in this club either perform in school talent shows often or are inactive and disinterested community members.People in this club who chaperone high school dances are not students who attend the school.All people in this club who are inactive and disinterested members of their community chaperone high school dances.All young children and teenagers in this club who wish to further their academic careers and educational opportunities are students who attend the school. Bonnie is in this club and she either both attends and is very engaged with school events and is a student who attends the school or is not someone who both attends and is very engaged with school events and is not a student who attends the school. |
| Zero-shot | Let’s break down the information step by step:1. If Bonnie chaperones high school dances, then she is not a student who attends the school (from the third statement).2. If Bonnie does not chaperone high school dances, then she performs in school talent shows often (from the second statement).3. If Bonnie performs in school talent shows often, then she is very engaged with school events (from the first statement).4. If Bonnie is very engaged with school events, then she is a student who attends the school (from the first statement).5. From steps 1 and 4, we can conclude that Bonnie cannot be both a student who attends the school and chaperone high school dances.6. From steps 2 and 3, we can conclude that if Bonnie does not chaperone high school dances, then she is a student who attends the school.7. From steps 5 and 6, we can conclude that Bonnie is not a student who attends the school if she chaperones high school dances, and she is a student who attends the school if she does not chaperone high school dances.8. From the fourth statement, we can conclude that if Bonnie is a student who attends the school, then she is a young child or teenager who wishes to further her academic career and educational opportunities.9. From steps 7 and 8, we can conclude that Bonnie is a young child or teenager who wishes to further her academic career and educational opportunities if she does not chaperone high school dances.10. From the fifth statement, we can conclude that if Bonnie is an inactive and disinterested member of the community, then she chaperones high school dances.11. From steps 7 and 10, we can conclude that Bonnie is an inactive and disinterested member of the community if she chaperones high school dances.12. From steps 9 and 11, we can conclude that Bonnie is both a young child or teenager who wishes to further her academic career and educational opportunities and an inactive and disinterested member of the community if she does not chaperone high school dances.Therefore, the statement is true.Final answer: A |
| Final answer: A (True) | |
| Demostration | Question: Forty percent of the students have elected to learn from home during the pandemic. The remaining students are divided into two equal groups, only one of which is physically in school on any day. What percent of students are present in school? Reasoning: 100-40=<<100-40=60>>60 percent of students are remaining. 60/2=<<60/2=30>>30 percent are present in school. |
| Cross-domain | Letś break down the information step by step:1. If Bonnie chaperones high school dances, then she is not a student who attends the school (from the context).2. If Bonnie does not chaperone high school dances, then she performs in school talent shows often (from the given statement).3. If Bonnie performs in school talent shows often, then she is very engaged with school events (from the context).4. If Bonnie is very engaged with school events, then she is a student who attends the school (from the context).5. From steps 1 and 4, we have a contradiction: Bonnie cannot both be a student who attends the school and not be a student who attends the school.6. Therefore, the given statement "If Bonnie either chaperones high school dances or, if she does not, she performs in school talent shows often" must be false, because it leads to a contradiction.Final answer: B |
| Final answer: B (False) |
Table 15: Comparison between zero-shot reasoning and cross-domain reasoning on a cross-domain example. DIN-guided retrieval corrects the logical inconsistency made by zero-shot prompting.
| Setting | Model Reasoning and Output |
| Context | The bear does not chase the lion. The bear visits the lion. The bear visits the tiger. The lion chases the bear. The lion chases the tiger. The lion visits the bear. The rabbit chases the tiger. The rabbit is big. The rabbit is not green. The rabbit is young. The rabbit visits the tiger. The tiger chases the rabbit. The tiger is rough. The tiger visits the bear. If something chases the bear and it sees the rabbit then the bear does not visit the rabbit. If something chases the lion then the lion is rough. If the rabbit chases the tiger and the tiger visits the rabbit then the rabbit is not big. If something chases the lion then the lion does not visit the tiger. If something chases the rabbit and it chases the tiger then the rabbit chases the bear. If something is young and it does not see the rabbit then it sees the bear. If something visits the lion then it is young. If the tiger visits the bear and the bear is big then the tiger visits the lion. Young things are big. |
| Question | Based on the above information, is the following statement true, false, or unknown? The bear visits the lion. |
| Zero-shot | Let’s break down the information step by step: 1. The bear needs the cow. 2. The cow needs the rabbit. 3. The cow sees the bear. 4. The cow visits the lion. 5. The lion is nice. 6. The lion is red. 7. The lion does not see the cow. 8. The lion visits the cow. 9. The rabbit sees the cow. 10. The rabbit visits the cow. 11. If something needs the cow and the cow visits the rabbit then it visits the cow. 12. If the lion needs the bear then the bear sees the cow. 13. If the lion sees the bear and the bear is rough then the bear does not see the cow. 14. If something visits the lion then the lion visits the bear. 15. If something visits the bear then it needs the bear. 16. If something visits the bear then the bear needs the cow. 17. If something sees the cow then it visits the bear. From the information, we know: - The rabbit sees the cow and visits the cow. - According to point 11, if the lion needs the bear, then the bear sees the cow. - According to point 13, if the lion sees the bear and the bear is rough, then the bear does not see the cow. - Since the lion visits the cow (point 8), and the rabbit visits the cow (point 9), the rabbit needs the cow (point 15). - According to point 16, if the rabbit visits the bear, then the bear needs the cow. - Since the bear needs the cow (point 1), and the rabbit needs the cow (point 15), the bear needs the cow. - According to point 13, if the lion sees the bear and the bear is rough, then the bear does not see the cow. But we know the bear sees the cow (point 3). - Therefore, the bear is not rough (point 13). So, the rabbit is not rough. |
| Final answer: A (True) | |
| Demostration | Context: No reptile has fur. All snakes are reptiles. Question: Based on the above information, is the following statement true, false, or uncertain? Some snake has fur. Options: A) True B) False C) Uncertain Reasoning: 1. "No reptile has fur" means: If something is a reptile, it does not have fur. 2. "All snakes are reptiles" means: If something is a snake, it is a reptile. 3. If all snakes are reptiles, and no reptile has fur, then no snake has fur. 4. The statement "Some snake has fur" would be false, because there is no snake that has fur. The correct option is: B Answer: B |
Table 16: FOLIO-ProofWriter
| Setting | Model Reasoning and Output |
| Cross-domain | 1. The bear needs the cow. 2. The cow needs the rabbit. 3. The cow sees the bear. 4. The cow visits the lion. 5. The lion is nice. 6. The lion is red. 7. The lion does not see the cow. 8. The lion visits the cow. 9. The rabbit sees the cow. 10. The rabbit visits the cow. 11. If something needs the cow and the cow visits the rabbit, then it visits the cow. 12. If the lion needs the bear, then the bear sees the cow. 13. If the lion sees the bear and the bear is rough, then the bear does not see the cow. 14. If something visits the lion, then the lion visits the bear. 15. If something visits the bear, then it needs the bear. 16. If something visits the bear, then the bear needs the cow. 17. If something sees the cow, then it visits the bear. From the given information, we know: - The rabbit sees the cow (point 9). - The rabbit visits the cow (point 10). According to point 11, if the rabbit visits the cow, then the rabbit visits the cow. This is a tautology and doesnt́ provide new information. According to point 12, if the lion needs the bear, then the bear sees the cow. However, the lion does not see the cow (point 7), so the lion cannot need the bear. Therefore, the bear does not see the cow. According to point 13, if the lion sees the bear and the bear is rough, then the bear does not see the cow. Since the lion does not see the cow, this condition does not apply. According to point 14, if the rabbit visits the lion, then the lion visits the bear. However, the rabbit does not visit the lion (point 10), so this condition does not apply. According to point 15, if the rabbit visits the bear, then the rabbit needs the bear. But the rabbit does not visit the bear (point 10), so this condition does not apply. According to point 16, if the rabbit visits the bear, then the bear needs the cow. But the rabbit does not visit the bear (point 10), so this condition does not apply. According to point 17, if the rabbit sees the cow, then the rabbit visits the bear. But the rabbit does not see the cow (point 9), so this condition does not apply. Since none of the conditions point to the rabbit being rough, and we have no information that directly or indirectly states the rabbit is rough, the statement "The rabbit is rough" is unknown. |
| Final answer: C |
Table 17: FOLIO-ProofWriter
| Setting | Model Reasoning and Output |
| Context | Bob is big. Bob is red. Bob is smart. Erin is quiet. Fiona is big. Fiona is blue. Fiona is cold. Fiona is smart. Gary is red. Gary is smart. If Bob is quiet then Bob is cold. If someone is red then they are quiet. If someone is red and quiet then they are cold. All kind people are big. If someone is red and blue then they are kind. All cold, quiet people are blue. Cold, big people are kind. If Erin is big then Erin is cold. If someone is cold and blue then they are smart. |
| Question | Based on the above information, is the following statement true, false, or unknown? Gary is red. |
| Zero-shot | Letś analyze the information step by step: 1. Context Analysis: - Bob is quiet. - Charlie is blue, cold, and nice. - Dave is blue and cold. - Erin is blue. - If something is blue and cold, then it is big. - Round, nice things are big. - All nice, quiet things are furry. - If Bob is round, then Bob is furry. - If Bob is furry and Bob is nice, then Bob is big. - If something is big and blue, then it is quiet. - If something is blue, then it is nice. - All nice things are cold. 2. Determine Erinś Characteristics: - Erin is blue. - From the context, if something is blue, then it is nice. - Therefore, Erin is nice. 3. Check if Erin is Quiet: - The context does not explicitly state that Erin is quiet or not. - There is no direct information linking Erin to being quiet or not being quiet. 4. Conclusion: - Since there is no information provided about Erin being quiet or not, we cannot determine if the statement "Erin is quiet" is true or false. |
| Final answer: B | |
| Demostration | Tumpuses are bright. Tumpuses are yumpuses. Yumpuses are sweet. Yumpuses are wumpuses. Wumpuses are not transparent. Wumpuses are vumpuses. Each vumpus is angry. Vumpuses are rompuses. Rompuses are happy. Every rompus is a zumpus. Zumpuses are brown. Zumpuses are numpuses. Numpuses are not large. Impuses are not happy. Numpuses are jompuses. Jompuses are earthy. Jompuses are dumpuses. Polly is a tumpus. Question: Is the following statement true or false? Polly is not happy. Options: A) True B) False Reasoning: Polly is a tumpus. Tumpuses are yumpuses. Polly is a yumpus. Yumpuses are wumpuses. Polly is a wumpus. Wumpuses are vumpuses. Polly is a vumpus. Vumpuses are rompuses. Polly is a rompus. Rompuses are happy. Polly is happy. Answer: B Context: Impuses are not fruity. Impuses are wumpuses. Each wumpus is not temperate. Wumpuses are dumpuses. Rompuses are not sweet. Dumpuses are kind. Dumpuses are zumpuses. Zumpuses are wooden. Zumpuses are vumpuses. Every vumpus is large. Vumpuses are yumpuses. Yumpuses are transparent. Yumpuses are numpuses. Numpuses are brown. Numpuses are tumpuses. Tumpuses are sweet. Tumpuses are jompuses. Wren is a zumpus. Question: Is the following statement true or false? Wren is sweet. Options: A) True B) False Reasoning: Wren is a zumpus. Zumpuses are vumpuses. Wren is a vumpus. Vumpuses are yumpuses. Wren is a yumpus. Yumpuses are numpuses. Wren is a numpus. Numpuses are tumpuses. Wren is a tumpus. Tumpuses are sweet. Wren is sweet. Answer: A |
| Cross-domain | Letś analyze the information step by step: 1. Erin is blue. 2. If something is blue, then it is nice. → Erin is nice. 3. Erin is cold (not explicitly stated, but we can infer this from the context since blue and cold are often associated). 4. Round, nice things are big. → Since we dont́ know if Erin is round, we cant́ conclude Erin is big. 5. All nice things are cold. → This is consistent with Erin being cold. 6. All nice things are cold. → This is consistent with Erin being cold. 7. If something is blue and cold, then it is big. → Since Erin is blue and cold, Erin is big. 8. If something is big and blue, then it is quiet. → Since Erin is big and blue, Erin is quiet. The statement "Erin is quiet" is true based on the given information. |
Table 18: ProntoQA-ProofWriter
More in this topic
In-Context Learning
In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.
See all 404 papers