Transforming Behavioral Neuroscience Discovery with In-Context Learning and AI-Enhanced Tensor Methods
Authors: Paimon Goulart, Jordan Steinhauser, Dawon Ahn, Kylene Shuler, Edward Korzus, Jia Chen, Evangelos E. Papalexakis
Organizations: UC Riverside Computer Science & Engineering Riverside, CA, USA · UC Riverside Psychology Riverside, CA, USA · UC Riverside Electrical & Computer Engineering Riverside, CA, USA
Scientific discovery pipelines typically involve complex, rigid, and time-consuming processes, from data preparation to analyzing and interpreting findings. Recent advances in AI have the potential to transform such pipelines in a way that domain experts can focus on interpreting and understanding findings, rather than debugging rigid pipelines or manually annotating data. As part of an active collaboration between data science/AI researchers and behavioral neuroscientists, we showcase an example AI-enhanced pipeline, specifically designed to transform and accelerate the way that the domain experts in the team are able to gain insights out of experimental data. The application at hand is in the domain of behavioral neuroscience, studying fear generalization in mice, an important problem whose progress can advance our understanding of clinically significant and often debilitating conditions such as PTSD (Post-Traumatic Stress Disorder). We identify the emerging paradigm of "In-Context Learning" (ICL) as a suitable interface for domain experts to automate parts of their pipeline without the need for or familiarity with AI model training and fine-tuning, and showcase its remarkable efficacy in data preparation and pattern interpretation. Also, we introduce novel AI-enhancements to tensor decomposition model, which allows for more seamless pattern discovery from the heterogeneous data in our application. We thoroughly evaluate our proposed pipeline experimentally, showcasing its superior performance compared to what is standard practice in the domain, as well as against reasonable ML baselines that do not fall under the ICL paradigm, to ensure that we are not compromising performance in our quest for a seamless and easy-to-use interface for domain experts. Finally, we demonstrate effective discovery, with results validated by the domain experts in the team.
Figures & tables
Figure 1. Overview of the proposed pipeline. Our pipeline streamlines time-consuming and expertise-intensive steps, easing the workload for scientists, in contrast to conventional workflow.
Figure 2. Proposed AR-ICL method for behavioral labeling at the (t) -th frame.
Figure 3. Performance of VLM-based calcium activity. F1 score (left axis) and accuracy (right axis).
Method
Macro F1
Balanced Acc
MCC
DINO Baseline
0.203
0.441
0.128
Qwen3 – No ICL
0.370
0.365
0.130
Qwen3 – ICL
0.492
0.782
0.451
Qwen3 – Temporal ICL
0.490
0.777
0.424
Qwen3 – AR-ICL
0.545
0.801
0.517
Table 1. Overall performance comparison. Best values per column are bolded.
CPD
NeAT
Subjects
NC
C
NC
C
1
0.1457
0.2077
0.1099
0.1113
2
0.1255
0.1483
0.0937
0.0919
Table 2. Test RMSE of tensor completion. Note that C and NC indicate if tensors are coupled or not.
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
1
High neuron coactivity during the threatening and familiar environments . Unexpected result, but might be due to (1) the mouse knows the familiar environment poses no threat, and (2) the threatening environment has a strong negative valence associated with it — where the animal is afraid (demonstrating high freezing behavior) throughout learning to discriminate the environments.
High neuron co-activity during early learning phase and sustained activity throughout the learning process , with peaks occurring in both safe and threatening environments, suggesting a general engagement during environmental exploration and initial learning. This latent variable is not directly supported by previous research, but some studies report broad prelimbic cortex activation during early learning stages.
1
2
2
Threatening environment responding component, with high neuron co-activity during threatening environment exposure ; especially near the end of the behavior paradigm, where the mouse knows the threatening environment signals threat.
High neuron co-activity during the threatening environment , with a peak occurring near the end of learning, suggesting increased engagement as the animal consolidates threat-related memory.
4
4
3
High neuron co-activity during familiar environment with some activity in the safe and threatening environments towards the end of learning (suggesting mouse is recognizing the environments).
High neuron co-activity during the initial phase of learning and again during the later phase of learning , with minimal activity during the middle phase. This suggests a biphasic engagement pattern, possibly reflecting early encoding and late consolidation or reactivation processes.
3
3
Table 3. Examples of AI-driven interpretation of latent neural components. We compare expert-written interpretations with VLM-generated hypotheses and discovery scores.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Freezing
Fleeing
Grooming/Exploring
Method
Precision
Recall
F1
F2
Precision
Recall
F1
F2
Precision
Recall
F1
F2
DINOv2 Baseline
0.251
0.612
0.356
0.475
0.007
0.571
0.013
0.031
0.930
0.138
0.241
0.166
Qwen3 – No ICL
0.306
0.144
0.196
0.161
0.000
0.000
0.000
0.000
0.878
0.952
0.914
0.936
Qwen3 – ICL
0.422
0.941
0.583
0.755
0.042
0.714
0.080
0.171
0.986
0.692
0.813
0.736
Qwen3 – Temporal ICL
0.484
0.917
0.634
0.778
0.027
0.762
0.052
0.119
0.980
0.653
0.784
0.700
Qwen3 – AR-ICL
0.529
0.905
0.668
0.792
0.051
0.714
0.096
0.199
0.980
0.784
0.871
0.817
Appendix
Table 4. Per-class performance on behavioral video labeling. Best values per column are bolded.
Figure 4. AR-ICL prompt used for temporally consistent behavioral labeling. In addition to fixed ICL examples, the model receives the previous second with its predicted label and the next second as unlabeled temporal context.
Figure 5. Discovery ICL prompt used for AI-driven interpretation of latent neural components. The model receives retrieved neuroscience literature (RAG component), followed by several example latent factors paired with expert interpretations and discovery scores (ICL), and then a new latent component for analysis.
Figure 6. ICL prompt used for coarse calcium activity extraction. The VLM is given example calcium imaging clips paired with binary activity matrices over a spatial grid and predicts the activity matrix for a new input segment.
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
1
Looking at the third factor (across-trial factor), previous research supports the conclusion that a specific neuronal population will be engaged in all environments that an animal is exposed to if that animal does not associate any valence to the environments. Discovery Score = 5; this latent variable is supported by previous research
Looking at the factor (across-trial factor), there is a clear pattern of neural engagement that occurs in three distinct, non-overlapping time windows, each marked by a burst of activity in a subpopulation of neurons. These bursts appear to be temporally separated and occur at approximately equal intervals across the trials, suggesting a periodic or phase-locked response to specific events or transitions within the behavioral paradigm. The activity is sparse and transient, with minimal engagement outside these windows. This pattern is consistent with findings in the prelimbic cortex during learning tasks, where neural populations exhibit phasic, event-locked activity corresponding to discrete stages of learning, such as cue presentation, decision-making, or reward expectation. Specifically, studies have shown that prelimbic neurons can encode trial-specific events or transitions between behavioral states, often with sparse, transient bursts that are temporally aligned with task-relevant cues or outcomes. The three distinct peaks may reflect engagement during three distinct phases of learning or three separate trials within a session, where the animal is processing or updating its internal representation of the environment. Discovery Score = 4; this latent variable is supported by previous literature, which demonstrates phasic, event-locked neural activity in the prelimbic cortex during learning, though the exact triphasic structure observed here is not explicitly reported in the literature.
5
4
2
Looking at the third factor (across-trial factor), previous research supports the existence of a subpopulation of neurons that are highly co-engaged within an environment that is very familiar to the animal and which the animal can recognize as its homecage. Discovery Score = 5; this latent variable is supported by previous research
Looking at the factor (across-trial factor), there is a clear and progressive increase in neural engagement over the course of the behavioral sessions, with minimal activity at the beginning and a sharp rise toward the end, particularly in the final few trials. This suggests that a specific subpopulation of neurons becomes increasingly active as learning progresses, potentially reflecting the consolidation of a learned association or the emergence of a behavioral strategy. This pattern is consistent with findings in the prelimbic cortex during fear conditioning and extinction paradigms, where neural activity in this region is known to increase with learning and is associated with the formation of context-outcome associations. Specifically, studies have shown that prelimbic neurons exhibit ramping activity during learning, with increased firing rates as animals acquire task-relevant information, particularly during the transition from early to late stages of training. This ramping pattern is thought to reflect the encoding of predictive value or the updating of internal models. Discovery Score = 4; this latent variable is supported by previous literature, particularly in the context of learning-related ramping activity in the prelimbic cortex, though direct evidence for this exact pattern in the same behavioral paradigm is not fully established.
5
4
Appendix
Table 5. Full comparison of expert-written interpretations and VLM-generated hypotheses for latent components (Part 1 of 5).
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
3
Looking at the third factor (across-trial factor), some research supports the notion that there is a reorganization of network activity, where a subpopulation of neurons might become highly active when the animal is in a novel, ambiguous environment that might signal safety or it might signal threat. However, other conclusions can be made that there should be a division of this population with one group of neurons co-engaged during exposure to the threatening environment, and a different subpopulation of neurons that become co-engaged when the animal is exposed to the safe environment, or the environment that does not pose a threat to the animal. Discovery Score = 3; this latent variable is supported by previous literature; however, there is also research that concludes the opposite as to what this component displays.
Looking at the factor (across-trial factor), there is a distinct pattern of neural engagement that emerges during the middle portion of the behavioral session, with two prominent peaks of activity occurring in the second and third blocks of trials, followed by a sharp decline in activity in the final block. This suggests a transient population response that is not sustained across the entire session and may reflect a phase of active learning or consolidation. In the provided neuroscience literature, the prelimbic cortex is known to exhibit dynamic population coding during learning, particularly during the acquisition phase of associative tasks, where neural activity often increases as animals form cue-outcome associations. However, the specific pattern of two distinct peaks followed by a decline is not directly reported in the literature for this brain region or task. While some studies show increased activity during learning, the precise biphasic structure and subsequent suppression are not well-documented. Scientific interpretation: This factor may represent a transient population response associated with the initial encoding of cue-outcome associations, followed by a reduction in activity as the task becomes more automated or as the animal transitions to a different cognitive state (e.g., consolidation or disengagement). The decline in the final block may reflect a shift in neural strategy or a reduction in attentional demand. Discovery Score = 3; this latent variable is partially supported by literature, but the specific biphasic pattern is not directly reported.
3
3
4
Looking at the third factor (across-trial factor), there is high engagement of a group of neurons during the first day of the behavior paradigm within all 3 environments. There is currently no literature that supports the idea that there is a subpopulation of neurons that responds specifically to the first exposure of the 3 different environments and that this population would not respond to these 3 environments similarly 24 hours later. No associative learning is occurring between habituation exposure 1, 2, or 3 with the three exposures to all 3 environments separated by 24 hours. Discovery Score = 1; this latent variable is unsupported by current literature
Looking at the factor (across-trial factor), the pattern shows two distinct peaks of neural engagement occurring at specific trials, with minimal activity in between, suggesting a sparse, event-locked response pattern. This may reflect a population of neurons that are selectively activated during discrete, salient events or transitions within the behavioral task, such as cue presentation or reward delivery. This pattern is consistent with findings in the prelimbic cortex during associative learning tasks, where specific neuronal ensembles are recruited during critical moments of learning, such as cue-outcome associations or decision points. The sparse, phasic activation aligns with known roles of prelimbic neurons in encoding task-relevant events rather than sustained environmental representation. Discovery Score = 4; this latent variable is supported by previous literature, particularly in the context of event-related encoding in the prelimbic cortex during learning, though direct evidence for this exact sparsity and timing pattern is not fully established.
1
4
Appendix
Table 6. Full comparison of expert-written interpretations and VLM-generated hypotheses for latent components (Part 2 of 5).
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
5
Looking at the third factor (across-trial factor), there is no discernable pattern of the engagement for the subpopulation of neurons. There seems to be no recognizable correlation to the exposure to the different environments or the time period across the behavioral paradigm where precise learning of cue associations occurs. Discovery Score = 1; this latent variable is unsupported by current literature
Looking at the factor (across-trial factor), the pattern shows sparse and sporadic neural engagement across trials, with no consistent activation in any particular trial or group of trials. The majority of the data points remain near the baseline ( y≈0 ), with only a few isolated peaks occurring at random trial positions. This suggests a lack of sustained or structured population activity, possibly reflecting noise, low signal-to-noise ratio, or a non-task-related neural state. In the provided neuroscience literature, such sparse and unstructured activity patterns in the prelimbic cortex are not typically associated with learning or task-relevant processing. Instead, the literature emphasizes the emergence of structured, environment-specific, or reward-predictive population dynamics during learning. While some studies report transient or sparse activity during early learning phases, the complete absence of sustained or clustered engagement across trials is not a well-supported or commonly observed phenomenon in the context of associative learning paradigms. Scientific interpretation: This factor may represent a non-informative or noise component, or perhaps a population that is not engaged in the task at all. It does not reflect a coherent neural representation of environment, valence, or learning state, and thus may not correspond to a meaningful latent variable in the context of the behavioral task. Discovery Score = 1; this latent variable is unsupported by current literature.
1
1
Appendix
Table 7. Full comparison of expert-written interpretations and VLM-generated hypotheses for latent components (Part 3 of 5).
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
6
High neuron coactivity during the threatening and familiar environments. Unexpected result, but might be due to (1) the mouse knows the familiar environment poses no threat, and (2) the threatening environment has a strong negative valence associated with it — where the animal is afraid (demonstrating high freezing behavior) throughout learning to discriminate the environments.
Looking at the factor (across-trial factor), High neuron co-activity during early learning phase and sustained activity throughout the learning process, with peaks occurring in both safe and threatening environments, suggesting a general engagement during environmental exploration and initial learning. Discovery Score = 2; this latent variable is not directly supported by previous research, but some studies report broad prelimbic cortex activation during early learning stages.
1
2
7
High neuron co-activity during habituation phase (when environments hold no meaning or valence to the mouse) and near the end of learning when the animal knows that the safe environment and threatening environments signal safety and threat, respectively.
Looking at the factor (across-trial factor), High neuron co-activity during the early phase of learning, particularly before the first transition (around trial 10), with sustained but lower activity during the later phase of learning. Discovery Score = 3; this latent variable is supported by previous literature; however, there is also research that concludes the opposite as to what this component displays.
2
3
8
Safe environment responding component, with some familiar environment response towards end of training when the mouse learns the safe environment poses no threat.
Looking at the factor (across-trial factor), High neuron co-activity during the initial phase of learning (before environmental valence is established) and during the final phase of learning (when the animal has learned to discriminate environments), with minimal activity during the middle phase. Discovery Score = 3; this latent variable is supported by previous literature; however, there is also research that concludes the opposite as to what this component displays.
4
3
9
Threating environment responding component, with high neuron co-activity during threatening environment exposure; especially near the end of the behavior paradigm, where the mouse knows the threatening environment signals threat.
Looking at the factor (across-trial factor), High neuron co-activity during the threatening environment, with a peak occurring near the end of learning, suggesting increased engagement as the animal consolidates threat-related memory. Discovery Score = 4; this latent variable is supported by previous literature.
4
4
Appendix
Table 8. Full comparison of expert-written interpretations and VLM-generated hypotheses for latent components (Part 4 of 5).
Comp.
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
10
High neuron co-activity during familiar environment with some activity in the safe and threatening environments towards the end of learning (suggesting mouse is recognizing the environments?).
Looking at the factor (across-trial factor), High neuron co-activity during the initial phase of learning and again during the later phase of learning, with minimal activity during the middle phase. This suggests a biphasic engagement pattern, possibly reflecting early encoding and late consolidation or reactivation processes. Discovery Score = 3; this latent variable is supported by previous literature; however, the specific biphasic pattern is not consistently reported and may reflect task-specific dynamics not fully captured in prior studies.
3
3
11
Safe environment responding component, with high neuron co-activity during safe environment exposure, and also high activity during familiar environment exposure.
Looking at the factor (across-trial factor), High neuron co-activity during the initial phase of learning and again near the end of learning, with minimal activity during the middle phase. This suggests a pattern of engagement during early encoding and late consolidation or decision-making stages. Discovery Score = 3; this latent variable is supported by previous literature; however, the specific timing and dual-peak structure are not consistently reported across studies.
3
3
12
Unspecific activity, as high neuron co-activity is being detected across all 3 environment exposures and across learning epochs.
Looking at the factor (across-trial factor), High neuron co-activity during the initial phase of learning (early trials) and again during the later phase of learning (after the second transition), suggesting a pattern of engagement during both early exploration and late consolidation or re-evaluation. Discovery Score = 2; this latent variable is not really supported by previous research.
1
2
Appendix
Table 9. Full comparison of expert-written interpretations and VLM-generated hypotheses for latent components (Part 5 of 5).
Subject
Expert Interpretation
VLM Interpretation
Expert
VLM
Score
Score
1
Time 1 has high variability during the trial showing unspecific neuron activity for the majority of the environment exposure. However, a noticeably sharp increase towards the end of the trial stands out. This may indicate recognition of the environment. Alternatively, since this is the threatening environment, and the mouse did not receive the footshock it expected, this high activity may be due to a reward-type stimulus caused by the lack of threat.
Looking at the factor (across-trial factor), Highly variable and noisy neural engagement with no clear temporal structure or alignment to behavioral epochs, suggesting either non-specific activity or potential artifact. Discovery Score = 1; this latent variable is unsupported by current literature.
1
1
2
Time 0 has select time periods during the trial where neurons are highly co-engaged. This may be during different behavior output, such as the mouse engaging in freezing or grooming behavior. Although the values here cannot indicate valence of the environment, this component may be providing latent information about behavior planning and/or engagement during the trial.
Looking at the factor (across-trial factor), High and variable neuron co-activity across the session, with multiple sharp peaks distributed throughout early, middle, and late phases, suggesting a non-specific or exploratory neural response pattern that does not align with distinct behavioral epochs or environmental valence. Discovery Score = 2; this latent variable is not well-supported by previous research, as most studies report more structured, environment- or learning-phase-specific dynamics in the prelimbic cortex.
3
2
Appendix
Table 10. Full comparison of expert-written interpretations and VLM-generated hypotheses for components obtained from the end-to-end pipeline on two previously unlabeled subjects.
Figure 7. Behavioral environments used in the fear conditioning and discrimination task. From left to right: Safe environment (CS − ) which signals safety, Threatening environment (CS + ) paired with mild foot shocks during conditioning, and a Familiar home-cage analog (NS) environment.
CPD
NeAT
Subjects
NC
C
NC
C
Control
1
0.1457
0.2077
0.1099
0.1113
2
0.1255
0.1483
0.0937
0.0919
3
0.1231
0.1280
0.1002
0.1010
4
0.0928
0.1071
0.0724
0.0729
5
0.1067
0.1085
0.0781
0.0792
Appendix
Table 11. Test RMSE of tensor completion. Note that C and NC indicate if tensors are coupled or not.
Modern neuroscience relies on integrating multi-scale, multimodal datasets to uncover the neural principles underlying intelligence. However, analytical challenges posed by highly heterogeneous data and fragmented workflows increasingly constrain discoveries. Here we introduce SeekBrain, an autonomous multi-agent framework designed to accelerate neuroscience discovery through domain-grounded hierarchical planning and cross-modal data analysis. SeekBrain dynamically constructs a repertoire of analysis recipes extracted from code-paper pairs. By coupling this codified expertise with agentic planning and execution engines, the framework scalably generates hypotheses and analytical pipelines on demand. Systematic evaluation on the expert-annotated BrainArena benchmark demonstrates that SeekBrain substantially outperforms state-of-the-art agent baselines across various analysis tasks. Crucially, when deployed in real-world research, SeekBrain integrated behavioral, neural, and anatomical data to reveal structured, distributed neural representations of larval zebrafish behavior and a shared axis of regional decoding strength across the brain in a mouse decision-making task. These results establish SeekBrain as a scalable and practical tool for accelerating data-driven discoveries in neuroscience.
Jiamin Wu, Peishan Xiang, Jingyang Chen +26
Shanghai Artificial Intelligence Laboratory · The Chinese University of Hong Kong · State Key Laboratory of Brain Cognition and Brain-Inspired Intelligence Technology, Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences +3
Analyzing neuroimaging data requires specialized coding and statistical expertise, which limits accessibility for researchers without computational backgrounds. We present the AI Neuroscientist, a language agent for interactive data exploration. The system integrates a large language model (LLM) with a neuroimaging toolset to perform quality control, modeling, and visualization. This allows researchers to query data quality and specify analysis parameters directly in natural language, providing a transparent and interactive alternative to conventional scripted pipelines for small-scale data exploration. We demonstrate these capabilities using functional near-infrared spectroscopy (fNIRS) data, and evaluate the agent on a custom fNIRS benchmarking suite against general-purpose LLM agents with code sandboxes. Future extensions will generalize the architecture to additional modalities, including functional magnetic resonance imaging (fMRI) data, and expand the benchmarking suite to additional fNIRS tasks.
Aakash Patel, Panos Ketonis, Shreya Saxena +2
Department of Computer Science, Yale University · Wu Tsai Institute, Yale University · Department of Biomedical Engineering, Yale University +2
Agentic AI tools offer a promising path to automating software development bottlenecks in scientific research pipelines, particularly for stages that take domain experts days to months to build, where scientists care about correctness and robustness, not implementation details. We present an empirical study of general-purpose coding agents on a fly optogenetics data-to-discovery pipeline. We assess agents on tasks substantially larger than existing benchmarks, datasets orders of magnitude bigger, and evaluation criteria grounded in domain expert standards. We show that agents can solve several individual pipeline stages, suggesting stage-level automation is tractable. By analyzing agents' code iterations, we show that they struggle most when there is not a pre-defined criterion to iterate on, and they must instead use their scientific judgment to assess their current solution, a key open challenge. Mirroring scientific practice, they sometimes attempt visual inspection of intermediate outputs for self-evaluation, but largely fail to interpret what they see or act on it appropriately. Solving the end-to-end pipeline correctly requires stringing together successes across all pipeline stages, and this is beyond agents' current abilities. We identify challenges largely absent from existing benchmarks, including computational resource management and generalization to large held-out data collections. Finally, we distill principles for constructing scientific tasks and rigorous evaluation criteria for open-ended problems.