In-context operator learning enables neural networks to infer solution operators from contextual examples without weight updates. While prior work has demonstrated the effectiveness of this paradigm in leveraging vast datasets, a systematic comparison against single-operator learning using identical training data has been absent. We address this gap through controlled experiments comparing in-context operator learning against classical operator learning (single-operator models trained without contextual examples), under the same training steps and dataset. To enable this investigation on real-world spatiotemporal systems, we propose GICON (Graph In-Context Operator Network), combining graph message passing for geometric generalization with example-aware positional encoding for cardinality generalization. Experiments on air quality prediction across two Chinese regions show that in-context operator learning outperforms classical operator learning on complex tasks, generalizing across spatial domains and scaling robustly from few training examples to 100 at inference.
Figures & tables
Figure 1: Illustration of In-Context Operator Networks (ICON). Given k contextual examples, each consisting of a key-value pair related by the governing operator F , ICON infers the underlying operator from these in-context examples and applies it to map a new query input to its predicted output—all in a single forward pass without any weight updates.
Figure 2: Overview of the Graph In-Context Operator Network (GICON) architecture. Left : The overall pipeline selects contextual examples from historical frames, producing an interleaved sequence of keys and values along with the query. Each frame is processed by separate key and value encoders, passed through N GICON layers, and decoded at key and query positions to produce predictions (during inference, only the query prediction is used). Right : Detailed structure of a single GICON layer, consisting of (1) spatial update via message passing that aggregates information from neighboring nodes within each graph, and (2) contextual per-node update using transformers that perform in-context learning across the example sequence with positional encodings.
Figure 3: Example cardinality generalization on BTHSA for simple to moderate operators. Top: PM 2.5 . Bottom: O 3 . Left to right: Δt=1,4,12 h. Classical single-operator learning achieves lower RMSE for simple operators ( Δt=1,4 h), while ICON with operator diversity outperforms the baseline at Δt=12 h given sufficient examples. All models are evaluated with up to 100 examples despite training with at most 5.
Figure 4: Example cardinality generalization on BTHSA at Δt=24 h. Left: PM 2.5 . Right: O 3 . For this complex operator, ICON with operator diversity surpasses the single-operator baseline with sufficient examples, with error decreasing for PM 2.5 and a sharp initial drop followed by stable performance for O 3 .
Figure 5: Operator extrapolation to Δt=48 (out-of-distribution) on BTHSA. Left: PM 2.5 . Right: O 3 . Single-operator shows flat curves, while example-trained ICON models improve with examples. Models with k=5 achieve best extrapolation, with sustained improvement for PM 2.5 and a sharp initial drop for O 3 .
Figure 6: Geometric generalization: models trained on BTHSA or YRD, both evaluated on YRD at Δt=24 h without fine-tuning. Left: PM 2.5 . Right: O 3 . BTHSA-trained models transfer to YRD with moderate performance gap for PM 2.5 and minimal gap for O 3 , suggesting cross-region generalization.
Figure 7: Single-operator setting ( Δt=24 , on BTHSA). Left: PM 2.5 . Right: O 3 . Models trained without contextual examples show no difference between noise and quality examples. The model trained with k=5 approaches and slightly surpasses the baseline for PM 2.5 at 100 quality examples. However, when given noise, they show clear degradation, confirming that these models attend to example content.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Table 8
Figure 8: Training dynamics: multi-operator vs. single-operator comparison at Δt=24 h. Left: PM 2.5 . Right: O 3 . Top: BTHSA. Bottom: YRD. ICON with operator diversity exhibits negligible overfitting while single-operator models overfit.
Figure 9: Training dynamics: single-operator ablation at Δt=24 h with varying k . Left: PM 2.5 . Right: O 3 . Top: BTHSA. Bottom: YRD. All settings exhibit overfitting, with k=0 achieving the best validation performance across all single-operator settings.
Figure 10: Example cardinality generalization on YRD for simple operators. Left: PM 2.5 . Right: O 3 . Top: Δt=1 h. Bottom: Δt=4 h. Classical single-operator learning achieves lower RMSE for these simple operators.
Figure 11: Example cardinality generalization on YRD for complex operators. Left: PM 2.5 . Right: O 3 . Top: Δt=12 h. Bottom: Δt=24 h. ICON with operator diversity surpasses the baseline with sufficient examples.
Department of Mathematics, University of Texas at Austin, Austin, Texas · Department of Aerospace Engineering and Engineering Mechanics, Oden Institute for Computational Engineering and Sciences, University of Texas at Austin, Texas