High-content multi-channel imaging, from Cell Painting assays to remote sensing, is central to modern scientific pipelines. Because these channels are highly heterogeneous and experimental configurations frequently evolve, a practical vision backbone must be fundamentally channel-agnostic. Current channel-adaptive vision transformers rely on global self-attention with learnable channel embeddings, which can be computed only for previously known channels. Attempts to overcome this limitation of new channels either discard the embeddings, losing channel identity, or process each channel independently, thereby eliminating critical cross-channel interactions. To address this, we introduce ChanSFormer, an efficient Vision Transformer that replaces rigid embeddings with disentangled spatial-channel attention and individual channel CLS tokens. This architecture natively preserves both distinct channel identities and cross-channel information flow. Furthermore, the individual channel class token design unlocks more informative feature representations for self-supervised learning and enables representation-based channel sampling while remaining channel-agnostic. We evaluate ChanSFormer on two biology multi-channel datasets, CHAMMI, JUMP-CP and a satellite dataset, So2Sat. Experimental results show that ChanSFormer outperforms state-of-the-art methods by 3.93%-4.27% on JUMP-CP in classification accuracy. It also demonstrates exceptional cross-dataset transferability under a self-supervised setting, outperforming previous methods by up to 2.72% in accuracy in the same classification tasks. Furthermore, the disentangled attention reduces the quadratic complexity of channel-by-spatial sequence length of the global attention baseline, thus improving throughput by 55%-313%. Code is available at https://github.com/cvlab-stonybrook/ChanSFormer.
Figures & tables
Figure 1: (a) Overall framework of ChanSFormer . Each channel of an input image is independently projected by a shared patch embedding; a shared spatial positional embedding is added, and individual channel class (CLS) tokens, copied from a single learnable CLS token, are concatenated. The resulting tokens pass through L ChanSFormer encoders. (b) Supervised classification . The channel CLS tokens are aggregated via attention into an image-level representation. (c) Self-supervised DINOv2 pretraining . The channel CLS tokens are treated as distinct samples to generate the DINO loss [ 11 ] , while the remaining patch tokens generate the iBOT [ 17 ] loss. (d) Architecture of the ChanSFormer encoder . A single global attention block is replaced by channel attention (across channels, per spatial position) followed by spatial attention (across spatial positions, per channel).
CHAMMI
JUMP-CP
So2Sat
Method
#Params
Avg
Full
Flu.
Full
Sen-1
HyperViT [ 15 ]
24.7M
54.54
47.07
42.43
60.73
41.88
DepthwiseViT [ 12 ]
22.6M
60.94
49.86
44.98
60.41
43.41
TemplateViT [ 13 , 14 ]
21.5M
57.02
52.48
43.85
55.86
37.28
ChAda-ViT [ 7 ]
21.5M
63.88
65.03
42.15
56.98
12.38
ChannelViT [ 6 ]
21.5M
64.90
67.51
56.49
62.75
47.70
Table 1: Supervised classification on three multichannel benchmarks. For JUMP-CP and So2Sat, Full evaluates on all channels, Flu. evaluates on the fluorescence channels for JUMP-CP and Sen-1 evaluates on the Sentinel-1 channels for So2Sat. The highest number is bold, the second highest is underlined. #Params is the number of parameters. The reported numbers are the mean of 3 runs.
CHAMMI
JUMP-CP
So2Sat
Pretrain
Method
Avg
Full
Flu.
Full
Sen-1
CHAMMI
BoC [ 10 ]
48.92
39.36 †
36.42 †
67.50 †
46.46 †
ChannelViT [ 6 ]
44.29
23.94 †
24.71 †
66.65 †
44.54 †
ChanSFormer
50.24
40.82 †
39.14 †
68.08 †
46.23 †
JUMP-CP Flu.
BoC [ 10 ]
40.85 †
46.00 †
42.24
65.47 †
45.28 †
ChannelViT [ 6 ]
32.88 †
30.72 †
32.97
65.00 †
44.47 †
Table 2: Self-supervised linear-probe transfer across multichannel datasets. Models are pretrained with DINOv2 [ 11 ] on the dataset listed in the Pretrain column, then evaluated by linear probe on the target dataset. Flu. and Sen-1 represent the fluorescence and Sentinel-1 channels respectively. For ChannelViT, the channel embeddings for unseen channels are zero-padded. † marks evaluations involving channels absent from SSL pretraining.
Table 4
Number of channels for evaluation ( k )
Method
8
7
6
5
4
3
2
1
ChannelViT [ 6 ]
67.51
60.36±9.1
52.74±12.2
44.89±13.2
36.88±12.3
29.36±9.3
23.70±5.0
20.78±1.6
DiChaViT [ 8 ]
69.19
61.91±9.3
54.49±12.4
46.35±13.4
38.00±12.4
30.09±9.3
23.97±4.9
20.90±1.6
ChanSFormer
73.46
65.61±10.0
57.32±12.8
48.52±13.7
39.50±12.5
30.92±9.2
24.17±4.6
20.65±1.3
Table 5: Test accuracy under partial channels of JUMP-CP. Each column reports the mean ± std over all (k8) combinations of k available channels out of 8 [ 6 , 8 ] . ChanSFormer achieves comparable or better performance in all 8 cases.
CHAMMI
JUMP-CP
Pretrain
Method
Avg
Full
Flu.
CHAMMI: Allen + CP (HPA held out)
BoC [ 10 ]
35.72
19.63
19.65
ChannelViT [ 6 ]
33.92
19.54
19.53
ChanSFormer
46.00
38.38
37.25
CHAMMI: HPA + CP (Allen held out)
BoC [ 10 ]
46.76
37.75
35.67
ChannelViT [ 6 ]
40.93
19.62
19.62
Table 6: Leave-one-source-out self-supervised transfer. Models are pretrained with DINOv2 [ 11 ] on CHAMMI with one source held out, then evaluated by linear probe (JUMP-CP) or the official CHAMMI protocol. All evaluations in this table involve channels absent from SSL pretraining. Bold denotes the best result under each pretraining source.
CHAMMI
JUMP-CP
So2Sat
Method
Avg
Full
Flu.
Full
Sen-1
ChanSFormer
68.83
73.46
61.91
64.76
49.75
w. channel embedding
69.03
73.78
62.36
64.78
49.55
ChannelViT [ 6 ]
64.90
67.51
56.49
62.75
47.70
w/o channel embedding (CA-MAE [ 9 ] )
50.37
61.93
51.56
62.58
46.73
Spatial-only (BoC [ 10 ] + CLS channel attn)
61.73
41.56
39.10
63.41
47.70
Table 7: Ablation on channel embeddings and cross-channel interactions. Channel embeddings are not critical for ChanSFormer compared with ChannelViT with global attention, and cross - channel interactions prove essential. Flu. and Sen-1 represents the fluorescence and Sentinel-1 channels respectively.
Setting
Method
CHAMMI
JUMP-CP Full
JUMP-CP Flu.
Supervised
ChanSFormer
68.83
73.46
61.91
w. HCS [ 6 ] channel sampling
66.17
72.81
61.78
w. HCS [ 6 ] w/o channel CLS token [ 22 ]
64.58
69.77
58.00
DINOv2
ChanSFormer (channel-level)
50.24
40.82
39.14
(PT: CHAMMI)
w/ attention pooling (image-level)
48.61
38.97
38.52
DINOv2
ChanSFormer (channel-level)
42.86
46.13
45.15
Table 8: Ablation of ChanSFormer’s design choices. The Setting column distinguishes between supervised classification and DINOv2 linear probes. For the DINOv2 ablation, models are pretrained on either CHAMMI or JUMP-CP (indicated as PT) to compare feature granularity. Flu. represents the fluorescence channels.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Method
CHAMMI
JUMP-CP Full
JUMP-CP Flu.
So2Sat Full
So2Sat Sen-1
ChanSFormer
68.83 ± 0.51
73.46 ± 0.14
61.91 ± 0.62
64.76 ± 0.30
49.75 ± 0.33
Appendix
Table 9: Standard deviation of ChanSFormer on 3 datasets.
Method
#Params
Allen
HPA
CP
Avg
HyperViT [ 15 ]
24.7M
45.17
63.90
26.23
54.54
DepthwiseViT [ 12 ]
22.6M
50.35
71.52
27.74
60.94
TemplateViT [ 13 ]
21.5M
49.51
64.52
25.65
57.02
ChAda-ViT [ 7 ]
21.5M
67.08
60.67
24.60
63.88
ChannelViT [ 6 ]
21.5M
67.66
62.14
27.62
64.90
DiChaViT [ 8 ]
21.5M
75.69
63.67
28.98
69.68
Appendix
Table 10: Detailed breakdown of performance on CHAMMI subsets (Allen, HPA, and CP). The Avg column represents the mean of Allen and HPA. Baseline results are reported from DiChaViT [ 8 ] .
Pretrain
Method
Allen
HPA
CP
Avg
CHAMMI
BoC [ 10 ]
49.88
47.96
27.91
48.92
ChannelViT [ 6 ]
46.60
41.98
26.93
44.29
ChanSFormer
48.96
51.51
25.10
50.24
JUMP-CP Flu.
BoC [ 10 ]
43.31
38.40
27.83
40.85
ChannelViT [ 6 ]
32.48
33.29
28.77
32.88
ChanSFormer
44.48
41.25
27.99
42.86
Appendix
Table 11: Detailed breakdown of performance on CHAMMI subsets (Allen, HPA, and CP) for DINOv2 linear-probe. The Avg column represents the mean of Allen and HPA.
Method
#Params
CHAMMI
JUMP-CP
So2Sat
HyperViT [ 15 ]
24.7M
62.28
39.64
61.22
DepthwiseViT [ 12 ]
22.6M
76.52
46.97
61.55
TemplateViT [ 13 , 14 ]
21.5M
70.30
35.01
60.84
ChannelViT [ 6 ]
21.5M
80.68
80.81
68.01
DiChaViT [ 8 ]
21.5M
78.83
82.99
67.18
ChanSFormer
28.9M
82.18
84.11
68.77
Appendix
Table 12: Supervised classification with ImageNet ViT-S/16 pretraining. Top-1 accuracy on the full channel set of each dataset. The highest number per column is bold and the second highest is underlined.
T
So2Sat Full
Sen-1
0.01
64.85
48.35
0.1
64.76
49.75
1.0
64.99
49.46
Appendix
Table 13: Sensitivity to the sampling temperature T in supervised training. The selected value for each dataset is shown in bold in the T column; bold numbers denote the best result in each column.
CHAMMI
JUMP-CP
Method
# Channels per view
Avg
Full
Flu.
ChannelViT [ 6 ]
2
32.88
30.72
32.97
4
34.09
31.97
35.53
ChanSFormer
2
42.86
46.13
45.15
4
45.04
48.18
46.49
Appendix
Table 14: Number of channels per view in DINOv2 pretraining on JUMP-CP Flu., evaluated as in Table 2 . The image batch size is halved for four channels to keep the number of channel views per iteration fixed.
SSL source
Target
ChannelViT (zero-pad)
ChannelViT (CE adapt.)
ChanSFormer
CHAMMI
JUMP-CP Full
23.94
25.54
40.82
JUMP-CP Flu.
24.71
24.99
39.14
So2Sat Full
66.65
66.91
68.08
So2Sat Sen-1
44.54
45.14
46.23
JUMP-CP Flu.
JUMP-CP Full
30.72
31.10
46.13
So2Sat Full
65.00
65.19
66.33
Appendix
Table 15: Zero-padded vs. adapted channel embeddings for ChannelViT [ 6 ] on unseen-channel transfer. Models are pretrained with DINOv2 on the SSL source and evaluated by linear probe on the Target .
Figure 2: Learned channel relations on JUMP-CP. Left: the EMA channel-similarity matrix S used by our sampler. Right: channel-attention weights (row: query, column: key) at patch positions, averaged over all heads, all 12 layers, and the full test set with all 8 channels present; each row sums to 1 (uniform =0.125 ). White lines separate the fluorescence (Mito, AGP, RNA, ER, DNA) and brightfield (HighZBF, LowZBF, Brightfield) channels.
Figure 3: Learned channel relations on So2Sat. Left: the EMA channel-similarity matrix S used by our sampler. Right: channel-attention weights (row: query, column: key) at patch positions, averaged over all heads, all 12 layers, and the full test set with all 18 channels present; each row sums to 1. White lines separate the 8 Sentinel-1 channels from the 10 Sentinel-2 bands. Sentinel-1: real (re) and imaginary (im) parts of the unfiltered VH and VV channels, intensity (int) of the refined-Lee-filtered VH and VV channels, and real and imaginary parts of the refined-Lee-filtered covariance-matrix off-diagonal element (Cov).