How Architecture and Training Affect TPC Representations Across Experiments
Authors: Tyler Wheeler, Michelle P. Kuchera, Raghuram Ramanujan, William Sieland, Ryan Krupp, Yassid Ayyad, Daniel Bazin, Connor L. Cross, Hoi Yan Ian Heung, Andrew J. Jones, Ruchi Mahajan, Saiprasad Ravishankar, Pranjal Singh, Benjamin Votaw, Chris Wrede
Organizations: Department of Computational Math, Science, and Engineering, Michigan State University, East Lansing, MI, USA · Department of Physics, Grand Valley State University, Allendale, Michigan, USA · Department of Physics, Davidson College, Davidson, NC, USA · Department of Mathematics and Computer Science, Davidson College, Davidson, NC, USA · Department of Particle Physics, Universidad de Santiago de Compostela, Santiago de Compostella, Spain · Facility for Rare Isotope Beams, Michigan State University, East Lansing, Michigan 48824, USA · Department of Physics and Astronomy, Michigan State University, East Lansing, Michigan 48824, USA · Department of Physics and Astronomy, University of Kentucky, Lexington, KY, USA
Deep-learning efforts have increasingly shifted toward foundation model approaches. In experimental physics, this allows models and learned representations to be reused beyond the experiments in which they were developed. This work evaluates the reusability of representations across experiments and detector systems using probes on frozen encoders. These probes reveal task-relevant structure before downstream adaptation, complementing fine-tuning. Together with random-weight controls, they distinguish contributions from architecture and encoder training that downstream performance alone cannot resolve. Time projection chamber (TPC) data provide a useful testbed because events from TPC systems can be represented as variable-length sparse tensors, while detector geometries, event topologies, and scientific tasks can differ substantially. We investigate whether fixed-dimensional TPC event representations can be reused across classification tasks, experiments, and detector systems. Sparse ResNet and PointNet-style encoders produce 512-dimensional embeddings for four datasets from the GADGET II TPC and AT-TPC. Randomly initialized encoders isolate the contribution from architecture before supervised training. We then train each encoder on a classification task, freeze its parameters, and train a linear or nonlinear probe for each downstream task. We find that this architecture-induced structure remains useful across experiments and detector systems. The randomly initialized PointNet-style representation is highly informative on several tasks. The two architectures organize their embedding spaces differently, but neither exhibits a large, systematic loss of utility cross-detector. These results show that architecture is a major source of task-relevant structure in TPC embeddings and should be treated explicitly when assessing representation learning and developing reusable detector models.
Figures & tables
Figure 1: Example event topologies from the two TPC systems used in this work. (a) A 20 Mg, 1600 keV beta-delayed proton event from the GADGET II TPC. (b) A 4-track event from the AT-TPC.
Figure 2: (a) Schematic of the GAseous Detector with GErmanium Tagging (GADGET) II TPC—a detector optimized for measuring low-energy β -delayed particle decays, surrounded by a germanium array. (b) Schematic of the Active-Target TPC (AT-TPC)—a detector designed for nuclear reaction studies in inverse kinematics operated in a solenoid magnet providing up to 4 T of magnetic field.
Feature
GADGET II TPC
AT-TPC
Primary use case
β -delayed charged-particle decays
Inverse-kinematics reactions
Detector mode
Implant-decay TPC
Active-target TPC
Active/drift length
40 cm drift region
100 cm drift region
Active diameter
8 cm pad-plane diameter
55 cm pad-plane diameter
Gas/target role
Stopping/decay medium
Reaction target and tracking medium
Typical gas
P10 at 800 Torr
Experiment-dependent target gas (e.g., 4 He, 2 H)
Table 1: Comparison of the two TPC systems used in this work.
Dataset
Detector
Isotope
Task name
Classes
Events per class
Total events
E21072
GADGET II
20 Mg
20 Mg, 2 class
Proton; alpha
25,078; 9,501
34,579
E21072
GADGET II
20 Mg
20 Mg, 3 class
770 keV p; 1596 keV p; 2153 keV α
18,770; 6,308; 9,501
34,579
E25058
GADGET II
21 Mg
21 Mg, 3 class
1285 keV p; 1930 keV p; 2153 keV α
1,261; 1,351; 3,363
5,975
E20020
AT-TPC
16 O
16 O +α , 2 class
0–2 tracks; 3–5 tracks
1,184; 1,274
2,458
E20020
AT-TPC
16 O
16 O +α , 3 class
0–2 tracks; 3 tracks; 4–5 tracks
1,184; 476; 798
2,458
E20009
AT-TPC
10 B
10 B +d , 2 class
3 tracks; 4 tracks
2,310; 2,151
4,461
Table 2: Summary of the datasets used in this work.
Figure 3: Sparse ResNet14 architecture used to produce event-level TPC embeddings. The lower panel shows the structure of each residual block.
Figure 4: Minkowski PointNet-style architecture used to embed TPC events. Top: a shared multilayer perceptron is applied independently to each of the input points. Global max pooling aggregates the resulting N×512 features into the 512-dimensional event embedding. The post-pooling linear layer, dropout, and final classifier are used during source-task training but are not included in the extracted embedding. Bottom: each pointwise layer applies the same learned transformation independently to every active input; information is combined across inputs only by global pooling.
Evaluation task
Encoder training
Architecture
Linear SVM
RBF SVM
Accuracy
Macro-F1
Accuracy
Macro-F1
20Mg , 3 class
Naïve baseline
—
0.5428±0.0000
0.2346±0.0000
0.5428±0.0000
0.2346±0.0000
Random weights
ResNet
0.9109±0.0052
0.8840±0.0068
0.9321±0.0061
0.9077±0.0091
Random weights
PointNet
0.9979±0.0005
0.9977±0.0004
0.9933±0.0013
0.9928±0.0015
20Mg , 2 class
ResNet
0.9782±0.0059
0.9732±0.0073
0.9803±0.0042
0.9753±0.0053
20Mg , 2 class
PointNet
0.9999±0.0001
0.9999±0.0001
0.9999±0.0001
0.9999±0.0001
Table 3: Within-experiment probe performance. Values are the mean ± standard deviation over five runs. Boldface marks the largest mean in each evaluation-task and metric column. The naïve baseline predicts the most frequent class in the training set.
Evaluation task
Encoder training
Architecture
Linear SVM
RBF SVM
Accuracy
Macro-F1
Accuracy
Macro-F1
21Mg , 3 class
Naïve baseline
—
0.5632±0.0000
0.2402±0.0000
0.5632±0.0000
0.2402±0.0000
Random weights
ResNet
0.8524±0.0051
0.7985±0.0065
0.8499±0.0142
0.7922±0.0221
Random weights
PointNet
0.9955±0.0025
0.9942±0.0031
0.9945±0.0027
0.9937±0.0030
20Mg , 2 class
ResNet
0.9506±0.0026
0.9246±0.0042
0.9622±0.0065
0.9422±0.0098
20Mg , 2 class
PointNet
0.9967±0.0032
0.9958±0.0039
0.9985±0.0006
0.9983±0.0008
Table 4: Cross-experiment probe performance. Values are the mean ± standard deviation over five runs. Boldface marks the largest mean in each evaluation-task and metric column. The naïve baseline predicts the most frequent class in the training set.
Evaluation task
Encoder training
Architecture
Linear SVM
RBF SVM
Accuracy
Macro-F1
Accuracy
Macro-F1
16O+α , 3 class
Naïve baseline
—
0.4817±0.0000
0.2167±0.0000
0.4817±0.0000
0.2167±0.0000
Random weights
ResNet
0.6008±0.0081
0.5107±0.0107
0.6646±0.0121
0.4873±0.0070
Random weights
PointNet
0.8008±0.0173
0.7579±0.0197
0.8390±0.0081
0.8044±0.0101
20Mg , 2 class
ResNet
0.6955±0.0083
0.6162±0.0060
0.7817±0.0068
0.6960±0.0172
20Mg , 2 class
PointNet
0.8248±0.0115
0.7841±0.0175
0.8720±0.0110
0.8489±0.0135
Table 5: Cross-detector probe performance. Values are the mean ± standard deviation over five runs. Boldface marks the largest mean in each evaluation-task and metric column. The naïve baseline predicts the most frequent class in the training set.
Figure 5: PCA projections of the latent representations for the 16O+α three-class task. Panels compare randomly initialized and 16 O-trained ResNet and PointNet-style encoders. Points are colored according to track-multiplicity class.
Figure 6: PCA projections of the latent representations for the 21 Mg three-class task. Panels compare randomly initialized and 16 O-trained ResNet and PointNet-style encoders. Points are colored according to the three decay-event classes.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Setting
ResNet14
PointNet-style
Loss
Cross-entropy
Cross-entropy
Optimizer
AdamW
AdamW
Initial learning rate
5×10−4
5×10−4 ( 16O+α ); 5×10−6 ( 20Mg )
Weight decay
10−4
10−4
Batch size
128
128
Learning-rate scheduler
Reduce on plateau
Reduce on plateau
Appendix
Table 6: Optimization, regularization, and augmentation settings used to train the ResNet14 and PointNet-style encoders. Settings were used across encoder-training datasets unless otherwise indicated.
A learned representation is useful beyond its source task only if later tasks can make use of what the encoder has already learned. Source-task performance, however, gives no guarantee that the representation itself will remain useful once the original head is removed. We study representation reuse with respect to a specified family of future tasks. For such a family, we define the task-relevant state as the coarsest deterministic state sufficient for all of its tasks. A larger task family can require a finer state, and losing a required distinction makes at least one future task unrecoverable on a target support containing that conflict. Exact reuse also depends on how the retained information is presented to the downstream rule. We separate this into structural accessibility, which concerns whether the required distinctions remain recoverable, and coordinate consistency, which concerns whether those distinctions can be interpreted by the same downstream rule across samples. Both failure modes appear separately in the experiments. Reuse improves as task-relevant distinctions become more accessible. Reversible sample-dependent coordinate changes can sharply reduce reuse even when the representation remains recoverable, and inverting the applicable coordinate change restores performance. The same accessibility-reuse ordering appears in prospectively specified CIFAR-100 experiments with both ResNet-18 and DenseNet-121. Finally, preserving a candidate state that contains the distinctions needed by the anticipated task family improves reuse on compatible held-out tasks, with the clearest benefit when target supervision is limited. Representation reuse is therefore relative to the tasks that follow source learning: a useful representation must retain the distinctions those tasks need and make them available in a form that a shared downstream rule can use.
Picking the frozen image encoder for a 3D~CT vision--language model (VLM), together with the token-compression scheme on top of it, is a search over many candidates. There are several encoders, several ways to compress their tokens, and several token budgets, and the combinations grow fast. Comparing them the usual way means fine-tuning a large language model (LLM) on each combination, and running the whole sweep this way needs far more compute than most groups can spend. We ask whether a cheap probe on the encoder's cached embeddings can stand in for that comparison. We build an image-grounded probing benchmark over (encoder × compression) cells, with clinical attribute families and two validation gates, scale-sanity and probe-separability, that keep each attribute well-scaled and decodable. These gates are the main methodological contribution. On this benchmark we compare a range of read-out heads, and in a preliminary study we pair each probe with its matched downstream task. The early signal is encouraging: the cheap probe orders the candidates in close agreement with expensive fine-tuning, at about r≈0.95 on the cells measured so far. We read this as an ordinal claim, a ranking predictor rather than an exact estimate, and we are explicit about where it stays preliminary. If it holds up, encoder and compression choices can be screened in minutes with frozen-token probes, with full training spent only on the finalists.
Pathology foundation models improve transferable representation learning for histopathology, but recent gains often rely on encoders with hundreds of millions of parameters and high inference cost. We propose TAP-Path, a task-adaptive compression framework that directly restructures a pretrained Virchow2 encoder rather than distilling it into a separate student. TAP-Path combines validation-driven transformer-block selection, physical removal of redundant blocks, input-adaptive patch-token pruning, multi-depth feature recovery, and a lightweight gated task head. The final model retains 24 of 32 transformer blocks and 70% of patch tokens after pruning, reducing encoder parameters by 24.96% (631.24M to 473.70M) and analytical encoder compute by 35.20% (340.13G to 220.40G FLOPs). Across three task-head optimization seeds, TAP-Path achieved 87.98±0.067 test accuracy, 81.26±0.49 balanced accuracy, and 82.38±0.48 macro-F1 on a 32-class histopathology benchmark, compared with 86.89% for full Virchow2 and 87.67% for UNI2-h. TAP-Path achieved a Brier score of 0.1800±0.0005 and failure-detection AUROC of 0.9047±0.0060. A validation-only rare-aware objective improved rare-class balanced accuracy in a secondary operating analysis. Frozen external evaluation on 433 CPTAC samples yielded 91.22±0.83 accuracy and 91.10±0.81 balanced accuracy. These results show that task-adaptive structural and token sparsification can improve the accuracy-efficiency trade-off of large pathology foundation models while preserving reliability under internal and external evaluation.
Mehedi Hasan, Ashfak Yeafi, Md Khairul Islam
Department of Computer Science and Engineering, Brac University, Dhaka, Bangladesh · EEE, Khulna University of Engineering & Technology, Khulna, Bangladesh · Mathematics and Computer Science, Hobart and William Smith Colleges, Geneva, NY, USA