cs.CVMar 17, 2026

Understanding Cell Fate Decisions with Temporal Attention

Authors: Florian Bürger, Martim Dias Gomes, Adrián E. Granada, Noémie Moreau, Katarzyna Bozek

Organizations: Institute for Biomedical Informatics, Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany · Faculty of Mathematics and Natural Sciences, University of Cologne, Cologne, Germany · Cologne Excellence Cluster on Cellular Stress Responses in Aging-Associated Diseases (CECAD), University of Cologne, Cologne, Germany · Center for Molecular Medicine Cologne (CMMC), Faculty of Medicine and University Hospital Cologne, University of Cologne, Cologne, Germany · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Understanding non-genetic determinants of cell fate is critical for developing and improving cancer therapies, as genetically identical cells can exhibit divergent outcomes under the same treatment conditions. In this work, we present a deep learning approach for cell fate prediction from raw long-term live-cell recordings of cancer cell populations under chemotherapeutic treatment. Our Transformer model is trained to predict cell fate directly from raw image sequences, without relying on predefined morphological or molecular features. Beyond classification, we introduce a comprehensive explainability framework for interpreting the temporal and morphological cues guiding the model's predictions. We demonstrate that prediction of cell outcomes is possible based on the video only, our model achieves balanced accuracy of 0.94 and an F1-score of 0.93. Attention and masking experiments further indicate that the signal predictive of the cell fate is not uniquely located in the final frames of a cell trajectory, as reliable predictions are possible up to 10 h before the event. Our analysis reveals distinct temporal distribution of predictive information in the mitotic and apoptotic sequences, as well as the role of cell morphology and p53 signaling in determining cell outcomes. Together, these findings demonstrate that attention-based temporal models enable accurate cell fate prediction while providing biologically interpretable insights into non-genetic determinants of cellular decision-making. The code is available at https://github.com/bozeklab/Cell-Fate-Prediction.

Figures & tables

Figure 1: Overview of the proposed cell fate prediction framework and the explainability using temporal attention analysis. 1) From time-lapse microscopy recordings, we extract single-cell trajectories. Each trajectory starts at the first appearance of a cell and ends at the last frame prior to the fate event (division or death). 2) For each cell patch, frame-level embeddings are computed using a ResNet-50 backbone. We randomly mask a portion of frames at the end of each trajectory during training. 3) The sequence of frame embeddings is processed by a Transformer encoder, which integrates temporal information to predict the final cell fate. 4) To enable interpretability, we extract the attention weights assigned to individual frames and categorize them into high-attention and low-attention frames. 5) In parallel, we compute handcrafted cell features for each patch (e.g., area, eccentricity, p53 concentration). 6) Finally, we compare the feature distributions of high- and low-attention frames to identify biologically meaningful patterns associated with the model’s decision-making process.
Figure 2: Confusion matrix normalized over the ground-truth labels, with absolute counts shown in parentheses.
Figure 3: Temporal truncation analysis to assess the importance of late parts of the cell trajectory for its fate prediction. a) Balanced accuracy when progressively truncating sequences by removing an increasing number of frames from the end of each trajectory. b) Balanced accuracy when restricting the model input to only the last kk frames of each sequence. c) Class-wise recall under progressive truncation from the sequence end, reported separately for apoptosis and mitosis. d) Class-wise recall when predictions are based solely on the last kk frames, highlighting the contribution of late temporal information for each cell fate.
Figure 4: Aggregated attention weights across correctly classified sequences, aligned to the final frame and normalized by the global minimum and maximum attention values.
Figure 5: Effect size analysis based on Cliff’s delta for handcrafted cell features. For each feature, the median Cliff’s delta and its 95% confidence interval are shown separately for mitosis and apoptosis. Positive values indicate higher feature values in high-attention frames compared to low-attention frames, whereas negative values indicate the opposite. The background shading denotes commonly used effect size categories (negligible, small, medium, and large effects), facilitating interpretation of the magnitude and direction of the observed differences.
Figure 6: Illustrative mitotic trajectory with corresponding temporal attention weights. The sequence depicts progressive cell growth followed by characteristic mitotic rounding in the frames immediately preceding division. Frames assigned high attention by the Transformer model are highlighted in green, whereas representative low-attention frames are marked in red.
More in this topic

Microscopy Image Analysis

Microscopy image analysis uses computational methods to detect, segment, track, reconstruct, and quantify structures in scientific images.

See all 198 papers

Explore similar work

CardsList

  1. Applications of temporal graph learning for predicting the dynamics of biological systems

    May 27, 2026Manuel Dileo, Andrea SottorivaTemporal GNNsTemporal Link Prediction

    Center for Computational Biology, Human Technopole Foundation, Milan, Italy

  2. BLINK: Behavioral Latent Modeling of NK Cell Cytotoxicity

    Mar 5, 2026Iman Nematollahi, Jose Francisco Villena-Ossa, Alina Moter +6Toxicity DetectionLatent Dynamics Modeling

    Department of Computer Science, University of Freiburg, Freiburg, Germany · Institute for Transfusion Medicine and Gene Therapy, University Medical Center Freiburg, Freiburg, Germany · Goethe University, Department of Pediatrics, Experimental Immunology and Cell Therapy, Frankfurt am Main, Germany +2

  3. Predictive Multiplicity in Cell-Fate Assignment: Label-Free Rashomon Sets and the Limits of Per-Cell Certification

    Oct 8, 2026Arjun Bhupatiraju, Abhiram BhupatirajuSingle-Cell TranscriptomicsSelective Prediction

    McNeil High School, Austin, Texas