Graph Neural Network-Based Reinforcement Learning for Controlling Biological Networks - the GATTACA Framework

Andrzej Mizera; Jakub Zarzycki

cs.LGMay 5, 2025

Graph Neural Network-Based Reinforcement Learning for Controlling Biological Networks - the GATTACA Framework

Authors: Andrzej Mizera, Jakub Zarzycki

Organizations: Faculty of Mathematics, Informatics and Mechanics, University of Warsaw, Banacha 2, 02-097 Warsaw, Poland · IDEAS Research Institute, Królewska 27, 00-060 Warsaw, Poland · IDEAS NCBR Sp. z o.o., Chmielna 69, 00-801 Warsaw, Poland

Abstract

Cellular reprogramming, the artificial transformation of one cell type into another, has been attracting increasing research attention due to its therapeutic potential for complex diseases. However, identifying effective reprogramming strategies through classical wet-lab experiments is hindered by lengthy time commitments and high costs. In this study, we explore the use of deep reinforcement learning (DRL) to control Boolean network models of complex biological systems, such as gene regulatory and signalling pathway networks. We formulate a novel control problem for Boolean network models under the asynchronous update mode, specifically in the context of cellular reprogramming. To solve it, we devise GATTACA, a scalable computational framework. To facilitate scalability of our framework, we consider previously introduced concept of a pseudo-attractor and improve the procedure for effective identification of pseudo-attractor states. We then incorporate graph neural networks with graph convolution operations into the artificial neural network approximator of the DRL agent's action-value function. This allows us to leverage the available knowledge on the structure of a biological system and to indirectly, yet effectively, encode the system's modelled dynamics into a latent representation. Experiments on several large-scale, real-world biological networks from the literature demonstrate the scalability and effectiveness of our approach.

Figures & tables

Panel of Figure 1 — Fig. 1: Schematic illustration of the concept behind Attractor-Target Control. The source attractor $\mathcal{A}_{S}$ (blue dot) of a 4-gene network is not aligned with the target configuration specifying the target expression of two genes (expression profile encoding: olivegreen - gene OFF, aquamarine - gene ON, white - gene expression profile unspecified, i.e. either OFF or ON). The target can be any of the attractors (red dots) containing a state that is aligned with the target configuration. Transitions that follow the original network dynamics are shown with dashed arrows. Red lightnings indicate interventions (control actions) and red arrows indicate the change of state due to interventions. The depicted control strategy is of length 3, which from $\mathcal{A}_{S}$ traverses two intermediate attractor states (green dots) to finally reach one of the target attractors. Ellipses represent basins of attraction of the respective attractors on the control trajectory (path).

Fig. 2: Schematic representation of the ATGC DRL agent of the GATTACA framework.

Model	BD Id	Size	Target Configuration
Bladder	183	35	Apoptosis_b1=1 or Apoptosis_b2=1
MAPK	70	53	Apoptosis=1
T-LGL	14	61	Apoptosis=1
Bortezomib	59	67	JNK=0, p21=Cas3=1
T-diff	-	68 / 71	TBET=RORGT=FOXP3=1, GATA3=0
ABA	84	81	Closure=1

TABLE I: BN models of real-world biological networks with their target configurations used for the performance evaluation of the GATTACA framework. The BD Id column provides model identification numbers in the BioDivine repository [ 31 ] .

Model	Size (Inputs)	Environmental Condition
Bladder	35 (4)	EGFR_stimulus = 0, DNAdamage = , FGFR3_stimulus = , GrowthInhibitors = 1
MAPK	53 (4)	EGFR_stimulus = 0, FGFR3_stimulus = 0, Growth_Arrest = , DNA_damage =
T-LGL	61 (7)	CD45 = , IFN = , IL15 = , PDGF = , Stimuli = , Stimuli2 = , TAX = *
Bortezomib	67 (5)	XX = 0, SHP1 = 0, TNFAR = 0, Bort = 0, TNFA = 0
T-diff	68 / 71 (24)	APC = 1, IFNB_e = 0, IFNG_e = 0, IL2_e = 1, IL4_e = 0, IL6_e = 0, IL10_e = 0, IL12_e = 0, IL15_e = 0, IL21_e = 0, IL23_e = 0, IL27_e = 0, TGFB_e = 1, IFNGR1 = 1, IFNGR2 = 1, GP130 = 1, IL6RA = 1, CGC = 1, IL12RB2 = 1, IL10RB = 1, IL10RA = 1, IL15RA = 1, IL2RB = 1, IL27RA = 1
ABA	81 (23)	PC = , PtdInsP3 = , DAGK = , CPK23 = , NAD = , Sph = , SCAB1 = , Nitrite = , ERA1 = , ABH1 = , SPP1 = , GEF1_4_10 = , ARP_Complex = , MRP5 = , RCN1 = , NtSyp121 = , ABA = , NADPH = , GCR1 = , PtdInsP4 = , GAPC1_2 = , GTP = , CPK6 = *

TABLE II: Environmental conditions for the BN models of real-world biological networks used in the evaluation of GATTACA. Unspecified node values are indicated with *.

Model	Size (Inputs)	iPASIP PA States			CABEAN
Model	Size (Inputs)	F-A	C-A	Spurious	F-A	C-A	Failed
Bladder	35 (4)	9 [20]	0 [5]	0 [0]	9 [20]	0 [4]	1 of 16
MAPK	53 (4)	6 [12]	0 [3]	0 [0]	6 [12]	0 [3]	3 of 16
T-LGL	61 (7)	149	182	2?	166	130	6 of 128
Bortezomib	67 (5)	5 [83]	0 [0]	0 [0]	5 [83]	0 [0]	0 of 32
T-diff	68 (24)	12	0	0	12	0	0 of 1

TABLE III: Comparison of the numbers of pseudo-attractor (PA) states identified by iPASIP with the numbers of attractors obtained with CABEAN. The numbers of fixpoint attractors (F-A), cyclic or complex attractors (C-A), and spurious PA states are presented for the environmental conditions in Table II (plain numbers) and for input nodes left unspecified, i.e. all possible environmental conditions (numbers in square brackets). The numbers of environmental conditions for which CABEAN failed to compute a result are shown in the ‘Failed’ column. The ‘?’ indicates that our verification of whether the PA states are indeed spurious failed.

Fig. 3: Bar plot of the number of revisits to individual states of the Bladder BN model attractor containing the extra PA state identified by iPASIP under the environmental condition of EGFR_stimulus set to 1 and all remaining input nodes set to 0. For readability, only states with at least 1,000 revisits are shown in the decreasing order of revisit count. State of index 0 is the PA state found by iPASIP and the remaining attractor states are indexed arbitrarily.

Model	Size (Inputs)	PA states	Attractors	GATTACA	CABEAN
Bladder	35 (4)	9	9	1.0	1.0
MAPK	53 (4)	6	6	1.0	1.0
T-LGL	61 (7)	333	296	1.02	1.0
Bortezomib	67 (5)	5	5	1.99	1.05
T-diff	68 (24)	12	12	2.55	1.18
ABA	81 (23)	40,000	N/A	2.23	N/A

TABLE IV: Comparison of the average control strategy lengths required to drive the network from the source state to the target configuration obtained with the GATTACA framework and the exact ASI method implemented in CABEAN. Results that could not be obtained with CABEAN are indicated as not available (N/A).

Model	Size	PA states	GATTACA
MCF-7	117	1,500	2.11
Original T-diff in PS env. cond.	71	156,864	6.17
Original T-diff in FS env. cond.	71	25,856	6.17

TABLE V: Average control strategy lengths obtained with the GATTACA framework for the MCF-7 model under all environmental conditions and the original 71-node T-diff model of [ 36 ] under two environmental conditions (env. cond.), i.e. the partially specified (PS) one in [ 36 ] and the fully specified (FS) one in [ 25 ] . Both conditions are characterised by large number of PA states identified by iPASIP.

Panel of Figure 9 — 1.Fig.: State transition graph with highlighted attractor states of the Boolean network in under the asynchronous update mode.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 274 papers

Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference
May 30, 2026Tianyang Xu, Tianci Liu, Niraj Rayamajhi +4Transformer Gene Regulatory Network Inference
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA · Department of Horticulture and Landscape Architecture, Purdue University, West Lafayette, IN, USA · School of Biological Sciences, Illinois State University, Normal, IL, USA
Applications of temporal graph learning for predicting the dynamics of biological systems
May 27, 2026Manuel Dileo, Andrea SottorivaTemporal GNNs Temporal Link Prediction
Center for Computational Biology, Human Technopole Foundation, Milan, Italy
CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning
Mar 23, 2026Dongxia Wu, Shiye Su, Yuhui Zhang +4Constrained Generative Modeling RL for Image Generation
Stanford University, Stanford, CA

May 30, 2026J/K move · Enter open · S save

Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference

Tianyang Xu, Tianci Liu, Niraj Rayamajhi, Ryan Patrick+3

Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN, USA · Department of Horticulture and Landscape Architecture, Purdue University, West Lafayette, IN, USA · School of Biological Sciences, Illinois State University, Normal, IL, USA

Transformer Gene Regulatory Network Inference Genomics Single-Cell Transcriptomics

Open paper AskPDF arXiv

Gene regulatory networks (GRNs) capture transcription factor-target interactions and are central to understanding cell-state regulation and disease. Reconstructing GRNs from paired single-cell transcriptomic and chromatin accessibility data is promising but challenging: scATAC is extremely sparse, and most methods rely on fixed peak-to-gene links and weak supervision. We present EpiAwareNet, a prior-guided multi-omic Transformer framework that reconstructs GRNs from paired single-cell data using only lightweight biological priors. In Stage 1, EpiAwareNet learns joint gene-peak representations with a gene-peak cross-attention module, enabling data-driven, gene-specific aggregation of accessibility signals rather than hard-coded peak-to-gene assignments. In Stage 2, EpiAwareNet incorporates a bulk-derived GRN prior as noisy positive edges to provide weak supervision under label scarcity, refining regulatory scores while remaining robust to prior noise. In our experiments, EpiAwareNet improves GRN reconstruction over representative single- and multi-omic baselines and yields GRNs with greater biological plausibility, such as improved recovery of known regulatory interactions, suggesting that lightweight biological priors from bulk data can effectively guide single-cell GRN inference when combined with adaptive cross-modal representation learning. Code and data will be available at https://github.com/tianyang-x/EpiAwareNet_pub.

Abstract

RL Control

Explore similar work

Prior-Guided Multi-Omic Transformers for Single-Cell Gene Regulatory Network Inference

Applications of temporal graph learning for predicting the dynamics of biological systems

CellFluxRL: Biologically-Constrained Virtual Cell Modeling via Reinforcement Learning