CNA: An AI-Oriented Comprehensive Normalized Assessment for Healthy Status and Application to Optimize RRT Strategies by Reinforcement Learning

Jiang Liu; Chan Zhou; Yujie Li; Di Wu; Yihao Xie; Peiwei Li; Xin Shu; Jiaqi Zhu

cs.LGSep 3, 2026

CNA: An AI-Oriented Comprehensive Normalized Assessment for Healthy Status and Application to Optimize RRT Strategies by Reinforcement Learning

Authors: Jiang Liu, Chan Zhou, Yujie Li, Di Wu, Yihao Xie, Peiwei Li, Xin Shu, Jiaqi Zhu,

Organizations: Chongqing Institute of Green and Intelligent Technology, CAS · Chongqing School, University of Chinese Academy of Sciences · Department of Anesthesiology, Southwest Hospital, Third Military Medical University (Army Medical University) · College of Computer and Information Science, Southwest University ·

Abstract

Millions worldwide require Renal Replacement Therapy (RRT) as a treatment essential for survival. However, optimizing RRT strategies via AI is challenging due to heterogeneous patient dynamics, missing data, and the absence of an AI-oriented health assessment criterion. We propose an AI-Oriented Comprehensive Normalized Assessment (CNA) for healthy status and apply it to optimize RRT strategies by using offline reinforcement learning (RL). The key idea of CNA is transforming vital-sign distributions into a standard normal space, enabling a unified, data-driven health-status score defined by deviations from referent intervals, which also provides an AI-oriented criterion to assess strategy quality and supports RL termination. We further design a structured 23-dimensional state representation that integrates 19 indicators with 4 RRT descriptors, and employ matrix decomposition to reconstruct missing vital signs, improving data completeness for learning. These components are incorporated into multiple offline RL algorithms and validated via systematic ablation studies on RRT feature subsets. Compared with physicians' observed treatments, the best learned strategy reduces mortality from 13.2% to 5.0% (reducing 62.24%) and shortens average in-hospital stay from 308.5 to 250.1 hours (reducing 18.93%), demonstrating both methodological innovation and the potential of CNA-guided RL to improve RRT outcomes in nephrology.

Figures & tables

Figure 1: Overview of the new framework. It comprises four main modules: 1) data processing, 2) reinforcement learning (RL) training, 3) model selection based on the Treatment Effect Comparison Matrix (TECM), and 4) results analysis. “Data Pre” denotes operations performed on initial data including cleansing, preliminary imputation, and window segmentation; MF represents matrix factorization, for data reconstruction. CNA is the new notion of Comprehensive Normalized Assessment proposed in this study. TECM and $\eta$ -TECM* are performance metrics of RL algorithms posed in [ 25 ] .

Indicators	Ref-Vals	MIMIC-IV		eICU		Rate
Indicators	Ref-Vals	$miss_{1}$	$rate_{1}$	$miss_{2}$	$rate_{2}$	Rate
Age	$*$	0	0.00%	0	0.00%	0.00%
Sex	$*$	0	0.00%	0	0.00%	0.00%
MV	$*$	0	0.00%	0	0.00%	0.00%
Vaso_Value	$*$	0	0.00%	0	0.00%	0.00%
Weight	$*$	0	0.00%	0	0.00%	0.00%

Table 1: Data information

Figure 2: Indicators selected in various medical scores and guidelines

Panel of Figure 3 — Figure 3: Flow diagram of the patient enrollment and data preprocessing

Panel of Figure 4 — Figure 4: Illustration of matrix factorization-based missing-value reconstruction

Indicator	Ref-Vals	Minimum		Maximum		$\#$ Out
		min o	min $\star$	max o	max $\star$
Age	$*$	18	$*$	91	$*$	$*$
Sex	$*$	0	$*$	1	$*$	$*$
Height	$*$	96.2	$*$	203	$*$	$*$
Weight	$*$	34.5	$*$	332.7	$*$	$*$
Vaso_Value	$*$	0	$*$	300	$*$	$*$

Table 2: Indicators’ distribution changes with data cleaning and reconstruction

Action	Treatment status	Clinical interpretation
$a_{0}$	NULL	No RRT intervention was recorded within the current time window.
$a_{1}$	CRRT	Continuous renal replacement therapy was delivered or ongoing within the current time window.
$a_{2}$	IRRT	Intermittent renal replacement therapy was delivered within the current time window.

Table 3: Clinical interpretation of the action space in the RRT-MDP

	Optimal strategy: $\pi^{*}$	Worst strategy: $\pi^{\prime}$
“Good” actual treatments: $P_{G}$	OG	WG
“Bad” actual treatments: $P_{B}$	OB	WB

Table 4: Treatment Effect Comparison Matrix (TECM)

Panel of Figure 5 — Figure 5: TECM* of models via action similarity rate. Subfigure a-h represent the results for OG, OB, WG, WB, O-Gap $\Xi^{o}$ , W-Gap $\Xi^{w}$ , comprehensive confidence $\sigma$ , and comprehensive bias $\mu$ of eight models

Metric	PAT	DQN-		DDQN-		BCQ-		CQL-
Metric	PAT	MF	SOFA	MF	SOFA	MF	SOFA	MF	SOFA
OG	$\times$	0.636	0.540	0.583	0.570	0.482	0.463	0.635	0.635
OB	$\times$	0.471	0.511	0.450	0.460	0.370	0.361	0.477	0.486
WG	$\times$	0.562	0.688	0.286	0.670	0.626	0.577	0.381	0.434
WB	$\times$	0.637	0.761	0.736	0.754	0.716	0.685	0.599	0.617
$\sigma$	$\times$	1.06	1.043	1.694	1.047	1.057	1.075	1.293	1.213

Table 5: Performances of MF-SOFA & SOFA models with

\tau=0.65

Panel of Figure 6 — Figure 6: Mortality rates with MF-SOFA & MF-CNS. Subfigure a represents the mortality rate corresponding to DQN; Subfigure b represents the mortality rate corresponding with DDQN; Subfigure c represents the mortality rate corresponding with BCQ; Subfigure d represents the mortality rate corresponding with CQL

Panel of Figure 7 — Figure 7: Average length of ICU stays with MF-SOFA & MF-CNS. Subfigure a represents the average length of ICU stay of DQN; Subfigure b represents the average length of ICU stay of DDQN; Subfigure c represents the average length of ICU stay of BCQ; Subfigure d represents the average length of ICU stay of CQL

Metric	Strategy
	PAT	DQN-MF-		DDQN-MF-		BCQ-MF-		CQL-MF-
	PAT	SOFA	CNS	SOFA	CNS	SOFA	CNS	SOFA	CNS
MoR	13.24	9.76	9.35	9.16	8.50	7.52	7.23	7.12	5.00
MoR	13.24	$\uparrow$ 29.38%	$\uparrow$ 30.82%	$\uparrow$ 26.28%	$\uparrow$ 35.77%	$\uparrow$ 43.20%	$\uparrow$ 45.39%	$\uparrow$ 46.22%	$\uparrow$ 62.24%
AiHS	308.48	279.09	275.59	275.83	271.82	274.6	254.78	266.35	250.07
AiHS	308.48	$\uparrow$ 9.53%	$\uparrow$ 10.66%	$\uparrow$ 10.58%	$\uparrow$ 11.88%	$\uparrow$ 10.98%	$\uparrow$ 17.41%	$\uparrow$ 13.66%	$\uparrow$ 18.93%

Table 6: Mortality rates and average length of ICU stays of various strategies

Metric	PAT	DQN-MF-		DDQN-MF-		BCQ-MF-		CQL-MF-
Metric	PAT	SOFA	CNS	SOFA	CNS	SOFA	CNS	SOFA	CNS
OG	$\times$	0.781	0.600	0.595	0.581	0.647	0.632	0.628	0.573
OB	$\times$	0.627	0.401	0.559	0.388	0.479	0.471	0.458	0.464
WG	$\times$	0.361	0.4	0.429	0.297	0.393	0.299	0.213	0.250
WB	$\times$	0.625	0.597	0.547	0.652	0.649	0.631	0.567	0.665
$\sigma$	$\times$	1.403	1.182	1.139	1.141	1.328	1.382	1.699	1.610

Table 7: Performances of MF-SOFA & MF-CNS models with

\tau=0.65

Panel of Figure 8 — Figure 8: TECM* of models via action similarity rate. Subfigure a-h represent the results for OG, OB, WG, WB, O-Gap $\Xi^{o}$ , W-Gap $\Xi^{w}$ , comprehensive confidence $\sigma$ , and comprehensive bias $\mu$ of eight models

Panel of Figure 9 — Figure 9: Mortality and hospital stay of the CQL-MF-CNS model with varied RRT information dimensions (lower values indicate better performance)

Dimension	Contains				Mortality	In-hospital stay
	R1	R2	R3	R4
0	0	0	0	0	5.31	253.79
1	1	0	0	0	5.57	253.98
	0	1	0	0	5.91	258.62
	0	0	1	0	5.32	253.42
	0	0	0	1	5.28	256.95

Table 8: Performance of CQL model with different dimensional combinations of RRT features

Healthcare

Healthcare AI develops and evaluates models and systems for clinical care, medical research, public health, and healthcare operations.

See all 621 papers

Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management
May 10, 2026Prabhjot Singh, Abhishek Gupta, Chris Betz +4Reinforcement Learning Type 1 Diabetes
Altitude
Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits
Aug 11, 2026Hangqi Ren, Junyi LiaoCounterfactual Evaluation Offline RL
School of Engineering, Vanderbilt University, Nashville, TN, USA · Pratt School of Engineering, Duke University, Durham, NC, USA
Rolling Day-Wise Mortality Prediction in Critically Ill Patients With AKI on CRRT Utilizing Machine Pressure Waveforms
Sep 11, 2026Shehan Irteza Pranto, Joanna Yang, Joshua Lambert +6Time Series Modeling Mortality Prediction
Department of Biomedical Informatics and Data Science, University of Alabama at Birmingham, AL, USA · Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine at Mount Sinai, New York, NY, USA · College of Nursing, University of Cincinnati, OH, USA +4

May 10, 2026J/K move · Enter open · S save

Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management

Prabhjot Singh, Abhishek Gupta, Chris Betz, Abe Flansburg+3

Altitude

Reinforcement Learning Type 1 Diabetes Healthcare Offline RL

Open paper AskPDF arXiv

Reinforcement learning (RL) in healthcare has had mixed results, with reward sparsity, unreliable off-policy evaluation, and deployment-simulation gap as recurring failure modes. We argue that chronic disease management is structurally a more tractable RL setting than the acute-care problems the field has primarily studied, but only if the problem is formalized to exploit chronic care's properties. We propose such a formalization. The agent's objective is to compress time-to-control (TTC) under a tiered reward calibrated to the CMS ACCESS Model. Two quantities from our companion preference-learning paper [Singh et al. 2026] enter as load-bearing structural elements: the execution intensity εbounds action availability under a constrained Markov Decision Process, and the clinician capability κweights offline-data transitions during RL training. Together they couple preference learning and RL into a two-loop architecture. We present simulation results on synthetic state machines for hypertension and type 2 diabetes. Capability-weighted offline RL outperforms uniform-weighted offline RL and the behavior policy by 15 percentage points on T2D TTC; the uniform-weighted formulation (the standard in existing healthcare RL) underperforms even the heterogeneous behavior policy. \Epsilon-aware policies generalize across deployment regimes while ε-naive policies do not.

Abstract

Healthcare

Explore similar work

Learning to Compress Time-to-Control: A Reinforcement Learning Framework for Chronic Disease Management

Unmasking Toxic Mimicry in Medical Offline Reinforcement Learning for ICU Sepsis Management via Counterfactual Clinical Audits

Rolling Day-Wise Mortality Prediction in Critically Ill Patients With AKI on CRRT Utilizing Machine Pressure Waveforms