RINS: Residual-Image Neural Subspace Solvers for Large Sparse Linear Systems

Zhongyan Ouyang; Weixin Liao; Mingquan Feng; Yehui Tang; Junchi Yan

cs.CESep 9, 2026

RINS: Residual-Image Neural Subspace Solvers for Large Sparse Linear Systems

Authors: Zhongyan Ouyang, Weixin Liao, Mingquan Feng, Yehui Tang, Junchi Yan

Organizations: Shanghai Innovation Institute, Shanghai, China · Faculty of Computation, Harbin Institute of Technology, Harbin, China · School of Artificial Intelligence and School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · School of Smart Governance, Renmin University of China, Suzhou, China

Abstract

Large sparse linear systems from PDE discretizations require correction subspaces whose operator images explain the current residual. We study this residual-image viewpoint and propose Gate-RINS, a neural subspace solver that generates polynomial correction bases from cached residual probes and modulates them with a lightweight residual- and coordinate-dependent pointwise gate. The projected least-squares update remains unchanged, so the neural component only chooses the expansion directions while the numerical closure tests them through $\operatorname{range}(AQ_t)$ . We also introduce a hybrid controller schedule that composes GRANS-style graph controllers with Gate-RINS under the same projected solver. Across six PDE-derived benchmark tasks and two scales, Gate-RINS reaches fixed relative-residual thresholds faster in synchronized wall-clock time than GMRES and a recent graph-only neural baseline in most settings, and hybrid schedules further improve the residual trajectory. Difficult-mode diagnostics and trajectory visualizations support the interpretation that these gains are associated with operator-image subspaces that align more effectively with the current residual.

Figures & tables

Figure 1: A neural solver can generate a correction space whose operator image leads to a smaller projected residual than an Arnoldi-generated image space.

PDE	RINS	GRANS	GMRES	GRANS	GMRES
Poisson	$0.105\pm 0.013$	$0.219\pm 0.041$	$0.292\pm 0.035$	2.09x	2.79x
Helmholtz	$0.121\pm 0.010$	$0.194\pm 0.036$	$0.335\pm 0.037$	1.61x	2.77x
Heat	$0.058\pm 0.004$	$0.136\pm 0.025$	$0.128\pm 0.019$	2.34x	2.20x
Advection-Diffusion	$0.105\pm 0.007$	$0.232\pm 0.024$	$0.320\pm 0.032$	2.20x	3.04x
Klein-Gordon	$0.061\pm 0.009$	$0.120\pm 0.014$	$0.138\pm 0.045$	1.97x	2.27x
Wave	$0.061\pm 0.008$	$0.130\pm 0.031$	$0.146\pm 0.045$	2.15x	2.41x

Table 1: Time in seconds to reach relative residual

10^{-5}

n\approx 1000

. Solver-time entries are the pooled mean and standard deviation over all valid seed/sample observations. The last two columns report the speedup of RINS over GRANS and GMRES, computed from pooled mean times. The

n\approx 10000

table is in Appendix C .

Figure 2: Main benchmark relative-residual curves at

n\approx 1000

. Panels correspond to the six PDE-derived sparse linear-system tasks. RINS and GRANS are run for 200 neural block iterations, generating 20 new basis vectors at every neural iteration; one neural iteration is therefore not the same computational unit as one GMRES Arnoldi step. GMRES is run until it reaches

\min(\mathrm{RelRes}_{\mathrm{RINS}},\mathrm{RelRes}_{\mathrm{GRANS}})

. Curves use one fixed training seed and report median residual over 10 held-out samples, with shaded bands showing the sample interquartile range. The dotted horizontal line marks

10^{-5}

; curves that do not cross this line within the neural budget are reported as dashes in Table 1 .

Figure 4: Hybrid inference acceleration on the Poisson problem. Hybrid(GRANS

\to

G-RINS) uses a GRANS prefix trained for the first hybrid bootstrap step and Gate-RINS for the remaining steps, while Hybrid(GRANS

\to

S-RINS) uses the same frozen GRANS first block followed by Static-RINS with the same

m

and

L

. Both schedules keep the same projected minimum-residual update as the standalone solvers.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Continuous/discrete form
Poisson	$-\nabla\!\cdot(\kappa\nabla u)=f$ .
Helmholtz	$-\nabla\!\cdot(\kappa\nabla u)+\alpha(x)u=f$ .
Heat	Backward Euler step $(M+\Delta tK)u^{n+1}=Mu^{n}+\Delta t\,b$ .
Advection–diffusion	$-\nabla\!\cdot(\kappa\nabla u)+v\cdot\nabla u=f$ .
Klein–Gordon	$(\Delta t^{-2}M+c^{2}K+M_{m})u^{n+1}=f^{n+1}+\Delta t^{-2}M(2u^{n}-u^{n-1})$ .
Wave	$(\Delta t^{-2}M+c^{2}K)u^{n+1}=f^{n+1}+\Delta t^{-2}M(2u^{n}-u^{n-1})$ .

Appendix

Table 2: PDE-derived linear systems used in the main benchmark. Klein–Gordon and wave are implicit one-step systems from second-order time-evolution equations.

Task	Train	Test
Poisson	$929.35\pm 67.81$ (785–1117)	$913.70\pm 93.47$ (786–1091)
Helmholtz	$918.72\pm 66.86$ (741–1113)	$926.10\pm 70.60$ (781–1035)
Heat	$909.88\pm 60.63$ (764–1051)	$922.80\pm 55.06$ (829–1011)
Advection-Diffusion	$913.74\pm 55.55$ (761–1042)	$918.00\pm 67.28$ (818–1026)
Klein-Gordon	$919.78\pm 64.89$ (766–1092)	$935.50\pm 50.59$ (892–1076)
Wave	$919.57\pm 70.94$ (741–1133)	$941.90\pm 60.14$ (842–1020)

Appendix

Table 3: Actual mesh-size distributions for the main benchmark at

n\approx 1000

. Entries are mean

\pm

standard deviation with the observed min–max range in parentheses.

Task	Train	Test
Poisson	$11139.11\pm 779.56$ (9504–12964)	$10794.50\pm 1049.34$ (9036–12519)
Helmholtz	$11178.78\pm 802.02$ (9228–13384)	$10707.10\pm 658.70$ (9864–11592)
Heat	$11158.86\pm 802.72$ (8886–13347)	$11257.00\pm 1044.04$ (9935–13319)
Advection-Diffusion	$11128.69\pm 832.55$ (9099–13188)	$11674.40\pm 901.93$ (10582–13381)
Klein-Gordon	$11165.38\pm 832.44$ (9002–13012)	$11336.40\pm 656.58$ (10243–12460)
Wave	$11172.76\pm 777.30$ (8955–13166)	$11032.60\pm 810.63$ (9810–12194)

Appendix

Table 4: Actual mesh-size distributions for the main benchmark at

n\approx 10000

. Entries are mean

\pm

standard deviation with the observed min–max range in parentheses.

PDE	RINS	GRANS	GMRES	GRANS	GMRES
Poisson	$2.687\pm 0.465$	$4.816\pm 1.580$	$4.313\pm 0.690$	1.79x	1.61x
Helmholtz	$2.699\pm 0.391$	$7.026\pm 3.209$	$4.693\pm 0.664$	2.60x	1.74x
Heat	$0.867\pm 0.114$	$4.557\pm 2.121$	$1.602\pm 0.221$	5.26x	1.85x
Advection-Diffusion	$3.080\pm 0.644$	$5.656\pm 2.232$	$5.139\pm 1.058$	1.84x	1.67x
Klein-Gordon	$1.003\pm 0.213$	$1.888\pm 1.139$	$1.611\pm 0.669$	1.88x	1.61x
Wave	$1.373\pm 0.263$	$2.355\pm 0.790$	$2.181\pm 0.962$	1.72x	1.59x

Appendix

Table 5: Time (s) to reach relative residual

10^{-5}

n\approx 10000

. Solver-time entries are the pooled mean and standard deviation over all valid seed/sample observations across the five seeds. The last two columns report the speedup of RINS over GRANS and GMRES, computed from pooled mean times.

Figure 5: Main benchmark relative-residual curves at

n\approx 10000

. The layout and stopping rules are the same as in Figure 2 .

Figure 7: One-basis limit test at

n\approx 1000

. Solid curves show median relative residual versus cumulative solve time, shaded bands show interquartile ranges, and numeric markers indicate 50 and 100 generated directions or GMRES iterations. Static-RINS converges rapidly in a low-dimensional accumulated subspace, whereas GRANS does not converge under this one-basis constraint.

Neural PDE Solvers

Neural PDE solvers use trainable models to approximate solutions or solution operators for partial differential equations, often with physical constraints.

See all 271 papers

RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections
May 26, 2026Yali Fink, Ido Ben-Yair, Lars Ruthotto +1Graph Neural Networks Partial Differential Equations
Institute for Interdisciplinary Computational Sciences, Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Be’er Sheva, Israel · Department of Mathematics and Computer Science, Emory University, Atlanta, GA, USA.
Error-Conditioned Neural Solvers
Jun 25, 2026Haina Jiang, Liam Wang, Peng-Chen Chen +4Neural PDE Solvers PDE Surrogate Modeling
University of Michigan · KAIST AI · Los Alamos National Laboratory
Linearized subspace refinement framework to expose hidden accuracy in trained neural networks
Jan 20, 2026Wenbo Cao, Weiwei ZhangSubspace Optimization Neural Network Optimization
Institute of AI for Industries, Chinese Academy of Sciences, Nanjing 211135, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · School of Aeronautics, Northwestern Polytechnical University, Xi’an 710072, China +2

May 26, 2026J/K move · Enter open · S save

RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections

Yali Fink, Ido Ben-Yair, Lars Ruthotto, Eran Treister

Institute for Interdisciplinary Computational Sciences, Faculty of Computer and Information Science, Ben-Gurion University of the Negev, Be’er Sheva, Israel · Department of Mathematics and Computer Science, Emory University, Atlanta, GA, USA.

Graph Neural Networks Partial Differential Equations

Open paper AskPDF arXiv

The scalable solution of large sparse linear systems is a bottleneck in scientific computing and graph analysis. While algebraic multigrid (AMG) offers optimal linear scaling, its performance is severely constrained by the trade-off between the sparsity and convergence quality of coarse-grid operators. Classical AMG heuristics struggle to balance these objectives, often sacrificing stability or performance for sparsity. We propose RAPNet, a graph neural network (GNN) framework that resolves this trade-off by learning to generate sparse, robust coarse operators directly from the sparse algebraic system. Key to our approach is a level-wise training strategy that enables learning from small subgraphs and generalization to million-node domains, bypassing the bottlenecks of prior neural AMG attempts. RAPNet executes exclusively during the solver setup phase, ensuring that the solve phase retains its favorable computational properties. We show that our method outperforms classical non-Galerkin baselines on diverse PDE discretizations and graph Laplacians, making it particularly effective for multi-query tasks such as eigenproblems, time-dependent simulations, and inverse or design problems.

Abstract

Neural PDE Solvers

Explore similar work

RAPNet: Accelerating Algebraic Multigrid with Learned Sparse Corrections

Error-Conditioned Neural Solvers

Linearized subspace refinement framework to expose hidden accuracy in trained neural networks