cs.LGMar 21, 2026
SaveCFNN: Continued Fraction Neural Network
Organizations: School of Future Technology, Shanghai University, Shanghai, 200444, China. · Institute of Artificial Intelligence, Shanghai University, Shanghai, 200444, China. · School of Computer Science and Technology, Shanghai University, Shanghai, 200444, China. · College of Sciences, Shanghai University, Shanghai, 200444, China. ·
Abstract
Accurately characterizing non-linear functional manifolds with singularities is a fundamental challenge in scientific computing. While Multi-Layer Perceptrons (MLPs) dominate, their spectral bias hinders resolving high-curvature features without excessive parameters. We introduce Continued Fraction Neural Networks (CFNNs), integrating continued fractions with gradient-based optimization to provide a rational inductive bias.'' This enables capturing complex asymptotics and discontinuities with extreme parameter frugality. We provide formal approximation bounds demonstrating exponential convergence and stability guarantees. To address recursive instability, we develop three implementations: CFNN-Boost, CFNN-MoE, and CFNN-Hybrid. Benchmarks show CFNNs consistently outperform MLPs in precision with one to two orders of magnitude fewer parameters, exhibiting up to a 47-fold improvement in noise robustness and physical consistency. By bridging black-box flexibility and white-box transparency, CFNNs establish a reliable grey-box'' paradigm for AI-driven scientific research.
Figures & tables
| Function | Expression | Task category |
|---|---|---|
| Bilinear Ratio Function | Conventional-NN-challenging | |
| Runge Function | Conventional-NN-challenging | |
| Rational Interaction Function | Continued-fraction-aligned | |
| Nested Rational Function | Continued-fraction-aligned |
Table 1: Synthetic benchmark functions used to evaluate function approximation performance. The four functions are organized into two task categories: continued-fraction-aligned functions, used to assess whether the CFNN inductive bias matches nested rational structure, and conventional-neural-network-challenging functions, used to test approximation accuracy on manifolds with sharp transitions or nonlinear ratio interactions. Here, denotes a small positive constant introduced to avoid numerical singularity in the denominator.
| Model | Bilinear Ratio | Runge | Rational Interaction | Nested Rational |
|---|---|---|---|---|
| MLP | 1.08 | 4.05e-1 | 1.34 | 8.65e-1 |
| CFNN | 1.22 | 4.53e-1 | 1.19 | 5.51e-1 |
| CFNN-Boost | 1.97e-1 | 4.97e-3 | 3.96e-2 | 2.12e-2 |
| CFNN-MoE | 2.04e-2 | 5.98e-3 | 3.08e-2 | 1.98e-2 |
| CFNN-Hybrid | 2.27e-2 | 5.09e-3 | 3.09e-2 | 2.07e-2 |
| KAN | 2.18e-3 | 2.03e-3 | 6.63e-3 | 9.8e-5 |
Table 2: RMSE for function approximation on the four synthetic benchmark functions at a matched model scale of approximately 200 learnable parameters. Lower values indicate better performance. The best result among the MLP and CFNN-family models is highlighted in bold; KAN is reported separately as an external reference baseline.
| Model | Waveform | Magic | Credit Card | CIFAR10 | Sentiment | Quora |
|---|---|---|---|---|---|---|
| CoFrNet-DL | 0.87 | 0.86 | 0.71 | 0.87 | 0.84 | 0.88 |
| CoFrNet-D | 0.69 | 0.76 | 0.66 | 0.38 | 0.80 | 0.75 |
| MLP | 0.34 | 0.65 | 0.50 | 0.35 | 0.83 | 0.85 |
| CFNN | 0.84 | 0.85 | 0.82 | 0.44 | 0.80 | 0.92 |
| CFNN-Boost | 0.88 | 0.88 | 0.82 | 0.44 | 0.82 | 0.88 |
| CFNN-MoE | 0.89 | 0.87 | 0.82 | 0.45 | 0.87 | 0.94 |
Table 3: Classification accuracy across six real-world benchmark datasets spanning tabular, image, and text modalities. Higher values indicate better performance. Results are reported under the standardized train-validation-test protocol described in Methods. The best result in each dataset is highlighted in bold; tied best results are both highlighted.
| Function family | Notation / definition | Input domain |
|---|---|---|
| Jacobian elliptic function | ||
| Incomplete elliptic integral of the first kind | ||
| Incomplete elliptic integral of the second kind | ||
| Modified Bessel function of the first kind |
Table 4: Complex analytical benchmark functions used in the Pareto frontier and training-dynamics experiments. These four two-variable function families were selected to probe complementary challenges for approximation, including double periodicity, near-singular integral structure, and rapidly varying special-function behaviour. Together, they provide a stringent test bed for evaluating the trade-off between parameter count, fitting accuracy, and optimization stability across model architectures.
| Model | NSR ( ) | Top-5 accuracy ( ) | MIR ( ) | Suppression ( ) |
|---|---|---|---|---|
| MLP | 0.474 | 50.0% | 6.2 | 47.4 |
| CFNN | 0.019 | 100.0% | 2.5 | 1.9 |
| CFNN-Boost | 0.071 | 100.0% | 2.8 | 7.1 |
| CFNN-MoE | 0.067 | 100.0% | 3.2 | 6.7 |
| CFNN-Hybrid | 0.010 | 100.0% | 3.0 | 1.0 |
Table 5: SHAP-based evaluation of noise suppression and attribution fidelity on the synthetic feature-noise benchmark. NSR denotes the noise-to-signal attribution ratio, computed as the total absolute attribution assigned to nuisance features divided by that assigned to the four ground-truth signal covariates. Top-5 accuracy reports the fraction of true signal features recovered within the five highest-ranked features. MIR denotes the mean importance rank of the four true signal variables. Suppression reports NSR normalized by the CFNN-Hybrid value. Lower NSR, MIR, and Suppression are better; higher Top-5 accuracy is better. The best result in each metric is highlighted in bold; tied best results are all highlighted.
| Noise ratio | MLP | KAN | CFNN | CFNN-Boost | CFNN-MoE | CFNN-Hybrid |
|---|---|---|---|---|---|---|
| 20% | 800 | 400 | 50 | 400 | 400 | 50 |
| 40% | 800 | 1200 | 50 | 400 | 800 | 100 |
| 60% | 1200 | - | 100 | - | 1200 | 50 |
| 80% | 800 | 1200 | 200 | 1200 | 400 | 50 |
| 90% | 800 | - | 100 | - | 400 | 400 |
Table 6: Minimum parameter count required for each model to achieve prediction error below the threshold under increasing levels of noise-feature interference. Parameter values are reported over the discrete search grid described in Methods. Lower values indicate greater parameter efficiency. The best result at each noise ratio is highlighted in bold; tied best results are both highlighted. A dash indicates that the model did not reach the threshold within the evaluated parameter range.
| Model | MSE ( ) | MAE ( ) | ( ) | Relative MSE ( ) |
|---|---|---|---|---|
| MLP | 0.3037 | 0.4179 | 0.6668 | 63.3 |
| CFNN | 0.0087 | 0.0717 | 0.9904 | 1.8 |
| CFNN-Boost | 0.0296 | 0.0843 | 0.9214 | 6.2 |
| CFNN-MoE | 0.0164 | 0.0954 | 0.9820 | 3.4 |
| CFNN-Hybrid | 0.0048 | 0.0553 | 0.9948 | 1.0 |
Table 7: Predictive performance under the fixed high-dimensional feature-redundancy configuration described in Methods. Lower MSE, MAE, and Relative MSE indicate better predictive accuracy; higher indicates better goodness of fit. Relative MSE is reported with respect to the CFNN-Hybrid value. The best result in each metric is highlighted in bold.
| Model | MSE ( ) | MAE ( ) | ( ) |
|---|---|---|---|
| MLP | 411.60 | 16.56 | -2.9449 |
| CFNN | 61.64 | 5.85 | 0.4092 |
| CFNN-Boost | 64.01 | 5.99 | 0.3865 |
| CFNN-MoE | 81.16 | 6.03 | 0.4829 |
| CFNN-Hybrid | 7.13 | 1.93 | 0.9317 |
Table 8: Predictive performance on the UCI Energy Efficiency dataset under the single-target thermal-load regression setting described in Methods. Lower MSE and MAE indicate smaller prediction errors, whereas higher indicates better goodness of fit. The best result in each metric is highlighted in bold.
| Model | Spearman ( ) | Ranking Consistency ( ) | Top-3 Accuracy ( ) |
|---|---|---|---|
| Standard CFNN | 0.5401 | 64.3% | 33.3% |
| Hybrid CFNN | 0.3086 | 60.7% | 33.3% |
| Boost CFNN | 0.6944 | 71.4% | 66.7% |
| MLP | 0.6172 | 75.0% | 33.3% |
Table 9: Alignment between model-derived feature-importance rankings and thermodynamic domain knowledge on the UCI Energy Efficiency dataset. Spearman quantifies rank correlation with the ground-truth physical hierarchy, Ranking Consistency reports the overall agreement between the estimated and domain-informed orderings, and Top-3 Accuracy measures the fraction of truly important variables recovered among the three highest-ranked features. Higher values indicate better interpretability alignment. The best result in each metric is highlighted in bold.
Figure 1: Training-loss trajectories for the foundational CFNN and its three stabilized variants, measured by mean squared error. The dashed CFNN curve exhibits severe optimization instability, with abrupt spikes and extended oscillatory plateaus that reach the scale. In contrast, CFNN-Hybrid and CFNN-Boost descend smoothly to the lowest final losses, while CFNN-MoE remains stable but converges to a higher error floor. The inset magnifies the late training stage (epochs 160–200), making the separation among the three stable variants explicit.
Figure 2: Comparison of gradient standard deviation across the foundational CFNN and its three stabilized variants. Lower values indicate greater optimization stability. The foundational CFNN exhibits an extreme gradient fluctuation of 9.69, whereas CFNN-Hybrid, CFNN-MoE, and CFNN-Boost reduce this statistic to 0.68, 0.33, and 0.01, respectively. The CFNN bar is truncated at 1 for visual clarity, with its true value annotated above the bar. These results provide a quantitative explanation for the severe loss oscillations in Fig. 1 .
Figure 3: Test-loss scaling with continued-fraction depth for the foundational CFNN and its three stabilized variants, measured by mean squared error on a logarithmic scale. The foundational CFNN exhibits strongly non-monotonic behaviour: performance improves sharply from depth 2 to 4, deteriorates at depth 6, and only partially recovers at larger depths. In contrast, CFNN-Hybrid shows the most favorable scaling trend, with test loss decreasing consistently to the regime as depth increases. CFNN-MoE and CFNN-Boost also improve more smoothly than the foundational CFNN, but converge to higher loss plateaus in the to range.
Figure 4: Frequency-domain comparison of approximation residuals for CFNN-Hybrid and competing baselines. (a) Mean relative error in low-frequency ( ) and high-frequency ( ) bands. MLP exhibits pronounced spectral bias, with high-frequency error ( ) substantially exceeding low-frequency error ( ), whereas SIREN displays the opposite imbalance, achieving lower high-frequency error ( ) at the cost of the worst low-frequency error ( ). CFNN-Hybrid attains the lowest error in both bands (0.0094 and 0.0106), indicating the most balanced frequency response. (b) Cumulative error spectrum across normalized frequency. CFNN-Hybrid maintains the lowest cumulative error throughout the full frequency range, while MLP accumulates the largest error and the other high-frequency-aware baselines remain intermediate.
Figure 5: Spatial manifestation of spectral bias in function approximation. Top row: Cross-sectional fitting on high-curvature (left, ) and low-curvature (right, ) slices. Insets highlight critical failure modes: MLP exhibits peak flattening in high-curvature regions, while SIREN and Chebyshev-KAN introduce spurious oscillations in smooth regions. Bottom row: Residual error distributions. CFNN-Hybrid maintains near-zero error across both regimes, demonstrating adaptive resolution without trade-offs.
Figure 6: Lead-metric comparison between CFNN-family architectures and matched MLP baselines across structural configurations. Positive values indicate improvement over the corresponding MLP. The foundational CFNN shows strongly configuration-dependent behaviour, whereas CFNN-Boost, CFNN-MoE, and CFNN-Hybrid maintain broadly positive RMSE lead across most settings. Error bars denote variability across benchmark functions.
Figure 7: Pareto-style comparison of parameter efficiency on four challenging analytical function families. Each panel reports test MSE as a function of parameter count, so curves closer to the lower-left indicate better accuracy-efficiency trade-offs. Across all four tasks, CFNN-Hybrid forms the strongest frontier, maintaining lower error than the MLP baseline over nearly the full parameter range and continuing to improve where the MLP begins to saturate. CFNN-MoE also remains competitive and typically outperforms MLP at moderate-to-large scales, but generally plateaus above CFNN-Hybrid.
Figure 8: Training-loss dynamics of CFNN-Hybrid compared with matched-capacity MLP and a tenfold-wider MLP baseline across four analytical benchmark functions. Lower curves indicate faster convergence and better optimization performance. CFNN-Hybrid reaches low-error regimes earlier than both MLP baselines on all four tasks and maintains the lowest loss for most of training, despite the presence of occasional transient spikes on the more difficult manifolds. The enlarged MLP (10x) improves over the matched-scale MLP but typically remains above CFNN-Hybrid, indicating that the observed gain is not explained by parameter count alone.
Figure 9: Pareto-style comparison of parameter efficiency under increasing noise-feature interference. Each panel reports test MSE as a function of parameter count for a fixed noise ratio, with lower curves indicating better robustness-efficiency trade-offs. Across the 20%–80% noise regimes shown here, CFNN-Hybrid remains on or near the strongest frontier and is closely followed by the foundational CFNN, whereas MLP and KAN generally require substantially more parameters to approach comparable error levels. CFNN-MoE remains competitive at moderate scales, while CFNN-Boost exhibits more configuration-dependent behaviour.
Figure 10: Feature importance analysis results on the UCI Energy Efficiency dataset. Features are annotated as high, medium, or low importance based on their true relevance, with distinct colors used to indicate each importance level in the figure.
Figure 11: Feature importance correlation analysis on the UCI Energy Efficiency dataset. Ideally, features within each importance category should cluster around the gray reference line, indicating accurate alignment between estimated and true importance rankings.