Modern self-supervised predictive architectures excel at capturing complex statistical correlations from high-dimensional data but lack mechanisms to internalize verifiable human logic, leaving them susceptible to spurious correlations and shortcut learning. Conversely, traditional rule-based inference systems offer rigorous, interpretable logic but suffer from discrete boundaries and NP-hard combinatorial explosion. To bridge this divide, we propose a bidirectional neuro-symbolic framework centered around Rule-informed Joint-Embedding Predictive Architectures (RiJEPA). In the first direction, we inject structured inductive biases into JEPA training via Energy-Based Constraints (EBC) and a multi-modal dual-encoder architecture. This fundamentally reshapes the representation manifold, replacing arbitrary statistical correlations with geometrically sound logical basins. In the second direction, we demonstrate that by relaxing rigid, discrete symbolic rules into a continuous, differentiable logic, we can bypass traditional combinatorial search for new rule generation. By leveraging gradient-guided Langevin diffusion within the rule energy landscape, we introduce novel paradigms for continuous rule discovery, which enable unconditional joint generation, conditional forward and abductive inference, and marginal predictive translation. Empirical evaluations on both synthetic topological simulations and a high-stakes clinical use case confirm the efficacy of our approach. Ultimately, this framework establishes a powerful foundation for robust, generative, and interpretable neuro-symbolic representation learning.
Figures & tables
Figure 1: The three-step pipeline for Rule-Based JEPA (RbJEPA), progressing from raw rule extraction to structured encoding and finally to representation learning via predictive loss minimization.
Figure 2: The Multi-Modal Dual-Encoder Architecture bridging continuous data and discrete logic. Continuous data contexts ( x ) and symbolic rule antecedents ( A ) are encoded on the left and converge on a universal predictor g . The predictions are compared against target encodings of data ( y ) and consequents ( C ) generated on the right, enabling joint minimization of data loss ( LJEPA ) and rule energy constraints ( LEBC ). SG denotes stop gradient .
Figure 3: Geometric illustration of the Energy-Based Constraints (EBC). Valid rule antecedents (gray boxes) are pulled into stable, low-energy basins corresponding to their correct consequents. Corrupted rule pairings are actively repelled into high-energy regions via the contrastive margin.
Figure 4: Energy landscapes of Classic JEPA and RiJEPA over the feature space. While the baseline learns an unstructured, flat functional mapping, RiJEPA carves deep, logically valid energy basins while actively repelling negative OOD rules.
Model
ID Energy (Valid)
OOD Energy (Invalid)
Classic JEPA
0.003
0.150
RiJEPA (EBC)
0.006
12.713
Table 1: Mean Energy Assigned to the Rule Mapping.
Figure 5: Mean energy assignment by latent region. RiJEPA successfully applies a massive energy penalty to invalid rules outside the data distribution.
Figure 6: Global PCA projection of latent representations. RiJEPA actively clusters valid rules and strictly isolates invalid regions, whereas the Classic JEPA leaves the semantic space entangled.
Variable
Type
Description & Encoding Schema
age
Integer
Patient age in years
sex
Categorical
Patient sex ( 1=male , 0=female )
cp
Categorical
Chest pain type ( 1:typical angina,2:atypical angina,3:non-anginal,4:asymptomatic )
trestbps
Integer
Resting blood pressure on hospital admission (in mm Hg)
Table 2: Summary of the 14 Clinical and Non-invasive Test Variables
Figure 7: The universal predictor g(⋅) . The predictor facilitates modality-agnostic reasoning by utilizing a standardized latent bottleneck of d=32 . It accepts either continuous data context embeddings ( zc_data ) or symbolic rule antecedent embeddings ( zc_rule ), processing them through a shared non-linear transformation to yield their respective target predictions.
Risk Profile
Symbolic Logic (Antecedent → Consequent)
Support
Confidence
Low Risk
age_group=Middle AND thalach_level=High → target_risk=0.0
21.5%
70.8%
Low Risk
cp=1.0 AND restecg=2.0 → target_risk=0.0
4.2%
83.3%
High Risk
cp=4.0 AND slope=3.0 → target_risk=1.0
4.2%
100.0%
Table 3: Examples of Extracted Clinical Rules via FP-Growth
Figure 8: t-SNE visualization comparing the Classic JEPA (upper) and RiJEPA (bottom) latent spaces. While Classic JEPA yields an unstructured, continuous manifold where risk profiles are visually entangled, RiJEPA’s Energy-Based Constraints explicitly pull continuous patient data (circles) into highly separable clusters strictly anchored around discrete symbolic poles (crosses).
Evaluation Metric
Classic JEPA
RiJEPA
Clinical Significance
Downstream Linear Probing
100%
100%
Both models allow for ex-post high-dimensional linear separation.
Zero-Shot Logical Accuracy
NA
100%
Only RiJEPA forms explicit logical clusters, enabling zero-shot diagnosis.
Fallback Latent Norm (Ablation)
NA
5.87
RiJEPA maintains geometric stability even without symbolic input.
Table 4: Quantitative Evaluation on the UCI Heart Risk Test Set
Figure 9: The Logic-Accuracy Trade-off. The Classic JEPA purely optimizes for continuous data correlation, achieving a lower raw reconstruction MSE. RiJEPA absorbs a slightly higher data prediction error in order to structurally satisfy the injected logical rule constraints.
Paradigm
Query Logic
Discovered Clinical Profile
Joint Discovery
Unconditional
IF Normal Chol + Elevated BP + LV Hypertrophy + Downsloping ST + Typical Angina ⟹ High Risk + Downsloping ST + High FBS + (Mixed Age Markers)
Forward Inference
IF Senior THEN…
High Risk + Medium Heart Rate + Exercise Angina + Asymptomatic Pain + LV Hypertrophy
Abductive Reasoning
High Risk BECAUSE…
Exercise Angina + Low/Medium Heart Rate + Downsloping ST + Senior Age
Marginal-Predictive
Real Patient Vitals (e.g., Young, High Max HR)
Low Risk + Normal ECG + Upsloping ST + High Heart Rate + No Angina
Table 5: Generative Clinical Discovery through Langevin Diffusion Paradigms
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China · Institute of Deep Perception Technology, JITRI, Wuxi 214000, China