cs.LGSep 26, 2026

Analytic-Walk Rotary Positional Encodings for Graphs

Authors: Jiaqing Xie, Yuxin Wang, Xipeng Qiu

Organizations: Shanghai Innovation Institute · Fudan University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Rotary position encodings make attention sensitive to relative position, but extending them to graphs requires choosing how graph structure enters the rotation. Previous works assign each node a rotation from spectral coordinates, so the rotary factor between two nodes depends only on their endpoints and cannot distinguish the routes connecting them. We introduce Analytic-Walk Rotary Positional Encodings (AW-RoPE), which place the rotations on edges and sum the transported features over all walks, so contributions along different routes can reinforce or cancel. An exact variant evaluates the complete sum by a differentiable linear solve, and a sparse variant truncates it at a finite depth. We prove forward and parameter-derivative truncation bounds at fixed inputs and parameters. Both variants act on projected queries and keys, and the sparse recurrence also augments message-passing networks. Across five synthetic tasks both variants reduce nRMSE by 1515--58%58\% relative to the strongest baseline, and on real superpixel, peptide and OGB benchmarks the sparse recurrence attains the best mean on every dataset with Performer kernels and on twelve of thirteen datasets with GIN. Analysis shows that AW-RoPE can distinguish routes whose only cue is how two endpoints are connected, while node-wise rotary encodings cannot.

Figures & tables

Figure 1: Node-phase cancellation and controlled edge-phase changes. Edge labels are phases in radians ( ω=1\omega=1 ). (a) With node-wise rotations, every route between the same endpoints accumulates the same phase, so the route gap Δ=θ2−θ1\Delta=\theta_{2}-\theta_{1} is zero for any node phases. (b) The first AW example uses exactly the same phases. The next two add π/4\pi/4 or π/2\pi/2 to each forward lower edge. Reverse phases negate. The powers are normalized AW responses with prescribed phases, without fitting the scorer.
Figure 2: AW-RoPE implementation. (1) A shared scorer maps node features H\mathbf{H} to edge displacements auva_{uv} , with ωℓ\omega_{\ell} the channel frequency and θ\theta collecting learned parameters. (2) For fixed input X\mathbf{X} , one-step transport T\mathbf{T} and decay zz , Exact solves for Y∞\mathbf{Y}_{\infty} , with identity I\mathbf{I} . Sparse sums Zk=zkTkX\mathbf{Z}_{k}=z^{k}\mathbf{T}^{k}\mathbf{X} from depth zero through LL to approximate it by YL\mathbf{Y}_{L} . (3) Either operator F\mathbf{F} transforms queries and keys, while V bypasses transport. GIN uses a gated Sparse residual with output H′\mathbf{H}^{\prime} . Tiles denote features, and channel indices are suppressed.
DatasetNoPEWIREExactSparse
Monochromatic-00.060±0.0010.060_{\pm 0.001}0.053±0.0020.053_{\pm 0.002}0.044±0.006\mathbf{0.044}_{\pm 0.006}0.045±0.0090.045_{\pm 0.009}
Monochromatic-50.087±0.0010.087_{\pm 0.001}0.068±0.0050.068_{\pm 0.005}0.036±0.003\mathbf{0.036}_{\pm 0.003}0.036±0.0020.036_{\pm 0.002}
Monochromatic-100.081±0.0010.081_{\pm 0.001}0.063±0.0020.063_{\pm 0.002}0.027±0.000\mathbf{0.027}_{\pm 0.000}0.032±0.0090.032_{\pm 0.009}
Monochromatic-150.068±0.0020.068_{\pm 0.002}0.056±0.0040.056_{\pm 0.004}0.026±0.003\mathbf{0.026}_{\pm 0.003}0.028±0.0030.028_{\pm 0.003}
Watts–Strogatz SPD0.065±0.0050.065_{\pm 0.005}0.038±0.0060.038_{\pm 0.006}0.016±0.001\mathbf{0.016}_{\pm 0.001}0.017±0.0050.017_{\pm 0.005}
Table 1: Synthetic graph regression. Test nRMSE ↓\downarrow ; published SEs, local sample SDs. Exact and Sparse use a four-layer, width-32 dense-attention GraphGPS backbone (Table G.1 ); NoPE and WIRE are published reference means under this protocol [ 11 ] . Bold compares the matched pair.
Figure 4
MethodEndpointAttention
Mixing (Zero)50.00±0.0050.00_{\pm 0.00}97.92±1.7797.92_{\pm 1.77}
WIRE50.00±0.0050.00_{\pm 0.00}82.43±0.7582.43_{\pm 0.75}
WIRE + mixing50.00±0.0050.00_{\pm 0.00}93.89±6.9793.89_{\pm 6.97}
Dynamic local gradient49.99±0.0249.99_{\pm 0.02}100.00±0.00\mathbf{100.00}_{\pm 0.00}
Sparse AW, phases zeroed50.00±0.0050.00_{\pm 0.00}50.00±0.0050.00_{\pm 0.00}
Exact AW, phases zeroed50.00±0.0050.00_{\pm 0.00}50.00±0.0050.00_{\pm 0.00}
Table 3: Constructed route-interference task. Test accuracy (%), mean ±SD over five runs. Endpoint reads a marked node after transport; Attention uses one full block. Truncated walks use L=8L=8 , z=0.8z=0.8 ; Exact uses the complete resolvent. Protocol in Appendix I.2 .
Figure 5: Approximation at all 30 trained runs. Columns are the five tasks. Rows are relative output and local VJP error. Each curve compares Sparse with Exact at identical parameters and local inputs. Thin lines retain all three seeds, and thick lines average seed-level context and probe means. Values below 10−1610^{-16} use a plotting floor. Full settings: Appendix E.1 .
Appendix figures & tables23 assets

Supplementary material from the paper’s appendix.

Appendix

SymbolMeaning
N,MN,Mnumber of nodes and directed edges
P\mathbf{P}row-normalized nonnegative transition
auva_{uv}antisymmetric directed-edge displacement
ω\omegarotary frequency of one complex channel
Tω\mathbf{T}_{\omega}P\mathbf{P} lifted by eiωauv\mathrm{e}^{\mathrm{i}\omega a_{uv}}
zzgeometric walk-decay factor in [0,1)[0,1)
Appendix
Table C.1 : Notation used in the derivations.
IdentityMaximum error
Gradient operator factorization1.11×10−161.11\times 10^{-16}
Gradient resolvent factorization4.44×10−164.44\times 10^{-16}
Gauge covariance, operator1.31×10−161.31\times 10^{-16}
Gauge covariance, resolvent4.48×10−164.48\times 10^{-16}
Gauge-invariant magnitude4.44×10−164.44\times 10^{-16}
Permutation equivariance1.49×10−151.49\times 10^{-15}
Appendix
Table D.1 : Maximum absolute discrepancies in the independent numerical identity audit.
Figure D.1 : Deterministic Exact -AW versus sparse-AW diagnostics on a fixed graph, input, and linear readout, in double precision. Left: maximum complex-feature error relative to the complete output. Right: relative Euclidean error of the readout gradient jointly with respect to inputs, edge displacements, frequencies, edge weights, and decay. Each curve changes only LL at the indicated zz ; 32 configurations including L=0L=0 are audited. These are operator/derivative checks, not trained-model comparisons. Near-unit decay requires greater depth; numerical floors need not decrease monotonically.
  Exact AW-RoPESparse AW-RoPE
DatasetSeedTest nRMSEEpochTest nRMSEEpoch
Monochromatic-000.0402240.040224
Monochromatic-010.0402190.039219
Monochromatic-020.0512120.055215
Monochromatic-500.0351950.035230
Monochromatic-510.0342140.035208
Appendix
Table E.1 : Every run in the completed Exact – Sparse study. Test nRMSE is paired with the best validation epoch (zero-based); lower is better. No run is removed. Three-decimal values are displayed here; the artifact retains full precision.
  Training epoch (s)Peak allocated (MiB)
DatasetLLExactSparseExactSparse
Monochromatic-0816.659±0.25316.659_{\pm 0.253}22.974±0.31022.974_{\pm 0.310}157.676±0.000157.676_{\pm 0.000}135.950±0.000135.950_{\pm 0.000}
Monochromatic-51616.572±0.07516.572_{\pm 0.075}34.561±0.14034.561_{\pm 0.140}156.801±0.000156.801_{\pm 0.000}161.184±0.000161.184_{\pm 0.000}
Monochromatic-101616.615±0.19016.615_{\pm 0.190}34.641±0.92034.641_{\pm 0.920}155.925±0.000155.925_{\pm 0.000}153.277±0.000153.277_{\pm 0.000}
Monochromatic-15817.273±1.04417.273_{\pm 1.044}22.980±0.08222.980_{\pm 0.082}155.050±0.000155.050_{\pm 0.000}123.479±0.000123.479_{\pm 0.000}
Watts–Strogatz SPD816.932±0.22916.932_{\pm 0.229}23.355±0.15423.355_{\pm 0.154}107.952±0.000107.952_{\pm 0.000}107.347±0.000107.347_{\pm 0.000}
Appendix
Table E.2 : Isolated full-model H200 cost for the matched synthetic study. Entries are three-decimal mean and sample SD over three timing repeats. LL is the Sparse depth; Exact is untruncated. Smaller is better.
TaskCheckpoint originL=8L=8L=16L=16L=32L=32L=64L=64
M0Exact-trained(7.244±3.230)×10−3(7.244_{\pm 3.230})\times 10^{-3}(1.971±1.284)×10−4(1.971_{\pm 1.284})\times 10^{-4}(2.545±2.514)×10−7(2.545_{\pm 2.514})\times 10^{-7}(8.153±12.055)×10−13(8.153_{\pm 12.055})\times 10^{-13}
M0Sparse-trained(6.738±3.501)×10−3(6.738_{\pm 3.501})\times 10^{-3}(2.095±1.904)×10−4(2.095_{\pm 1.904})\times 10^{-4}(4.470±6.379)×10−7(4.470_{\pm 6.379})\times 10^{-7}(3.682±6.271)×10−12(3.682_{\pm 6.271})\times 10^{-12}
M5Exact-trained(6.530±4.548)×10−2(6.530_{\pm 4.548})\times 10^{-2}(8.044±6.941)×10−3(8.044_{\pm 6.941})\times 10^{-3}(1.399±1.465)×10−4(1.399_{\pm 1.465})\times 10^{-4}(5.112±6.865)×10−8(5.112_{\pm 6.865})\times 10^{-8}
M5Sparse-trained(5.961±4.192)×10−2(5.961_{\pm 4.192})\times 10^{-2}(7.044±6.310)×10−3(7.044_{\pm 6.310})\times 10^{-3}(1.247±1.407)×10−4(1.247_{\pm 1.407})\times 10^{-4}(6.065±8.981)×10−8(6.065_{\pm 8.981})\times 10^{-8}
M10Exact-trained(1.014±0.126)×10−1(1.014_{\pm 0.126})\times 10^{-1}(1.539±0.193)×10−2(1.539_{\pm 0.193})\times 10^{-2}(4.049±0.313)×10−4(4.049_{\pm 0.313})\times 10^{-4}(3.794±1.652)×10−7(3.794_{\pm 1.652})\times 10^{-7}
M10Sparse-trained(1.075±0.081)×10−1(1.075_{\pm 0.081})\times 10^{-1}(1.743±0.246)×10−2(1.743_{\pm 0.246})\times 10^{-2}(5.857±2.700)×10−4(5.857_{\pm 2.700})\times 10^{-4}(1.281±1.553)×10−6(1.281_{\pm 1.553})\times 10^{-6}
Appendix
Table E.3 : Relative output error at trained checkpoints. Means and sample SDs across all three run-level context averages, with three-decimal mantissas. These are approximation errors, not nRMSE or errors between independently trained models.
Nodes/graphMethodLLForward msBackward msTotal msPeak MiB
64NoPE–1.430±0.0541.430_{\pm 0.054}1.276±0.0491.276_{\pm 0.049}2.706±0.1012.706_{\pm 0.101}99.006±0.00099.006_{\pm 0.000}
64WIRE–1.621±0.0181.621_{\pm 0.018}1.577±0.0241.577_{\pm 0.024}3.198±0.0383.198_{\pm 0.038}99.195±0.00099.195_{\pm 0.000}
64Exact AW-RoPE–3.133±0.0683.133_{\pm 0.068}2.460±0.0652.460_{\pm 0.065}5.593±0.1275.593_{\pm 0.127}218.241±0.000218.241_{\pm 0.000}
64Sparse AW-RoPE163.495±0.0943.495_{\pm 0.094}6.096±0.2146.096_{\pm 0.214}9.591±0.3069.591_{\pm 0.306}180.102±0.000180.102_{\pm 0.000}
128NoPE–1.469±0.0491.469_{\pm 0.049}1.311±0.0331.311_{\pm 0.033}2.780±0.0822.780_{\pm 0.082}101.854±0.000101.854_{\pm 0.000}
128WIRE–1.591±0.0561.591_{\pm 0.056}1.566±0.0311.566_{\pm 0.031}3.157±0.0733.157_{\pm 0.073}102.231±0.000102.231_{\pm 0.000}
Appendix
Table F.1 : Matched learnable positional-layer cost on H200. ReLU linear attention, batch size eight, width 64, four heads, z=0.8z=0.8 . Forward/backward times and peak allocated memory are summarized over three independent repeats, each averaging ten warmed steps. Exact is untruncated; Sparse uses L=16L=16 . All values have three decimals. All 72 small-graph cells (including the companion FAVOR+ table) are complete. The 256-node Exact timing jump is discussed below.
NodesMethodLLForward msBackward msTotal msPeak MiB
64NoPE–1.532±0.0351.532_{\pm 0.035}1.511±0.0371.511_{\pm 0.037}3.043±0.0683.043_{\pm 0.068}99.702±0.00099.702_{\pm 0.000}
64WIRE–2.009±0.5132.009_{\pm 0.513}1.753±0.0231.753_{\pm 0.023}3.762±0.4923.762_{\pm 0.492}99.891±0.00099.891_{\pm 0.000}
64Exact AW-RoPE–3.717±0.8253.717_{\pm 0.825}2.703±0.0182.703_{\pm 0.018}6.420±0.8346.420_{\pm 0.834}218.241±0.000218.241_{\pm 0.000}
64Sparse AW-RoPE163.799±0.2133.799_{\pm 0.213}6.618±0.4676.618_{\pm 0.467}10.417±0.67810.417_{\pm 0.678}180.797±0.000180.797_{\pm 0.000}
128NoPE–1.565±0.0231.565_{\pm 0.023}1.543±0.0041.543_{\pm 0.004}3.108±0.0193.108_{\pm 0.019}103.246±0.000103.246_{\pm 0.000}
128WIRE–1.708±0.0081.708_{\pm 0.008}1.774±0.0201.774_{\pm 0.020}3.482±0.0153.482_{\pm 0.015}103.622±0.000103.622_{\pm 0.000}
Appendix
Table F.2 : Companion softmax FAVOR+ positional-layer timings. Protocol, hardware, initialization and reporting match the ReLU measurements in Table F.1 . Entries are mean ±SD , three decimals; Exact is untruncated.
HyperparameterValue
Backbone layers4
Hidden width32
Attention heads1
Dropout0.2
Batch size16
Attentionfull
Appendix
Table G.1 : Shared hyperparameters of the matched synthetic study (Section 4.2 ). Both Exact and Sparse arms use these settings with matched initializations.
DatasetMethodLRLLzzValidation
Mono-0NoPE1e−41\mathrm{e}{-4}––0.046553
Mono-0AW2e−42\mathrm{e}{-4}160.60.031604
Mono-0AW-NB2e−42\mathrm{e}{-4}160.80.029987
Mono-0AW-NB-MS2e−42\mathrm{e}{-4}80.80.032441
Mono-5NoPE2e−42\mathrm{e}{-4}––0.044055
Mono-5AW2e−42\mathrm{e}{-4}160.80.026159
Appendix
Table G.2 : Validation-selected configurations in the completed GIN synthetic grid. Metric is normalized RMSE (lower is better).
DatasetGIN-NoPEGIN-AW-RoPEGIN-AW-NBGIN-AW-NB-MSWinner
Monochromatic-00.047±0.0000.047_{\pm 0.000}0.035±0.0010.035_{\pm 0.001}0.034±0.002\mathbf{0.034_{\pm 0.002}}0.036±0.0020.036_{\pm 0.002}AW-NB
Monochromatic-50.041±0.0000.041_{\pm 0.000}0.026±0.001\mathbf{0.026_{\pm 0.001}}0.031±0.0010.031_{\pm 0.001}0.032±0.0050.032_{\pm 0.005}AW
Monochromatic-100.034±0.0000.034_{\pm 0.000}0.020±0.000\mathbf{0.020_{\pm 0.000}}0.022±0.0010.022_{\pm 0.001}0.022±0.0000.022_{\pm 0.000}AW
Monochromatic-150.024±0.0000.024_{\pm 0.000}0.016±0.000\mathbf{0.016_{\pm 0.000}}0.016±0.0000.016_{\pm 0.000}0.016±0.0000.016_{\pm 0.000}AW
Watts–Strogatz SPD0.013±0.0010.013_{\pm 0.001}0.006±0.001\mathbf{0.006_{\pm 0.001}}0.007±0.0020.007_{\pm 0.002}0.011±0.0010.011_{\pm 0.001}AW
Appendix
Table G.3 : GIN synthetic confirmation. Test nRMSE ↓\downarrow , mean ±SD over two matched runs after 250 epochs, with configurations frozen from the validation-only grid. Bold marks the lowest unrounded mean.
InjectionValidation ↑\uparrowTest ↑\uparrowΔ\Delta
NoPE0.4487±0.02860.4487{\pm}0.02860.4502±0.02760.4502{\pm}0.0276–
AW-RoPE ReZero0.6457±0.0022\mathbf{0.6457{\pm}0.0022}0.6461±0.0024\mathbf{0.6461{\pm}0.0024}+0.1959+0.1959
AW-RoPE ReZero + norm.0.6265±0.00570.6265{\pm}0.00570.6263±0.00570.6263{\pm}0.0057+0.1762+0.1762
AW-RoPE Single + norm.0.5402±0.01280.5402{\pm}0.01280.5382±0.01380.5382{\pm}0.0138+0.0880+0.0880
Appendix
Table G.4 : AW-RoPE Cluster stability ablation, four matched runs. All arms are paired and budget-matched. This table isolates Sparse AW-RoPE injection. Δ\Delta is relative to NoPE test accuracy.
Figure H.1 : Prediction and circulation under different edge fields. Top: paired test metrics at each validation-best checkpoint, mean and sample SD over three runs. Bottom: one point per run, obtained by averaging raw fundamental-cycle RMS across insertion layers with cycles in the first validation batch. The symmetric-log axis retains exact zeros. The one unrestricted Monochromatic-0 run is flat on this diagnostic batch.
Figure H.2 : Exploratory L×zL\times z sensitivity, fixed run. Each cell shows the best validation score of one run, to three decimals. Color scales are independent across tasks. Brighter cells indicate better scores. All 27 cells are retained, including the three reused mechanism cells.
DatasetMethodTrain s/epochPeak GiBLoader s
Monochromatic-0NoPE7.818±0.0767.818_{\pm 0.076}0.097±0.0000.097_{\pm 0.000}8.601±0.0498.601_{\pm 0.049}
Monochromatic-0WIRE10.292±0.53510.292_{\pm 0.535}0.098±0.0000.098_{\pm 0.000}8.580±0.0478.580_{\pm 0.047}
Monochromatic-0AW-RoPE, L=4L=416.548±0.20216.548_{\pm 0.202}0.115±0.0000.115_{\pm 0.000}8.807±0.1028.807_{\pm 0.102}
Monochromatic-0AW-RoPE, L=8L=821.516±0.11521.516_{\pm 0.115}0.130±0.0000.130_{\pm 0.000}8.730±0.2958.730_{\pm 0.295}
Monochromatic-0AW-RoPE, L=16L=1631.770±0.36531.770_{\pm 0.365}0.159±0.0000.159_{\pm 0.000}8.789±0.1968.789_{\pm 0.196}
Peptides-structNoPE3.493±0.0433.493_{\pm 0.043}2.801±0.0262.801_{\pm 0.026}31.090±0.67331.090_{\pm 0.673}
Appendix
Table H.1 : Isolated H200 full-model cost. Three timing runs, mean ±SD to three decimals. Peak GiB is each run’s maximum allocated memory over its three measured epochs. Loader seconds use existing prepared caches and are listed separately. The real tasks use FAVOR+.
Figure I.1 : Route phase changes transport on a fixed graph. The diamond has both edge orientations, uniform row transition weights, and a unit real input at vv . Arrows show walk indices from uu toward vv ; the row action gathers that input back to uu . Only aub=abv=Δ/2a_{ub}=a_{bv}=\Delta/2 and their negatives on reversed edges vary; all other displacements are zero, with ω=1\omega=1 . Every displacement remains in [−π,π][-\pi,\pi] . At L=2L=2 , the two routes have equal weight 1/41/4 . AW’s normalized response power changes from reinforcement to cancellation. The dashed line is the magnitude of an endpoint-factorized coefficient with the same phase-free mixing weights, not a measured WIRE network.
Field classIngredientsRouteFeature-cond.
Mixing (Zero)transport only×\times×\times (proof)
WIREendpoint phases×\times×\times
Static field (random, learned)circulation–×\times (proof)
Dynamic local gradientadaptivity, local×\times×\times (proof)
Dynamic nonlocal Flatadaptivity, nonlocal–✓ (unstable)
Sparse AW (ours)circulation + adaptivity✓✓
Appendix
Table I.1: Capability ladder on the constructed tasks. Each row adds one ingredient. Check marks solve at high accuracy, crosses sit at chance. “(proof)” marks entries covered by an impossibility statement; the route task uses endpoint readout and full networks, the feature-conditional task uses full networks on unseen sizes. The ladder compares field classes within the walk-transport operator interface; learned restriction-map transports are discussed in Appendix B.9 .
TaskCollision or empirical failureSuccessful construction
Route interference, endpointMixing (Zero), WIRE and pointwise gradient transportAW route-phase interference
Feature-conditioned cycle, full blockStatic shell transport and pointwise gradients; WIRE and random fixed fields fail empiricallyLocal edge AW or dynamically projected nonlocal Flat
Appendix
Table I.2: The two constructed tasks test different capabilities under specified readouts. These are comparison axes, not a nested hierarchy of all possible graph models.
ArmEvaluation2021222324
Sparse AWTest100.00100.00100.0093.4599.91
Sparse AW48/64 nodes92.9781.64100.0050.0071.88
Dynamic nonlocal FlatTest99.91100.0099.9674.26100.00
Dynamic nonlocal Flat48/64 nodes50.7882.81100.0050.0097.27
Appendix
Table I.3: All AW and dynamic nonlocal Flat runs, accuracy (%). Test is the primary unseen-size split; 48/64 nodes is a separate extrapolation diagnostic. Zeroing phases gives 50%50\% for every run on the primary test.
Figure I.2 : Learned fields and Flat run variability. (a) AW edge displacements for the first paired validation example at n=18n=18 . (b) The dynamic Flat control’s reconstructed zero-mean potential on that pair. Cycle indices are used only for this display, not supplied as model features. (c) All five Flat validation trajectories; the partially fitted run is highlighted. No smoothing is applied.
DatasetMetricWIREMixingWIRE + mixingSparse AW-RoPE
Monochromatic-0nRMSE ↓\downarrow0.073±0.0080.073_{\pm 0.008}0.044±0.0050.044_{\pm 0.005}0.045±0.0060.045_{\pm 0.006}0.043±0.006\mathbf{0.043}_{\pm 0.006}
Peptides-structMAE ↓\downarrow0.257±0.0020.257_{\pm 0.002}0.255±0.0020.255_{\pm 0.002}0.257±0.0010.257_{\pm 0.001}0.253±0.001\mathbf{0.253}_{\pm 0.001}
Appendix
Table I.4 : Completed matched mixing controls. Test at validation-best, mean ±sample SD{}_{\pm\mathrm{sample\ SD}} over three matched runs, three decimals. Mixing is phase-free propagation; WIRE + mixing applies propagation before WIRE rotation. Bold marks the best unrounded mean.
DatasetArmAllocatedPositionalUnused encoder
Monochromatic-0WIRE396443200
Monochromatic-0Mixing39324085
Monochromatic-0WIRE + mixing396443200
Monochromatic-0Sparse AW-RoPE3965633285
Peptides-structWIRE50672919200
Peptides-structMixing5048090270
Appendix
Table I.5 : Parameter accounting for the matched mixing study. Counts are identical across the three runs of each row. Allocated includes modules whose outputs are unused. The final column identifies unused coordinate-encoder parameters.
Figure I.3 : Attention on fixed Monochromatic-0 validation examples. Left: the three categorical node colors supplied to the task. Center and right: final-layer attention probabilities from the circled node for WIRE and Sparse AW-RoPE. Rows are the first three validation graphs in their stored order. All six attention panels use one common color scale, without per-panel rescaling. Seed 0 and validation-best checkpoints are fixed.
More in this topic

Graph Positional Encoding

See all 15 papers

Explore similar work

CardsList

  1. Rotary Position Encodings for Graphs

    Sep 26, 2025Isaac Reid, Arijit Sehanobish, Cederik Höfs +7Graph Positional EncodingRotary Positional Embeddings

    University of Cambridge · Google DeepMind · Independent Researcher +3

  2. Understanding Truncated Positional Encodings for Graph Neural Networks

    Jun 11, 2026James Flora, Mitchell Black, Weng-Keen Wong +1GNN ExpressivityPositional Encoding

    School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, Oregon, USA · Halıcıoğlu Data Science Institute, University of California San Diego, San Diego, California, USA.

  3. RoPE is Dead, Long Live RoPE: Towards Scalable Data-aware Positional Encodings

    Sep 28, 2026Jarod Lévy, Mathurin Videau, Jad Yehya +3Rotary Positional EmbeddingsLong-Context Language Modeling

    Meta AI, Paris · Inria, Université Paris-Saclay, Palaiseau, France