cs.CLNov 17, 2026

Zero-Dimensional Attention Topology for Linguistic Acceptability

Authors: Saeed AlMarri

Organizations: Khalifa University · Department of Engineering Science and Management, Khalifa University of Science and Technology, Abu Dhabi, UAE

Abstract

Topological summaries of Transformer attention provide sentence representations for linguistic acceptability classification. We examine a zero-dimensional persistent-homology (H0H_0) representation that summarizes minimum-spanning-tree merge values from 144 symmetrized BERT attention graphs into 1,440 features. The retrospective evaluation uses 8,517 downstream training sentences and 521 in-domain and 497 out-of-domain CoLA validation sentences after data-integrity filtering. The recorded out-of-domain Matthews correlation coefficients are 0.438 for the topological representation and 0.508 for attention statistics, compared with 0.545 for BERT. Attention-feature results are retained aggregate measurements; BERT predictions are rescored and length controls are refitted from the accompanying records. Analytical results show that the representation retains token-count information and discards a class of directional attention differences. Source resampling of BERT predictions also demonstrates the sensitivity of uncertainty estimates to the sampling unit. The reported results do not demonstrate a predictive advantage for this fixed topological representation over simpler attention summaries. They identify representation properties and evaluation controls relevant to subsequent studies of attention topology.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Which Question Is Your Attention Metric Answering? Attention Rows as Compositional Data

    Aug 11, 2026Marios Papamichalis, Regina RuaneAttention Head AnalysisTransformer Interpretability

  2. Global and Local Topology-Aware Attention with Persistent Homology and Euler Biases for Time-Series Forecasting

    May 4, 2026Usef Faghihi, Amir SakiTime Series ForecastingPersistent Homology

  3. Expressivity of Transformers: A Tropical Geometry Perspective

    Apr 16, 2026Ye Su, Yong LiuTransformer ExpressivityTransformer