Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces

Hossein Javidnia

cs.LGMar 21, 2026

Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces

Authors: Hossein Javidnia

Organizations: School of Computing, Dublin City University, Ireland

Abstract

Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.

Figures & tables

Figure 1: Semantic sections on a cyclic chart subsystem. (A) A globalizable section, where transporting the local representative

s_{u}

around a loop

\gamma

returns a representative

T_{\gamma}s_{u}

that remains nearly aligned with

s_{u}

, yielding small loop defect

\Delta_{\gamma}\approx 0

. (B) A twisted section, where the same loop transport returns a rotated representative, so

T_{\gamma}s_{u}

no longer aligns with

s_{u}

and the loop defect is nonzero,

\Delta_{\gamma}>0

. This illustrates the difference between cycle-consistent semantic globalization and holonomy-obstructed local coherence.

Quantity	Value
Seed charts per model	$16$
Seed axes per chart	$32$
Raw seeded discoveries per model	$512$
Maximum propagation depth	$8$
Minimum overlap size	$256$
Transport conditioning threshold	$s_{\min}=0$

Table 1: Section-discovery protocol used for the cross-model section census. All runs use the same discovery and classification settings unless otherwise stated.

Model	Globalizable	Twisted	Tree-local	Fragile	Total unique
Llama 3.2 3B Instruct	1	1	2	32	36
Qwen 2.5 3B Instruct	1	2	5	16	24
Gemma 2 2B IT	14	4	8	40	66

Table 2: Deduplicated cross-model census of semantic section types at layer 16. Counts are reported after merging near-duplicate sections discovered from multiple seeds.

Model	Section type	Count	Mean support	Mean cycles	Mean edge defect	Mean alignment
Llama 3.2 3B Instruct	Globalizable	1	6.0000	1.0000	0.1367	0.9892
Llama 3.2 3B Instruct	Twisted	1	4.0000	1.0000	0.1232	0.9908
Llama 3.2 3B Instruct	Tree-local	2	4.5000	0.0000	0.1486	0.9887
Llama 3.2 3B Instruct	Fragile	32	1.9375	0.0938	0.0720	0.9946
Qwen 2.5 3B Instruct	Globalizable	1	9.0000	1.0000	0.1106	0.9915
Qwen 2.5 3B Instruct	Twisted	2	8.0000	1.5000	0.1217	0.9902

Table 3: Type-wise geometric profile of deduplicated semantic sections at layer 16. Support is the number of charts in the section support; cycles counts independent cycles in the induced support subgraph. Lower edge defect and higher alignment indicate stronger transport compatibility.

Model	Type	Seed	Support	Cycles	Mean edge defect	Mean holonomy defect	Representative chart support
Llama 3.2 3B Instruct	Globalizable	$(12,11)$	6	1	0.1367	0.3199	$\{12,26,38,78,108,111\}$
Llama 3.2 3B Instruct	Twisted	$(15,0)$	4	1	0.1232	0.3750	$\{15,27,30,101\}$
Qwen 2.5 3B Instruct	Tree-local	$(5,0)$	15	0	$\approx 0$	—	$\{0,3,5,8,10,15,28,43,72,77,79,83,93,101,115\}$
Qwen 2.5 3B Instruct	Globalizable	$(14,10)$	9	1	0.1106	0.3363	$\{1,13,14,52,63,64,96,97,126\}$
Qwen 2.5 3B Instruct	Twisted	$(14,17)$	9	2	0.1218	0.3954	$\{1,13,14,21,52,63,96,97,126\}$
Gemma 2 2B IT	Globalizable	$(14,4)$	23	9	0.0960	0.2245	$\{3,4,9,13,14,15,\dots,107,111,116\}$

Table 4: Representative deduplicated semantic sections used as exemplars in Section 5.3 , illustrating globalizable, twisted, and tree-local regimes across the three model families.

Model	Type	Seed	Support	Cycles	Mean edge defect	Mean path defect	Mean holonomy defect	Pairwise mean cosine	Pairwise min cosine	Fraction $\geq 0.5$	Fraction $\geq 0.7$
Llama 3.2 3B Instruct , layer 16	Globalizable	$(12,11)$	6	1	0.1367	0.1003	0.3199	0.2139	0.0796	0.0000	0.0000
Llama 3.2 3B Instruct , layer 16	Twisted	$(15,0)$	4	1	0.1232	0.0832	0.3750	0.2475	0.0615	0.1667	0.0000
Llama 3.2 3B Instruct , layer 16	Tree-local	$(11,9)$	5	0	0.1419	0.1366	—	0.3028	0.0701	0.3000	0.0000
Llama 3.2 3B Instruct , layer 16	Tree-local	$(0,5)$	4	0	0.1553	0.1294	—	0.1246	0.0284	0.1667	0.0000
Qwen 2.5 3B Instruct , layer 16	Tree-local	$(5,0)$	15	0	0.0000	0.0000	—	0.3483	0.0055	0.3143	0.0571
Qwen 2.5 3B Instruct , layer 16	Globalizable	$(14,10)$	9	1	0.1106	0.0942	0.3363	0.1814	0.0069	0.0000	0.0000

Table 5: Semantic-section coherence versus raw cross-chart vector similarity for the representative exemplars in Section 5.3 , together with a small number of additional tree-local baseline sections. Even when a candidate object is section-coherent, its raw local representatives are not close as a single shared vector across charts.

Model	Type	Seed	Support	Cycles	Pairs	Section recovery	Raw @ $0.3$	Raw @ $0.5$	Raw @ $0.7$	Mean cosine	Min cosine
Llama 3.2 3B Instruct , layer 16	Globalizable	$(12,11)$	6	1	15	1.0000	0.2667	0.0000	0.0000	0.2139	0.0796
Llama 3.2 3B Instruct , layer 16	Twisted	$(15,0)$	4	1	6	1.0000	0.3333	0.1667	0.0000	0.2475	0.0615
Llama 3.2 3B Instruct , layer 16	Tree-local	$(11,9)$	5	0	10	1.0000	0.5000	0.3000	0.0000	0.3028	0.0701
Llama 3.2 3B Instruct , layer 16	Tree-local	$(0,5)$	4	0	6	1.0000	0.1667	0.1667	0.0000	0.1246	0.0284
Qwen 2.5 3B Instruct , layer 16	Tree-local	$(5,0)$	15	0	105	1.0000	0.5619	0.3143	0.0571	0.3483	0.0055
Qwen 2.5 3B Instruct , layer 16	Globalizable	$(14,10)$	9	1	36	1.0000	0.2500	0.0000	0.0000	0.1814	0.0069

Table 6: Semantic identity recovery: sections versus raw global similarity. The section criterion recovers all within-section pairs by construction, while raw signed-cosine similarity recovers only a small fraction of them.

Model	Type	$n$ sections	Mean support	Mean cycles	Total pairs	Section	Raw @ $0.3$	Raw @ $0.5$
Llama 3.2 3B Instruct , layer 12	Globalizable	1	8.0000	8.0000	28	1.0000	0.1071	0.0714
Llama 3.2 3B Instruct , layer 12	Tree-local	5	6.4000	0.0000	101	1.0000	0.0792	0.0000
Llama 3.2 3B Instruct , layer 16	Globalizable	1	6.0000	1.0000	15	1.0000	0.2667	0.0000
Llama 3.2 3B Instruct , layer 16	Twisted	1	4.0000	1.0000	6	1.0000	0.3333	0.1667
Llama 3.2 3B Instruct , layer 16	Tree-local	2	4.5000	0.0000	16	1.0000	0.3750	0.2500
Llama 3.2 3B Instruct , layer 20	Tree-local	2	4.5000	0.0000	16	1.0000	0.2500	0.0000

Table 7: Aggregate semantic identity recovery by section type. Across models and layers, section-based recovery is perfect on certified supports, while raw global-similarity recovery remains poor and collapses to zero at stricter thresholds.

Figure 2: Aggregate semantic identity recovery across section types. Section-based recovery remains perfect on certified supports, while raw global-similarity recovery is consistently lower and typically collapses at stricter thresholds.

Model / layer	Globalizable	Twisted	Tree-local	Fragile	Total unique
Llama 3.2 3B Instruct , layer 12	1	0	5	17	23
Llama 3.2 3B Instruct , layer 16	1	1	2	32	36
Llama 3.2 3B Instruct , layer 20	0	0	2	21	23

Table 8: Deduplicated semantic-section census across Llama layers.

Model / layer	Globalizable	Twisted	Tree-local	Fragile	Total unique
Llama 3.2 3B Instruct , layer 12	$0$ – $1$	$0$ – $0$	$5$ – $5$	$17$ – $18$	$23$ – $23$
Llama 3.2 3B Instruct , layer 16	$0$ – $1$	$1$ – $1$	$2$ – $2$	$32$ – $33$	$36$ – $36$
Qwen 2.5 3B Instruct , layer 16	$0$ – $1$	$1$ – $2$	$4$ – $5$	$16$ – $17$	$22$ – $24$
Gemma 2 2B IT , layer 16	$13$ – $18$	$2$ – $5$	$8$ – $8$	$40$ – $47$	$64$ – $74$

Table 9: Range of deduplicated section counts under threshold variation. Each entry reports the minimum and maximum count across the

27

threshold settings.

Mechanistic Interpretability

Mechanistic interpretability studies how neural networks implement behaviors through internal features, circuits, and causal information flow.

See all 391 papers

Oct 20, 2026

OV-Foam: How 3D Representations Shape Semantic Feature Lifting

Existing feature lifting approaches for open-vocabulary segmentation rely on simple backprojection to assign features to primitives. In the case of 3D Gaussian splatting reconstruction, the primitives overlap heavily, leading to suboptimal reconstructions, governed by many primitives per ray. Backprojection, however, reduces to a least-squares solvable problem, exactly when no two primitives share a ray. Motivated by this insight, we propose OV-Foam, which uses PowerFoam as the underlying representation. This representation uses a power-diagram to partition the scene, resulting in zero overlap between primitives and a substantial reduction in primitives per ray, which ultimately simplifies the lifting problem and allows for closer-to-optimal feature assignment. We show that the space partitioning primitives of OV-Foam reach near-optimal assignment, and our experiments demonstrate improved open-vocabulary segmentation over Gaussian baselines.

Yousef Rajeh, Alejandra Esparza Pelcastre, Ivan Viola +1

King Abdullah University of Science and Technology · Universidad Panamericana

Open-Vocabulary 3D Segmentation 3D Scene Representation

Ask PDF

May 7, 2026

OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention

Understanding open-vocabulary 3D scenes with Gaussian-based representations remains challenging due to fragmented and spatially inconsistent semantic predictions across multi-view observations. In this paper, we present OpenGaFF, a novel framework for open-vocabulary 3D scene understanding built upon 3D Gaussian Splatting. At the core of our method is a Gaussian Feature Field that models semantics as a continuous function of Gaussian geometry and appearance. By explicitly conditioning semantic predictions on geometric structure, this formulation strengthens the coupling between geometry and semantics, leading to improved spatial coherence across similar structures in 3D space. To further enforce object-level semantic consistency, we introduce a structured codebook that serves as a set of shared semantic primitives. Furthermore, a codebook-guided attention mechanism is proposed to retrieve language features via similarity matching between query embeddings and learned codebook entries, enabling robust open-vocabulary reasoning while reducing intra-object feature variance. Extensive experiments on standard 2D and 3D open-vocabulary benchmarks demonstrate that our method consistently outperforms prior approaches, achieving improved segmentation quality, stronger 3D semantic consistency and a semantically interpretable codebook that provides insight into the learned representation.

Kunyi Li, Michael Niemeyer, Sen Wang +3

1Technical University of Munich · 3Munich Center for Machine Learning · 2Google +1

3D Gaussian Splatting Open-Vocabulary 3D Segmentation 3D Scene Understanding

Ask PDF

Jun 8, 2026

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability

As Vision-Language Models are increasingly deployed in safety-critical applications, the trustworthiness of their explanations becomes crucial. Explainable AI (XAI) methods for Vision-Language Models often suffer from semantic hallucination, where attribution maps highlight prominent image regions even when prompted with incorrect text descriptions (e.g., highlighting a dog when prompted ``cat''). Although this problem is widespread, a formal mathematical analysis of XAI methods and CLIP embeddings is largely missing in the literature. We demonstrate that this phenomenon is not specific to a single architecture but is a fundamental consequence of Linear Semantic Leakage in high-dimensional embedding spaces. We propose a unified theoretical framework, Linear Semantic Attribution (LSA), which generalizes across discriminative methods. We introduce OSP, a geometric intervention that utilizes the residual property of OMP to disentangle unique semantic signals from shared concepts. We prove theoretically and demonstrate empirically that OSP minimizes hallucination by orthogonalizing the query vector against distractor concepts, rendering the attribution model blind to shared features while preserving fidelity for correct prompts. Our code is available at: https://github.com/emirhanbilgic/Orthogonal-Semantic-Projection

Emirhan Bilgiç, Baptiste Caramiaux, Zhi Yan +1

U2IS, ENSTA, Institut Polytechnique de Paris, Palaiseau · ISIR, Université Sorbonne, Pierre et Marie Curie, Paris · AMIAD, Pôle Recherche, Palaiseau

VLM Interpretability Multimodal Model Interpretability VLM Hallucination Mitigation

Ask PDF

Abstract

Mechanistic Interpretability

Explore similar work

OV-Foam: How 3D Representations Shape Semantic Feature Lifting

OpenGaFF: Open-Vocabulary Gaussian Feature Field with Codebook Attention

Disentangling Hallucinations: Orthogonal Semantic Projection for Robust Interpretability