Generative Interpretability via Scalable Neuro-Symbolic Models

Xiaocong Yang

cs.LGSep 11, 2026

Generative Interpretability via Scalable Neuro-Symbolic Models

Authors: Xiaocong Yang

Abstract

As the use of Large Language Models moves from chatbots into agentic systems, where outputs become actions with irreversible consequences on reality, the existing paradigm on AI Interpretability research, post-hoc interpretability, is structurally inadequate for safe and trustworthy model deployment: it explains behavior after the fact but cannot audit or intervene in an inference computation before it commits to an output. We therefore argue for a shift toward \emph{generative interpretability}, an architectural property under which a model's inference pass natively exposes semantically meaningful checkpoints that are human-understandable and amenable to causal intervention. We show the merits of generative interpretability as comparison to other interpretability research paradigms, and propose Neuro-Symbolic Models as a concrete instantiation.

Figures & tables

Figure 1. An example of a Neuro-Symbolic Model handling a customer’s refund request. Audit and intervene are designed at the refund amount checkpoint, as it is high-stake for a company. An inference DAG from input to output in which neural steps alternate with symbolic states: s1 claim(c7, \$42), s2 duplicate(c7), s3 within\_limit(c7), and s4 refund(c7). An audit panel at s3 shows that \$42 exceeds the \$25 limit; the auditor rewrites s3 to over\_limit(c7), which changes s4 from refund(c7) to escalate(c7). — Figure 1. An example of a Neuro-Symbolic Model handling a customer’s refund request. Audit and intervene are designed at the refund amount checkpoint, as it is high-stake for a company. An inference DAG from input to output in which neural steps alternate with symbolic states: s1 claim(c7, $42), s2 duplicate(c7), s3 within_limit(c7), and s4 refund(c7). An audit panel at s3 shows that $42 exceeds the $25 limit; the auditor rewrites s3 to over_limit(c7), which changes s4 from refund(c7) to escalate(c7).

Panel of Figure 2 — Figure 2. Interpretability paradigms organized by two temporal and spatial axes. Shaded cells admit audit and intervention before undesired content is generated. Neuro-symbolic models realize generative interpretability within this region while preserving distributed, high-capacity representations.

Explore similar work

Scaling Inherently Interpretable Language Models
Aug 6, 2026Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7Interpretability Diffusion Language Models
Guide Labs Team
Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes
Aug 3, 2026Maryam Rezaee, Pooriya Safaei, Maryam Asgarinezhad +1Post-Hoc Free-Energy Landscape
Department of Mathematical Sciences, Sharif University of Technology, Tehran, Iran
Automated Interpretability and Feature Discovery in Language Models with Agents
May 2, 2026Arnau Marin-Llobet, Javier FerrandoMechanistic Interpretability Interpretability
Harvard University

Aug 6, 2026, cs.CLJ/K move · Enter open · S save

Scaling Inherently Interpretable Language Models

Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen+6

Guide Labs Team

Interpretability Diffusion Language Models Causal Attention Scaling

Open paper AskPDF arXiv

Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.

Abstract

Explore similar work

Scaling Inherently Interpretable Language Models

Interpreting Black-Box Large Language Models with Sentence-Level Energy Landscapes

Automated Interpretability and Feature Discovery in Language Models with Agents