Distributed Interpretability and Control for Large Language Models

Dev Arpan Desai; Shaoyi Huang; Zining Zhu

cs.LGApr 7, 2026

Distributed Interpretability and Control for Large Language Models

Authors: Dev Arpan Desai, Shaoyi Huang, Zining Zhu

Abstract

Large language models that require multiple GPU cards to host are usually the most capable models. It is necessary to understand and steer these models, but the current technologies do not support the interpretability and steering of these models in the multi-GPU setting as well as the single-GPU setting. We present a practical implementation of activation-level interpretability (logit lens) and steering (steering vector) that scales up to multi-GPU language models. Our system implements design choices that reduce the activation memory by up to 7x and increase the throughput by up to 41x compared to a baseline on identical hardware. We demonstrate the method across LLaMA-3.1 (8B, 70B) and Qwen-3 (4B, 14B, 32B), sustaining 20-100 tokens/s while collecting full layer-wise activation trajectories for sequences of 1,500 tokens. Using label-position steering vectors injected post-LayerNorm, we show controllable, monotonic shifts in model outputs with a mean steerability slope of 0.702 across evaluated datasets, without fine-tuning or additional forward passes. We release detailed benchmarks, ablations, and a reproducible instrumentation recipe to enable practical interpretability and real-time behavioral control for frontier LLMs at https://github.com/Devdesai1901/LogitLense.

Figures & tables

Figure 1: Distributed single-pass logit-lens architecture. Selected transformer layers are instrumented with lightweight wrappers that expose post-attention activations under tensor-parallel execution. Hidden states for the most recent token are recorded during a single KV-cached forward pass. After generation, all captured activations are decoded in batch via the final normalization layer and shared LM head, enabling scalable per-token, per-layer analysis without additional forward passes.

Tokens	LL4L–TPtime (s)	SP–TPtime (s)	Speedup ( $\times$ )	Baseline tok/s	Ours tok/s
100	$166.0$ 4.8	$4.0$ 0.3	$41.5$	$0.60$ 0.02	$25.0$ 1.8
300	$498.0$ 11.2	$15.0$ 0.9	$33.2$	$0.60$ 0.01	$20.0$ 1.2
500	$833.0$ 19.5	$20.0$ 1.1	$41.7$	$0.60$ 0.01	$25.0$ 1.4

Table 1: Baseline (LL4L–TP) vs. our method (SP–TP) logit-lens runtime comparison on LLaMA-3.1–8B. Wall-clock time and throughput are reported as mean

\pm

std over

N=3

runs. Speedup is computed as the ratio of (LL4L–TP) time to (SP–TP) time. Both methods use identical multi-GPU tensor-parallel inference (4

\times

RTX A6000, bfloat16). The baseline performs per-layer re-forwarding, while ours method uses single-pass capture with deferred batched projection.

Figure 2: Logit lens heatmap for the token “university” across all 80 layers of the LLaMA 3.1–70B model. For each layer, three types of activations are visualized attention , MLP , and block output representing the intermediate computations captured after the post-LayerNorm stage. Each layer displays the model’s top- $k=3$ most probable tokens, and the color shade indicates their probability: darker shades correspond to higher token probabilities within that layer. Due to space constraints, only the first three and last three layers are shown here. Some cells display non-English or alphanumeric characters (e.g., “\u06\u0644”), which occur when the model emits tokens from other scripts or symbols not supported by the visualization font.

Figure 3: Steering-vector propensity analysis on LLaMA 3.1–70B. Mean propensity vs. steering multiplier $\alpha$ for selected layers. Steerability peaks at mid layer ( $L{=}35$ ) and is weaker at early layers ( $L{=}25$ ), consistent with directional sensitivity patterns reported by Tan et al. (2024) .

Model	Params	Mean tok/s	Std
LLaMA-3.1–8B	8B	23.3	2.6
LLaMA-3.1–70B	70B	21.0	1.2
Qwen3–4B	4B	28.5	1.9
Qwen3–14B	14B	21.2	1.4
Qwen3–32B	32B	17.0	1.1

Table 2: Cross-model throughput summary. comparison of average decoding throughput for SP–TP. Mean tok/s is aggregated across the token budgets used for each model; Std is across runs (

N=3

). (Replace dummy values with measured results.)

Model	Params	Tokens	Time (s) (mean $\pm$ std, N=3)	tok/s (mean $\pm$ std)
LLaMA-3.1–8B	8B	100	4.0 $\pm$ 0.3	25.0 $\pm$ 1.8
LLaMA-3.1–8B	8B	300	15.0 $\pm$ 0.9	20.0 $\pm$ 1.2
LLaMA-3.1–8B	8B	500	20.0 $\pm$ 1.1	25.0 $\pm$ 1.4
LLaMA-3.1–70B	70B	500	25.0 $\pm$ 1.5	20.0 $\pm$ 1.1
LLaMA-3.1–70B	70B	1500	61.3 $\pm$ 3.8	20.0 $\pm$ 1.0
LLaMA-3.1–70B	70B	2500	125.0 $\pm$ 6.2	20.0 $\pm$ 1.0

Table 3: Runtime scaling across models. Wall-clock time and throughput for SP–TP over varying token budgets. Reported values are mean

\pm

std over

N=3

runs on identical hardware (4

\times

RTX A6000).

Model	Layer	Mean slope	$p$ -value
LLaMA-3.1–8B	35	0.68	0.004
LLaMA-3.1–8B	62	0.07	0.180
LLaMA-3.1–70B	35	0.72	0.002
Qwen3–14B	35	0.55	0.011
Qwen3–32B	35	0.49	0.019

Table 4: Steering statistics for each (model, dataset, layer), we fit a per-prompt linear regression of the steering metric versus multiplier

m\in[-1.5,1.5]

, then report the mean slope

\pm

std across prompts. The

p

-value is computed via a paired test comparing the metric at

m=-1.5

vs.

m=+1.5

LLM Interpretability

Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.

See all 776 papers

Scaling Inherently Interpretable Language Models
Aug 6, 2026Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail +7Language Model Steering LLM Interpretability
Guide Labs Team
Measuring Activation Control in Large Language Models
Aug 21, 2026Marek Mateusz Kowalski, Joshua Fonseca Rivera, Uzay Macar +1LLM Safety Evaluation LLM Auditing
UK AI Security Institute
Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models
Aug 9, 2026Muhammad Faishal Adly Nelwan, Alfan Farizki WicaksonoLanguage Model Steering Controllable Text Generation
Faculty of Computer Science Universitas Indonesia

Aug 6, 2026J/K move · Enter open · S save

Scaling Inherently Interpretable Language Models

Guide Labs Team, Andreas Madsen, Aya Abdelsalam Ismail, Giang Nguyen+6

Guide Labs Team

Language Model Steering LLM Interpretability Diffusion Language Models Representation Geometry in Language Models LLM Training

Open paper AskPDF arXiv

Interpretability is often treated as a tax on capability: language models are trained as opaque systems, then explained after the fact, with methods whose reliability is difficult to establish. In this work, we challenge this premise. Rather than reverse-engineering a model, we make interpretability a constraint of the training pipeline, optimized alongside the language modeling objective. Across three orders of magnitude of compute, on both autoregressive and diffusion language models, interpretability scales with capability rather than against it. Surprisingly, model representations become more disentangled and aligned with human-understandable concepts with scale. We instantiate the training-time recipe with Steerling-8B, a diffusion language model with a causal attention mask. For any group of generated tokens, Steerling-8B attributes the output to relevant input tokens, human-understandable concepts, and training data. This enables closed-loop intervention: diagnose an output through its concept or feature attribution, retrieve similar training data, and correct the behavior through concept steering without retraining. Steerling-8B remains competitive with open peer models trained on substantially 2-16x more compute, suggesting a different scaling paradigm: interpretability can be designed into training, and it improves with scale.

Abstract

LLM Interpretability

Explore similar work

Scaling Inherently Interpretable Language Models

Measuring Activation Control in Large Language Models

Deployable Per-Instance Multi-Layer Activation Steering for Large Language Models