cs.AISep 4, 2026

Technical Manual for a Toolkit for Measuring Contextual Individuation in Transformer Language Models

Authors: José Luciano Verçosa Marques, Frederico Jorge Heitmann, Daniel Omar Perez, Marcelo Vinicius de Paula, Tárcio André dos Santos Barros

Organizations: Center for Electric Mobility Research (CEMOBE) / Power Electronics Laboratories (LEPO), University of Campinas (Unicamp) · Institute of Computing (IC), University of Campinas (Unicamp) · Center for Logic, Epistemology and History of Science (CLE), University of Campinas (Unicamp)

Abstract

A transformer language model assigns a single, context-independent vector to a word type at its embedding layer, yet is widely believed to individuate that word's occurrences by context in its later layers. Testing this belief cleanly requires a construct that holds the word form fixed while its context and intended sense vary in a controlled, labeled way. This manual documents an open toolkit built around such a construct, which we call a bridge form: a single written word that recurs, unchanged, across two or more subject domains with a different sense in each. We describe, and justify, every stage of the pipeline: the declarative specification of bridge forms and their source domains, corpus acquisition from Wikipedia, occurrence localization, layer-wise representation extraction, a domain-pairwise silhouette measurement of separation in the model's representation space, and a paired visualization protocol. Each design choice is presented together with the methodological failure mode it is meant to avoid (sense contamination from overly broad category labels, the multi-group bias of the silhouette coefficient, subword-tokenization misalignment, and axis-comparability artifacts in dimensionality-reduced plots, among others). This manuscript is a methodological and implementation reference: it does not report or interpret empirical outcomes of running the toolkit on any particular model or bridge-form set. The toolkit, its full source, and the corpora used to exercise it are archived separately (Section 9) under a persistent identifier, and are intended to be cited as an instrument by studies that use it to produce and interpret empirical results.

Figures & tables

Explore similar work

CardsList
  1. Transformer-Based Language Models Across Domain Verticals: Architectures, Applications and Critical Assessment

    Jun 23, 2026Guruprakash J, Krithika L. BLLM EvaluationLLM Alignment

  2. Unveiling the Mechanisms of Multi-Hop Reasoning in Transformers via Identity Bridge

    Sep 29, 2025Pengxiao Lin, Zheng-An Chen, Zhi-Qin John XuTransformer InterpretabilityOOD Generalization