Multimodal language models can simulate different observers, yet it remains unclear how assigned perspectives reshape their internal representations and whether these changes govern their judgements. Here, we investigate the representational geometry of persona-conditioned visual interestingness in Qwen3-VL-8B-Instruct. Across 100 images and 53 prompting conditions, we systematically vary persona content and the linguistic framing used to introduce it, while recording behavioural ratings and layer-wise hidden states. We observe a systematic reorganisation of representational geometry across processing depth: early layers primarily encode image identity, whereas persona-specific structures emerge in later layers. Strikingly, prompt framing reshapes internal representations more strongly than persona identity, whereas persona content has a greater impact on behavioural judgements. This dissociation is further illustrated by minimal identity statements that alter representations even without changing responses. These findings demonstrate that representational distance does not necessarily reflect behavioural relevance. More broadly, they reveal how simulated perspectives emerge as distinct structures within the representational space of artificial cognitive systems, highlighting a fundamental distinction between the geometry of internal representations and their functional consequences.
Figures & tables
Figure 1: Experimental design. (1) Two persona schemas, socio-demographic and Big Five, with five prototypical personas each (P1–P5). (2) Each prompt is assembled from a preamble, the persona and the task; five prompt styles (S1–S5), which differ only in how the persona is introduced and where it appears, and three reference conditions give 53 prompts per image. (3) Qwen3-VL rates each image, and its hidden state is read at every decoder layer. (4) The hidden states yield MDS projections and layer-wise GDV curves, and the ratings enter the behavioural analysis.
Figure 2: Persona identity and prompt framing shape multimodal representations across Qwen3-VL-8B layers. Multidimensional scaling projections of the hidden states at decoder layers 2 (A), 10 (B), 28 (C) and 36 (D) for the 20-image subset under all 53 prompts. Cool colours denote the five socio-demographic personas and warm colours the five Big Five personas; circles, squares, upward triangles, diamonds and downward triangles distinguish the five prompting styles (S1–S5). Black crosses mark the task-only condition (Base), grey squares the minimal identity statement “You are an AI” (D1) and olive diamonds “You are a human being” (D2); marker shapes are reused between the reference conditions and the styles, which are told apart by colour. The value in each panel title is the GDV of the two-dimensional projection shown, computed over the 53 prompts. Base, D1 and D2 lie apart from the persona × style conditions, which occupy a shared but internally structured representational space.
Figure 3: Prompting style and persona identity differentially structure multimodal representations across Qwen3-VL-8B layers. Multidimensional scaling projections of the hidden states at decoder layers 2, 10, 22, 28 and 36 (columns A–E) for the 20-image subset. Only the 25 persona × style conditions of each schema are shown; the reference conditions are shown in Fig. 2 . The five rows show the same projections in different colourings, and the value in each panel title is the GDV of the two-dimensional projection shown under that row’s labelling. Row 1: prompting style, revealing pronounced clustering according to the five prompt formulations. Row 2: persona schema, contrasting socio-demographic with Big Five personas and showing increasing separation in deeper layers. Row 3: individual persona identity, with the socio-demographic personas in cool and the Big Five personas in warm colours, revealing finer persona-specific structure. Row 4: image identity, clearly clustered only in the earliest layer shown. Row 5: assigned interestingness rating, whose clustering varies across layers and is strongest in the final layer.
Figure 4: Layer-wise transition from image to persona and rating structure in Qwen3-VL-8B (Big Five schema). GDV values quantify clustering in the original high-dimensional hidden-state space at every decoder layer; more negative values indicate stronger cluster separation, and each curve is one prompting style (S1–S5). The projection panels show two-dimensional MDS projections at layer 36 for a 20-image subset, one per style, with the GDV of that projection in each title. (A1) GDV by image. Under every style, image structure dominates the first half of the network. (A2) Projections coloured by image. (B1) GDV by persona (P1–P5). Persona structure appears only as image structure recedes, and its strength depends on the prompting style. (B2) Projections coloured by persona. (C1) GDV by assigned rating. Rating structure also grows in later layers, mainly under S2 and S5. (C2) Projections coloured by rating; Extremely Interesting is marked absent because it does not occur in the 20-image subset. The same analysis for the socio-demographic schema is shown in Fig. 7 .
Figure 5: Ratings depend more on the persona than on the prompting style (Big Five schema). (A) Distributions of the 100 image ratings for the task-only condition (Base), the minimal identity statements “You are an AI” (D1) and “You are a human being” (D2), and the five Big Five personas (P1–P5); colours indicate the prompting style (S1–S5). The distributions differ more between personas than between the styles that introduce them, and Base, D1 and D2 are indistinguishable. (B) Share of rating variance explained by the three main effects (image, persona, prompting style), their two-way interactions (P × S, I × P, I × S) and the three-way interaction (I × P × S). Bars show the mean over 500 image-level bootstrap replicates, whiskers the 95% interval. All testable terms were significant in the mixed-effects model ( p<0.001 ; Table 1 ). Persona explains the largest share and style the smallest of the three main effects. Most of the persona effect is a shift in a persona’s overall rating level, but the I × P term shows that personas also change which images they rate higher. Prompting style, which dominated the global structure of the representations (Fig. 3 ), thus has little effect on the ratings, whereas the persona, which structured them only finely, has the largest. The same analysis for the socio-demographic schema is shown in Fig. 8 .
Figure 6: Rating patterns follow the persona more than the prompting style, and minimal identity statements leave them unchanged (Big Five schema). (A) Ratings under Base and the minimal identity statements D1 and D2 plotted against each other on the 1–5 scale; marker area is proportional to the number of images. The three conditions rate all 100 images identically, although D1 and D2 move the representations away from those of Base (Fig. 2 ). (B) Pearson correlations of the 100 image ratings between all 28 conditions: Base, D1, D2 and the 25 persona × style combinations, ordered by persona (P1–P5) and, within each persona, by style (S1–S5). Each outlined block compares one persona across its five styles. Ratings agree more for the same persona across styles than for the same style across personas, visible as darker blocks along the diagonal that are clearest for P2 and P4; the persona thus shapes which images are rated higher more than the style does. The same analysis for the socio-demographic schema is shown in Fig. 9 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Layer-wise transition from image to persona and rating structure in Qwen3-VL-8B (socio-demographic schema). As in Fig. 4 , GDV values quantify clustering in the original high-dimensional hidden-state space at every decoder layer; more negative values indicate stronger cluster separation, and each curve is one prompting style (S1–S5). The projection panels show two-dimensional MDS projections at layer 36 for a 20-image subset, one per style, with the GDV of that projection in each title. (A1) GDV by image. Image clustering dominates the early layers and weakens abruptly at layer 12 under every style except S5, where it weakens only around layer 20. (A2) Projections coloured by image. (B1) GDV by persona (P1–P5). Persona clustering emerges at layer 12, earlier than for the Big Five personas, abruptly under every style except S5, where it grows slowly. It is strongest under S4, apart from a peak under S3 at layer 1 that falls off sharply at layer 2. (B2) Projections coloured by persona. (C1) GDV by assigned rating. Rating clustering increases markedly in later layers only under S2. (C2) Projections coloured by rating; Extremely Interesting is marked absent because it does not occur in the 20-image subset.
Figure 8: Ratings depend most on the image, then on the persona, and least on the prompting style (socio-demographic schema). (A) Distributions of the 100 image ratings for Base, the minimal identity statements D1 and D2, and the five socio-demographic personas (P1–P5); colours indicate the prompting style (S1–S5), as in Fig. 5 . The distributions differ less between personas than under the Big Five schema, with P4 rating lowest. (B) Share of rating variance by component, as in Fig. 5 B; all testable terms were significant in the mixed-effects model ( p<0.001 ; Table 1 ). Unlike under the Big Five schema, the image explains the largest share and the persona the second largest, while style remains the smallest main effect; personas change which images they rate higher (I × P) slightly more than under the Big Five schema.
Figure 9: Rating patterns follow the persona more than the prompting style (socio-demographic schema). (A) Ratings under Base and the minimal identity statements D1 and D2 plotted against each other, as in Fig. 6 A. (B) Pearson correlations of the 100 image ratings between all 28 conditions, ordered as in Fig. 6 B. As under the Big Five schema, ratings agree more for the same persona across styles than for the same style across personas, with the clearest blocks for P2 and P5, and they stay closest to those of Base when the persona is placed after the task (S3).
Figure 1: Clustering by prompting style exceeds clustering by persona at every layer. GDV in the original hidden-state space at every decoder layer, computed on the representations of all five prompting styles together (2,500 per layer) and grouped by prompting style (S1–S5), by persona (P1–P5), by image and by assigned rating; more negative values indicate stronger clustering. (A) Big Five schema. (B) Socio-demographic schema. Style and persona each form five classes of equal size, so their values are directly comparable. Clustering by style is strong from the first layers on and peaks around layers 18–22, whereas clustering by persona appears only in later layers and remains several times weaker. Rating clustering reflects persona and style together, since both shift the rating level.
Pairs of conditions
Socio-demographic
Big Five
Same persona, different style
.69
.63
Same style, different persona
.62
.51
Neither shared
.56
.48
Appendix
Table 2: Mean pairwise Pearson correlation of the 100 image ratings, per schema, averaged over the off-diagonal cells of Fig. 6 B and Fig. 9 B. The rows group the 25 persona × style conditions by what a pair shares.
Prompting steers large language models (LLMs) and vision-language models (VLMs) without weight updates, but it remains unclear how instruction changes reshape internal representations to produce behavior. We introduce a nested geometric decomposition framework that treats prompting as a transformation of the representational geometry of the content following the prompt. For each prompt pair, we align representations of the same stimuli under two prompts using increasingly expressive stimulus-invariant maps: translation, rigid transformation with uniform scaling, sequential axis scaling, affine transformation, and nonlinear transformation. We then causally test each map by replacing a single layer's prompt-A hidden state for held-out stimuli with its mapped counterpart and measuring recovery of prompt-B representational geometry and behavior. Across three LLMs, three VLMs, and six text or image datasets spanning style, emotion, scene content, and number, prompts consistently reshape representations toward the instructed task structure. Cross-validated variance decomposition shows that much prompt-induced activation change is captured by shape-preserving maps, especially translation and rigid transformation with uniform scaling, while tier profiles reveal model- and task-specific routing strategies across layers. Crucially, although translation and rigid tiers already improve behavioral agreement, affine transformation is the first tier to nearly recover target-prompt task geometry and yields corresponding behavioral gains. This suggests that cross-dimensional linear mixing is a key mechanism by which prompts reorganize representations toward instructed task structure. Our framework decomposes prompt-induced representational change into interpretable geometric components and reveals how models route task-relevant structure to produce prompt-driven behavior.
Common methods for monitoring features in large language models (LLMs) such as linear probes and sparse autoencoders (SAEs) assume that features maintain a consistent representation across contexts. LLMs, however, can flexibly adopt different perspectives or "personas" that suggest different underlying systems of conceptual relations. In this paper, we examine whether prompting an LLM to adopt a persona alters the representational geometry of the model's latent space. We extract representations of over 9,000 common words and 33 semantic axes from models adopting liberal and conservative political personas and compare these against two neutral baselines. We find that persona conditioning systematically alters both the projections of words onto semantic axes and the alignments among the axes themselves in directions consistent with the specified ideology. We then test the implications of this reorganization for two safety-relevant features: truthfulness and morality. Linear probes, SAE-based classifiers, and classifiers built from manually selected word representations, all trained without a persona, produce divergent results when applied to models conditioned on political personas, producing politically slanted ratings of truth and morality on controversial statements. These findings suggest that internal representations, as they are conventionally extracted, can exhibit different geometric relations when the model adopts personas, potentially undermining monitoring efforts.
This study examines how persona prompting shapes language generated by two multimodal large language models in urban perception, a setting for examining subjective interpretations of shared visual evidence. We organize outputs into three functional layers: descriptive grounding (captions), intermediate semantic layer (perception tags), and interpretive framing (justifications). Using approximately 60,000 persona-conditioned annotations from each of two MLLMs, Qwen3-VL and Gemma4, we find that captions converge strongly across persona profiles and show only small attribute-associated differences. Justifications vary substantially more: economic status produces the largest difference in both models, with political orientation and personality also prominent. Paired image-level comparisons confirm larger justification than caption differences for these three attributes. For perception tags, personas sharing the same attribute level produce more similar tag sets than personas with different attribute levels, with the largest separation observed for economic status. Exploratory topic analysis further suggests persona-specific evaluative emphasis. Across models, profile-pair similarity patterns are strongly correlated for all three output types, although agreement is lowest for justifications. Overall, persona prompting affects interpretive framing more strongly than descriptive grounding.
Neemias da Silva, Matt Ratto, Myriam Delgado +3
University of Toronto, Toronto, Canada · Universidade Tecnologica Federal do Parana, Curitiba, Brazil