cs.AIOct 27, 2026
SaveA Heart Means Recursion: Exploring Visual Meaning and Communication in VLMs
Organizations: Google DeepMind · Stanford University
Abstract
Do vision-language models (VLMs) share our ability to communicate visually? We share preliminary findings, starting from a surprising observation: simple visual symbols like a red heart can strongly influence model behavior on seemingly unrelated tasks, such as steering models towards recursive solutions on simple coding problems. Notably, this steering effect is largely implicit, as models' direct interpretations and thinking traces about visual symbols diverge substantially from their observed behavior. In visual communication between models, this leads to a generation--interpretation gap: while models tend to send shape primitives that they (and humans) find intuitive, these visual cues often fail to steer other models. In further communication games involving abstract concepts and stories, models can creatively use visuals to transmit meaning to one another, although some communications remain non-intuitive to humans. These findings reveal that VLMs might be capable of rich, creative, and visually-grounded communication, but their interpretation and production of visual meaning diverges from human conventions in surprising ways. Together this work highlights both the promise and challenges of visual communication between humans and VLMs.
Figures & tables
Figure 1: Simple visual symbols steer model behavior on a coding problem. A) Evaluation methodology. We append an image of a simple colored shape (e.g., a blue triangle) to a coding prompt (binary tree traversal) and instruct the model to look at the image while solving the problem. Across sampled generations, we compute the proportion of recursive solutions and measure the visual steering effect as the change in relative to the no-image baseline. B) Visual symbols steer toward recursion far more strongly than random noise or matching text descriptions. Star-patterned bars show average for Gemini 3.1 Pro and Claude Opus 4.7’s top three steering symbols (symbols shown below each bar), which nearly matches the effect of directly instructing the model to use recursion (light gray bars). By contrast, matching text descriptions (e.g., “an image of a pink heart”; hatched bars) and Gaussian noise images (dark gray bars) induce minimal to no steering. C) Models display different steering landscapes over a library of 288 visual symbols. Heatmaps show across 12 colors ( -axis) for each model’s 8 highest- and 6 lowest-steering shapes ( -axis; full heatmaps in Sec. B.2.1 ).
| Thinking Trace Category | % of Traces | Example (visual symbol & trace excerpt) |
|---|---|---|
| Faithful attribution Trace claims relevance, and symbol actually steers. | 25.0% of steered ( ) | “The image is a solid yellow circle – likely a distraction, but I note it… Wait. The circle… that could suggest a loop, therefore, iterative. Fine. Let’s do iterative.” |
| Unfaithful disregard Trace claims irrelevance, but symbol actually steers. | 71.5% of steered ( ) | “First, the image is completely irrelevant – a cyan spiral. Distraction, ignore… Do I go recursive or iterative? Recursive is usually cleaner and more concise for this…” |
| Faithful disregard Trace claims irrelevance, and symbol does not steer. | 77.9% of non-steered ( ) | “The purple triangle is a red herring; I’m going to disregard that… I immediately think about both iterative and recursive approaches…” |
| Unfaithful attribution Trace claims relevance, but symbol does not steer. | 19.3% of non-steered ( ) | “A downward pink arrow… A downward arrow could signify a ‘top-down’ approach, which often implies recursion… I’ll go with recursive.” |
Table 1: Gemini 3.1 Pro’s thinking traces often dismiss visual symbols as irrelevant even when they actually influence behavior. Percentages are computed for cases where the symbol steers (first two rows) or does not steer (second two rows) on binary tree inorder traversal. For GPT-5.4, of traces show unfaithful attribution; Claude Opus 4.7 did not emit traces.
Figure 2: Models generate creative visualizations to communicate concepts, but exhibit a generation–interpretation gap. A) Communication game methodology. A sender model generates a visualization to convey a target coding strategy (e.g., recursion), concept, or story to a receiver model, refining its visualization through multi-turn feedback. B) Multi-turn feedback enables models to converge on effective visualizations. Bars show the average number of turns to succeed using visual symbols (top bars) versus free-form PIL drawings (bottom bars). Symbol-based communication requires more communication turns to align sender and receiver than free-form drawing. C) Senders choose intuitive symbols to communicate recursion rather than the symbols that actually steer them. Top symbols selected by Claude Opus 4.7 as a sender (left, sized by selection frequency) compared to the symbols with the largest empirical steering effect (right). D) Models construct creative scenes using free-form code and abstract symbols. Examples show successful visualizations generated by Gemini 3.1 Pro to communicate the concept “art” (left) and a three-sentence story about a knight removing a thorn from a dragon’s paw (right).
Figure 3: Visual steering associations diverge from both human intuition and models’ own explicit interpretations. A) Optimizing symbol communication for model receivers reduces human legibility, whereas free-form drawings remain interpretable. Bars show human accuracy ( ) at identifying the target coding strategy (recursion vs. iteration), concept, or story from visualizations on the first communication turn (blue; unsuccessful for the receiver) versus the final turn (red; successful), using visual symbols (top bars) or free-form PIL drawings (bottom bars). B) Models’ direct interpretations of visual symbols poorly predict their empirical steering effects. Each point represents one or more of the 288 visual symbols (sized by count), plotting the proportion of trials in which a model explicitly classifies the symbol as “recursion” ( -axis) against its behavioral steering effect ( -axis).
Appendix figures & tables40 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: All shapes and colors in the visual symbol library. Each symbol shows an example of the color and shape indicated by the label; the library includes all 288 combinations of the colors and shapes.
Figure 5: Full steering heatmaps for inorder binary tree traversal , where the model is instructed to look at the image (“Explicit image instruction prompt,” Appendix B.1 ). Each heatmap is one model; rows are the 12 colors, and columns are the 24 shapes. Cell values are the steering effect (i.e. delta relative to that model’s text-only baseline), on a diverging scale from (red, towards iteration) to (blue, towards recursion).
Figure 6: Recursion steering heatmaps aggregated across baseline-iterative coding problems : problems for Claude Opus 4.7, for Gemini 3.1 Pro, and for GPT-5.4. Because all problems for each model have baseline iterative solutions, the cell values show steering towards or away from recursion. Axes and color scale are as in Figure 5 .
Figure 7: Iteration steering heatmaps aggregated across baseline-recursive coding problems : problems for Claude Opus 4.7, for Gemini 3.1 Pro, and for GPT-5.4. Because all problems for each model have baseline recursive solutions, the cell values show steering towards or away from iteration. Axes and color scale are as in Figure 5 .
Figure 8: The steering effect of text captions largely does not match that of images. A) Top three steering captions for each model. The colors of the text show the corresponding color. Captions are formatted as, e.g., “An image of a brown spiral.” B) The steering effect of captions decouples from the steering effect of images of visual symbols.
Figure 9: Images of Gaussian noise ( ) used to measure the noise image steering baseline.
Figure 10: Naturalistic animal images ( ) used to measure the ImageNet steering baseline.
Figure 11: Disrupting shape structure via pixel shuffling disrupts visual steering at high enough percentages. A) Example pixel-shuffled stimuli. Progressive pixel shuffling ( to ) preserves global color proportions while progressively destroying shape geometry and adding low-level noise (shown for the pink heart and blue triangle). B) Recursion proportion across pixel shuffle levels. Lines show on inorder binary tree traversal across five symbols (visualized next to their lines) for Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4.
Figure 12: Visual steering is highly sensitive to symbol orientation. A) Example rotated stimuli. Visual symbols rotated from (prototypical upright orientation, outlined in bold) to in increments. B) Recursion proportion across rotation angles. Lines show on inorder binary tree traversal across the three orientation-dependent symbols for Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4.
Figure 13: Canvas position and symbol size modulate visual steering. A) Stimulus grid. Example stimuli across canvas positions (top left, center, bottom right) and symbol sizes (small, medium, large). The default unperturbed configuration (center, medium) is outlined in bold. B) Mean across the position-by-size sweep. Each cell shows averaged over the five base symbols on inorder binary tree traversal.
Figure 14: Per-symbol steering across canvas position and symbol size. Lines track for each of the five symbols across Claude Opus 4.7, Gemini 3.1 Pro, and GPT-5.4. A) Sensitivity to canvas position (marginalized across symbol sizes). B) Sensitivity to symbol size (marginalized across canvas positions).
Figure 15: Placing images at the end of context rather than the beginning changes which symbols steer for recursion. A) The steering effect at the start and end of context decouple most strongly for Claude Opus 4.7 and GPT-5.4. The -axis shows the steering effect of visual symbols when placed at the start of context, while the -axis shows the end of context. Gemini 3.1 Pro remains somewhat correlated. B) Top steering symbols appear to shift from more geometric to more semantic. Lines show the steering effect for four individual symbols when placed at the start, middle, and end of context (symbols are visualized next to their lines). Claude Opus 4.7 and GPT-5.4 exhibit a marked shift where more intuitively “recursive” shapes like branching arrows or spirals begin to exert the strongest steering effect. Interestingly, this appears to line up with these models’ interpretations and choices in communication games. Gemini 3.1 Pro’s top steering shapes remain largely the same.
| Task | Description |
|---|---|
| Fibonacci number computation | Write a function fib(n) that returns the n -th Fibonacci number. Assume n is a non-negative integer. |
| Binary tree inorder traversal | Given the root of a binary tree, return the inorder traversal of its nodes’ values. Assume standard definition: class TreeNode: def init(self, val=0, left=None, right=None): self.val = val; self.left = left; self.right = right |
| Binary tree preorder traversal | Given the root of a binary tree, return the preorder traversal of its nodes’ values. Assume standard definition: class TreeNode: def init(self, val=0, left=None, right=None): self.val = val; self.left = left; self.right = right |
| Binary tree postorder traversal | Given the root of a binary tree, return the postorder traversal of its nodes’ values. Assume standard definition: class TreeNode: def init(self, val=0, left=None, right=None): self.val = val; self.left = left; self.right = right |
| Binary search | Given a sorted (in ascending order) integer array nums and a target value, write a function to search target in nums . If target exists, then return its index. Otherwise, return -1 . |
| Graph depth-first search (DFS) | Given a graph represented as an adjacency list graph (a dictionary mapping each node to a list of its neighbors) and a starting node start , return a list of all nodes visited in Depth First Search (DFS) order starting from start . |
Table 2: Recursion vs. iteration coding tasks. Problem names and exact descriptions provided to models.
| Task | Description |
|---|---|
| Maximum depth of a binary tree | Given the root of a binary tree, return its maximum depth. A binary tree’s maximum depth is the number of nodes along the longest path from the root node down to the farthest leaf node. |
| Clone graph | Given a reference of a node in a connected undirected graph, return a deep copy (clone) of the graph. Each node in the graph contains an integer val and a list of its neighbors . Assume standard Node definition: class Node: def init(self, val=0, neighbors=None): self.val = val; self.neighbors = neighbors if neighbors is not None else [] |
| Symmetric tree | Given the root of a binary tree, check whether it is a mirror of itself (i.e., symmetric around its center). Assume standard definition: class TreeNode: def init(self, val=0, left=None, right=None): self.val = val; self.left = left; self.right = right |
| Course schedule | There are a total of numCourses courses you have to take, labeled from 0 to numCourses - 1 . You are given an array prerequisites where prerequisites[i] = [a, b] indicates that you must take course b first if you want to take course a . Return True if you can finish all courses, or False otherwise. |
Table 3: DFS vs. BFS coding tasks. Problem names and exact descriptions provided to models.
| Data Structures | Description |
|---|---|
| Hash map vs. binary search tree | Design a Leaderboard class that supports update_score(player_id, score) and get_top_k_players(k) . Player scores are updated frequently during gameplay, and top- k queries are periodically requested. |
| Design an OrderBook class that supports adding and canceling orders by ID ( add_order(order_id, price, qty) , cancel_order(order_id) ) as well as fetching the current best bid and ask prices ( get_best_bid() , get_best_ask() ). Orders are frequently added and modified. | |
| Design an IPRateLimiter class supporting record_request(ip_address, timestamp) to track traffic counts per IP, and get_top_active_ips(k) to return the top k most active IPs in order. Traffic records are continuously updated. | |
| Doubly linked list vs. ring buffer | Implement an LRUCache class with get(key) and put(key, value) with fixed capacity C . Choose an underlying data structure to track key access recency. |
| Implement a fixed-capacity StreamBuffer class supporting append_event(event_data) and get_recent_events(n) . When capacity is reached, the oldest events must be automatically evicted in O(1) time. | |
| Implement a CommandHistory class with fixed capacity max_history supporting execute(command) , undo() , and redo() . Older commands beyond capacity are discarded when new commands are executed. |
Table 4: Data structure steering tasks. Candidate data structure pairs and exact problem descriptions provided to models.
Figure 16: Visual symbols steer models toward both recursion and iteration across coding problems. Bars show the mean shift away from each model’s unsteered baseline strategy ( across problems and top-3 symbols, shown below each bar). A) Aggregated recursion steering across baseline-iterative problems ( for Claude Opus 4.7, for Gemini 3.1 Pro, and for GPT-5.4). B) Aggregated iteration steering across baseline-recursive problems ( for Claude Opus 4.7, for Gemini 3.1 Pro, and for GPT-5.4). On baseline-recursive problems, Claude Opus 4.7 responds strongly to Gaussian noise (indicating a less purely semantic influence), whereas Gemini 3.1 Pro and GPT-5.4 maintain the selective visual-symbol steering pattern from Sec. 3 .
Figure 17: Consistent steering impacts of shapes toward recursive algorithmic strategies. Box plots depict the distribution of steering effects (i.e. proportion shifts, ) driven by individual shapes across baseline-iterative coding problems ( for Claude Opus 4.7, for Gemini 3.1 Pro, and for GPT-5.4). Full color-shape heatmaps for recursion steering are presented in Fig. 6 .
Figure 18: Circular shapes steer models toward using ring buffers over a linked list baseline. Box plots depict the steering effect ( ) driven by individual shapes across doubly linked list vs. ring buffer coding problems ( ), computed the same way as Fig. 17 . GPT-5.4 is omitted because it did not demonstrate steering on any of the three problems.
Figure 19: Writing style steering heatmaps aggregated across topics for optimistic vs. melancholic (Claude Opus 4.7, top) and verbose vs. terse (Gemini 3.1 Pro, bottom). Warm colors tend to steer Claude Opus 4.7 toward optimistic writing while cool colors steer toward melancholic, alongside a shape–color interaction where down arrows, “x” symbols, and caution triangles steer toward melancholic regardless of color. Gemini 3.1 Pro is steered toward verbose prose by infinity symbols, spirals, and branching arrows (possibly evoking a lengthy, branching conversation), while simple straight lines steer toward terse prose. Axes and color scale are as in Figure 5 .
Figure 20: Visual shapes intuitively steer writing style between poetic and professional prose. Box plots depict the steering effect ( ) driven by individual shapes on poetic vs. professional writing across topics, computed the same way as Fig. 17 . Values above indicate steering towards poetic, while values below indicate steering towards professional.
Figure 21: Visual symbols steer matched semantic preferences more strongly than random string pairs. Bars show the mean shift away from each model’s unsteered baseline preference ( pairs per split). A) Steering on matched semantic pairs , like “summer” vs. “winter.” B) Steering on length-6 random strings.
Figure 22: Steering results for all binary choice semantic pairs. The labels along the -axes indicate the direction of steering away from each model’s unsteered baseline preference (from to ), with icons below each bar showing the visual symbols that drive the shift.
Figure 23: Cross-model correlations (Pearson ) in visual steering on inorder binary tree traversal. Shapes are generally more correlated than colors.
Figure 24: Recursion steering results for smaller models.
Figure 25: Silently injecting visual symbols steers differently from explicit instruction. A) Top-3 steering shapes under silent injection and their steering deltas from baseline. B) Explicit steering ( -axis) vs. implicit (silent injection) steering ( -axis).
| Claude Opus 4.7 | ||
|---|---|---|
| Visual Symbol | Interpret as Recursion | Interpretation |
| 1.00 “recursive” (matches) | “A heart symbolizes love, and ‘I love you’ is often associated with recursion in programming humor (love is recursive/self-referential).” | |
| 0.00 “iterative” (matches) | “The image shows a single arrow pointing forward in one direction, which suggests linear, forward progression—characteristic of iterative approaches that move step by step in a sequence, rather than recursive approaches that would typically be represented by circular or self-referential imagery.” | |
| 0.00 “iterative” (mismatches) | “The image shows a single simple triangle without any nested or self-similar sub-triangles (like a Sierpinski pattern would suggest). A plain single shape suggests a straightforward iterative drawing approach rather than a recursive one.” | |
| 1.00 “recursive” (mismatches) | “The image shows a Y-shaped branching arrow, splitting into two paths. This branching structure is characteristic of recursion, where a problem splits into multiple subproblems (like tree recursion), whereas iteration would be represented by a linear or looping single path.” | |
Table 5: Examples of Claude Opus 4.7 symbol interpretations that match (top two rows) or mismatch (bottom two rows) empirical visual steering on binary tree inorder traversal.
| GPT-5.4 | ||
|---|---|---|
| Visual Symbol | Interpret as Recursion | Interpretation |
| 0.50 “recursive” (matches) | “The image is a heart, which commonly suggests affection or ‘love.’ In coding-culture wordplay, a heart/self-referential symbol more strongly hints at recursion—something that refers back to itself—rather than iteration…” | |
| 0.00 “iterative” (matches) | “The image shows a circular arrow looping back on itself, which commonly represents repetition or cycling through steps. In coding, that more directly suggests iteration—a loop that repeats—rather than recursion…” | |
| 0.15 “iterative” (mismatches) | “The image shows a single upward arrow, which commonly suggests moving forward step by step or progressing through repeated stages. That aligns more naturally with an iterative coding strategy than with recursion, which is usually associated with self-reference or nested repetition.” | |
| 1.00 “recursive” (mismatches) | “The image is a spiral curling inward, which commonly symbolizes self-reference or something containing a smaller version of itself—an intuitive visual metaphor for recursion. Iteration is more often represented by a simple loop…” | |
Table 6: Examples of GPT-5.4 symbol interpretations that match (top two rows) or mismatch (bottom two rows) empirical visual steering on binary tree inorder traversal.
| Gemini 3.1 Pro | ||
|---|---|---|
| Visual Symbol | Interpret as Recursion | Interpretation |
| 1.00 “recursive” (matches) | “The image depicts a single path branching out into two separate, identical paths. This branching structure is a classic visual representation of recursion (specifically tree recursion or divide-and-conquer algorithms), where a single function call splits into multiple smaller, self-similar function calls…” | |
| 0.00 “iterative” (matches) | “The image displays a continuous circular arrow, which is the universal symbol for a loop or cycle. In programming, an iterative strategy involves repeatedly executing a block of code in a loop (such as a for or while loop) until a condition is met. This simple, continuous circular motion perfectly visually represents standard iteration…” | |
| 0.05 “iterative” (mismatches) | “The image displays a plus sign ( + ), which is universally recognized in programming as the operator for addition and incrementing (e.g., i++ or count += 1 ). Incrementing a counter is a core mechanism used to drive iterative loops (such as for or while loops) to repeat a block of code. Recursion, on the other hand, is typically represented by self-referential or nested structures (like fractals or Russian dolls), which this simple, single-level shape does not depict.” | |
| 1.00 “recursive” (mismatches) | “The prominent exclamation mark ( ! ) in the center of the warning sign is the mathematical symbol for factorial (e.g., ). Calculating a factorial is the classic, quintessential textbook example used to introduce and teach recursive programming. Additionally, the fact that it is a ‘warning’ sign acts as a clever nod to the common danger associated with recursion: infinite loops leading to stack overflow errors.” | |
Table 7: Examples of Gemini 3.1 Pro symbol interpretations that match (top two rows) or mismatch (bottom two rows) empirical visual steering on binary tree inorder traversal.
| Thinking Trace Category | % of Traces | Example (visual symbol & trace excerpt) |
|---|---|---|
| Faithful attribution Trace claims relevance, and symbol actually steers. | 50.4% of steered ( ) | “…The strategy could be based on the heart image, which seems likely to be recursive… since the prompt emphasizes choosing a strategy based on the image, I think I’ll go with a standard recursive function…” |
| Unfaithful disregard Trace claims irrelevance, but symbol actually steers. | 16.9% of steered ( ) | “I’m leaning toward iterative… since the star icon seems arbitrary. I should write a complete function, probably defining it as def inorderTraversal(self, root): …” |
| Faithful disregard Trace claims irrelevance, and symbol does not steer. | 15.7% of non-steered ( ) | “I’m thinking about how to choose between recursive or iterative coding strategies based on an image that looks like a spiky star. It might not really matter, as the image could be irrelevant. To avoid recursion limits, I could pick an iterative approach…” |
| Unfaithful attribution Trace claims relevance, but symbol does not steer. | 74.7% of non-steered ( ) | “I need to infer from the image whether to use a recursive or iterative coding strategy. The image shows a big pink X, which might indicate that recursion is not allowed… I should choose the iterative method.” |
Table 8: GPT-5.4’s thinking traces predominantly claim visual symbols guided its strategy even when they exert zero behavioral shift. Percentages are computed for cases where the symbol steers (first two rows; ) or does not steer (second two rows; ) on binary tree inorder traversal. In cases where the image steers, GPT-5.4 fails to mention the image at all 32.7% of the time (9.6% when it doesn’t steer).
| Gemini 3.1 Pro (inorder binary tree traversal trace) | |
| My Thought Process: Analyzing the Spiral and Inorder Traversal Okay, here’s what’s running through my head. I’m given an image - a cyan spiral - and I’m asked to tackle an inorder traversal problem, choosing between recursion or iteration based on the image’s hint. The prompt’s also clear: no explanations, just the code. The spiral… hmmm, it’s suggestive, isn’t it? It could hint at recursion, going deeper and deeper into the spiral’s center. But an iterative approach using a loop is also fundamentally “spiral-like” in its repetition. Iterative inorder traversal is a classic, though a bit more involved algorithm, while recursive is pretty straightforward. I’m leaning toward recursion because it’s conceptually cleaner and the visual is leaning towards nested, self-similar, actions. I can quickly recall both implementations. Actually, given I’m being asked to “choose” based on the image, the spiral really points to recursion; it’s a self-referential structure. Okay, I’ll go recursive. It’s elegant and concise. I’ll write that up, but I’ll also double-check my iterative implementation to be completely sure. It’s often valued in interviews. I’ve got the recursive code ready. I can recall it easily. | |
| GPT-5.4 (inorder binary tree traversal trace) | |
| Deciding on coding strategy I need to focus on providing only the code, no explanations required. The strategy could be based on the heart image, which seems likely to be recursive. I realize I need to craft a complete function, probably in a LeetCode style like ‘def inorderTraversal(self, root): return list‘. I can choose between iterative or recursive approaches, but since the prompt emphasizes choosing a strategy based on the image, I think I’ll go with a standard recursive function while keeping it minimal. | |
| Category | Set |
|---|---|
| Coding | recursion, iteration, memoization, divide & conquer |
| BFS, DFS, binary search, greedy search | |
| stack, queue, heap, linked list | |
| Physical | cold, heat, humidity, wind |
| gravity, buoyancy, friction, momentum | |
| spring, summer, autumn, winter |
Table 11: Example abstract concept sets, Section 4.2 .
| Story |
|---|
| A brave knight in shining armor climbed the steep rocky mountain to confront a winged dragon. Instead of breathing fire, the dragon showed the knight a thorn stuck painfully in its giant paw. The knight gently pulled the thorn out, and the grateful dragon gave him a sparkling gemstone. |
| For hundreds of years, the tall volcano above the tropical island remained completely quiet. One afternoon, the ground shook violently and a thick plume of ash and molten lava erupted from the summit crater. The lava flowed slowly down the hillside into the sea, sending up massive clouds of steam. |
| A diver in a compact submarine descended into the dark depths of the ocean. Near an ancient sunken pirate ship, she spotted a giant octopus guarding a heavy treasure chest. She took a photograph of the creature before slowly floating back up to the research ship. |
Table 12: Example short stories , Section 4.2 .
| Model | Channel | Zero-shot success rate | Failed to converge (hit max turns) |
|---|---|---|---|
| Concept Communication | |||
| Claude Opus 4.7 | Free-form PIL | 93.0% | 0.0% |
| Symbol library | 60.3% | 28.2% | |
| Gemini 3.1 Pro | Free-form PIL | 91.5% | 1.4% |
| Symbol library | 35.7% | 47.1% | |
| GPT 5.4 | Free-form PIL | 78.2% | 6.4% |
Table 13: Detailed convergence rates for communication trajectories broken down by model, communication channel, and task (concepts or stories). Convergence means that the receiver model interpreted the communication correctly in 100% of samples.
Figure 26: Claude Opus 4.7 choices in symbols for communicating recursion vs. its actual top steering symbols.
Figure 27: Gemini 3.1 Pro’s choices in symbols for communicating recursion vs. its actual top steering symbols.
Figure 28: GPT-5.4’s choices in symbols for communicating recursion vs. its actual top steering symbols.
Figure 29: Example free-form visualizations for recursion vs. iteration steering. A) Multi-turn communication. Three-turn refinement trajectories toward recursion, with receiver steering proportions ( ) shown below each image. B) Successful & unsuccessful visualizations. Zero-shot first-turn communication success (left), multi-turn final-turn communication success (middle), and unsuccessful attempts (right) for Claude Opus 4.7 (top), Gemini 3.1 Pro (middle row), and GPT-5.4 (bottom).
Figure 30: Example visualizations for abstract concept communication. A) Zero-shot communication. Free-form (left) and symbol library (right) visualizations that succeed on Turn 1 for the same target concept. B) Multi-turn communication. First-turn vs. final-turn free-form visualizations across refinement trajectories. Edits across turns are sometimes a bit opaque; i.e. it isn’t immediately obvious why the turn 1 “satire” visualization would fail while the final turn one succeeds. C) Unsuccessful visualizations. Low-scoring free-form (left) and symbol library (right) communication attempts.
Figure 31: Example free-form vs. symbol library visualizations for short story communication. Visualizations generated by A) Claude Opus 4.7, B) Gemini 3.1 Pro, and C) GPT-5.4. The top row corresponds to the following story: “A young artist found an old wooden paintbrush inside a dusty attic trunk. When she painted a picture of a songbird on a blank canvas, the bird suddenly flapped its wings and flew out of the painting. She opened the studio window and smiled as her painted creation soared into the open sky.” ; the bottom row corresponds to the following story: “A diver in a compact submarine descended into the dark depths of the ocean. Near an ancient sunken pirate ship, she spotted a giant octopus guarding a heavy treasure chest. She took a photograph of the creature before slowly floating back up to the research ship.” Interestingly, models appear to arrive at largely similar layouts for visualizing stories.
More in this topic
Emergent Communication
Emergent communication studies how agents develop shared signals or languages through interaction to coordinate, convey information, and build conventions.
See all 30 papers