Unlocking the Latent Canvas: Eliciting and Benchmarking Symbolic Visual Expression in LLMs

Yiren Zheng; Shibo Li; Jiaming Liu; Haofan Wang; Yiren Song

cs.CVMar 15, 2026

Unlocking the Latent Canvas: Eliciting and Benchmarking Symbolic Visual Expression in LLMs

Authors: Yiren Zheng, Shibo Li, Jiaming Liu, Haofan Wang, Yiren Song

Organizations: Tsinghua University · Fudan University · Alibaba Group · Lovart AI ·

Abstract

Current multimodal approaches predominantly treat visual generation as an external process, relying on pixel rendering or code execution, thereby overlooking the native visual representation capabilities latent within Large Language Models (LLMs). In this work, we unlock this potential through ASCII art, a compact, efficient, and text-native visual format. We introduce SVE-ASCII, a unified framework designed to elicit and benchmark Symbolic Visual Expression directly within the pure text space. To address the scarcity of systematic resources, we construct ASCIIArt-7K, a high-quality dataset synthesized via a novel "Seed-and-Evolve" pipeline that augments human-curated anchors through in-context stylistic editing. We further implement a unified instruction-tuning strategy that jointly optimizes for both Generation (Text-to-ASCII) and Understanding (ASCII-to-Text). Crucially, our experiments reveal a critical phenomenon regarding task duality: while it is established that perception aids generation, we provide compelling evidence that generative training significantly enhances visual comprehension. This confirms a mutually reinforcing cycle in symbolic visual processing, a relationship previously hypothesized but rarely empirically demonstrated in the visual domain. We release our dataset, the ASCIIArt-Bench benchmark, and the SVE-ASCII model, establishing a robust baseline for native text-based visual intelligence.

Figures & tables

Figure 2: Overview of SVE-ASCII. First, we construct the ASCIIArt-7K using a scalable synthesis pipeline. Subsequently, we fine-tune Qwen2.5-7B-Instruct via Understanding-Generation Joint Training. By switching supervision signals between tasks, our model achieves efficient bidirectional capabilities in both understanding and generating ASCII art.

Figure 3: Generation from Scratch vs. Imitation. The quality of the generated ASCII art cup improves significantly when an imitation example is provided in the prompt.

Panel of Table 1 — Table 1: Style Rules Decomposition. Taking a dog ASCII art as an example, this stage extracts the specific set of characters used and the topological features describing the dog’s shape.

Figure 4: Impact of Topology on Style Transfer. We synthesize variants for different seeds. Visually insensitive subjects (e.g., trucks) demonstrate robust style transfer across diverse shapes. Conversely, visually sensitive subjects (e.g., rabbits) struggle with large morphological changes, often failing to transform into distinct animals such as foxes or cats.

Model	Recall Subset (In-Distribution)						Generalization Subset (Out-of-Distribution)
Model	SA	IF	SC	SL	CE	Comp.	SA	IF †	SC	SL	CE	Comp.
GPT-5.2	0.558	0.544	0.785	0.690	0.825	0.633	0.604	0.568	0.756	0.713	0.801	0.650
Kimi K2	0.383	0.420	0.812	0.607	0.814	0.537	0.449	0.392	0.808	0.596	0.789	0.539
Doubao-Seed-1.6	0.259	0.265	0.694	0.450	0.838	0.413	0.332	0.347	0.698	0.476	0.845	0.465
Qwen-Plus	0.346	0.351	0.804	0.577	0.836	0.500	0.398	0.371	0.782	0.595	0.834	0.519
Gemini 3.0 Flash	0.474	0.453	0.544	0.585	0.754	0.522	0.490	0.403	0.535	0.590	0.723	0.505

Table 2: Main results on our ASCII Art Benchmark comparing Proprietary (Closed-Weight) and Open-Weight models. Consistent with the benchmark definition, we evaluate models on the Recall Subset (In-Distribution) and the Generalization Subset (Out-of-Distribution). The metrics reported are Semantic Alignment (SA), Instruction Faithfulness (IF), Structural Coherence (SC), Spatial Logic (SL), Character Efficiency (CE), and the Weighted Composite Score (Comp.). Bold indicates the best performance, and blue text indicates the second best. Inst. denotes the instruction-tuned version of the model, and Qwen2.5-7B-Inst. refers to the base model in this benchmark.

Model	Task I (Direct)			Task II (Select)
Model	Seen	Unseen	Avg.	Seen	Unseen	Avg.
GPT-5.2	11.0	17.0	14.0	18.0	17.0	17.5
Kimi-K2	9.0	6.0	7.5	10.0	5.0	7.5
Doubao-Seed-1.6	2.0	4.0	3.0	5.0	4.0	4.5
Qwen-Plus	9.0	10.0	9.5	8.0	6.0	7.0
Claude-Sonnet-4.5	16.0	14.0	15.0	19.0	21.0	20.0

Table 3: Results on the ASCII Art Understanding Benchmark comparing Proprietary (Closed-Weight) and Open-Weight models. We evaluate models on two tasks: Task I: Direct Understanding and Task II: Category Selection. All metrics are reported as Accuracy (%), evaluating performance on Seen, Unseen, and Overall. Bold indicates the best performance, and blue text indicates the second best.

Figure 5: Qualitative comparison with SOTA methods on the Generation Task. We compare the proposed method with SOTA baselines on our evaluation benchmarks, presenting results from the Recall Subset on the left and the Generalization Subset on the right.

Metric / Task	Single-Task	SVE-ASCII	$\Delta$ (%)
Part I: Generation Task (Composite Score)
Set A: Semantic Align.	0.838	0.936	$\uparrow$ 11.7%
Set A: Instr. Faith.	0.841	0.946	$\uparrow$ 12.5%
Set A: Spatial Logic	0.833	0.926	$\uparrow$ 11.2%
Set A: Composite	0.841	0.929	$\uparrow$ 10.5%
Set B: Instr. Faith. †	0.549	0.619	$\uparrow$ 12.8%

Table 4: Effect of Understanding-Generation Joint Training. We compare SVE-ASCII with models trained in a single-task setting. Set A: Recall Subset. Set B: Generalization Subset. † IF (Instruction Faithfulness) is the critical metric for the Generalization Subset, measuring robustness to novel prompts.

\Delta

denotes the relative improvement percentage.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 6: More results of ours and baselines.

ASCIIBench: Evaluating Language-Model-Based Understanding of Visually-Oriented Text
Dec 2, 2025Kerry Luo, Michael Fu, Joshua Peguero +6VLM Evaluation LLM Evaluation
Algoverse AI Research
ASCII Art Turns LLMs into VLA Controllers
Jun 19, 2026Yitao Jiang, Roy Xing, Luyang Zhao +3Language Model-Based Control AI Control
Dept. of Computer Science, Dartmouth College, Hanover, NH, USA. · Dept. of Electrical and Computer Engineering, Clemson University, Clemson, SC, USA. · Dept. of Mechanical and Aerospace Engineering, University of Houston, Houston, TX, USA.
Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs
Aug 7, 2026Hongyu Luo, He Wang, Huihao Jing +6Creativity Assessment LLM Evaluation
The Hong Kong University of Science and Technology, Hong Kong SAR, China

Dec 2, 2025J/K move · Enter open · S save

ASCIIBench: Evaluating Language-Model-Based Understanding of Visually-Oriented Text

Kerry Luo, Michael Fu, Joshua Peguero, Husnain Malik+5

Algoverse AI Research

VLM Evaluation LLM Evaluation

Open paper AskPDF arXiv

Large language models (LLMs) have demonstrated several emergent behaviors with scale, including reasoning and fluency in long-form text generation. However, they continue to struggle with tasks requiring precise spatial and positional reasoning. ASCII art, a symbolic medium where characters encode structure and form, provides a unique probe of this limitation. We introduce ASCIIBench, a novel benchmark for evaluating both the generation and classification of ASCII-text images. ASCIIBench consists of a filtered dataset of 5,135 class-labeled ASCII images and is, to our knowledge, the first publicly available benchmark of its kind. Alongside the dataset, we release weights for a fine-tuned CLIP model adapted to capture ASCII structure, enabling the evaluation of LLM-generated ASCII art. Our analysis shows that cosine similarity over CLIP embeddings fails to separate most ASCII categories, yielding chance-level performance even for low-variance classes. In contrast, classes with high internal mean similarity exhibit clear discriminability, revealing that the bottleneck lies in representation rather than generational variance. These findings position ASCII art as a stress test for multimodal representations and motivate the development of new embedding methods or evaluation metrics tailored to symbolic visual modalities. All resources are available at https://github.com/ASCIIBench/ASCIIBench.

Abstract

Explore similar work

ASCIIBench: Evaluating Language-Model-Based Understanding of Visually-Oriented Text

ASCII Art Turns LLMs into VLA Controllers

Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs