Organizations: ByteDance · Shenyang Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Science · Westlake University · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences)
Solving complex geometric problems inherently requires interleaved reasoning: a tight alternation between constructing diagrams and performing logical deductions. Although recent Multimodal Large Language Models (MLLMs) have demonstrated strong capabilities in visual generation and plotting, we identify a counter-intuitive and underexplored phenomenon. Naively applying Supervised Fine-Tuning (SFT) on interleaved plot-solution data leads to a substantial degradation in reasoning performance compared to text-only baselines. We argue that this failure stems from a fundamental limitation of SFT, which primarily induces distributional alignment: the model learns to reproduce the surface format of interleaved plotting but fails to internalize the causal dependency between the generated plot and reasoning steps. To overcome this limitation, we propose Faire (Functional alignment for interleaved reasoning), a reinforcement learning framework that enforces three casual constraints to move beyond superficial imitation toward functional alignment. Extensive experiments show that Faire induces a qualitative shift in model behavior in which the plotting is effectively internalized, yielding competitive performance on challenging geometric reasoning benchmarks.
Figures & tables
Figure 1: Illustration of challenges of geometric interleaved reasoning on several sub-tasks between SFT-Text only and SFT-Interleaved, which can only be tackled by RL post-training.
Figure 2: Illustration of Faire framework. The model generates a reasoning trace and GeoGebra code from a geometry problem. As for reward designs, a gated reward enforces answer correctness Cans and code executability Cexe , then aggregates perceptual Cperc , semantic Csem , and formal verification signals Cgeo .
Figure 3: The data construction pipeline employing a tri-perspective verification mechanism—Visual Alignment, Semantic Consistency, and Geometric Assertion—to curate rigorous interleaved geometric reasoning samples.
Figure 4: Stage and category distribution. Inner ring shows educational stages; outer ring shows category shares within each stage.
Figure 5: Distribution of top-100 entropy-increased tokens after RL. Tokens are grouped by semantic function.
Figure 6: RL vs. SFT Text–Diagram Alignment in a Geometry problem, where RL model learns when to draw during reasoning.
Figure 7: RL vs. SFT in Function Graph. The SFT model draws incorrect quadrant distributions with text rationales, while the RL model draws an accurate image to infer the correct answer.
Table 4: Reward ablation (leave-one-out). Executability-only RL provides limited gains; Faire requires all verifiers to achieve functional alignment.
Paradigms
SFT
RL
Acc (%)
Draw
Acc (%)
Draw
Text-only
68.13
-
71.21
-
Interleaved
62.48
35.23
74.83
45.37
Table 5: Ablation on Training Paradigms. Comparing the interplay between drawing and solving. Note that while drawing initially hurts SFT accuracy, it boosts performance post-RL.
Figure 8: Verifier score vs. human rating ( r=0.9578 ).
Figure 9: Prompt used for the VLM-as-Verifier in Faire. The verifier performs pixel-level, mathematical, and logical consistency checks to determine whether a rendered diagram faithfully supports the intended reasoning step.
Figure 10: Prompt used for parser-based semantic verification, which evaluates whether a textual geometric description faithfully instantiates the intended reasoning and relations.
Figure 11: Prompt used for the GGB-as-Verifier in Faire. The verifier synthesizes executable GeoGebra assertions from visual relations and evaluates them deterministically to enforce formal geometric correctness.
Model
Stage-wise scores
Image similarity
Overall
Planning (VLM-T) ↑
Mid (VLM-I-Mid) ↑
Final (VLM-I-Res) ↑
LPIPS ×10−2↓
PSNR ↑
SSIM ×10−2↑
VLM-I ↑
End-to-end UMMs
Qwen-Image [ 59 ]
—
—
22.75
56.39
58.23
48.06
22.75
Janus [ 72 ]
33.85
21.69
19.76
57.74
57.76
60.97
20.73
Nano Banana [ 55 ]
58.54
44.83
22.81
51.85
64.53
59.51
33.82
GPT-image-1 [ 53 ]
—
—
39.11
50.67
16.95
64.37
39.11
Table 7: Results on GGBench. Stage-wise interleaved evaluation (higher is better) and image similarity (lower LPIPS is better).
Figure 12: Category-wise verification scores.
Figure 13: Interleaved function visualization (line + reciprocal curve). Faire constructs both graphs in the same coordinate system and marks the key points needed by the reasoning, so the option decision is supported by a checkable geometric state.
Figure 14: Analytic-geometry construction with projections. The diagram materializes auxiliary points and projection relations, turning abstract constraints into explicit incidences that the subsequent deduction can rely on.
Figure 15: Plane-geometry construction via auxiliary intersections. The construction exposes hidden structure (intersections and induced sub-triangles), providing a concrete state for validating proportional or area arguments.
Figure 16: Reasoning–diagram consistency comparison on a judgement problem. Faire keeps deductions and construction aligned to the same geometric state; baselines may produce answer-correct text with diagram drift (orientation/clutter) or fail in both reasoning and construction.
Figure 17: Function graph selection: grounded structure versus ungrounded calculation. Faire anchors the decision in a state-faithful plot under the given sign constraints, while large generalist models may compute extensively yet still select an option inconsistent with the intended configuration.
College of Computer Science and Artificial Intelligence, Fudan University · School of ECE, Peking University · School of Software and Microelectronics, Peking University +1