We introduce Evo, a duality latent trajectory model that bridges autoregressive (AR) and diffusion-based language generation within a continuous evolutionary generative framework. Rather than treating AR decoding and diffusion generation as separate paradigms, Evo reconceptualizes text generation as a latent flow: each token is associated with a vector-valued embedding that evolves over a progression variable ti∈[0,1], indicating its semantic maturity. Low ti values correspond to confident AR-like refinement, while high values invoke diffusion-style planning, allowing the model to adaptively balance AR and diffusion based on uncertainty. Theoretically, we show that both AR and diffusion models emerge as discretizations of a shared probability flow, and we derive Evo's training objective from a unified variational ELBO. The model is implemented as a time-conditioned Transformer governed by a shared vector field, trained end-to-end to jointly infer latent codes and their progression times. During decoding, Evo performs efficient, semantics-aware refinement, achieving high-quality outputs without sacrificing speed. Empirically, Evo 8B achieves state-of-the-art or highly competitive results on 15 diverse benchmarks, including reasoning (GSM8K, ARC-C), code generation (HumanEval, MBPP), and general language understanding, while maintaining fast inference speed. Our results demonstrate that Evo delivers a new paradigm for LLM design with strong generation quality, robust symbolic reasoning, and decoding efficiency.
Figures & tables
Figure 1: Evo generates text by evolving a diffusion-based semantic scaffold into token-level autoregressive realizations along a shared latent trajectory, allowing different tokens to reach semantic maturity at different times. Unlike prior autoregressive, diffusion, and AR+diffusion models, which either generate strictly left-to-right or apply diffusion in a fixed autoregressive order, Evo adaptively balances diffusion-style global planning and autoregressive next-token prediction by semantic granularity, selectively refining mature tokens while continuing to finalize the remaining scaffold within the same sentence.
Model
Evo 8B
BD3-LM 7B
ARD 7B
LLaDA 8B
MDLM 7B
LLaMA3 8B
Qwen2.5 7B
E2E Latency (s)
8.6
14.2
32.5
21.8
18.9
7.4
8.1
Inference Speed (tokens/s)
52
28
12
16
22
58
46
General Tasks
MMLU
76.8 (5)
70.0 (5)
46.2 (5)
65.6 (5)
64.0 (5)
65.1 (5)
74.5 (5)
BBH
68.4 (3)
61.8 (3)
36.9 (3)
49.6 (3)
55.8 (3)
57.9 (3)
70.1 (3)
ARC-C
65.6 (25)
60.2 (25)
46.7 (0)
48.2 (0)
59.6 (25)
52.8 (0)
64.0 (25)
Table 1: Benchmark Results of Pre-trained LLMs. Evo 8B , BD3-LM 7B , ARD 7B (AR + Diffusion hybrid), LLaDA 8B , MDLM 7B (Diffusion-based), LLaMA3 8B , and Qwen2.5 7B (AR-based) are evaluated under the same protocol. The numbers in parentheses represent the number of shots.
Model
Evo 8B
BD3-LM 7B
ARD 7B
LLaDA 8B
MDLM 9B
LLaMA3 8B
Qwen2.5 7B
E2E Latency (s)
8.6
14.2
32.5
21.8
18.9
7.4
8.1
Inference Speed (tokens/s)
52
28
12
16
22
58
46
Post-training
SFT
SFT+RL
SFT+RL
SFT
SFT+RL
SFT+RL
SFT+RL
General Tasks
MMLU
78.6 (5)
75.1 (5)
48.9 (5)
65.5 (5)
70.2 (5)
68.4 (5)
73.8 (5)
MMLU-pro
57.2 (0)
48.1 (5)
12.8 (0)
37.0 (0)
52.1 (5)
41.9 (0)
56.3 (5)
Table 2: Benchmark Results of Post-trained LLMs. Evo 8B and LLaDA 8B post-trained via SFT. The numbers in parentheses represent the number of shots.
Figure 2: The scaling capability of Evo on MMLU.
Ablation Variant
PPL ↓
Task Acc ↑
w/o Time Conditioning
9.6
69.1
Fixed ti=0 (AR-only)
10.2
69.6
Fixed ti=1 (Diff-only)
8.6
77.0
Fixed ti=0.5 (Uniform)
8.8
77.9
Heuristic ti (entropy)
8.4
78.2
Evo (full)
8.1
81.6
Figure 3: Ablation study of Evo on text modeling and reasoning tasks. PPL: perplexity on validation set; Task Acc: average task accuracy (MMLU and GSM8K). Results highlight the importance of time conditioning, learned token-wise ti , and adaptive refinement.
Ablation Variant
PPL ↓
Task Acc ↑
Two-Module (AR + Diff)
8.6
77.9
No Shared Embeddings
8.4
78.3
No Pretrained Init
9.1
76.2
Frozen AR Backbone
8.7
80.4
Evo (full)
8.1
81.6
Figure 4: Ablation on architecture and pretraining. Evo benefits from a unified model, embedding sharing, and pretrained initialization. Task accuracy is averaged across MMLU and GSM8K.
User: Lily can run 12 kilometers per hour for 4 hours. After that, she runs 6 kilometers per hour for another 3 hours. How many kilometers does she run in total?
Evo: First, at 12 km/h for 4 hours, Lily covers 48 kilometers. Then, at 6 km/h for 3 hours, she covers 18 kilometers. Adding them gives 66 kilometers in total.
Table 3: Mathematical reasoning with adaptive refinement.
User: Could you share the first two lines of the poem “The Road Not Taken”?
Evo: “Two roads diverged in a yellow wood, And sorry I could not travel both.”
User: Translate into German.
Evo: “Zwei Wege trennten sich im gelben Wald, und leider konnte ich nicht beide gehen.”
User: Now write a short poem about life choices, each sentence starting with “C”.
Evo: Choices shape the path we walk. Courage guides us through the unknown. Curiosity opens hidden doors. Commitment carries us forward. Clarity shows the way home.
Table 4: Multi-turn dialogue with translation and constrained generation.
User: Write a short science-fiction story about a time-traveling botanist.
Evo: Dr. Amina travelled to the year 3050, carrying seeds from the past. She planted them in Martian soil, curious to see what might grow. Under the twin moons, the seedlings shimmered with an unearthly glow. The Martian settlers gathered to witness the strange blossoms. In that moment, Amina realized she had brought not just plants, but hope, across centuries.
Table 5: Step-adaptive story generation. Italics indicate refined sentences.
User: A tank is filled at 15 liters per minute for 8 minutes, then at 10 liters per minute for 12 minutes. How many liters are in the tank?
Evo: First stage: 15×8=120 liters. Second stage: 10×12=120 liters. Adding gives 240 liters in total.
Table 6: Mathematical reasoning with token refinement.
User: What is the proverb “A stitch in time saves nine” in German?
Evo: German: “Wer rechtzeitig handelt, erspart sich viel Mühe.”
User: Write a haiku in English using the word “time”.
Evo: Time flows like a stream, Leaves drift in a quiet dream, Dawn breaks, gold and clean.
Table 7: Multi-turn multilingual dialogue.
User: Write a Python function to check if a number is prime.
Evo: def is_prime(n): if n < 2: return False for i in range(2, int(n**0.5)+1): if n % i == 0: return False return True
Table 8: Python function generation.
User: Write a short story about a lost map in the desert.
Evo: The wind buried the caravan’s trail in minutes. Karim unfolded the old map, only to find half the ink faded. The sun blazed overhead, turning sand into a sea of gold. He followed the faint lines until they ended at a lone acacia tree. Beneath its roots, a small tin box held the real map, untouched by time.
Table 9: Short story with refinement. Italics indicate refined sentences.
User: Given the line “And sorry I could not travel both”, write the preceding line from Robert Frost’s “The Road Not Taken”.