Ruyi2 Technical Report

Huan Song; Shuyu Tian; Junyi Hao; Minxiu Xu; Hongjun An; Yiliang Song; Jiawei Shao; Xuelong Li

cs.CLFeb 26, 2026

Ruyi2 Technical Report

Authors: Huan Song, Shuyu Tian, Junyi Hao, Minxiu Xu, Hongjun An, Yiliang Song, Jiawei Shao, Xuelong Li

Organizations: Institute of Artificial Intelligence (TeleAI), China Telecom

Abstract

Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies. Building upon the AI Flow framework, we introduce Ruyi2 as an evolution of our adaptive model series designed for efficient variable-depth computation. While early-exit architectures offer a viable efficiency-performance balance, the Ruyi model and existing methods often struggle with optimization complexity and compatibility with large-scale distributed training. To bridge this gap, Ruyi2 introduces a stable "Familial Model" based on Megatron-LM. By using 3D parallel training, it achieves a 2-3 times speedup over Ruyi, while performing comparably to same-sized Qwen3 models. These results confirm that family-based parameter sharing is a highly effective strategy, establishing a new "Train Once, Deploy Many" paradigm and providing a key reference for balancing architectural efficiency with high-performance capabilities.

Figures & tables

Model	Layers	Hidden Size	Tie Embedding	Heads (Q / KV)	Context Length
Ruyi2–1.7B	3	5120	No	40 / 8	8K
Ruyi2–8B	22	5120	No	40 / 8	8K
Ruyi2–14B	40	5120	No	40 / 8	8K

Table 1: Model architecture details for Ruyi2.

Figure 1: The data engineering pipeline for constructing a high-quality training-ready corpus. The pipeline consists of four main stages: filtering, deduplication, decontamination, and data mixture, ensuring the high density and quality of the dataset.

Figure 2: Overview of the Staged Capability Improvement Pipeline

Figure 3: SFT Data Mixture Strategy: Evolution from Diverse Generalization to Focused Domain Specialization

Panel of Figure 5 — Figure 5: Input/output cosine similarity score of each token per layer for text "A fox sat on a box".

Figure 6: Training loss dynamics on the new model (0-3k steps). The Layer Cloning baseline (red) exhibits optimization inertia with a slower descent rate. In contrast, our Randomized Internal Initialization (purple) consistently achieves lower loss values , demonstrating superior convergence efficiency and a stronger capacity for learning novel representations without being constrained by the backbone’s existing patterns.

Model	MMLU	MMLU-P	CMMLU	BBH	ARC-c	Hella	IFEval	Human	Math	GSM8K	Avg.
Qwen3-1.7B	39.31	39.82	61.61	35.81	67.12	54.25	67.65	62.20	70.32	75.89	57.40
Qwen3-8B	48.64	55.63	78.84	55.11	82.03	77.32	81.89	87.20	82.30	85.37	73.43
Qwen3-14B	60.43	64.11	82.06	64.70	81.69	80.80	85.77	87.80	84.42	85.90	77.77
Ruyi2-1.7B	62.77	9.60	22.68	19.95	27.46	58.77	47.13	47.56	39.14	75.97	41.10
Ruyi2-8B	79.68	56.12	74.72	59.38	82.71	78.19	73.94	71.95	72.96	92.19	74.18
Ruyi2-14B	81.84	71.55	82.15	77.86	84.41	83.94	81.52	84.76	86.52	94.24	82.88

Table 2: Performance Comparison between Ruyi2 and Qwen3 Models across benchmarks.

LLM Inference Acceleration

LLM inference acceleration develops algorithms and systems that reduce language-model latency, memory use, and serving cost while preserving output quality.

See all 1,048 papers

AE-LLM: Adaptive Efficiency Optimization for Large Language Models
Mar 20, 2026Kaito Tanaka, Masato Ito, Yuji Nishimura +2LLM Inference Efficiency Energy-Efficient ML
SANNO University
Stepped MoE: Segment-Level Routing with Configurable Inference Complexity
Oct 5, 2026Arnav Kundu, Zhaoyang Xu, Bairu Hou +3On-Device Language Model Inference Dynamic Neural Networks
Apple
BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inference
Jun 8, 2026Yuhua Zhou, Shaoqi Yu, Shichao Weng +4LLM Pruning Cost-Aware Inference
College of Computer Science and Technology, Zhejiang University · Zhejiang Lab · Ant Group +1

Mar 20, 2026J/K move · Enter open · S save

AE-LLM: Adaptive Efficiency Optimization for Large Language Models

Kaito Tanaka, Masato Ito, Yuji Nishimura, Keisuke Matsuda+1

SANNO University

LLM Inference Efficiency Energy-Efficient ML

Open paper AskPDF arXiv

Large Language Models (LLMs) have achieved remarkable success across diverse applications, yet their deployment remains challenging due to substantial computational costs, memory requirements, and energy consumption. Recent empirical studies have demonstrated that no single efficiency technique is universally optimal; instead, the effectiveness of methods such as efficient attention mechanisms, mixture-of-experts (MoE), parameter-efficient fine-tuning, and quantization varies significantly depending on task characteristics, resource constraints, and model scales. Building upon these insights, we propose AE-LLM, a unified framework that automatically selects and combines optimal efficiency techniques tailored to specific deployment scenarios. Our approach introduces a multi-objective optimization framework that jointly considers accuracy, latency, memory footprint, and energy consumption, while accounting for hardware constraints and task requirements. We develop an efficient search algorithm that explores the combinatorial space of efficiency techniques across architecture, fine-tuning, and inference stages, identifying Pareto-optimal configurations. Extensive experiments across 15 models (0.5B-70B parameters) and 10 diverse tasks demonstrate that AE-LLM achieves an average of

2.8\times

improvement in efficiency metrics while maintaining competitive accuracy (within 1.2% of baseline), compared to static efficiency configurations. Furthermore, our framework generalizes effectively to vision-language models, achieving similar efficiency gains. Our contributions provide practitioners with an automated tool for navigating the complex trade-off landscape of LLM efficiency optimization.

Abstract

LLM Inference Acceleration

Explore similar work

AE-LLM: Adaptive Efficiency Optimization for Large Language Models

Stepped MoE: Segment-Level Routing with Configurable Inference Complexity

BUDDY: BUdget-Driven DYnamic Depth Routing for Adaptive Large Language Model Inference