MNN-Pocket-0.3B Technical Report
Organizations: Taobao and Tmall Group of Alibaba
Abstract
This technical report presents MNN-Pocket-0.3B, a compact 300M-parameter dense language model designed for efficient on-device deployment and supporting context lengths of up to 32K tokens. With an INT4 deployment footprint of only 173 MB and decoding throughput of up to 466 tokens/s on iPad Pro M5 (Metal), it can be seamlessly integrated into applications on edge devices such as smartphones and smart glasses. MNN-Pocket-0.3B is developed through the co-design of its architecture, data, training pipeline, and inference system. Hardware-aware architecture selection and dependency-aware vocabulary pruning balance model capability with on-device memory and compute constraints. A four-stage pre-training pipeline covering approximately 4.1 trillion high-quality Chinese and English tokens equips the model with broad knowledge and reasoning capabilities. This is followed by a three-stage post-training pipeline comprising instruction fine-tuning, multi-teacher on-policy distillation, and quantization-aware training. Finally, hardware-specific optimizations in the MNN inference engine enable fast, memory-efficient inference and long-context processing. Extensive evaluations demonstrate that MNN-Pocket-0.3B achieves frontier-level performance among models of comparable size, with particularly strong results in Chinese and English knowledge and reasoning, mathematics, coding, and instruction following. It consistently outperforms similarly sized open-source models and even surpasses several 0.5B-parameter models on instruction-following benchmarks. We release MNN-Pocket-0.3B with open weights and ready-to-use MNN deployment packages. We hope it will provide a practical foundation for capable, responsive, and memory-efficient edge applications while advancing research and development in on-device language models.
Figures & tables
| Hyperparameter | Stable Stage 1 | Stable Stage 2 | Short Decay | Long Decay |
|---|---|---|---|---|
| Context length | 4,096 | 4,096 | 4,096 | 32,768 |
| Total tokens | 3T | 800B | 200B | 100B |
| Global batch size | 1024 | 1024 | 1024 | 128 |
| Tokens per step | 4.2M | 4.2M | 4.2M | 4.2M |
| Peak learning rate | ||||
| LR schedule | Constant | Constant | Cosine | Cosine |
| Category | Samples | Tokens | Percent |
| Family | |||
| Multilingual Creation & Open-Ended Generation | 607K | 289M | |
| Mathematics & Quantitative Reasoning | 722K | 393M | |
| Programming & Algorithmic Coding | 801K | 433M | |
| Application Development & General QA | 247K | 148M | |
| General Chinese-Language Tasks | 450K | 337M | |
| Metric | MNN-Pocket 0.3B | Gemma-3 270M | ERNIE-4.5 0.3B | LFM2.5 350M | SmolLM2 360M | MiniCPM4 0.5B | Hunyuan 0.5B | Qwen3 0.6B |
|---|---|---|---|---|---|---|---|---|
| MNN inference file size (MB) | 173 | 162 | 207 | 202 | 206 | 247 | 307 | 340 |
| Peak Memory (MB) | 228 | 272 | 287 | 258 | 257 | 305 | 449 | 427 |
| Device | Backend | Stage | MNN-Pocket 0.3B | Hunyuan 0.5B | Qwen3 0.6B | MiniCPM4 0.5B | SmolLM2 360M | ERNIE-4.5 0.3B | LFM2.5 350M | Gemma-3 270M |
|---|---|---|---|---|---|---|---|---|---|---|
| iPad Pro (M5) | Metal | Prefill | 8721 | 5330 | 6027 | 6973 | 7067 | 7218 | 8026 | 9519 |
| Decode | 466 | 262 | 286 | 357 | 397 | 395 | 504 | 455 | ||
| CPU | Prefill | 2228 | 1474 | 1396 | 1818 | 1824 | 2185 | 2234 | 3793 | |
| Decode | 231 | 149 | 135 | 197 | 187 | 224 | 262 | 290 | ||
| Android (8 Gen 5) | OpenCL | Prefill | 3403 | 1757 | 1627 | 2368 | 2291 | 2532 | 3070 | 4209 |
| Decode | 138 | 88 | 93 | 117 | 127 | 119 | 175 | 118 |
| Draft Precision | Training Source | Verification Model | Math | Code | Chat | Avg. | |||
|---|---|---|---|---|---|---|---|---|---|
| MATH-500 | AIME25 | MBPP | HumanEval | MT-Bench | A-Hard | ||||
| BF16 | MNN | MNN | 5.34 | 4.59 | 5.55 | 5.83 | 3.55 | 3.18 | 4.67 |
| BF16 | MNN | MNN-INT4 | 5.30 | 4.49 | 5.57 | 5.75 | 3.62 | 3.16 | 4.65 |
| BF16 | MNN-INT4 | MNN-INT4 | 5.34 | 4.55 | 5.64 | 5.82 | 3.70 | 3.18 | 4.70 |
| INT4 | MNN | MNN-INT4 | 5.16 | 4.37 | 5.48 | 5.65 | 3.50 | 3.06 | 4.54 |
| INT4 | MNN-INT4 | MNN-INT4 | 5.22 | 4.47 | 5.55 | 5.76 | 3.65 | 3.10 | 4.63 |
| Device | Workload | AR tok/s | Mean AL | Spec. tok/s | Speedup |
|---|---|---|---|---|---|
| iPad Pro (M5) | Math | 435 | 4.49 | 709 | 1.63 |
| Code | 5.73 | 895 | 2.06 | ||
| Chat | 3.51 | 547 | 1.26 | ||
| Avg. | 4.57 | 717 | 1.65 | ||
| Android (8 Gen 5) | Math | 135 | 4.48 | 159 | 1.17 |
| Code | 5.72 | 186 | 1.38 |
| 300M Models | 500M Models | ||||||||
| Benchmark | MNN-Pocket 0.3B | MNN-Pocket 0.3B-INT4 | Gemma-3 270M | ERNIE-4.5 0.3B | LFM2.5 350M | SmolLM2 360M | Hunyuan 0.5B | MiniCPM4 0.5B | Qwen3 0.6B |
| Average | 50.00 | 49.59 | 6.42 | 31.99 | 31.28 | 18.43 | 31.22 | 46.81 | 44.28 |
| Knowledge | |||||||||
| C-Eval | 38.86 | 40.27 | 6.84 | 32.17 | 32.10 | 23.77 | 35.29 | 67.98 | 44.06 |
| CMMLU | 40.11 | 39.65 | 4.94 | 33.91 | 31.65 | 24.87 | 35.69 | 66.04 | 44.50 |
| MMLU-Redux | 44.00 | 43.40 | 13.58 | 32.68 | 41.01 | 24.17 | 36.29 | 52.85 | 51.63 |
| 300M Models | 500M Models | ||||||
| Benchmark | MNN-Pocket 0.3B-Base | Gemma-3 270M Base | ERNIE-4.5 0.3B Base | LFM2.5 350M Base | SmolLM2 360M Base | Hunyuan 0.5B-Pretrain | Qwen3 0.6B Base |
| Average | 33.58 | 16.94 | 34.00 | 29.86 | 22.25 | 42.60 | 44.86 |
| General Tasks | |||||||
| C-Eval | 36.55 | 23.25 | 40.56 | 35.66 | 22.07 | 57.95 | 54.68 |
| CMMLU | 38.13 | 25.26 | 39.54 | 36.78 | 25.30 | 54.54 | 53.29 |
| MMLU-Redux | 47.07 | 24.47 | 46.59 | 48.44 | 30.19 | 59.34 | 57.45 |
| System | 你是一个聪明的AI电商助手。 |
|---|---|
| Input | 请在商品评论中抽取: 1.评论中被评价的商品属性(AspectTerms),并为其归入对应类目(Categories)。可选类目为:[’包装’, ’成分’, ’尺寸’, ’服务’, ’功效’, ’价格’, ’气味’, ’使用体验’, ’物流’, ’新鲜度’, ’真伪’, ’整体’, ’其他’]; 2.消费者用以评价的表述(OpinionTerms),并判定其情感倾向(Polarities)。可选倾向为:[’正面’, ’负面’, ’中性’]。 若评论未明确提及被评价的属性,AspectTerms 置为空字符串 ”。 供参考的评论示例:’很好,遮暇功能差一些’ 输出:[{’AspectTerms’: ”, ’OpinionTerms’: ’很好’, ’Categories’: ’整体’, ’Polarities’: ’正面’}, {’AspectTerms’: ’遮暇功能’, ’OpinionTerms’: ’差一些’, ’Categories’: ’功效’, ’Polarities’: ’负面’}] 下面是需要处理的评论:包装太随便了,感觉很不好 输出: |
| Output | [{’AspectTerms’: ’包装’, ’OpinionTerms’: ’太随便了’, ’Categories’: ’包装’, ’Polarities’: ’负面’}, {’AspectTerms’: ”, ’OpinionTerms’: ’很不好’, ’Categories’: ’整体’, ’Polarities’: ’负面’}] |
| Language backbone | Params | F1 (%) |
|---|---|---|
| MNN-Pocket-0.3B | 0.3B | 80.15 |
| Qwen3-1.7B | 1.7B | 78.59 |
| Qwen3-0.6B | 0.6B | 77.47 |
| Hunyuan-0.5B | 0.5B | 77.27 |
| ERNIE-4.5-0.3B | 0.3B | 76.73 |
| LFM2.5-350M | 0.35B | 76.19 |
| Metric | iPhone | iPad | Android |
|---|---|---|---|
| Device / SoC | 13 Pro / A15 | Pro / M5 | Canoe / 8Gen5 |
| Model storage (MB) | 2,018 | 2,018 | 2,018 |
| Peak memory (MB) | 1,591 | 1,858 | 1,704 |
| ASR RTF | 0.073 | 0.047 | 0.045 |
| LLM decoding speed (token/s) | 130.2 | 324.9 | 231.7 |
| TTS RTF | 0.653 | 0.366 | 0.651 |
| Language backbone | Common | WildVoice | SD-QA | OBQA | MMSU | BBH | IFEval | AdvBench | AlpacaEval | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| MNN-Pocket-0.3B | 40.23 | 36.72 | 5.75 | 32.75 | 27.91 | 58.90 | 64.11 | 97.69 | 44.91 | 45.44 |
| Gemma3-270M | 37.17 | 31.54 | 6.43 | 25.93 | 25.02 | 49.40 | 18.32 | 93.27 | 38.56 | 36.18 |
| ERNIE-4.5-0.3B | 37.33 | 32.50 | 4.87 | 28.57 | 25.63 | 50.60 | 33.98 | 39.42 | 38.46 | 32.37 |
| LFM2.5-350M | 47.17 | 39.96 | 11.06 | 22.86 | 25.76 | 46.30 | 57.17 | 88.65 | 50.47 | 43.27 |
| SmolLM2-360M | 37.93 | 32.26 | 15.78 | 21.98 | 24.59 | 46.40 | 32.59 | 23.46 | 41.20 | 30.69 |
| Hunyuan-0.5B | 22.67 | 22.19 | 2.02 | 29.67 | 28.24 | 48.70 | 18.06 | 23.85 | 23.73 | 24.35 |
Small Language Models
Small language models are compact neural language systems designed to deliver useful text and reasoning capabilities with limited compute, memory, and energy.
See all 296 papers