HyperThink: Text-to-Parameter Hypernetworks for Efficient Reasoning

Long chain-of-thought~(CoT) reasoning improves the problem-solving capability of large reasoning models~(LRMs), but often incurs substantial inference cost and overthinking. Existing efficiency methods primarily rely on prompting, post-hoc compression, or length-regularized optimization; these approaches can be sensitive to model scale, depend on external estimators, or under-allocate reasoning to difficult instances. We propose Draft-Thinking, a lightweight adaptation framework that learns a model-adapted draft CoT: a compact, self-contained working derivation that directly supports the final answer. We identify a Cross-Scale Draft Mismatch: concise traces generated by a strong teacher can be too sparse for a smaller target model. To address this mismatch, Progressive Draft Adaptation first transfers a concise reasoning-form prior through supervised fine-tuning (SFT) and then applies two-stage reinforcement learning (RL) with progressively harder data and relaxed response budgets to calibrate the reasoning granularity to the target model. The method requires neither post-hoc trajectory compression nor an explicit length-penalty reward. At inference time, draft, adaptive, and step-by-step prompts provide efficiency-oriented, balanced, and detailed reasoning modes, respectively. The step-by-step mode largely preserves the original model's accuracy while using fewer tokens. With only 847 training problems and 120 RL optimization steps, Draft-Thinking substantially reduces reasoning cost across six mathematical and scientific reasoning benchmarks. On MATH500, its draft mode reduces the average response length from 5,668 to 986 tokens, an 82.6% reduction, while achieving 90.6% accuracy compared with 93.0% for the original long CoT model.

Extended chain-of-thought (CoT) traces improve LLM reasoning but incur substantial computational and memory costs. While existing CoT compression methods mitigate this by condensing thought steps into compact representations via memory tokens and retaining only these representations at inference time, the loss of fine-grained information makes subsequent steps more error-prone. To alleviate this, we propose HybridThinker, where in addition to preserved these representations, thought steps are also temporarily retained to provide fine-grained details. However, we observe that naively keeping thought steps accessible to subsequent steps during training lets the model bypass memory tokens by retrieving information directly from these steps, leaving the model's ability to compress and retrieve information through memory tokens insufficiently trained. We therefore introduce a hybrid training scheme, in which only some thought steps are directly accessible through attention to subsequent steps, while the other thought steps are masked, forcing the model to use memory tokens for compression and retrieval. Across 4 reasoning benchmarks, HybridThinker matches the uncompressed baseline, advancing the state of the art in CoT compression by 5.8 points on average accuracy with similar inference time. Ablation studies confirm that both temporary thought-step retention and the hybrid training scheme contribute to these gains.

Chain-of-thought (CoT) prompting improves reasoning in large language models (LLMs) by externalizing intermediate computation as discrete text tokens, but this textual interface also introduces redundancy and inference overhead. Latent reasoning offers a promising alternative by carrying part of the computation in continuous representations. However, existing methods typically predefine when latent computation is invoked and how it is allocated during decoding, leaving a key problem unresolved: when to invoke latent computation, what type of computation to perform, and how much budget to allocate. We propose Typed Latent Reasoning (Tyler), a typed and budget-aware framework for latent reasoning during autoregressive decoding. Tyler learns a policy that, at each decoding step, chooses between emitting a text token and switching to a latent computation module specialized for a particular reasoning function. Once invoked, an operator maps the current reasoning state into latent tokens that support global planning, local state updates, or reusable procedural abstraction. Across extensive experiments on three backbone LLMs, Tyler improves accuracy by up to 14.49 points over CoT and by up to 4.30 points over the strongest competing baseline. It further generalizes across diverse reasoning domains and achieves the best final-stage performance with the lowest forgetting.