cs.CLNov 16, 2026

Latent Bridge: Asking for Help Without Words

Authors: Qichao Wang, Tingyu Qu, Pengcheng Wu, Yiran Zhong, Feida Zhu, Zhiqi Shen, Peilin Zhao, Steven Hoi

Organizations: Nanyang Technological University · Alibaba Token Hub, Alibaba Group · Shanghai Jiao Tong University

Abstract

Text-based model collaboration requires a large model to generate intermediate responses before a smaller model can use its guidance, adding large-model decoding cost to inference. We introduce POND (Prefill-Only, No-Decode), a method that uses a frozen large sender for prefill and a small receiver for all autoregressive decoding. A trainable bridge maps the sender's hidden states to additional attention keys and values, allowing the receiver to use these representations throughout generation. POND also supports periodic review, in which the sender processes the receiver's partial response and updates its guidance through additional prefill passes while preserving the receiver's existing KV cache. Across reasoning and coding benchmarks, POND improves receiver performance with senders from different model families. In particular, GLM-5.3 provides gains comparable to those of Qwen3.8-Max when paired with a Qwen receiver, supporting the effectiveness of representation transfer between independently trained model families. POND further demonstrates its ability to generalize across domains. When trained on math and coding tasks, it transfers directly to general knowledge tasks without additional domain-specific training. These results demonstrate that large frontier model prefill representations can improve small-model generation without requiring large-model autoregressive decoding.

Figures & tables

Explore similar work

CardsList
  1. On the Opportunities and Challenges of Prefill-to-Autoregressive Cross-Model Transfer

    Nov 4, 2026Zhenyi Shen, Junru Lu, Wei Liu +8Multi-LLM CollaborationTransfer Learning

  2. Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge

    Jun 6, 2026Juntong Shi, Brian L. Trippe, Jure Leskovec +2Importance SamplingProduct of Experts

  3. Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs

    May 26, 2026Wenhui Tan, Minghao Li, Xiaoqian Ma +5Multi-Token PredictionLLM Compression