cs.CLNov 7, 2026

LongCapHarness: Capability-Guided Auto-Harnessing for Long-Context Reasoning

Authors: Tao Zhang, Yuhang He

Organizations: Microsoft Research Asia

Abstract

Existing methods enhancing an LLM's long-context reasoning capability often rely on data synthesis and follow-up LLM post-training. However, the data synthesis strategy and the post-training recipes are empirically driven and computationally intensive. In this work, we propose a new framework, dubbed LongCapHarness, to instead harness the underlying capabilities for long-context reasoning. The harnessing process recursively revises and updates capability primitive implementation, capability state and control logic through recursive self-improvement on the development set. With neither data synthesis nor LLM post-training, LongCapHarness is applied to Llama3-8B and transferred to Qwen2.5-3B. LongCapHarness substantially outperforms recent data-synthesis-based baselines on HELMET and RULER benchmarks; its estimated development cost is less than 50% of the estimated training costs of NExtLong and PolicyLong. We hope these results motivate further research into harnessing the latent capabilities of LLMs to solve tasks beyond their explicit training.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

    Oct 21, 2026Zhennan Wan, Jianfei ChenLong-Context Reasoning in Language ModelsRL for Language Model Reasoning

  2. LongHarness Bench: Stress-Testing Language Model Harnesses for Long-Context Reasoning

    Sep 29, 2026Quang Hieu Pham, Thuy Duong Nguyen, Jocelyn Qiaochu Chen +1Long-Context Language Model EvaluationLong-Context Language Model Inference