Paper ID: 2412.00353 • Published Nov 30, 2024
Enhancing Zero-shot Chain of Thought Prompting via Uncertainty-Guided Strategy Selection
Shanu Kumar, Saish Mendke, Karody Lubna Abdul Rahman, Santosh Kurasa, Parag Agrawal, Sandipan Dandapat
TL;DR
Get AI-generated summaries with premium
Get AI-generated summaries with premium
Chain-of-thought (CoT) prompting has significantly enhanced the capability of
large language models (LLMs) by structuring their reasoning processes. However,
existing methods face critical limitations: handcrafted demonstrations require
extensive human expertise, while trigger phrases are prone to inaccuracies. In
this paper, we propose the Zero-shot Uncertainty-based Selection (ZEUS) method,
a novel approach that improves CoT prompting by utilizing uncertainty estimates
to select effective demonstrations without needing access to model parameters.
Unlike traditional methods, ZEUS offers high sensitivity in distinguishing
between helpful and ineffective questions, ensuring more precise and reliable
selection. Our extensive evaluation shows that ZEUS consistently outperforms
existing CoT strategies across four challenging reasoning benchmarks,
demonstrating its robustness and scalability.