Large language models (LLMs) have a substantial capacity for high-level analogical reasoning: reproducing patterns in linear text that occur in their training data (zero-shot evaluation) or in the provided context (few-shot in-context learning). However, recent studies show that even the more advanced LLMs fail in scenarios that require reasoning over multiple objects or facts and making sequences of logical deductions. We propose a two-stage probabilistic inference paradigm, ThinkSum, which reasons over sets of objects or facts in a structured manner. In the first stage (Think - retrieval of associations), a LLM is queried in parallel over a set of phrases extracted from the prompt or an auxiliary model call. In the second stage (Sum - probabilistic inference or reasoning), the results of these queries are aggregated to make the final prediction. We demonstrate the possibilities and advantages of ThinkSum on the BIG-bench suite of LLM evaluation tasks, achieving improvements over the state of the art using GPT-family models on thirteen difficult tasks, often with far smaller model variants. We also compare and contrast ThinkSum with other proposed modifications to direct prompting of LLMs, such as variants of chain-of-thought prompting. Our results suggest that because the probabilistic inference in ThinkSum is performed outside of calls to the LLM, ThinkSum is less sensitive to prompt design, yields more interpretable predictions, and can be flexibly combined with latent variable models to extract structured knowledge from LLMs. Overall, our proposed paradigm represents a promising approach for enhancing the reasoning capabilities of LLMs.
Figures & tables
Figure 1: An example adapted from the Conceptual combinations (Invented words) task, in which models must select the most likely completion of a phrase that includes nonce words whose definitions are given. Top: Direct prompting evaluates completion likelihoods normalized over the four answer choices (‘people’, ‘animals’, ‘birds’, ‘researchers’). Middle: Chain-of-thought -like or auxiliary knowledge approaches would query a LLM or knowledge base for additional context. This example shows the brittleness entrusting all ‘reasoning’ to self-attention in linear text, especially in smaller models, which have stronger recency bias ( Whitammer et al., 2022 ) : if we simply list generated examples as the additional context in the prompt, the recency bias causes the LLM to still give a higher probability to ‘people’ than to ‘animals’, simply because ‘bam’ (simple dwelling) examples are given after the ‘binne’ examples. Bottom: Our ThinkSum approach to this task queries a LLM (GPT-2 XL) to produce sets of examples defining the nonce words, then marginalizes over substitutions of these examples into the target phrase.
GPT-3 (davinci) n -shot
ThinkSum
Task
Avg. H
n=0
1
2
3
GPT-3
InstructGPT
GPT-2 XL
Invented words (§ 3.1 )
N/A
0.29
0.14
0.14
0.21
0.64
0.71
0.29
Odd one out (§ 3.2 )
0.80
0.27
0.20
0.23
0.23
0.80
0.84
0.71
Five objects (§ 3.3 )
N/A
0.23
0.29
0.28
0.32
–
0.77
–
Sports understanding (§ A.1 )
0.71
0.50
0.50
0.50
0.50
0.71
0.74
0.54
Known unknowns (§ A.1 )
0.80
0.61
0.52
0.48
0.50
0.54
0.76
–
Table 1: Standard metric (BLEU for Codenames , accuracy for other tasks) for GPT-3 175B (davinci) and ThinkSum with 175B (davinci), InstructGPT and GPT-2 XL on BIG-bench tasks. A ‘–’ indicates that the model and task combination was not evaluated because the model does not reliably execute the appropriate Think prompt. We did not evaluate InstructGPT on Language identification due to the large dataset size and API quota.
Figure 2: Odd one out . Left: Performance of GPT-3 ( n -shot, n=0,1,2,3 ), auxiliary knowledge, and ThinkSum with various model sizes. Middle: Auxiliary knowledge vs. ThinkSum with varying number of differences. Right: Prompt used to generate knowledge statements.
Model
LoW
LVM
MV
text-davinci-002
0.84
0.67
0.70
text-davinci-001
0.74
0.77
0.70
Table 2: Different alternatives of probabilistic reasoning with ThinkSum for solving Odd one out : list of words, latent variable model, minority voting.
Figure 3: Details for Logical deduction . (a) Example question from the task, (b) demonstration for the Think prompt, (c) example LLM output. The demonstration induces the LLM to generalize from generic objects ordered by size to books ordered by position.
Odd one out
Phrase relatedness
Logical deduction ( N=5 )
ThinkSum
0.84
0.87
0.77
Aux. knowledge
0.71
0.75
0.50
Table 3: ThinkSum vs. auxiliary knowledge with text-davinci-002.
GPT-3 (davinci)
GPT-3 (davinci-002)
Task
Direct
CoT
ThinkSum
CoT
ThinkSum
Odd one out
0.27
0.33
0.80
0.64
0.84
Phrase relatedness
0.59
0.55
0.85
0.79
0.87
Logical deduction
0.32
0.25
–
0.39
0.77
Known unknowns
0.61
0.70
0.54
0.74
0.76
Invented words
0.29
0.50
0.64
0.64
0.71
Table 4: Comparison of ThinkSum with chain-of-thought prompting approaches.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A.1: Example posterior probabilities generated from text-davinci-002 for Sports understanding with the description “threw a touchdown” . The basketball player given in the question Draymond Green has a much lower posterior probability than the generated football players, from which we conclude the sentence “Draymond Green threw a touchdown.” is implausible.
Odd one out
Words: blue, pink, magenta, banana All words are colors except banana. The odd one out is banana. Words: pencil, eraser, baby, rule, notebook All words are office supplies except baby. The odd one out is baby.
Phrase relatedness
For each word or phrase, identify the most related choice from the listed options. Input: Ice Cream Option: Antarctica Option: Titanic Option: Dessert Option: Sour Cream Ice cream is a type of dessert. Therefore, ice cream and dessert are the most related. Answer: Dessert
Known unknowns
What was the population of San Francisco in 2018? Option: 879,676 Option: Unknown The question asks the population of San Francisco in 2018, for which data can be collected. Population data for cities on a yearly basis is available, and thus the answer is known, and it is 879,676. Answer: 879,676 What was the population of San Francisco yesterday? Option: 891,402 Option: Unknown The question asks the population of San Francisco yesterday. As it is not possible to know the exact population of a city on a daily basis, the answer for this question is unknown. Answer: Unknown
Logical deduction
On a table, there are five plates: a black plate, a white plate, a green plate, a blue plate, and a red plate. The white plate is bigger than the green plate. The red plate is the biggest. The black plate is bigger than the blue plate. The black plate is smaller than the green plate. Which plate is the smallest? Option: The red plate is the smallest. Option: The black plate is the smallest. Option: The white plate is the smallest. Option: The green plate is the smallest. Option: The blue plate is the smallest. The black plate is bigger than the blue plate. The black plate is smaller than the green plate, as a result the green plate is bigger than the blue plate as well. The white plate is bigger than the green plate, which is bigger than the blue plate. As a result, the green plate is bigger than the blue plate. The red plate is the biggest, so it is bigger than the blue plate. Since all other plates are bigger than the blue plate, the blue plate is smallest. Answer: The blue plate is the smallest.
Invented words
The word ’borger’ are animals who bite specific things for fun, and the word ’folpt’ is a type of a chewy toy. Question: Which of the following sentences best characterizes borger folpts? Option: Borger folpts are leashes for animals. Option: Borger folpts are toys for infants. Option: Borger folpts are hard to swallow. Option: Borger folpts are pet toys. Borgers are animals, and folpts are chewy toys. Therefore, borger folpts are chewy toys that animals, or pets, can play with. Therefore, the answer is borger folpts are pet toys. Answer: Borger folpts are pet toys.
Appendix
Table A.1: Few-shot demonstrations used for chain of thought (Table 4 ).
Figure B.1: Margin between 0-shot GPT-3 and average human performance for BIG-bench Lite tasks. Using ThinkSum , we address many of the tasks that have greater than 10% performance margin with average human, and significantly reduce and often overturn the margin.
Figure D.1: Probabilities of different (in)equalities according to GPT-3 text-davinci-002 (logit).
Figure D.2: Auxiliary knowledge prompting applied to Odd one out . Facts are generated using the “list differences” prompt described in Figure 2 (right) and post-processed according to § D.3 .
Task: Phrase relatedness
Input: For each word or phrase, identify the most related choice from the listed options.
Input: home town
Option: town center
Option: location
Option: native city
Option: home run
Appendix
Table D.1: List of examples for the studied BIG-bench tasks.