Algorithmic solver effort is often assumed to align with perceived puzzle difficulty, but this assumption is rarely tested against human solving data. We evaluate this assumption for Nonograms, a popular logic puzzle similar to Sudoku in which numeric clues along each row and column determine a unique solution grid. We formulate Nonograms as a constraint satisfaction problem and solve them using existing SAT solvers. We then conduct a user study in which we collect data on both participant interactions and reported difficulty. We find that neither participants' reported difficulty nor their behavioural signals correlate meaningfully with SAT solver metrics; however, we find evidence that expertise moderates the relationship between solver metrics and reported difficulty. In this process, we uncover distinct, recurring solving strategies that indicate human preference for complex propagation, diverging from solver-measured complexity.
Figures & tables
Figure 1: Example Nonogram. Numbers beside each row and column indicate the lengths of consecutive filled-cell runs, in order. Filled cells form the unique solution.
Figure 2: Distribution of number of conflicts over 1,000 randomly generated qualifying puzzles.
SAT metric
ρ
p
Decisions
-0.257
0.623
Propagations
-0.257
0.623
Conflicts
+0.000
1.000
Table 1: Spearman correlation between Bradley-Terry-adjusted puzzle difficulty ranking and SAT solver metrics.
Figure 3: The distribution of the z-mean expertise score across participants.
SAT metric
Metric β
Expertise β
Interaction β
Decisions
-0.0007
+0.0875
-0.0096*
Propagations
+0.0001
+0.2053
-0.0004**
Conflicts
+0.0067
+0.0891
-0.0247**
Table 2: Moderation of the SAT-metric → difficulty relationship by participant expertise: fixed effects from a crossed-random-effects LMM (difficulty ∼ metric × expertise, with crossed random intercepts for participant and puzzle). * p<.05 , ** p<.01 .
Outcome
ρ
p
Mean time to solve (s)
-0.541
< .001
Mean pause count
-0.507
< .001
Mean error count
-0.407
< .001
Mean hints used
-0.583
< .001
Table 3: Spearman correlation between participant expertise (composite score) and behavioural outcomes, aggregated to participant-level means.
Figure 4: Behavioural signals regressed on SAT solver metrics: fixed effects from a crossed-random-effects LMM (behavioural signal ∼ metric, with crossed random intercepts for participant and puzzle).
Figure 5: Relationship between each difficulty theme and participant-reported difficulty.
Figure 6: Heatmap of the first cell acted upon for each puzzle, by frequency.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Log-interval distribution of interaction gaps recorded from participant behaviour during puzzle solving, with a three-component GMM.
Category
Code
Theme / Strategy
Example
Difficulty
FOOT
Starting footholds / forced lines
“Not a lot of freebies”
CLUE
Clue magnitude
“Clue number too small”
PROP
Constraint propagation / chaining
“easier to solve using elimination”
AMBIG
Combinatorial ambiguity / overlap reasoning
“Multiple intermediate steps seemed possible, but only one worked in the end”
GUESS
Guessing / certainty
“never need to guess”
HINT
Hint usage
“I use many hints.”
Appendix
Table 4: Difficulty themes and strategy taxonomy, with example responses.