In this paper we try to answer the question "What constitutes Sudoku difficulty rating across different Sudoku websites?" Using two distinct methods that can both solve every Sudoku puzzle, I propose two new metrics to characterize Sudoku difficulty. The first method is based on converting a Sudoku puzzle into its corresponding Satisfiability (SAT) problem. The first proposed metric is derived from SAT Clause Length Distribution which captures the structural complexity of a Sudoku puzzle including the number of given digits and the cells they are in. The second method simulates human Sudoku solvers by intertwining four popular Sudoku strategies within a backtracking algorithm called Nishio. The second metric is computed by counting the number of times Sudoku strategies are applied within the backtracking iterations of a randomized Nishio. Using these two metrics, I analyze more than a thousand Sudoku puzzles across five popular websites to characterize every difficulty level in each website. I evaluate the relationship between the proposed metrics and website-labeled difficulty levels using Spearman's rank correlation coefficient, finding strong correlations for 4 out of 5 websites. I construct a universal rating system using a simple, unsupervised classifier based on the two proposed metrics. This rating system is capable of classifying both individual puzzles and entire difficulty levels from the different Sudoku websites into three categories - Universal Easy, Universal Medium, and Universal Hard - thereby enabling consistent difficulty mapping across Sudoku websites. The experimental results show that for 4 out of 5 Sudoku websites, the universal classification aligns well with website-labeled difficulty levels. Finally, I present an algorithm that can be used by early Sudoku practitioners to solve Sudoku puzzles.
Figures & tables
Figure 1: Two New York Times Sudoku Puzzles from July 23rd 2024 with candidates filled in
Figure 2: A rectangle is drawn between the cells where X-wing should be used
Website
Short Clauses (%)
Medium Clauses (%)
Long Clauses (%)
New York Times
Easy
67
33
0
Medium
17
71
13
Hard
15
72
13
Sudoku.org.uk
Gentle
24
67
9
Table 1: Clause Length Distribution across 5 websites
Website
Clause Length Distribution
Nishio Human Cycles with 4 Human Strategies
Nishio Human Cycles with 2 Human Strategies
NyTimes Sudokus
-0.80
0.84
0.68
Sudoku.org.uk
-0.47
0.84
0.81
Extreme Sudoku
-0.02
-0.002
-0.03
Sudoku of the Day
-0.75
0.89
0.83
Sudoku of the Day UK
-0.28
0.70
0.61
Table 2: Spearman’s Rank Correlation Coefficient between website-provided difficulty levels and two proposed metrics - Clause Length Distribution and Nishio Human Cycles. For Clause Length Distribution, I use the percentage of short clauses as the metric to calculate the correlation coefficient. Each puzzle is assigned an integer ranking based on the website’s difficulty level ordering (e.g. in New York Times, Easy puzzles are assigned 1, Medium puzzles are assigned 2, and Hard puzzles are assigned 3).
Nishio Human Cycles (50 Random Starts)
Percentage of Puzzles solved by Strategies
Difficulty
4 Strategies
2 Strategies
2 Strategies (%)
3 Strategies (%)
4 Strategies (%)
Easy
Mean
1.61
1.62
100
100
100
Median
1.58
1.60
Medium
Mean
6.74
12.40
0
52.5
62.5
Table 3: New York Times
Nishio Human Cycles (50 Random Starts)
Percentage of Puzzles solved by Strategies
Difficulty
4 Strategies
2 Strategies
2 Strategies (%)
3 Strategies (%)
4 Strategies (%)
Gentle
Mean
2.45
2.51
98
98
98
Median
2.38
2.4
Moderate
Mean
3.09
3.77
97
98
98
Table 4: Sudoku.org.uk
Nishio Human Cycles (50 Random Starts)
Percentage of Puzzles solved by Strategies
Difficulty
4 Strategies
2 Strategies
2 Strategies (%)
3 Strategies (%)
4 Strategies (%)
Evil
Mean
8.01
14.01
0
0
5
Median
6.63
10.09
Excessive
Mean
7.13
10.89
0
0
3
Table 5: Extreme Sudoku
Nishio Human Cycles (50 Random Starts)
Percentage of Puzzles solved by Strategies
Difficulty
4 Strategies
2 Strategies
2 Strategies (%)
3 Strategies (%)
4 Strategies (%)
Beginner
Mean
1.73
1.73
100
100
100
Median
1.72
1.70
Easy
Mean
2.09
2.14
100
100
100
Table 6: Sudoku of the Day
Nishio Human Cycles (50 Random Starts)
Percentage of Puzzles solved by Strategies
Difficulty
4 Strategies
2 Strategies
2 Strategies (%)
3 Strategies (%)
4 Strategies (%)
Easy
Mean
3.48
5.20
82
93
95
Median
2.83
3.43
Medium
Mean
4.01
5.49
78
90
90
Table 7: Sudoku of the Day UK
Figure 4: Nishio Human Cycles distribution over the 1320 Sudoku puzzle dataset. In this histogram, the width of each bin is is one Nishio Human Cycle.
Figure 5: Clause Length Distribution over the 1320 Sudoku puzzle dataset. In this histogram, the width of each bin is one percentage (short clauses).
Figure 6: Difficulty levels from the 5 websites classified into one of three universal categories based Mean and Median Nishio Human Cycles (from Tables 3 - 7 ).
Figure 7: Difficulty levels from the 5 websites classified into one of three universal categories based on Mean percentage of short clauses (from Table 1 ).
Figure 8: Three constructively identical Extreme Sudoku puzzles from three different difficulty levels: Egregious, Extreme, and Excessive. Given cells are shaded. Notice that the Nishio Human Cycles and Clause Length Distribution metrics are close across all three puzzles.
Difficulty Level
Constructively Identical Puzzles
Evil
44/60
Excessive
45/60
Egregious
50/60
Excruciating
54/60
Extreme
49/60
Table 8: Constructively identical puzzle counts for each Extreme Sudoku difficulty level (out of 60).
Nishio Human Cycles
Nishio Human Cycles
Website
4 Strategies
2 Strategies
Website
4 Strategies
2 Strategies
New York Times
Sudoku of the Day
Easy
Beginner
Mean
1
1
Mean
1
1
Median
1
1
Median
1
1
Medium
Easy
Table 9: Nishio Human Cycles (2 and 4 strategies) computed using the solver in Figure 9 on the five-website dataset.
Sudoku
Nishio Human Cycles
Platinum Blonde
1074.92
Golden Nugget
750.22
tarx0134
840.02
Red Dwarf
508.88
Table 10: Nishio Human Cycles for 4 Ultra-Hard Sudoku puzzles
Universal Category
Number of Puzzles from nikoli_100 dataset
Universal Easy
53
Universal Medium
24
Universal Hard
23
Table 11: Universal Classification via Nishio Human Cycles for the nikoli_100 dataset