Puzzles vs. Games

Listen, I don’t hate puzzles. I never said I hate puzzles. Nobody can prove I hate puzzles. All I’m saying is, most positions you get in a chess game aren’t like puzzles.

What makes a position a puzzle? Typically, there’s one and exactly one way to get a winning position. On average you might get about one position like that per game. It’s really important to get those right, but at the same time, you still have to play all the other positions. You can’t just pass every time you don’t have a clear win.

I’ve harped on this before, but this week I’m getting more specific. What kinds of positions are harder to get right in a game than as a puzzle, and vice versa? To answer this question I borrowed some ideas from my friend Martin who writes the Say Chess Newsletter.

Martin used the Lichess Puzzle API to find opening traps that occur frequently in real games as examples for his Blindfold Opening Visualization book. For this project I combined the Puzzle API with the Opening Explorer to find positions where the difference between the puzzle difficulty and the in-a-real-game difficulty was biggest. My goal was to get a clearer sense of how puzzles differ from real games.

The Setup

• First I downloaded all the puzzles in the Lichess database.
• Then I filtered for puzzles that occur on or before move ten. I wanted to make sure there were plenty of examples of the position occurring in real games and opening positions are the only ones that happen often enough.
• I took a random sample of 1000 of the opening puzzles.
• I looked up those 1000 positions in the Lichess Opening Explorer (excluding bullet games) to see how often players found the right move when the position happened in a real game.
• I filtered those for positions that had happened at least 100 times. This let me be fairly certain that the in-game solve percentage was representative of how hard the position was, not just a fluke from a few games.
• Finally, I multiplied the puzzle rating with the in-game solve percentage to get a single metric representing the difference between how hard a position was as a puzzle vs. in a game (more on this below).

You can check out the full results here.

The PuzzFuzz Metric

Every puzzle in the Lichess database has a puzzle rating that represents how hard it is to solve. The higher the rating, the more difficult the puzzle. Like normal ratings, puzzle ratings get adjusted based on performance. Every time someone solves the puzzle correctly, the rating goes down; every time someone gets it wrong, the rating goes up.

To get a metric representing the difference between how hard the position was in a game vs. as a puzzle, I multiplied the puzzle rating by the percentage of time players found the right move in a game.
Let’s think about how this works. Puzzles with higher ratings are supposed to be harder, so they should have a lower solve percentage in-game. Similarly, puzzles with low ratings should have a higher solve percentage. As long as all of this holds, it should more or less balance out and most positions should have a moderate PuzzFuzz rating.

To get a low PuzzFuzz rating, a position needs a low puzzle rating and a low in-game solve percentage. This means it’s supposed to be an easy puzzle, but players rarely find the right move in-game. So a low PuzzFuzz rating represents a position that’s much harder in-game than when presented as a puzzle.

The Positions

I sorted all of the positions by their PuzzFuzz rating. Let’s look at the lowest scorers, the positions that were easy as puzzles but often botched in real games.

You can try them as an interactive Lichess study. Take a shot at solving them, but eep in mind, you have an advantage over the players who missed these in real games: you know they’re puzzles.

What do these positions have in common? The right move is out of flow. By that I mean the best move doesn’t follow the progression of the game over the past few moves. It’s not what the player was planning to do, but an opportunity created by the opponent’s previous move in a hidden way.

For example, look at the first position. White was more or less minding his own business, developing his pieces. Black is developing too - Nbd7 brings out a knight. But this move also traps the queen allowing the tactical shot Bxf7+ when after Kxf7 Ng5+, White will win the queen, get a crushing attack, or both.

In the 126 times this position happened on Lichess, White only found the best move 7 times. Almost no one was alert to the possibility of a tactic - most players just castled. But as a puzzle it’s much easier. Indeed, knowing there’s a win in the position, Bxf7+ is virtually the only candidate.

Position number 6, despite being very simple, is another case of a move that goes against flow. White has just captured a pawn with fxe5 so the obvious thing to do is take back with Qxe5. Recaptures are the most in-flow moves of all.

Alert readers might notice a potential source of bias here. Given that both sides have to play some questionable moves to reach this position, the average rating here is probably on the lower side, which means that the in-game difficulty of the position may not be as hard as it seems. This is a valid observation, but not as clear-cut as you might think. This position occurred 178 times in the 2000 rating tier and Black played the best move less than half the time. Additionally, the puzzle rating for this one is low enough that even low-rated players are usually solving it correctly as a puzzle.

What makes this one hard in a real game is to find the best move you have to question the assumptions of your opponent’s previous move. Chess is a competition, but it’s also a conversation, and like in a conversation agreeing is easier than disagreeing. When your opponent captures something the easiest way to respond is taking back. But in this position the best move is Qh4+ when after g3 Qxe4+ Black will collect the rook on h1.

The tactical pattern is a common one, but in this position it’s out of flow for several reasons. First, White just took a pawn so the obvious move is to recapture. Second, until last move White’s pawn was blocking the fourth rank, so the tactical pattern didn’t exist until this very moment. Third, Black had previously moved the queen to f6, and it feels odd to now move it farther along the same diagonal.

Black players only managed to find Qh4+ 15% of the time in real games. But as a puzzle solution Qxe5 is clearly not enough - it just takes back a pawn restoring equal material. Once you know it’s a puzzle, Qh4+ is easy to find.

Takeaways

Puzzles are isolated little islands, but this isn’t how real chess games are played. In a real game, much of a player’s thought process comes down to flow - the thoughts, beliefs, and plans carried over from previous moves. Furthermore, a real game is a conversation. The players cannot help engaging with each other and at times taking each other at face value.

Knowing this, you should still do puzzles - they really are good training! - but also recognize that doing a puzzle is very different from playing a game. In a game, you should constantly remind yourself of two things:

1. Don’t take your opponent’s word for anything. They are your collaborator in a shared story you can’t help telling, but they’re also your mortal enemy.
2. After every move ask yourself, “What changed?” Not only the obvious, but the hidden.

If you liked this post, check out my newsletter for more chess writing