Lichess puzzles need some serious evaluation

I've had almost a string of puzzles lately which are just wrong. At times the puzzle itself is wrong (i.e. all the options are not considered, the opponent could easily get out of the trap with the moves which the computer wants you to play). Other times you can play an equally good (or better) move and you lose points because it is not what the computer has decided is right. Another thing is that the rating points gained or lost are sometimes completely unreflective of the difficulty of the puzzle.

I don't know how these puzzles are created, but if it is some kind of algorithm, to me it certainly needs some tweaking. Has anyone else had this experience?

@Eagle777 People say this here all the time. Usually they've missed something; sometimes the puzzles are weak. No way of guessing which is the case here as you've not posted any details. If they suck they can be removed. Vote them down if you feel it's bad; a high vote suggests you're wrong. Puzzles are created from an algorithm detecting tactics in real games played by good player on Lichess.

Exactly. I get my best insights trying to refute and then seeing my mistake. (And am totally smug when I can then vote "up" for a puzzle that has a low rating!)

otoh: I can't figure out how the solution to #62590 doesn't get refuted by Ng3 (check), exposing Qxh4 = up at least 2 points.

Do puzzles that get voted down to zero and stay there eventually get removed?

When it is non-zero I agree it usually means I've missed something but there are unfortunately a large number where the puzzle has already a rating of zero, study of the puzzle seems like the puzzle solution is not actually optimal, or at least seems equal with other solutions, and downvoting it seems to have no effect.

I enjoy the puzzles, though so I think they are a great feature. Would be nice to hear that the negative feedback is used somehow.


After RH4, NG3, the refutation is QXG3. At that point you are up a knight. If BXG3, then RXH6, and you are up a knight after trading queens. If the black Queen moves, then you move the white Queen and are still up the knight.

I get this all the time, I can show how my opponent can escape (albeit with a worse position) so I refute a mating line in favour of winning material or being up the exchange - I lose rating points as the computer blunders a mate in 3.

"Another thing is that the rating points gained or lost are sometimes completely unreflective of the difficulty of the puzzle."

This point is sort of relatable, although I assume that the rating of the puzzle is objectively based on the average performance of players of various ratings.

Still, I'm sometimes surprised that easier puzzles like are rated around 2200 when I've seen similar ones probably closer to 1500.

Some other examples of 2000+ rated puzzles that seem overrated to me: seems more like 1400 - 1500 seems like 1600 - 1700 seems like 1500 seems like 1300 - 1400 seems like 1600 seems around 1700

And this one is rated 2479 but it's also around 1600 difficulty:

But again, these are just subjective impressions. I assume there are objective metrics behind the rating.

I found a mating sequence once, and Lichess penalized me for not winning material instead. In puzzle #61628, move order should not matter, but Lichess penalized me for playing 1...Bxc3 2 bxc3 b6 3 Bxb6 Rb7 instead of 1...b6 first 2 Bxb6 Bxc3 3 bxc3 Rb7 . I lost 8 points. In puzzle #61689 I rejected Lichess answer 1...d4 because it does not win a piece: 1..d4 2 Bxf6 Bxc3+ 3Kd1 Bxa1 4 Bxh8. Both sides are down a rook and minor piece. I lost 18 points!

61628 You let the bishop slip away: 3.Bb8
61689 14... Rg8 15. Rb1 Qf5

Stockfish can help