- Blind mode tutorial
lichess.org
Donate

Puzzle database archive and updates?

#1

Hi, I would like to see how the puzzles are changing over time. The page at https://database.lichess.org/#puzzles is very helpful, but the puzzle database is about a year old. Is there a way to download a newer database?

Also, is there a way to download historical versions of the database?

Thanks very much!

Hi, I would like to see how the puzzles are changing over time. The page at https://database.lichess.org/#puzzles is very helpful, but the puzzle database is about a year old. Is there a way to download a newer database? Also, is there a way to download historical versions of the database? Thanks very much!
#2

Hi!

It seems the puzzle database was updated December 1st (i.e, not a year old),
according to the Last-Modified response header:

curl --head https://database.lichess.org/lichess_db_puzzle.csv.zst

HTTP/2 200
server: nginx/1.18.0 (Ubuntu)
date: Sat, 28 Dec 2024 18:37:11 GMT
content-type: application/octet-stream
content-length: 220660928
last-modified: Sun, 01 Dec 2024 17:31:15 GMT
etag: "674c9d63-d2704c0"
strict-transport-security: max-age=63072000; includeSubDomains; preload
accept-ranges: bytes

I would guess the archive is updated/grown every month:ish.
I wouldn't guess the non-updated/non-grown verisions of the archive is kept around.

Hi! It seems the puzzle database was updated December 1st (i.e, not a year old), according to the Last-Modified response header: curl --head https://database.lichess.org/lichess_db_puzzle.csv.zst HTTP/2 200 server: nginx/1.18.0 (Ubuntu) date: Sat, 28 Dec 2024 18:37:11 GMT content-type: application/octet-stream content-length: 220660928 last-modified: Sun, 01 Dec 2024 17:31:15 GMT etag: "674c9d63-d2704c0" strict-transport-security: max-age=63072000; includeSubDomains; preload accept-ranges: bytes I would guess the archive is updated/grown every month:ish. I wouldn't guess the non-updated/non-grown verisions of the archive is kept around.
#4

Thanks very much! Sorry for getting the date wrong: I saw "This file was last updated on 01/12/2024" and forgot Europeans put the month first!

Thanks very much! Sorry for getting the date wrong: I saw "This file was last updated on 01/12/2024" and forgot Europeans put the month first!
#5

BTW, in case anyone is interested, below are some results comparing puzzle ratings in 2023 vs 2024. I filtered on puzzles in both datasets that had at least 1000 plays in 2023, had at least 1000 more in 2024 than 2023, and had no more than 100 rating deviation:

min_bucket max_bucket avg rating 2023 avg rating 2024 count
500 600 565 618 2
600 700 661 726 14
700 800 762 925 36
800 900 869 1031 236
900 1000 969 1087 2175
1000 1100 1062 1126 11040
1100 1200 1156 1183 25398
1200 1300 1253 1262 31132
1300 1400 1352 1354 36883
1400 1500 1451 1449 39915
1500 1600 1551 1541 43158
1600 1700 1649 1633 41278
1700 1800 1750 1731 34811
1800 1900 1849 1829 29911
1900 2000 1950 1927 29519
2000 2100 2049 2022 23099
2100 2200 2148 2118 17977
2200 2300 2246 2213 14302
2300 2400 2345 2306 7885
2400 2500 2442 2393 3259
2500 2600 2538 2473 1148
2600 2700 2639 2545 177
2700 2800 2730 2662 26
2800 2900 2829 2853 3
2900 3000 2902 2778 1

It looks like there has been some compression in the puzzle ratings. A 1500ish puzzle is still a 1500ish puzzle, but lower rated puzzles (say, below 1200) are now rated harder than they used to be, and harder puzzles (say, above 1800) are now rated easier than they used to be.

In other words, if you have maintained your puzzle rating over the last year by doing hard puzzles, then it's possible your actual skills has increased very slightly (e.g. by 50 points if you've been doing 2300-2400 level puzzles)!

(And unfortunately, conversely, if you've been doing easy puzzles and are still stuck at the same puzzle rating score, then maybe you are worse than you were last year.)

BTW, in case anyone is interested, below are some results comparing puzzle ratings in 2023 vs 2024. I filtered on puzzles in both datasets that had at least 1000 plays in 2023, had at least 1000 more in 2024 than 2023, and had no more than 100 rating deviation: min_bucket max_bucket avg rating 2023 avg rating 2024 count 500 600 565 618 2 600 700 661 726 14 700 800 762 925 36 800 900 869 1031 236 900 1000 969 1087 2175 1000 1100 1062 1126 11040 1100 1200 1156 1183 25398 1200 1300 1253 1262 31132 1300 1400 1352 1354 36883 1400 1500 1451 1449 39915 1500 1600 1551 1541 43158 1600 1700 1649 1633 41278 1700 1800 1750 1731 34811 1800 1900 1849 1829 29911 1900 2000 1950 1927 29519 2000 2100 2049 2022 23099 2100 2200 2148 2118 17977 2200 2300 2246 2213 14302 2300 2400 2345 2306 7885 2400 2500 2442 2393 3259 2500 2600 2538 2473 1148 2600 2700 2639 2545 177 2700 2800 2730 2662 26 2800 2900 2829 2853 3 2900 3000 2902 2778 1 It looks like there has been some compression in the puzzle ratings. A 1500ish puzzle is still a 1500ish puzzle, but lower rated puzzles (say, below 1200) are now rated harder than they used to be, and harder puzzles (say, above 1800) are now rated easier than they used to be. In other words, if you have maintained your puzzle rating over the last year by doing hard puzzles, then it's possible your actual skills has increased very slightly (e.g. by 50 points if you've been doing 2300-2400 level puzzles)! (And unfortunately, conversely, if you've been doing easy puzzles and are still stuck at the same puzzle rating score, then maybe you are worse than you were last year.)
#6

Sorry for getting the date wrong: I saw "This file was last updated on 01/12/2024" and forgot Europeans put the month first!

If only there was a standard... like ISO 8601.

>Sorry for getting the date wrong: I saw "This file was last updated on 01/12/2024" and forgot Europeans put the month first! If only there was a standard... like ISO 8601.

This topic has been archived and can no longer be replied to.