lichess.org
Donate

Will Lichess run out of space?

This website seems to store every game, even the aborted and anonymous ones. How much longer do we have until lichess finally collapses under the number of all its games?
I don't believe Lichess will collapse @blind_man_walking with the number of Patrons, and @thibault himself utilizing his personal funds, I believe they update and will add servers as needed to keep Lichess up and running always.
Aborted games are deleted after 24 hours.

You can have access to the whole database of lichess here: database.lichess.org
You can see that all the games take less than 250 Gb.
@sheckley666 you are right.

But tens of thousands of games starts to consume some space. Luckily, it seems that Lichess has enough Patron helping to pay the bill.
@Nicals Thousands of thousands of thousands of games with a few bytes each will make it - a few gigabytes.
Never underestimate the amount of information a tiny USB stick is able to store.
@sheckley666 If you want to go more technical, you have ton consider the amount of computing power needed not only to store, but also to serve all those game at a high rate. A game may not only be stored once, but also on replicate servers.

A website like this one does not only need to store the raw data, they also need to get this data to the user. If you want to be quick, you will also need some solid indexes, etc... So even if a tiny USB stick can store a large quantity of files, this is not the same application.

You can check this out here: docs.google.com/spreadsheets/d/1CGgu-7aNxlZkjLl9l-OlL00fch06xp0Q7eCVDDakYEE/preview

There is clearly more here than just some random files in a directory.

Here is a technical post from the blog explaining how they did to store the games if you are interested in the topic.
lichess.org/blog/Wqa7GiAAAOIpBLoY/developer-update-275-improved-game-compression
@Nicals I don't want to belittle the effort lichess is making for all of us. Of course the lichess server farm is more than a USB stick and an abacus like cpu.
But for storing all the games it helps a lot that a chess game needs only a few bytes. A HD image consumes probably similar space as 100,000 chess games.
Lichess will not run out of space. Lichess will not ever need ads. Lichess will never charge money. Lichess will always stay Open-Source GNLv3 or up.
I don't think that it will be an issue. As stated above every game on here combined only takes up 242 GB. That's counting from January 2013 to now. Then another 4 or 5 GB if you count all of the variant games as well. The current trend is about 10 GB per month of data saved. My laptop which is not the best one on the market has a 1 TB HDD. All of the current games would take up about 1/4 of my HDD.

There are 16 TB HDD currently on the market for around $400. Assuming that they bought a brand new one of those today, and assuming that lichess stayed at the current 10 GB per month of storage... It would still be 1,600 months to fill such a HDD.

1,600 months = 133.333 years.

And even if lichess got 100 times more popular, it would still take over 1.3 years to fill such a big HDD. This is insane, but even if the storage of games starts to become an issue can always get a fresh HDD.

If they were limited to using HDD from like the 1990s there may be an issue, and new hard drives would be a constant issue as they would fill quickly. However with storage at it's current use and hard drives at their current capacity they can pretty much run it forever with minimal worries.

This topic has been archived and can no longer be replied to.