lichess.org
Donate

Developer update: 275% improved game compression

This site is amazing for chess, and for the technical solutions you guys come up with. Thank you for informing the community of the background changes. Unfortunately, only few developers do this, but it is a way to obtain and share knowledge. You are contributing to global education and art, which makes you truly stand out from the crowd. I am sincerely grateful that lichess.org exists.
@revoof

The article cites chessprogramming.wikispaces.com ...

It's been a while since I've been there. Following the link to read that section, there's a note: "It's time for us to say farewell…"

Any idea what will become of that? (I know this isn't a LiChess specific question, but, that is a valuable resource for the chess community at large.)

Would LiChess host a (static) mirror of all that information?

@tdunlap1310

There's a term, let's just say it is a 2-word composite term. The two-words are: "Back read." (Asked and answered.)
@revoof Ordering moves by PSTs from engines makes some sense, but couldn't you achieve higher prediction accuracy by generating PSTs from the lichess games themselves? (as humans might not play like engines ;) )
Also, how exactly do you utilize those? Sort by PST evaluation change?

Also in my engine I order captures by capture size (i.e. prefer capturing a Q over capturing a P) wouldn't that make sense here too? (similarily for promotions)
Really impressive :-)

I enjoyed every line of your post ^_^

P. S.
I'm a big data scientist and a Java developer myself. I will be very very happy if I can help in some way.
@Vetinari_Computer Yes, exactly (github.com/lichess-org/compression/blob/master/src/main/java/game/Move.java#L36). Now I wish I had taken more detailed notes:

- I think counter-intuitively capturing expensive pieces first was no improvement. Perhaps there are too many positions where pawns are traded but pieces could also have been traded. Or perhaps the other criteria are better suited to order capturing moves.
- Full static exchange evaluation for each move was worth 3 bytes per game, but too slow.
- I think a piece square table based only on how often a piece was on a given square peformed worse than the one we're using now (at least when combined in this order with the other criteria). I am sure a more clever way to build the table would improve compression, but I didn't explore this yet.

This topic has been archived and can no longer be replied to.