- Blind mode tutorial
lichess.org
Donate

pgn-extract: A great tool to search databases for the types of positions you want to learn

#1

Hello,

I came across pgn-extract a while back, and think it's a fantastic tool for your Chess studies, allowing you to find the exact types of positions you're looking for in massive databases. To anyone interested in this tool, I wrote a blog post about it with some examples of how it can be used, and thought some here might find it worth a read. If it sounds interesting, please check it out:

https://bigeatie.com/posts/pgn-extract/

Hello, I came across pgn-extract a while back, and think it's a fantastic tool for your Chess studies, allowing you to find the exact types of positions you're looking for in massive databases. To anyone interested in this tool, I wrote a blog post about it with some examples of how it can be used, and thought some here might find it worth a read. If it sounds interesting, please check it out: https://bigeatie.com/posts/pgn-extract/
#2

Great article, we saw it yesterday, I think.
I'll just add that you don't need to unpack the archive, you can just pipe it into pgn-extract, i.e.

zstdcat file.pgn.zst | pgn-extract ...

I think zstdcat is equivalent to "zstd -cdfq", so one can use it instead or pzstd in a similar way.

Great article, we saw it yesterday, I think. I'll just add that you don't need to unpack the archive, you can just pipe it into pgn-extract, i.e. zstdcat file.pgn.zst | pgn-extract ... I think zstdcat is equivalent to "zstd -cdfq", so one can use it instead or pzstd in a similar way.
#3

Thanks for letting me know, I'll update the post!

Thanks for letting me know, I'll update the post!
#4

I'll hop in to say I only took a look at this tool for some PGN to UCI convertions, and I found it convenient for that purpose. Who knows, maybe another dev will read this ;)

I'll hop in to say I only took a look at this tool for some PGN to UCI convertions, and I found it convenient for that purpose. Who knows, maybe another dev will read this ;)
#5

pgn-extract is an incredible swiss-knife. Recently, I wrote some filters and small scripts to extract games containing sacrifices and games containing exchange sacrifices from large databases. Just from the CCRL games, I found 38000 games containing exchange sacrifices won by the side making exchange sac. The idea is first extract games won by white, and from these games search for games matching a given material pattern that lasts for a certain number of moves, for example, white gives rook for minor piece or rook for minor piece and a pawn, and the material deficit lasts for 6 consecutive half moves, but in the end white wins, and then do a similar thing for games won by black after black sacrifices material. I also analysed games by top players like Karpov, Tal, Anand, Topalov, Kasparov, and to my surprise I found that Karpov is not behind others like Topalov or Anand when it comes to making sacrifices.

pgn-extract is an incredible swiss-knife. Recently, I wrote some filters and small scripts to extract games containing sacrifices and games containing exchange sacrifices from large databases. Just from the CCRL games, I found 38000 games containing exchange sacrifices won by the side making exchange sac. The idea is first extract games won by white, and from these games search for games matching a given material pattern that lasts for a certain number of moves, for example, white gives rook for minor piece or rook for minor piece and a pawn, and the material deficit lasts for 6 consecutive half moves, but in the end white wins, and then do a similar thing for games won by black after black sacrifices material. I also analysed games by top players like Karpov, Tal, Anand, Topalov, Kasparov, and to my surprise I found that Karpov is not behind others like Topalov or Anand when it comes to making sacrifices.
#6

@Dragon_Emperor_9774 said in #4:

I'll hop in to say I only took a look at this tool for some PGN to UCI convertions, and I found it convenient for that purpose. Who knows, maybe another dev will read this ;)

Could you elaborate? I don't understand what you mean by PGN to UCI conversion? UCI is a protocol?!

@Dragon_Emperor_9774 said in #4: > I'll hop in to say I only took a look at this tool for some PGN to UCI convertions, and I found it convenient for that purpose. Who knows, maybe another dev will read this ;) Could you elaborate? I don't understand what you mean by PGN to UCI conversion? UCI is a protocol?!
#7

A nice write up with some great examples. pgn-extract is a very nice utility. It's functionality reminds me of some of Chessbase search masks such as manoeuvre search capabilties https://help.chessbase.com/CBase/15/Eng/index.html?000134.htm without having to fork out $ for chessbase. Although the ability to build your search graphically is somewhat appealing.

I've long wondered about a hosted search (ie. https://lichess.org/games/search) to do searches like your exampless but it sounds like the required compute resources required make it too expensive/difficult.

A nice write up with some great examples. pgn-extract is a very nice utility. It's functionality reminds me of some of Chessbase search masks such as manoeuvre search capabilties https://help.chessbase.com/CBase/15/Eng/index.html?000134.htm without having to fork out $ for chessbase. Although the ability to build your search graphically is somewhat appealing. I've long wondered about a hosted search (ie. https://lichess.org/games/search) to do searches like your exampless but it sounds like the required compute resources required make it too expensive/difficult.
#8

I think they meant move output in UCI format, i.e. -Wuci flag.

I think they meant move output in UCI format, i.e. -Wuci flag.

This topic has been archived and can no longer be replied to.