Understanding the opening explorer: 1) Input variable

I would like to see a topic somewhere about how to undersantd and use the opening explorer.

I find it a great teaching tool (not rote memory, but hands-on memory) for myself about openings and their fate.

It is allowed in correspondance games.

I also find it interesting to study different chess player behavior dependig on cadence of timed games.

For those reasons, i suggest some kind of study or set of related discussions, anything. or give me pointers if already existing.

I will start with a first question: What is the actual input variable of the opening explorer tool.

Opening the menu for O.E. from scratch (no game), we have the choice of PGN or FEN as input.
In game through board analysis, opening explorer uses the current game's PGN and possibly FEN.

My attempt at answering: For a give depth of game (half-moves), there are more (or equal) PGNs paths than correponding FENs. Hence, i suspect that FEN is the most direct input variable, and our PGNs entries are translated to FENs.

For the first possible half-steps, this makes no difference. For i have an example PGNs for which the opening explorer support my answer (it may be obvious to some, but i hope to help those like me who might find this helpful).

example in future post here. when ready (anybody can beat me to it). Or i may have it wrong, we'll see.

I invite any possible pointers to related forum, faq, or study related to this topic.

D30 Queen's Gambit Declined
PGN:

1. d4 d5 2. c4 e6 3. Nf3 Bd6 4. g3 Nf6 5. Bg2 O-O 6. O-O h6 7. c5 Be7 8. Nc3 Nc6
   FENs:
   8.1 rnbq1rk1/ppp1bpp1/4pn1p/2Pp4/3P4/2N2NP1/PP2PPBP/R1BQ1RK1 b - - 2 8 (Nc3)
   8.2 r1bq1rk1/ppp1bpp1/2n1pn1p/2Pp4/3P4/2N2NP1/PP2PPBP/R1BQ1RK1 w - - 3 9 (Nc6)

at 8.1, the opening explorer shows Nc6 as having a population of 3
at 8.2, having chosen Nc6, the opening explorer shows at total 13+5+4+2+1+1+1 way more than 3.

so the 3 games shown by the o.e. at 8.1 were the subset of the many FENs having the same FEN at 8.2 but not necessarily the same FEN at 8.1, i.e. at 8.2 we are actually seeing the result of many PGNs, even if only our own path (above) is show on top the OE display. FEN1 (8.1) + FEN2 (8.1) = FEN3 (8.2). FEN1 and FEN3 for our own PGN.

This does not make sense if PGNs are the most direct input to those table results.

Not? any discussion? or additional explantion? boring?

I might add, for those interested by this kind of stuff, that one could build a index N of explored connectivity for each FENs. Maybe even add this to the results (tell the user, that N other PGNs lead to the same position you are in).

Maybe not, who cares?

I'm building books myself.

FEN's are not good keys for a book, because they contain the full move number, the half move clock ( possibly an ep square for which no ep capture is possible ), which make handling transpositions difficult.

Instead I personally use Zobrist hash keys ( https://en.wikipedia.org/wiki/Zobrist_hashing ) which do not suffer from the above problems and let you handle transpositions effortlessly.

Also when you build a book, you add play counts for a given Zobrist key. So if a move was played 3 times in a position with a certain key, then play count will be 3 for this move and this key. However if you proceed with this move, you can encounter a new position, which was reached from positions and moves other than yours, then this position can have way more total play count, then your original move.

@sefulesefarka Thanks you for your input. It jumped right in. which i like, as it suggest avenues of learning.

i will look into the hash keys question and transposition, which appear to be a dynamic programming type problem. An efficient way of travelling though a data structure (tree here).

my focus was not a programming one initially, but more of a chess question (interpretation), but at some point learning more than you ask for, can help. Some questions are not clear enough to limit which angle to adopt.

To converge on terminology:

( possibly an ep square for which no ep capture is possible )
could you please clarify the concept of ep, is it an abreviation? i really don't get the parenthesis.
Also, FEN's do not specify the half-move? what about the last digits (i have not read the specs).

Is lichess opening based on a book? book means database? or it is a wider concept. a data structure.

So my understanding was ok from the user point of view?
Are you saying that there are ambiguities in the opening explorer results because they use FEN as id?

I may be confused. thanks for replying if you have the energy. I may read through the link provided though. I did some computer science in the past, might come back to me.

@sefulesefarka

I get it, EP: en passant. One more source of ambiguity for the tables behind explorer.

EP should have been obvious (french is my 1st language), but in the context ...(i must be a robot)

I will come back here, as i learn more toward understanding your post, and my nebulous question.

hoping it does not get closed for time limit.

All these ideas are implemented in Pgn Editor [1]:

https://pgneditor.herokuapp.com

You can analyze positions using a cloud engine. This is unlike lichess which lets you analyze using your own client ( browser ) for running the engine. The engine runs on the server, so after starting an analysis, you can turn off your computer, and it will keep going on the server and keep storing analysis results. Of course there is only one engine instance, so if an other user requests analysis, this may interrupt yours. The evaluations are auto stored under the Zobrist key of the position. For this a Firebase real time database is used, which is essentially a large JSON blob ( Firebase has databases of data models other than realtime database, like Firestore, which is a document/collection based database, similar to MongoDb, and a storage database for storing large binary files like images ). The advantage of the JSON data model is that you can both set a key independently using a path, so you can say store the analysis of a single atomic position under path "analysis/atomic/[Zobrist-key of the position]", but you can also access the whole atomic book as a single blob under path "analysis/atomic", in the latter case you will get a dictionary in which the keys are the Zobrist keys and the values are the JSON representation of the analysis results.

[1] https://lishadowapps-blog.netlify.com/posts/pgn_editor

@dboing
"It is allowed in correspondance games."

No, it is cheating. Read the FAQ.

@Doofenshmirtz

I'm sorry for you. Perhaps, there should be a sub type of correspondance, for those who want to be rated on their rote memory skills. The 360 minutes category of games should suit you, otherwise.

I like to have books of openings at hand, so that the game is the best possible (not me, the game). remember that both players have access to those books. What is unerving about cheating is that one does not know if playing with a human or a machine. or that the game is not fair. Also, using the explorer blindly will not make you win, you need to understand what is happening, and the meaning of the data tables (books) presented. Books with very precise table of content, or index, if you want an analogy with the paper world.

Book assited player, i am. Engine assisted player, i am not. Perhaps you should expand your cheating vocabulary, might be less emotively charged. Less sinful.

Anyway, you have already started other threads to make your point. This is not the main topic here. Your objection is noted. I hope that i was careful in my response. Thank you for your contribution.

@Doofenshmirtz

FAQ: Is correspondence different from normal chess?

On Lichess, the main difference in rules for correspondence chess is that an opening book is allowed. The use of engines is still prohibited and will result in being flagged for engine assistance. Although ICCF allows engine use in correspondence, Lichess does not.

End of quote.

I guess i should have said book instead of books. And the opening explorer seems to constitute one book.