@ProgrammerAngrim Thank you for your good reply. Lots of clues from precision. Good understanding of what i might be missing.
Same exercise for me. How much of my previous fog remains? What of my previous guesses/questions remains, that I can figure out?
Rephrasing and recapping with my own words (not exclusively). also trying to deduce from your clues if options 1 and 2 are still plausible (compatible with your input). working on text file. as it can get ugly. wip. be back before archiving..
News:
I was getting pretty much satisfied with the Hash table explanation for lichess PV display of short PVs. Thinking i could start using that in interpreting engine analyses scores for what it really is, a remote position static evaluation, that I need to also consider if I want to use such analysis to understand any position from my games.
Trying to see the same thing as engine at current position, when itself is not doing any of that, but assigning what it actually saw on a remote position (that ought to be accessible along with score), is not really logical to me.
I would rather learn to see what it sees at that cherishable positions that made its containing branch the winner of the search. I think that should be not weird if analysis and learning with help of engine is valued. Like a newspaper editorial. Knowing who owns the paper, might be of some analysis value. Or its official stances on certain public issues.
So accident: snag. obstacle on the merry way to chess analysis by engine score....
I recently encountered 2 dissonant behaviours from a position in a study chapter.
turning on only manual engine or off. no plus. just 5 line display.
First problem: at 22 settled the first go. I get 5 lines all at 16 plies depth. not anywhere close to 22.
Second problem: if pushing root along one of the PVs, and going back to same exact position, drops the PV down to 6.
1) Is that lichess or SF?
2) What are the chances that all 5 displayed lines are short PV of the same length due to hash table transposition
(from within the same search)? I think very slim. this mean another type of cutoff. (back to 1).
Which brings another question about study context, manual engine trigger, and whole game sequence engine request, differential context effect of that short PV story.
and the Hash table persistency, in and across those context, possibly even cloud context:
Is there a mega cloud lichess hash table involved, even when i did not request any cloud (and local displayed).
Actually, maybe cloud means that:
A concatenation or merge of many hashtables. perhaps similar code as within one root SF tree search, or many roots tree searches.
The first problem is what puzzles me most.
The second could actually mean an engine toggle independent local SF storage for a hash table that persist across many calls. (don't know when I would get back my initial 16. browser reboot?). The closeby extra engine manual tree search, might likely have created more hash table entries that were actually encountered earlier in the previous root tree search. only then, and there, positions at plys depth shorter than horizon were not even considered for evaluation, one has to wait to the exact depths (modulo some quiescence test, and then material balance for NNue vs classical split). That would be the case with an empty hash table. (what is the hash table local life-cycle). This could be data about the prevalence of transposition between contiguous roots respective tree searches.
I went from 16 plies down to 12. I wonder if I were to explore around the best PV, if I could start saturating the hash table with positions from the very first root? who cares? i would if it could help me understand else where.
In any case. the 16 acroos all 5 not 22 with empty (?) hash table (unless same root tree search transpositions, all at 16 plies)., is the big question that i have no clue about. does any one. or is that 16 plies coincidence explanable?
I don't think the 16 versus 22 is explainable by UCI. IF you think so. please more clues.
But thinking about UCI angle more. 2004. But I know that SF has non-UCI command line access (and that SF14 has even improved on that front). I would suppose that lichess would also have such access.
It is not clear to me, what is not explicit in UCI. there is an entry about PV output. but I think it vaguely refers to string data. I don't recall specs on size limitation.
Anyone with UCI angle arguments to add?
PS:memo: find links to UCI PV entry. SF non-UCI. and lichess short PV closed issue.
groups.google.com/g/fishcooking/c/jFKq-aKJOu0/m/q6TbQuaZYGcJexcerpt from the beginning of thread (there was a branch associated I would need to read that long discussion to find out if ever implemented and what it was fixing that may actually be still there if never merged into main or master).
Oct. 23, 2014, 4:27:08 p.m.
to fishc...@googlegroups.com
(clipped branch github link)
This allows SF to report accurate PVs up to the tip of the search, so we can see the position the evaluation actually came from. The current implementation is a bit messy, storing the pv array in the Stack structure, and doing some hacks in Split nodes (PV is not totally accurate with threads > 1 yet, haven't tracked this down...). I haven't done a clean-up pass yet, this is just a proof-of-concept to see how hard it would be.
End of excerpt.
So when I say leaf, and static evaluation. I think here the word is tip (which would fit with a tentacle tree of future positions, with tactile receptors only at the tips). The octopuss is actually having sensory tissue all along .. but I think here the image is multiple human hands searching in the dark.... So tips only real source of information the rest being a compilation from the many other arms, themselves providing only tip information.
Edit: end of discussion there.
(Nov 26, same year)
It didn't pass regression tests unfortunately. I will take a look at it again soon.
Anyone with post-mortem resurrection news. Perhaps another implementation was not a "regression". (engine X engine time constraints lurking...?).
I guess pull request must come from issues and somewhere in there the case must be made that explains what this branch tried to improve. them maybe github history tools focusing on that could show if untouched to this day?
Or would I miss other workarounds. I assume no obsolete code is left hanging like "junk" DNA in living organisms...
I wonder if it is hard to implement or always getting killed by regression tests. If the regression tests failure are purely about time constraint differences. then could there be an analysis mode with different regression criteria (as here the speed criterion would be about human interactive time span, not engine X engine).
Anyone know why this is hard, or if the problem has been solved. or the lastest failed attempt after above?
The patch here works in getting the full but not full PV being displayed. but doubts about them being the actual tips.
UCI excerpts about PVs (may have left out relevant stuff. from gitub stockfish readme
www.shredderchess.com/download/div/uci.zip.
* info
.... (describing the info command)
Additional info:
....... (about ongoing search)
* pv <move1> ... <movei>
the best line found
* multipv <num>
this for the multi pv mode.
for the best move/pv add "multipv 1" in the string when you send the pv.
in k-best mode always send all k variants in k strings together.
Also all infos belonging to the pv should be sent together
e.g. "info depth 2 score cp 214 time 1242 nodes 2124 nps 34928 pv e2e4 e7e5 g1f3"
This is all we get about PVs. I think. So, now left to compare those on command line and lichess study experiments from a given position. mentioned above.
Just to let readers know. that I will make an effort, once i get answers, in less shorthand vocabulary. now more about skeleton and journalling. allowing those with courage and time on their hand to find the flaws and gaps of mine. ... PV= Principal Variations, that are usually displayed in lichess analysis type of pages (post-game, study, correspondence, learning section, puzzles).
They can be set to display from 0 variations up to 5 by SF engine manual toggle at position of interest. The point of this whole thread, is to actually see which positions in those variations which have scores attached to them on the left of the PV display (from SF via lichess), are the actual only position per variation that has been evaluated by the SF static classical evaluation function of input position.
By AB engine tree search algorithm or design, only the final node (=position in tree search) from each branch (=sub-variation from upstream move decision point = branching point) being searched. In other other word it is the last position in the PVs supposed to be displayed, in the top lichess module above user variations module.
The bug object of this thread, which is finally confirmed still there, is that they are displayed shorter than the actual PV that were searched. and the Hash table argument explaining some of those shorter PVs, is not responsible, at this point of my experiments (and others interested by this), for the latest set of same length truncation at 16 plies (engine counts half-moves as ply or plies, more precise than move which can mean both move and half-move).
If anyone would like me to explain better, you are welcome to ask. not with generic "what are you asking exactly", that would be too vague.... sorry. btw not only asking, journal-ling and reporting.
Best, would be to ask for words I forgot to translate from algorithm or programming speak to chess or lichess user experience with analysis tools. This would be my first audience widening attempt. a good exercise even for own clarity of process.
Sounds like what you may be after is stockfish running locally inside a homebrew wrapper customized to talk to the engine the way you want it ... I'd be schocked if this has not already been done before ... probably at many different levels of expertise ...
yes that is the idea, to be able to compare with lichess on same position challenges. to see how the P.V. output differ, in which case, it would become a lichess issue. I could go wake up a closed issue with such data or ask here for the full display length of SF provided PVs that command-line SF allows (UCI or not).
It might be some other lichess UI constraint that comes into play. or that users rarely want to know where do the score actually come from. Which I doubt would be the case, if all the information would be easily accessible, and well explained (I may not do the best job yet, first the information is needed).
This question is even more interesting since NNue is sharing some of those sparse evaluations with the classical evaluation function, for those remote position where classical eval would not see any imbalance (or so little). NNue apparently being trained to mimic a moderate depth search from classical eval (as far as i can infer from dispersed documentation on NNue hybridization with SF).
There is a 2 step process involving both classical evaluation. during engine use by us, and prior to SF14 production, NNue trained to act like no tree search predictor of tree search classical eval. This double usage of classical evaluation and correct interpretation, is going to make understanding which exact remote position in the search are being used as input to the those remote position scoring functions.
But that is extra motivation. even without NNue in the picture. I think it would help better interpret engine score attributed to a user currently looking at a position (realizing that this position is not actually being evaluated, only inheriting from the actual remote ones.
Perhaps the lichess UI was adapted to some older less complete output version of SF. anyway. I would get justification for this thread as feature request. The patch to dump engine PV display to user variations is half way to the full PV, it is a great step already. Now, the few missing plies to figure out there disappearance.
At the same time, one can explore through hash table reset command, at a fine grain level that happens within search for given root, and across searches for deeper root positions (from which to call the engine tree search and evaluation process). That would help understand how lichess uses such Hash table or other cache in its tab sessions in analyses pages. But that is secondary. We have to figure out how Hash table transposition can justify shorter PV display to distinguish from the cases where it is not the reason.
