@StrongPaleAle said in #9:
> Ok, "a loss is a loss" ...but why did the engine choose the fifth best line as the solution to deliver on the puzzle?
When several moves are close, what is seen as the best line will depend on different things such as exact engine version and depth it searches at. If using multithreading, it can give different results even at fixed depth with the same version.
The only reasonable way to properly fix these edge cases would be to have an engine predicting potential human moves (something along the lines of maia), then combining it with Stockfish using some kind of averaging search to maximize the odds of human failure until the end of a forced sequence. But that's a lot of work (and complication of actually running puzzle generation) for the benefit.
Don't get me wrong, I would like to see it done. But it's very understandable that it's far from a priority.
I'd argue there would be less work and more benefit looking at introducing other types of puzzles, such as expanding the "save equality puzzles" they exist but are rather hidden and limited, or introducing positions with multiple solutions (each of which is rated as a success) , as while that limits strict categorization of what a puzzle is about, it does open more type of tricky positions ; or even simply have completely normal positions (quiet and tactical), and just be rated on how close the picked move is in acpl to the engine-determined best move.
Or introducing variant puzzles (especially 960 puzzles).
> Ok, "a loss is a loss" ...but why did the engine choose the fifth best line as the solution to deliver on the puzzle?
When several moves are close, what is seen as the best line will depend on different things such as exact engine version and depth it searches at. If using multithreading, it can give different results even at fixed depth with the same version.
The only reasonable way to properly fix these edge cases would be to have an engine predicting potential human moves (something along the lines of maia), then combining it with Stockfish using some kind of averaging search to maximize the odds of human failure until the end of a forced sequence. But that's a lot of work (and complication of actually running puzzle generation) for the benefit.
Don't get me wrong, I would like to see it done. But it's very understandable that it's far from a priority.
I'd argue there would be less work and more benefit looking at introducing other types of puzzles, such as expanding the "save equality puzzles" they exist but are rather hidden and limited, or introducing positions with multiple solutions (each of which is rated as a success) , as while that limits strict categorization of what a puzzle is about, it does open more type of tricky positions ; or even simply have completely normal positions (quiet and tactical), and just be rated on how close the picked move is in acpl to the engine-determined best move.
Or introducing variant puzzles (especially 960 puzzles).