First of all this has little to do with engines playing. The games were mostly played by humans. The engines are there only to pick a random snapshot of the game and give an evaluation, then all the evaluations were grouped in 10 centipawn groups and the average win rate was calculated.
So to answer let's simply look at the graph and you can see that around 500 centipawn advantage usually results in above 95% conversion rate. Which is a very good approximation for a sure win.
"I believe, the graph should not be generalised", that's exactly what the graph is! A generalisation! A broadly valid one as far as I can see, he should have added error bars though. That's statistics for you and you can't argue with that.
What we can argue about is the specifics. So if you present me with an engine evaluation then from this graph I could give you a good guess on the result. If you provided me with additional information like the player ratings, we could filter out the data from our 750,000 games to gain further insight into this question. We would have data that better represents the situation at the cost of less data and thus a larger error.
Your example about Insights is a very good one to illustrate this. I have a win rate of 60% in classical, but in the B50 sicilian which is by far my best and most comfortable opening I am losing 67% of my games, does it mean I am terrible at it? No, the sample size is simply too small- 3 games. It's reminds me a little of quantum mechanics the better you know the position of a particle the less you know about it's momentum.
So there are clear trade-off's on how to segment/filter data. But statistics don't lie, we just need to be very pedantic and careful when interpreting it.
So to answer let's simply look at the graph and you can see that around 500 centipawn advantage usually results in above 95% conversion rate. Which is a very good approximation for a sure win.
"I believe, the graph should not be generalised", that's exactly what the graph is! A generalisation! A broadly valid one as far as I can see, he should have added error bars though. That's statistics for you and you can't argue with that.
What we can argue about is the specifics. So if you present me with an engine evaluation then from this graph I could give you a good guess on the result. If you provided me with additional information like the player ratings, we could filter out the data from our 750,000 games to gain further insight into this question. We would have data that better represents the situation at the cost of less data and thus a larger error.
Your example about Insights is a very good one to illustrate this. I have a win rate of 60% in classical, but in the B50 sicilian which is by far my best and most comfortable opening I am losing 67% of my games, does it mean I am terrible at it? No, the sample size is simply too small- 3 games. It's reminds me a little of quantum mechanics the better you know the position of a particle the less you know about it's momentum.
So there are clear trade-off's on how to segment/filter data. But statistics don't lie, we just need to be very pedantic and careful when interpreting it.
