Expected Score of Grandmasters based on Evaluation

I’ve often used the win percentage and accuracy functions provided by Lichess in the past to analyse games classical grandmaster games. But I was never quite happy because these functions were created to be used for a different purpose.
As Lichess uses them to analyse online games mostly played by amateur players, the functions are based on games by 2300 rated players on Lichess. This means that they will underestimate the winning chances by GMs in classical games. So I decided to find a function that better approximates the expected score of GMs given their advantage calculated by Stockfish.

Setup

Firstly I needed to analyse games using Stockfish to see how well grandmasters score in practical games given a certain advantage. I analysed around 4,500 games played in the last two years, taken from top tournaments, big open tournaments and the chess olympiad. I then only considered games played where one player was rated over 2500 and the rating difference between the two players was less than 50 points to avoid games where one side had a big rating advantage.
Finally I calculated how well the players scored in games where they reached a position with a certain advantage according to Stockfish.

Lichess Win Percentage

To calculate the expected score given an evaluation by Stockfish, Lichess uses the following formula:

where cp is the centipawn advantage calculated by Stockfish and they use the parameter k=0.00368208.
Plotting this functions together with the actual score of grandmasters when having a certain advantage according to Stockfish shows that the approximation by Lichess is based on weaker players.

While the general shape of the scores in grandmaster games follows a similar arc to the function. it’s clear that the function should be steeper to better approximate the scores for high level games. This is due to the fact that GMs convert an objectively better position more often and hence have a higher expected score when they have an advantage of +1.

Better Approximations

When I tried to find a function that better fits the data, the most obvious place to start was to take the same type of function but make it steeper. This means increasing the value of the parameter k until the new curve approximates the data from GM games well.
To find the best value for k, I minimised the mean squared error between the function and the datapoints gathered from the GM games. This lead to k=0.007545 and the following graph:

While this function approximates the score in grandmaster games much better, it overestimates the expected score for high evaluations. This means that the expected score in positions with an advantage of +3 or more get overestimated compared to the actual points scored in grandmaster games.
To address this issue I wanted to use a different function that has a similar shape, but is more “spread out”. After some experimentation, I decided to use the following function based on the arctan:

Again, I fitted this function to the data I got from the GM games by minimising the mean squared error. This gave k=0.007851 and the following graph:

This function is much closer to the average score in the GM games for high evaluations. While it’s slightly underestimating the expected score in much better positions, I think that this approach is better than overestimating the score since everyone can mess up a winning position.

Accuracy

Lichess uses its formula for the expected score to calculate the accuracy of the players. So I also want to look into adapting the accuracy for GM games in a future post. I hope that the accuracy percentage will then actually reflect the number of times that GMs play a move with this accuracy.
Let me know what you think about my approximations for the expected score.

If you enjoyed this post, check out my Substack.