Rating System Broken.

Well is the rating system broken?

I took the last 50 rated games of the first 3 players that complaint.
And calculated the performance elo, using this site: http://paxmans.net/performance_calc.php

What I got:

@Fischer_and_Chips88 Bullet performance 1727 vs his current rating of 1745
https://tinyurl.com/y8flabvw

@EnderShadow8 Blitz performance 1491 vs 1419
https://tinyurl.com/y8aprh9w

@n321 Blitz performance 1450 vs his current rating of 1348
https://tinyurl.com/yb5anobc

So even if we take a completely different way of rating, we are left in the same ballpark.

@BlackSalt

Better not to mix data from one to another system, we will get quite weird results and go nowhere with conclusions.

#101 Maybe if I just say, "The rating system is broken" people will be satisfied?

One thing which is strange about it is the definition of a "rating period" as set by system administrators as required by http://www.glicko.net/glicko/glicko2.pdf ... games during the rating period should be rated simultaneously, so the ordering of games played during that period should not impact the post-period ratings; but also each player's rating volatility (a measure of the predictive accuracy of the system, in that same paper) might have its accuracy impacted by having an abnormally low game count (1) per period. Thankfully Lichess does age (increase over time) player rating deviations, so ratings aren't completely fixed in place, and in practice ratings seem to fluctuate; that's not to say that improvements (especially those mentioned in the PDF and rigorously tested on other sites) cannot be discovered and applied through rigorous experimentation and observation.

I'm going to watch some more Cantrill talks since he describes just how rigorous the change control process can be...
https://www.youtube.com/watch?v=30jNsCVLpAE

@Toadofsky "Maybe if I just say, "The rating system is broken" people will be satisfied?"

Probably not. As I commented already, without clear rules of rating system and its parameters, all is still into deep "fog".

If you say:

• Player with the nearly same rating will get 4-6 points no matter what these ratings are
• Player with different rating will get points with this (whatever is) approx formula
• Player have ratio of W/L > 1, will get N% more points than usual, for W/L < 1 ...
• Player need to have standard deviation of N games larger than M .

Etc, then all will be much clearer. Otherwise we should "trust" this very complicated G2 system is implementated correctly and "fair". Key word is "transparency" of the implemented system, which in this case missing. All is quite arbitrarily and unknown.

FIDE rating system is at least clear.

BTW, Glickman have some JS example code of his system on the main page, then that may probably be used further for comparison.

I actually believe that G2 system is glorified and used because no body actually know "WTH exactly is happend under the hood". Slightly change in initial parameters is enough to make it useless, which is probably what is happening here.

#104 Here is Lichess' rating code (including parameter definitions):
https://github.com/ornicar/lila/tree/master/modules/rating/src/main

Here is an installation guide:
https://github.com/ornicar/lila/wiki/Lichess-Development-Onboarding

This system is not convenient or even close, but it is transparent... I offer no promises about "fairness" especially because ratings should measure performance, not skill. In addition the Glicko-2 PDF is a reasonable albeit imperfect description of Lichess' rating code. There are apparently about 100 reference implementations of Glicko-2 etc. as well:
https://github.com/search?q=glicko

and there is the game database with dates, player names, ratings, and results:
https://database.lichess.org

I have looked under the hood and there are specific concerns which I have stated already (perhaps not in this topic):

• Hyperparameter "tau" is not tuned per Glickman's suggestion
• Lichess defines a rating period differently than Glickman's suggestion (each game is treated as a rating period; consequently there are hacks in the code to cap a player's rating volatility)
• The rating refund program may be popular but is extremely complicated
• First-player advantage is not accounted for

@Toadofsky

I appreciate how much work you put in to explain the implementation of Glicko-2 in Lichess. Unfortunately this seemingly is not enough for some guys and they continue to claim there's something bad and/or foggy about it. And for what? Just because they only gain 4 - 8 rating points per game won instead of 15 - 20 as they seem to expect.

In my view the fixation with rating is quite unhealthy with some people. And no amount of explanations done will change that.

@Katzenschinken

You are missing the point. All is wrote here already. And #105 is what exactly was necessary for further research, comparison and testing.

@Toadofsky, thank you once again.

@n321

I stand by my comment. A lot of the Glicko-2 stuff was already mentioned in the thread but you continued to claim "foggyness" when in reality you are just unhappy with the amount of rating points awarded. Well, play more games then if you want to reach the rating adequate to your perceived skill a bit quicker.

@Katzenschinken

Please do not start personal accusations. Use PM instead.

Yes people's fixation on rating is unhealthy. Instead of trying to get better, which would improve their rating, they fixate on their rating and obsess over it, and even argue against the rating system as a whole for the reason that they don't have the rating that they want. The rating system is largely fair, and while it is not perfect, nobody failed to achieve certain milestones in chess because of their rating, for example beating this player or winning this tournament. You rating won't help you beat players, your skill and performance will, that is all. And working on your skill over time and then working on your performance, is the only thing that will improve your rating. You don't see any GMs complaining on here that the rating system here is unfair and they are not getting recognised for their skill, chess knowledge or execution because of their rating. They got their title through hard work, grit, not complaining that each win didn't give them enough rating points. That is all. Plus chess is fun; Why aren't you focused on having fun?