The mechanics and maths of the glicko2 system

""" If the cheaters quickly populate the top ranks, then the majority of their games will be against each other """.
No not like that, cheaters generally avoids cheaters so in my simulation I will put some limits on pairing them.

In fact I plan to capture a cheater randomly, and introduce a new cheater randomly too. Similar to Lichess cheaters do not exist forever, sooner or later they will be caught. But there are still cheaters in the system.

""" At this point, you cannot simply assume a 50% win rate between cheaters. The one with a better engine will win 100% of the time. """
Problem is both cheaters usually have the same engine (the strong free SF), probably different hardware though. But I think a 50% is a reasonable estimate.

""" How will you account for game selection? People will play those that are similar rank themselves. """ No not like that, I myself played in tournaments here, My rating is 1900+ and I get paired to as low as 1400+, couple of 1500+, 1600+. But generally I will use the rating interval as a guide in the pairing but not always, I will do a random match for as low as 400 rating difference too. """ If the cheaters quickly populate the top ranks, then the majority of their games will be against each other """. No not like that, cheaters generally avoids cheaters so in my simulation I will put some limits on pairing them. In fact I plan to capture a cheater randomly, and introduce a new cheater randomly too. Similar to Lichess cheaters do not exist forever, sooner or later they will be caught. But there are still cheaters in the system. """ At this point, you cannot simply assume a 50% win rate between cheaters. The one with a better engine will win 100% of the time. """ Problem is both cheaters usually have the same engine (the strong free SF), probably different hardware though. But I think a 50% is a reasonable estimate.

meeeep

#12

Rating inflation is taken to mean how much rating a player with constant strength (eg. an AI) is expected to gain. It does not have anything to do with percentiles and rankings. Rating inflation may not be even across the board (lower ratings could possibly inflate more or less than higher ratings). Either way, such rating systems are not fully accurate, and get progressively less so at the extremes (A 600 rated player on chess.com will probably be expected to win rating off Nakamura in bullet by playing random moves and hoping Nakamura's internet dies, for example), so there's probably no need to read too deep into the absolute meaning of ratings.
No idea. Post #3 offers a good guess.
It probably lowers the ratings for legitimate players -- if you win games off others, their rating goes down. However, it might not be as much as expected since initially the cheater's RD is high, and his rating adjusts quickly, so he reach his "correct" rating without too many net wins over other players. Add this to the fact that cheaters are outnumbered and I'd guess they don't have that much of an impact.
I find this a little strange, since ratings should deflate if people improve lots from experience. This improvement is particularly prominent in atomic. A clueless newbie might start a game as black with 1...Nf6 in response to Nf3. After losing to 2. Ne4, he might reconsider his response to white's first move. This could represent a 100 Elo point gain or something ridiculous. So what this means is that people generally start off at a low rating (1000 for example), and quickly learn the basics, shooting up to 1500 in 50 games (again unreliable numbers). So what happens is that imaginary newbie loses 500 points in his first 5 games (giving perhaps 30 rating points to older players) and then steals maybe 400 from his climb back towards 1500. This should result in rating deflation. Instead we're experiencing the opposite. I think it's weird.
As explained above I think it should deflate the ratings on principle. I have no idea whether the effect is observed.

"5") Regarding the simulation, no simulation can take into account everything that happens on the ground. So the most accurate data has to come from lichess.org. The problem is that there is nothing to compare it to (since we want to observe the effects of, say, no cheaters). However, my guess is that any reasonable simulation wouldn't be too far off the mark, especially when trying to find out the net effect of a group of players on the rating pool.

0) Rating inflation is taken to mean how much rating a player with constant strength (eg. an AI) is expected to gain. It does not have anything to do with percentiles and rankings. Rating inflation may not be even across the board (lower ratings could possibly inflate more or less than higher ratings). Either way, such rating systems are not fully accurate, and get progressively less so at the extremes (A 600 rated player on chess.com will probably be expected to win rating off Nakamura in bullet by playing random moves and hoping Nakamura's internet dies, for example), so there's probably no need to read too deep into the absolute meaning of ratings. 1) No idea. Post #3 offers a good guess. 2) It probably lowers the ratings for legitimate players -- if you win games off others, their rating goes down. However, it might not be as much as expected since initially the cheater's RD is high, and his rating adjusts quickly, so he reach his "correct" rating without too many net wins over other players. Add this to the fact that cheaters are outnumbered and I'd guess they don't have that much of an impact. 3) I find this a little strange, since ratings should deflate if people improve lots from experience. This improvement is particularly prominent in atomic. A clueless newbie might start a game as black with 1...Nf6 in response to Nf3. After losing to 2. Ne4, he might reconsider his response to white's first move. This could represent a 100 Elo point gain or something ridiculous. So what this means is that people generally start off at a low rating (1000 for example), and quickly learn the basics, shooting up to 1500 in 50 games (again unreliable numbers). So what happens is that imaginary newbie loses 500 points in his first 5 games (giving perhaps 30 rating points to older players) and then steals maybe 400 from his climb back towards 1500. This should result in rating deflation. Instead we're experiencing the opposite. I think it's weird. 4) As explained above I think it should deflate the ratings on principle. I have no idea whether the effect is observed. "5") Regarding the simulation, no simulation can take into account everything that happens on the ground. So the most accurate data has to come from lichess.org. The problem is that there is nothing to compare it to (since we want to observe the effects of, say, no cheaters). However, my guess is that any reasonable simulation wouldn't be too far off the mark, especially when trying to find out the net effect of a group of players on the rating pool.

jimj12

#13

@Meep

And how do you think you will measure constant strength? That was the entire point of introducing percentiles. They offer a measure of 'true' strength even as rating changes; over a decently sized population, the top X% will be expected to remain the top X% if there is a systematic (see: inflation) change to ratings.

@Meep And how do you think you will measure constant strength? That was the entire point of introducing percentiles. They offer a measure of 'true' strength even as rating changes; over a decently sized population, the top X% will be expected to remain the top X% if there is a systematic (see: inflation) change to ratings.

jimj12

#14

#11

""I myself played in tournaments here""
Ok, I did not think about tournaments. But then, you need to figure out what percentage of games tournaments account for on Lichess and still have to account for the selection effect on all other games (since I think at least as many games occur outside tournaments)

""No not like that, cheaters generally avoids cheaters""
How would they know who's cheating? And is there really any evidence for this? Consider a cheater that loses to someone. They could be vsing a very strong legit player and would want to try beat them (since presumably the only point of cheating is to beat players stronger than you). Thus, they would be inclined to keep vsing this cheater who is a better cheater than them, rather than avoid.

"probably different hardware though. But I think a 50% is a reasonable estimate."
Well then the one with better hardware will win 100% of the time. I think it is reasonable to assume that the ones with higher ranks after many games are the ones with better engines. Now, maybe when a cheater plays another that is a similar rank 50% is alright. But otherwise, I don't think so - beyond the provisional games of course.

#11 ""I myself played in tournaments here"" Ok, I did not think about tournaments. But then, you need to figure out what percentage of games tournaments account for on Lichess and still have to account for the selection effect on all other games (since I think at least as many games occur outside tournaments) ""No not like that, cheaters generally avoids cheaters"" How would they know who's cheating? And is there really any evidence for this? Consider a cheater that loses to someone. They could be vsing a very strong legit player and would want to try beat them (since presumably the only point of cheating is to beat players stronger than you). Thus, they would be inclined to keep vsing this cheater who is a better cheater than them, rather than avoid. "probably different hardware though. But I think a 50% is a reasonable estimate." Well then the one with better hardware will win 100% of the time. I think it is reasonable to assume that the ones with higher ranks after many games are the ones with better engines. Now, maybe when a cheater plays another that is a similar rank 50% is alright. But otherwise, I don't think so - beyond the provisional games of course.

RealKool

#15

@12
""" "5") Regarding the simulation, no simulation can take into account everything that happens on the ground. """
Of course we can agree on that.

@12 """ "5") Regarding the simulation, no simulation can take into account everything that happens on the ground. """ Of course we can agree on that.

RealKool

#16

@14

""" But then, you need to figure out what percentage of games tournaments account for on Lichess and still have to account for the selection effect on all other games (since I think at least as many games occur outside tournaments)"""

Regarding tournament games percentage I have an estimate, there is a menu play->tournaments, there are schedules in there that I can use.

Regarding the latter as I said the rating interval will take care of this and some randomness. Given 2 players the pairing algo will give higher pairing priority points for players that are close in strength.

""" How would they know who's cheating? And is there really any evidence for this?"""

Cheaters are not stupid, they also have a cheater protocol.

It is easy to detect a cheater, the only question is how many games are you going to consider before you sentence a player 20, 30, 50 games? There are players that deliberately cheats, 1, 2, 4 or 6 games for curiosity, after that they play normally.
Join the CIA team there are techniques there to detect cheaters. But the site has all the data to detect cheaters with very good accuracy.

""" Well then the one with better hardware will win 100% of the time."""
As I understand you don't know much about computer chess sorry. Never ever say 100% win. Try to visit or join chessbase's playchess, try to meet there some Centaurs - a human player that plays with computer assistance, or a player that allows a computer to play all its moves. One more, opening book and endgame tables are also a factor.

@14 """ But then, you need to figure out what percentage of games tournaments account for on Lichess and still have to account for the selection effect on all other games (since I think at least as many games occur outside tournaments)""" Regarding tournament games percentage I have an estimate, there is a menu play->tournaments, there are schedules in there that I can use. Regarding the latter as I said the rating interval will take care of this and some randomness. Given 2 players the pairing algo will give higher pairing priority points for players that are close in strength. """ How would they know who's cheating? And is there really any evidence for this?""" Cheaters are not stupid, they also have a cheater protocol. It is easy to detect a cheater, the only question is how many games are you going to consider before you sentence a player 20, 30, 50 games? There are players that deliberately cheats, 1, 2, 4 or 6 games for curiosity, after that they play normally. Join the CIA team there are techniques there to detect cheaters. But the site has all the data to detect cheaters with very good accuracy. """ Well then the one with better hardware will win 100% of the time.""" As I understand you don't know much about computer chess sorry. Never ever say 100% win. Try to visit or join chessbase's playchess, try to meet there some Centaurs - a human player that plays with computer assistance, or a player that allows a computer to play all its moves. One more, opening book and endgame tables are also a factor.

jimj12

#17

""It is easy to detect a cheater"'
When I said 'they' I was referring to the cheaters themselves. And this 'cheater protocol' do you have any actual proof?

Obviously I don't mean 100% literally. What I am saying is that the cheaters with a higher rating after many games will systematically beat those with lower ratings, because all else being equal, they are better cheaters

""It is easy to detect a cheater"' When I said 'they' I was referring to the cheaters themselves. And this 'cheater protocol' do you have any actual proof? Obviously I don't mean 100% literally. What I am saying is that the cheaters with a higher rating after many games will systematically beat those with lower ratings, because all else being equal, they are better cheaters

RealKool

#18

Started slowly this simulation thing.
A. Create a function to generate random players.
init_simulation()

Create random players including cheaters (around 2%) of total players, each with id, name, and starting R, RD, V.

I wanna make sure that the generated random player names have no duplicates.
names are from letters and numbers, excluding l, o, 0, and letter are all in upper case. name length will vary from 6 to 12 characters.
Ratings are generated randomly from 1500 to 2300 and RD from 50 to 350.

However during actual simulation start once there are duplicate names, the app will re-generate again.

Testing at 10, 000 players, here is the sample run. More test will be done later.

Under column ch, there are two values N and Y, if Y then the generated name is a cheater
A. All players:
id name rating rd vol ch
1 US5XNU 1917.0 251.0 0.05000 N
2 1VNXGLV 2143.0 122.0 0.05650 N
3 4X6N73H 2207.0 203.0 0.05000 N
4 81KUNVD5S5AB 1549.0 144.0 0.05650 N
5 1LRFZ4JCS 2038.0 252.0 0.05650 N
6 AVUVCW23 1859.0 115.0 0.05348 N
7 4RNQ22SYZ 1674.0 212.0 0.05348 N
8 VKCR4T 1538.0 245.0 0.05076 N
9 GYRB2BP6QM 1830.0 218.0 0.05348 N
10 C729J3M6 2050.0 292.0 0.05348 N
11 VXB465NR 1897.0 254.0 0.05650 N
12 7PPYVFEUXNY1 1569.0 200.0 0.05076 N
13 ZP8C6M2JQQ9P 2134.0 264.0 0.05650 N
...
9997 PCYGETB54 2060.0 63.0 0.05000 N
9998 N5LRRGGR7W6U 2297.0 322.0 0.05348 N
9999 P9U2741W 1643.0 237.0 0.05000 N
10000 1277QATYKTF 1956.0 57.0 0.05348 N

B. Good players:
id name rating rd vol ch
1 US5XNU 1917.0 251.0 0.05000 N
2 1VNXGLV 2143.0 122.0 0.05650 N
3 4X6N73H 2207.0 203.0 0.05000 N
4 81KUNVD5S5AB 1549.0 144.0 0.05650 N
...

C. Cheater players:
id name rating rd vol ch
37 1T4MR5GXXJU 1615.0 110.0 0.05650 Y
50 DFL81R92 1622.0 98.0 0.05650 Y
151 SVRXXCDKPS 2220.0 217.0 0.05076 Y
190 8BDZNJYU24 2233.0 283.0 0.05348 Y
302 X6A41ENM 1630.0 164.0 0.05000 Y
378 A275KEQUJLV 2289.0 344.0 0.05650 Y
390 WDH3FL 2002.0 338.0 0.05000 Y
...

D. Name duplicate check:
There are no duplicates in the generated names!!

E. Simulation starting data summary:
Total players: 10000
Good players : 9805 (98.05%)
Cheaters : 195 (1.95%)
Generation time: 67.7s

Started slowly this simulation thing. A. Create a function to generate random players. init_simulation() 1. Create random players including cheaters (around 2%) of total players, each with id, name, and starting R, RD, V. I wanna make sure that the generated random player names have no duplicates. names are from letters and numbers, excluding l, o, 0, and letter are all in upper case. name length will vary from 6 to 12 characters. Ratings are generated randomly from 1500 to 2300 and RD from 50 to 350. However during actual simulation start once there are duplicate names, the app will re-generate again. 2. Testing at 10, 000 players, here is the sample run. More test will be done later. Under column ch, there are two values N and Y, if Y then the generated name is a cheater A. All players: id name rating rd vol ch 1 US5XNU 1917.0 251.0 0.05000 N 2 1VNXGLV 2143.0 122.0 0.05650 N 3 4X6N73H 2207.0 203.0 0.05000 N 4 81KUNVD5S5AB 1549.0 144.0 0.05650 N 5 1LRFZ4JCS 2038.0 252.0 0.05650 N 6 AVUVCW23 1859.0 115.0 0.05348 N 7 4RNQ22SYZ 1674.0 212.0 0.05348 N 8 VKCR4T 1538.0 245.0 0.05076 N 9 GYRB2BP6QM 1830.0 218.0 0.05348 N 10 C729J3M6 2050.0 292.0 0.05348 N 11 VXB465NR 1897.0 254.0 0.05650 N 12 7PPYVFEUXNY1 1569.0 200.0 0.05076 N 13 ZP8C6M2JQQ9P 2134.0 264.0 0.05650 N ... 9997 PCYGETB54 2060.0 63.0 0.05000 N 9998 N5LRRGGR7W6U 2297.0 322.0 0.05348 N 9999 P9U2741W 1643.0 237.0 0.05000 N 10000 1277QATYKTF 1956.0 57.0 0.05348 N B. Good players: id name rating rd vol ch 1 US5XNU 1917.0 251.0 0.05000 N 2 1VNXGLV 2143.0 122.0 0.05650 N 3 4X6N73H 2207.0 203.0 0.05000 N 4 81KUNVD5S5AB 1549.0 144.0 0.05650 N ... C. Cheater players: id name rating rd vol ch 37 1T4MR5GXXJU 1615.0 110.0 0.05650 Y 50 DFL81R92 1622.0 98.0 0.05650 Y 151 SVRXXCDKPS 2220.0 217.0 0.05076 Y 190 8BDZNJYU24 2233.0 283.0 0.05348 Y 302 X6A41ENM 1630.0 164.0 0.05000 Y 378 A275KEQUJLV 2289.0 344.0 0.05650 Y 390 WDH3FL 2002.0 338.0 0.05000 Y ... D. Name duplicate check: There are no duplicates in the generated names!! E. Simulation starting data summary: Total players: 10000 Good players : 9805 (98.05%) Cheaters : 195 (1.95%) Generation time: 67.7s

RealKool

#19

Optimizing the function by not printing the data (but are saved in memory) and generating 500K players, I get this.

D. Name duplicate check:
There are no duplicates in the generated names!!

E. Simulation starting data summary:
Total players: 500000
Good players : 490045 (98.01%)
Cheaters : 9955 (1.99%)
Generation time: 18.4s

So I guess this function is doing its job.
End of Test.

Optimizing the function by not printing the data (but are saved in memory) and generating 500K players, I get this. D. Name duplicate check: There are no duplicates in the generated names!! E. Simulation starting data summary: Total players: 500000 Good players : 490045 (98.01%) Cheaters : 9955 (1.99%) Generation time: 18.4s So I guess this function is doing its job. End of Test.

RealKool

#20

@17

""" And this 'cheater protocol' do you have any actual proof?"""

If I have and show to you, somebody will destroy me, if I don't have you will just say I am stupid making up things.
So I guess we better stop this non sense you asking for proof. Hint Computers were already very strong in year 2005.

@17 """ And this 'cheater protocol' do you have any actual proof?""" If I have and show to you, somebody will destroy me, if I don't have you will just say I am stupid making up things. So I guess we better stop this non sense you asking for proof. Hint Computers were already very strong in year 2005.

This topic has been archived and can no longer be replied to.