- Blind mode tutorial
lichess.org
Donate

DANGER, rating system will break if bots play rated !

#21

since the day that human vs bot matches were made casual from the human's side there are already less and less high rated players playing Leela, the ones who actually stood a realistic chance for at least a draw - while cutting of rating changes was the easy solution I think this will just end up essentially killing off the whole bot API idea as it will become irrelevant, I think it would have been wiser to wait and see and/or implement additional safeguards against potential abuse, like banning bots whose operator allows sustained abuse (like playing same game over all the time for bots that do not have sufficient randomness) - or just even a more simple one just flagging users who gain say more than 150 points against bot/bots for moderator's to review, the bots and active bots will always stay relatively few owing to programming work involved to set them up, this could not have possibly caused the rating apocalypse as suggested, and saying that ICC has super high ratings because of bots is unsubstantiated, perhaps there is just entirely different scale there, like gaining 2000 points is same as gaining 1000 here etc. that is if bots there allowed to affect player ratings at all which I do not know

since the day that human vs bot matches were made casual from the human's side there are already less and less high rated players playing Leela, the ones who actually stood a realistic chance for at least a draw - while cutting of rating changes was the easy solution I think this will just end up essentially killing off the whole bot API idea as it will become irrelevant, I think it would have been wiser to wait and see and/or implement additional safeguards against potential abuse, like banning bots whose operator allows sustained abuse (like playing same game over all the time for bots that do not have sufficient randomness) - or just even a more simple one just flagging users who gain say more than 150 points against bot/bots for moderator's to review, the bots and active bots will always stay relatively few owing to programming work involved to set them up, this could not have possibly caused the rating apocalypse as suggested, and saying that ICC has super high ratings because of bots is unsubstantiated, perhaps there is just entirely different scale there, like gaining 2000 points is same as gaining 1000 here etc. that is if bots there allowed to affect player ratings at all which I do not know
#22

#21 "I think it would have been wiser to wait and see and/or implement additional safeguards against potential abuse, like banning bots whose operator allows sustained abuse (like playing same game over all the time for bots that do not have sufficient randomness) - or just even a more simple one just flagging users who gain say more than 150 points against bot/bots for moderator's to review,"

You think we already don't have all of those safeguards and more? We aren't chess.com .

#21 "I think it would have been wiser to wait and see and/or implement additional safeguards against potential abuse, like banning bots whose operator allows sustained abuse (like playing same game over all the time for bots that do not have sufficient randomness) - or just even a more simple one just flagging users who gain say more than 150 points against bot/bots for moderator's to review," You think we already don't have all of those safeguards and more? We aren't chess.com .
#24

i think lichess has obviously better cheating safeguard measures than chess.com

i think lichess has obviously better cheating safeguard measures than chess.com
#26

@Toadofsky

I disagree that there's any more potential for abuse against bots than there is against willing human opponents; and safeguards are already in place to both report and detect boosting or sandbagging.

Is it considered boosting to play always the same line against bot that is known to lead into winning position due to the bot weak opening knowledge? In case of the human opponent it would not, but going to this losing line and not trying to change anything would be a clear way of sandbagging. For bot it is not clear who is to blame if such a thing happens.

Moreover, there could be some weird bots who do things like "play the second or the first line of Stockfish with 50/50 probabilities". This seems to be one of the natural ways to reduce the strength of an engine and create and interesting bot, so hardly can be threated as sandbagging. At the same time, the human who just play usually against such engine and the human who tries to exploit its peculiarities (creating positions where the second move is much worse) will have completely different performance. And again, exploiting weaknesses of your opponent can hardly be called boosting.

@Toadofsky > I disagree that there's any more potential for abuse against bots than there is against willing human opponents; and safeguards are already in place to both report and detect boosting or sandbagging. Is it considered boosting to play always the same line against bot that is known to lead into winning position due to the bot weak opening knowledge? In case of the human opponent it would not, but going to this losing line and not trying to change anything would be a clear way of sandbagging. For bot it is not clear who is to blame if such a thing happens. Moreover, there could be some weird bots who do things like "play the second or the first line of Stockfish with 50/50 probabilities". This seems to be one of the natural ways to reduce the strength of an engine and create and interesting bot, so hardly can be threated as sandbagging. At the same time, the human who just play usually against such engine and the human who tries to exploit its peculiarities (creating positions where the second move is much worse) will have completely different performance. And again, exploiting weaknesses of your opponent can hardly be called boosting.
#27

@Wolfram_EP Honestly this is what I'm most concerned about. So...

  1. The ToS say penalties are at the moderators' discretion. I can't define it more clearly than that.
  2. IMHO it's the fault of both the booster and the sandbagger when ratings manipulation occurs. IMHO both should be playbanned.
  3. Bots now have Polyglot opening books, created from PGN files. Bots weighted-randomly selecting between GM opening moves should now be easy; but either way sandbagging bots could be playbanned until bugs are fixed.
@Wolfram_EP Honestly this is what I'm most concerned about. So... 1. The ToS say penalties are at the moderators' discretion. I can't define it more clearly than that. 2. IMHO it's the fault of both the booster and the sandbagger when ratings manipulation occurs. IMHO both should be playbanned. 3. Bots now have Polyglot opening books, created from PGN files. Bots weighted-randomly selecting between GM opening moves should now be easy; but either way sandbagging bots could be playbanned until bugs are fixed.
#28

@BermudaGhostShip ,

The problem in skewing human rating system is not that individual players can benefit from that. The problem is much more serious. Let me explain... The link below has the human rating distribution for blitz game:

https://lichess.org/stat/rating/distribution/blitz

As you can see, it resembles much what we know in statistics as a "standard normal distribution", or "bell curve", whose center is precisely in 1500, which is the default starting rating of Lichess. This is ideal. However, if Lichess allowed bots to change human ratings, then that curve would shift dramatically and transform the chart into a double peaked (bimodal) distribution. This is what happened to ICC (chessclub.com).

Other things to note:

  • When Andrew Tang played Leela, his rating didn't change. Lichess had already ceased to allow that. I don't think it affected the public.

  • I don't think that beserking is any major problem for our rating system. It is a temporary drop in the rating of an individual player, but the points he lost were given to the public and tend to be eventually returned.

  • I agree that loosing lots of rating on purpose to then enter a U1800 tournament is sandbagging, but there are rules to prevent this behavior and I don't think computers make this any easier. If you loose against very high rated opponent will hardly affect your rating, regardless of it being human or bot.

  • Playing against regular computers is totally different than playing against humans. It is more like a training, similar to puzzle solving, but so mechanical that it hardly resembles real chess. Individual performance against computers will vary dramatically for each human.

@BermudaGhostShip , The problem in skewing human rating system is not that individual players can benefit from that. The problem is much more serious. Let me explain... The link below has the human rating distribution for blitz game: https://lichess.org/stat/rating/distribution/blitz As you can see, it resembles much what we know in statistics as a "standard normal distribution", or "bell curve", whose center is precisely in 1500, which is the default starting rating of Lichess. This is ideal. However, if Lichess allowed bots to change human ratings, then that curve would shift dramatically and transform the chart into a double peaked (bimodal) distribution. This is what happened to ICC (chessclub.com). Other things to note: - When Andrew Tang played Leela, his rating didn't change. Lichess had already ceased to allow that. I don't think it affected the public. - I don't think that beserking is any major problem for our rating system. It is a temporary drop in the rating of an individual player, but the points he lost were given to the public and tend to be eventually returned. - I agree that loosing lots of rating on purpose to then enter a U1800 tournament is sandbagging, but there are rules to prevent this behavior and I don't think computers make this any easier. If you loose against very high rated opponent will hardly affect your rating, regardless of it being human or bot. - Playing against regular computers is totally different than playing against humans. It is more like a training, similar to puzzle solving, but so mechanical that it hardly resembles real chess. Individual performance against computers will vary dramatically for each human.
#30

@relevantproblem I dunno, but getting an opening book which randomly plays 3-4 moves based on GM moves (or even based upon the thousands of ECO openings) should entirely prevent users from repeating an opening line against you.

@relevantproblem I dunno, but getting an opening book which randomly plays 3-4 moves based on GM moves (or even based upon the thousands of ECO openings) should entirely prevent users from repeating an opening line against you.

This topic has been archived and can no longer be replied to.