Last month, my roommate couldn't curse Lichess enough because he was marked cheating while he was INDEED cheating( He turned on analysis in a mobile app while losing and made many comebacks, that's a kind of cheating imo). But he argued that there wasn't enough evidence. My major is statistics (obviously from my user name) while his is probability. So we had an debate and I nearly unfriended him.
Background: in statistics, you can reject some hypothesis when its probability is less than a certain significance level, say, 5% or 1% or 0.1%. Rejecting a true hypothesis is called type II error. But in probability theory, events with exactly 0 probability could happen and is happening everywhere. As in his best example, choose a real number randomly from 0 to 1, every number has a 0 probability to get chosen but of course one number will get chosen. So any event with positive probability is 2 big steps away from impossible. But some events with extremely small probability is effectively impossible imo, the critical probability to identify an event as impossible should be no more than 10^-10^10 of course, but more than 10^-10^10^10^10.
In fact, I admit when it comes to cheating, there is a rule called presumption of innocence or something, but that rule makes no sense in statistics. Any hypothesis testing must allow some type II error, although in crime confirming there should be some more strict significance level, say, 10^-10^10. (If that's not enough, 10^-10^10^3 should be good enough). Confirming cheating in a online chess website, however, doesn't need such a strict significance level. 10^-10^3, maybe even 10^-100, should be enough. Nobody want to get wrongfully accused, but nobody want to play against a despicable cheater like him. So higher type II error should be bearable.
But here comes the real problem. With some easy mathematics, I analyzed his games, he made 0 mistakes and 0 blunders after 20 moves in 17 games. that's 408 moves. There are about 10 reasonable legal moves in every position, so the possibility is 10^-408, which is good enough for confirming that he is cheating. But think this through. In some positions of his games, every move is the same, like one K+R v K+R, eight K+Q v K+some pawns, four K+R+pawns v K+pawns, two K+B/N+more pawns v K+less pawns, one K+B+B+some pawns v K+N and one K+P v K+N. He can find the winning or drawing moves of course. That will make the probability up to 10^-197, and some moves are obvious enough, some moves are inaccurate, which might bring the probability above 10^-100. (And he argued that the significance level should be 10^-10^10 rather than 10^-100)
Lack of direct evidence (Of course, if you ignore me as a witness against him) is the main problem. You can only use probability as a piece of evidence, but . I think that's why cheaters are hard to get caught. He said he didn't get caught at several other chess websites, but I don't know the websites and accounts so I can't report them. I hardly know anything about online chess, and I play terrible chess. Another friend of mine taught me chess and I lost 10 straight to the cheater on another chess website named chess.com.(Of course he didn't need engine to beat me, I am really terrible.) After my report he was banned there yesterday as well, hooray! But now he plays on Lichess again and hides his account from me. I think there must be some other way to detect cheaters other than just using probability.
TL;DR: How do Lichess lower the possibility of false cheating accusation while catching enough cheaters? Is it possible?
Background: in statistics, you can reject some hypothesis when its probability is less than a certain significance level, say, 5% or 1% or 0.1%. Rejecting a true hypothesis is called type II error. But in probability theory, events with exactly 0 probability could happen and is happening everywhere. As in his best example, choose a real number randomly from 0 to 1, every number has a 0 probability to get chosen but of course one number will get chosen. So any event with positive probability is 2 big steps away from impossible. But some events with extremely small probability is effectively impossible imo, the critical probability to identify an event as impossible should be no more than 10^-10^10 of course, but more than 10^-10^10^10^10.
In fact, I admit when it comes to cheating, there is a rule called presumption of innocence or something, but that rule makes no sense in statistics. Any hypothesis testing must allow some type II error, although in crime confirming there should be some more strict significance level, say, 10^-10^10. (If that's not enough, 10^-10^10^3 should be good enough). Confirming cheating in a online chess website, however, doesn't need such a strict significance level. 10^-10^3, maybe even 10^-100, should be enough. Nobody want to get wrongfully accused, but nobody want to play against a despicable cheater like him. So higher type II error should be bearable.
But here comes the real problem. With some easy mathematics, I analyzed his games, he made 0 mistakes and 0 blunders after 20 moves in 17 games. that's 408 moves. There are about 10 reasonable legal moves in every position, so the possibility is 10^-408, which is good enough for confirming that he is cheating. But think this through. In some positions of his games, every move is the same, like one K+R v K+R, eight K+Q v K+some pawns, four K+R+pawns v K+pawns, two K+B/N+more pawns v K+less pawns, one K+B+B+some pawns v K+N and one K+P v K+N. He can find the winning or drawing moves of course. That will make the probability up to 10^-197, and some moves are obvious enough, some moves are inaccurate, which might bring the probability above 10^-100. (And he argued that the significance level should be 10^-10^10 rather than 10^-100)
Lack of direct evidence (Of course, if you ignore me as a witness against him) is the main problem. You can only use probability as a piece of evidence, but . I think that's why cheaters are hard to get caught. He said he didn't get caught at several other chess websites, but I don't know the websites and accounts so I can't report them. I hardly know anything about online chess, and I play terrible chess. Another friend of mine taught me chess and I lost 10 straight to the cheater on another chess website named chess.com.(Of course he didn't need engine to beat me, I am really terrible.) After my report he was banned there yesterday as well, hooray! But now he plays on Lichess again and hides his account from me. I think there must be some other way to detect cheaters other than just using probability.
TL;DR: How do Lichess lower the possibility of false cheating accusation while catching enough cheaters? Is it possible?
