Why isn't cheat detection automated?

@Cezary_Wagner

"Presumably because this would require Lichess to have engine analysis running on every move of every game being played simultaneously. This is likely prohibitively costly to implement." that is bullshit - we can use post game analysis and openings books plus some "deep learning magic" - it is not magic if you understand how :)"

My impression was that the OP wanted an "on-the-fly" cheat detection system that could detect engine moves as they are played. Lichess already has a deep learning based cheat detection system that is very good.

@LaserGuy "My impression was that the OP wanted an "on-the-fly" cheat detection system that could detect engine moves as they are played."

I do want on-the-fly cheat detection and system which could detect engine moves - I want CHEATERS DETECTION (not CHEAT DETECTION) and not on-the-fly but post game. Engine not own moves I can play like engine that is not problem.

"Lichess already has a deep learning based cheat detection system that is very good." - how did evaluate that is "very good" and reference to source code or cheaters detection ratios?

I can evaluate by own if it is very good since I am programmer/designer and studied/have experience deep learning. Can you give some reference also to data - I am not believe that all is invented/I like to be inventor :)?

@AnExplosiveGuy I'm assuming you are asking how one becomes a moderator, to which I'm referring you to lichess.org/faq#mod-application

@Cezary_Wagner Move times in PGN wouldn't be accurate, since PGN doesn't support times under a second. You would need data in another format. That said, we already have an AI in place and no, we will not share training data. For obvious reasons.

@bufferunderrun If you do not want share data for developing AI - current AI is closed source :)
You can not train/improve AI without data - same like you can not improve code if it is closed source.

I think that there is no problem to ask people to share "suspected games" from any chess portal. Whatever why do not share such games? - it need only anonymization (probably only in EU - in USA they do not like cheaters and not care about privacy of cheaters but care about cheated people - different culture).

Accuracy with seconds is not need (I am not playing 30s/1m games so not care) AI can learn patterns from very rough data - sometimes more rough data is better for training.

I study little current AI but not enough to find if it good or not and what can be done. Writing such tools for players can take a lot of time so it is worth to think before.

The AI itself is open source. The training data is not. Anonymization wouldn't help here at all. You can take the data, plug it in and then train the AI and try to figure out ways to avoid detection. I mean, come on...

We won't share it.

@Cezary_Wagner github.com/clarkerubber/irwin database.lichess.org/

Every chess site has a constant stream of people asking about how that site detects cheats.They couch it in many different ways...but it always ends up as an attempt to gain knowledge about what defences need to be avoided for whatever reason.

@Cezary_Wagner My understanding is that you asked for a data sample, not the AI state or configuration of Lichess' Irwin? @bufferunderrun obviously has the near-mystical theory that if you are given a (presumably incomplete) data set of cheated and non-cheated games, then your build will reproduce Lichess' proprietary version with such precision that you would gain additional insights into how theirs works beyond all the obvious indicators that will be identified or input. This is basically nonsense.

@bunyip is accusing you of trying to get the data so you can cheat, sounds about right for this forum.

Nobody will ever see that data set because it would invite comparative analysis of Lichess' piss-poor performance on this issue. I have reported a good number of absolutely-obvious cheaters; a few have gotten banned, but most remain.

To my knowledge, Lichess doesn't use (or even store) data related to pre-moves, arrow usage, draw offers (except their existence in a game) or mouse tracking analytics. They have at times claimed to know if engine analysis is occurring in your browser, but sometimes claim they don't. In other words, there are still a lot of *basic* metrics they haven't bothered with; if they can't be arsed to do that, you really think they want to get in the weeds of a neural network? Cheat detection just isn't a priority for them, and their site continues to suffer for it.

I'm closing this topic, since OP's questions are already answered and recent posts suggest we are about to divert to a less friendly and less constructive direction.

Thank you for reaching out to us and expressing your thoughts.