Lichess AI May Be Misconfigured

It seems that the lichess AI may be misconfigured.

I started a game with the AI on level 8 from a position frequently reached in the Mieses Defense to the Evans Gambit, after the moves 1.e4 e5 2.Nf3 Nc6 3.Bc4 Bc5 4.b4 Bxb4 5.c3 Ba5 6.d4 exd4 7.O-O Nge7 8.Ng5 d5 9.exd5 Ne5.

I was playing Black with the assistance of locally installed Stockfish and, together, we defeated the lichess AI. After the game ended, I clicked for an automated analysis and the lichess analyzer (also Stockfish, I presume) tagged the lichess AI with 11 Inaccuracies, 3 Mistakes and 1 Blunder. This is a highly questionable performance for a chess engine claiming an Elo rating of 2600.

It gets worse, both the lichess AI and the lichess game analyzer missed a move (found by local Stockfish) that would have led to a draw by repetition.

The bug report portion of this post is simple. I am requesting that the AI be configured to take more time to analyze or to go to a greater depth. It should not be able to play a move that the game analyzer takes to be an inaccuracy, mistake or blunder.

I also have a feature request.

Anyone trying to verify the claims made above will have to reconstruct the game as follows. To the moves given above, add the moves in lichess.org/D7EU6nta and then the moves in lichess.org/LvtH3U1T.

What happened is that, although the time control was set to unlimited, the lichess AI claimed that I resigned while I was sleeping. Perhaps my session timed out. So the feature request is that users should have a way to save a game in progress so that the AI won't pretend that opponents who periodically need to sleep have resigned for the night.

Thanks,

PurpleMoonLight

For convenience, here is the PGN for the game:

[Event "Playing the Lichess AI"]
[Site "lichess.org"]
[Date "2019.02.11"]
[Round "1"]
[White "LiChess AI at Level 8"]
[Black "PurpleMoonLight"]
[Result "0-1"]
[WhiteElo "2600"]
[BlackElo "1500"]
[ECO "C52"]

1.e4 e5 2.Nf3 Nc6 3.Bc4 Bc5 4.b4 Bxb4 5.c3 Ba5 6.d4 exd4 7.O-O Nge7 8.Ng5 d5 9.exd5 Ne5 10.Qxd4 N7g6 11.Re1 O-O 12.Rxe5 b5 13.d6 Bb6 14.Bxf7+ Rxf7 15.Nxf7 Bxd4 16.Nxd8 Bxe5 17.Nc6 Bxd6 18.Ba3 Bd7 19.Bxd6 cxd6 20.Nd4 Rc8 21.Na3 Rxc3 22.Ndxb5 Rd3 23.Rc1 Nf4 24.Kf1 Rd5 25.Nc3 Rc5 26.Ne2 Nxe2 27.Rxc5 dxc5 28.Kxe2 Be6 29.Kd3 Bxa2 30.Nb5 a5 31.Nc3 Bb3 32.Ne4 Bd5 33.Nxc5 Bxg2 34.Kc4 Bc6 35.Ne6 a4 36.Kb4 Be8 37.Ng5 h5 38.Ne4 Kf7 39.Nd6+ Ke7 40.Nc4 g5 41.Nd2 Kf6 42.Ne4+ Kg6 43.Ng3 h4 44.Ne4 Bd7 45.Nc3 Kf5 46.Nd5 Be6 47.Nc3 Bb3 48.Ka3 Kf4 49.Nb5 Ke4 50.Kb4 Kf3 51.Nd6 g4 52.Nf5 h3 53.Nh6 Be6 54.Kc5 Kxf2 55.Kd6 g3 56.hxg3 h2 57.Kxe6 h1=Q 58.Nf7 a3 59.Ne5 Qe4 60.Kd6 a2 61.Ng4+ Qxg4 62.Kc5 a1=Q 63.Kd6 Qad4+ 64.Ke7 Qgd7+ 65.Kf8 Qh8# 0-1

The missed draw is shown in lichess.org/Tb1bM5Um0Jdp

The rest of this is over my head but I agree with the feature request:

"Users should have a way to save a game in progress so that the AI won't pretend that opponents who periodically need to sleep have resigned for the night."

The "play against the computer" option on Lichess isn't meant to be stockfish full strength. It's suppose to be a weaker engine that humans could feasibly defeat. It should be expected that this weaker engine playing against a locally ran stockfish would make mistakes, inaccuracies, and maybe even a blunder.

Clearly, if various levels of playing strength are offered, all but the highest would have to be impaired somehow. What is unclear is whether the highest level of playing strength offered has been impaired by choice or by mistake.

When you play against it, like you said, it displays a rating of 2600. Given that, it's very obvious to anyone with knowledge of engines would understand that the strongest engine in the world, playing at full strength, wouldn't display such a low rating.

By choice.

Here's the configuration settings:
github.com/niklasf/fishnet/blob/9018fc30bfdb975c4a5a35432c50b007c632d5ba/fishnet.py#L118-L120

I thought using a real engine against any stockfish level on lichess was not allowed altogether.

In unrated games with your opponent's permission, using an engine is allowed. The Lichess AI doesn't mind.

Re: #6

Thank you, Toadofsky, for the link to the code lichess uses. It looks like those three lines (118-120) limit the AI's time and depth per move at the various skill levels. That would explain why I was able to defeat it. I had local Stockfish set to take more time - usually 3 minutes per move, sometimes 5.

Clearly, I did not discover a bug after all.