lichess.org
Donate

Name the best chess engine

@sfumatosauce the most extreme I noticed someone trying is from the www.chess.com/computer-chess-championship
where they used
GPU: 4x RTX 2080ti (44 GB GPU memory)
CPU: 2x Intel Xeon Gold 6154
Cores: 36 physical
RAM: 96 GB
for the GPU engines like Leela.
However, speed grows by no means linear, my guess is the speed of 4x 2080ti is about twice or max. 2.5x as fast as one 2080ti.
So price/efficiency is pretty hilarious at that point.
@Quadbest @sfumatosauce
I am by no means an expert, and you should ask the nice moderators of
www.twitch.tv/tcec_chess_tv about it, but my impression is the general opinion suggests

Lc0 ~= Stockfish 10 > AlphaZero > Stockfish 8 on current high end CPU/GPU hardware.

Which we of course cannot prove since we don't have access to A0.

AlphaZeros games were impressive and it lead to the great Leela project but one should consider that
- Google DeepMind's marketing department did an impressive job too
- the hardware for sf was "not ideal"
- it was a few years ago / sf8 was used

If DeepMind decided to train a new net with advanced techniques, and for longer than their "4 hours", it might turn out slightly better. However, first of all they no longer work on chess afaik, and second the elo curve over training epochs is only very slowly growing at the later stages. In other words, I kinda doubt it's gonna reach way higher levels than let's say current highest values of ~3600, perhaps some +50 elo more. I might be wrong though, just my semi-knowledge.

At this point Lc0 is similar in strength to the latest development Stockfish which is a couple (~30 elo) stronger than Stockfish 10. Both of them are apparently stronger than AlphaZero according it's score against Stockfish 8 in the match published by Deepmind.

Which one of Lc0 or Stockfish is stronger than the other really boils down to hardware because they utilise different hardware. However, you can say that Brainfish (Stockfish with the Cerebellum opening book) is a stronger engine (tests show up to 50 elo gain) than Stockfish but I would argue that it isn't because the two are running the same base engine.
I think for the conventional layperson, Stockfish really is the best that you can get your hands on. Of course professionally, some variations of AlphaZero or Leela are much stronger, but you'd need a specific rig for that.
@s1mpl3x @Quantium Thanks for the hardware feedback. It's very tempting to build a powerful rig that runs Leela.

So when people talk about AI "networks" instead of individual machines or computers, are they referring to all of the other players and machines that contribute to the learning of a specific machine? iow, if I decided to build a super rig running Leela, would it be necessary to make it a server (allowing it to play against many other opponents) or can it achieve similar levels simply by playing itself? What exactly is happening on tcec.chessdom? Are those Leela-based rigs competing against each other primarily to see who has the best machine... or because it's necessary for each machine to learn? I assume if it's the latter, then both are necessary.
@sfumatosauce I am not sure I understand all your questions correctly.

What I can say is that "network" in this context simply refers to "artificial neural network" which is a rediscovered, now booming mathematical tool used for deep learning. Deep learning is a subcategory of Machine Learning, and Machine Learning is a subcategory of Artificial Intelligence.

Especially for various tasks in supervised learning (classifying images of cats vs dogs, or cars vs people), compression of information (via Autoencoders), and self learning systems ("reinforcement learning"), neural nets often are state of the art.

Now most classic engines like Stockfish, Komodo, Houdini use Graph-Based techniques like Alpha-Beta-Pruning, and evaluate the positions at the end of the search depth according to a long list of criteria, not only simply material, but material adjusted to playing phase (rooks are way more valuable in the endgame compared to the opening, a-/h-pawns are less worth than central pawns in the opening but more worth in several pawn and knight endgames,...), various positional elements like development, bishop pair, doubled pawns, open / closed pawn structures, knight outposts, king safety and so on. So human made criteria are developed by strong chess players and computer scientists, and directly implemented in the chess engine for the evaluation. For these heuristics many parameters need to be tuned, and this is a big part of the development process for future versions.

On the other hand very recent engines like Alpha0 and Leela follow a completely different approach, namely reinforcement learning. Over-simplified they somehow hardcode the rules of the game (in this case chess) and define a win (in this case checkmate), and you decide for a certain structure of an artificial neural net (referred to as the "architecture"), i.e. the number of layers and so on, and you let the engine play millions of games against itself. Now every few hundred games or so the net is allowed to vary certain parameters ("exploration"), and you get some feedback if the change was good or not, and you try to optimize the parameters on the way to win more games against the "older self". After many many games certain weights within the neural net have converged to fine-tuned values. So while the details have some mathematical black box character this net structure somehow manages to learn weights by itself, in a way that if any (so far unseen) position is fed in it can produce a very good idea of how good each possible move is in terms of winning probability (for classic engines this roughly translates to material advantage in number of pawns like +1.5).

So the trained net can be used to assess any position and suggest the best move, similar to a classic engine. However, contrary to the classic case no human heuristics have been used. Also special for the neural net approach is that it can take very long to train a good network, but evaluation is extremely fast. Both use some kind of matrix multiplications, and simply work faster with gpus, up to 100x or some incredible factor.

So as a user, you just download a recent version of Leela, which essentially is a trained neural net for chess. Depending on platform you might need to compile it and make a UCI-compatible chess engine out of it. Then you can use it as a chess engine, i.e. evaluating the current position and the ones within a little search depth.

However, perhaps you were referring to it, you can also help the Leela project with gpu / cpu ressources to build newer networks. You basically use a current net on your hardware to play against itself a number of games, and report back the games. There is a small part of the community that does this, but it's not for the normal user. So I think "network" rarely refers to the people doing this, it'd be very confusing for obvious reasons.

TCEC stands for Top Chess Engine Championship. Basically some people who like computer chess have come up with some rules like time controls, number of games, and so on to host tourneys where engines can battle each other. They also determine the hardware and so on. Some people try smaller projects with their own servers, but TCEC is by far the most established and large-scale project, one could compare it to the olympic games of chess engine "sport".

If you have different rigs but use the exact same version of Leela, essentially they should come up with the same evaluation and move suggestion if you let them calculate until the same depth. Of course some rigs will be faster than others. It would be different if each rig learned the net by itself from scratch since the algorithms have many random components for exploration and initial values. But as I said, you will just use an existing net and use it for evaluation.

And imo gpu rigs just for chess with more than let's say 1x the high end RTX 2080 Ti are only interesting if money has no meaning whatsoever to you ;)

This topic has been archived and can no longer be replied to.