This is interesting. I've been lurking on the discord...
Version 7 is out. Time to upgrade.
We have recovered from the bug, it seems. Exciting times ahead.
Hi all
I am not really sure what is going on here but there seems to be some dreadful tactical issues with early Leela Zero - either version ID 125 or later.
In TCEC it basically has zero points:
www.chessbomb.com/arena/2018-tcec-s12d4
And blunders like this:
www.youtube.com/watch?v=KEZTrgxFiP0&t=1s
And also blundered on a later version here against me:
www.youtube.com/watch?v=1ckIsa0GnDA&t=110s
But is paradoxically much stronger relative to me when playing quicker - and I play quickly as the day before showed:
www.youtube.com/watch?v=jrpF5iYYSv0
I am slightly concerned something is missing from the 'magic sauce' of Alpha Zero - and it may be that Leela zero will suffer tactically when against other engines for the foreseeable future until the issue is resolved. It does seem to have tactical achilles heels - so I do wonder about the claims of it being even over 2500 from an engine perspective.
I am excited if I am proven wrong, and she starts crushing the engines in a TCEC later season. This will make for great drama if it can inflict revenge on the engines.
Fundamentally to me it seems that a neural network approach is weaker tactically than a brute force approach. This is a classic tradeoff in many ways. Trying to emulate real intelligence rather than trying to play on the strength of fast processors and what engines do best - number crunch.
Fun stuff, K
I am not really sure what is going on here but there seems to be some dreadful tactical issues with early Leela Zero - either version ID 125 or later.
In TCEC it basically has zero points:
www.chessbomb.com/arena/2018-tcec-s12d4
And blunders like this:
www.youtube.com/watch?v=KEZTrgxFiP0&t=1s
And also blundered on a later version here against me:
www.youtube.com/watch?v=1ckIsa0GnDA&t=110s
But is paradoxically much stronger relative to me when playing quicker - and I play quickly as the day before showed:
www.youtube.com/watch?v=jrpF5iYYSv0
I am slightly concerned something is missing from the 'magic sauce' of Alpha Zero - and it may be that Leela zero will suffer tactically when against other engines for the foreseeable future until the issue is resolved. It does seem to have tactical achilles heels - so I do wonder about the claims of it being even over 2500 from an engine perspective.
I am excited if I am proven wrong, and she starts crushing the engines in a TCEC later season. This will make for great drama if it can inflict revenge on the engines.
Fundamentally to me it seems that a neural network approach is weaker tactically than a brute force approach. This is a classic tradeoff in many ways. Trying to emulate real intelligence rather than trying to play on the strength of fast processors and what engines do best - number crunch.
Fun stuff, K
Few weeks ago during a match against handicapped version of SF (from level 9 to 14 or something, see www.twitch.tv/y_sensei), it was observed that Leela lacked tactical awareness indeed, despite being able to get very pleasant positions (even dominating spatially, A0-style).
I'm not sure anyone (not even the authors) know exactly why that is.
I doubt this problem will remain though. Leela was absolutely terrible at tactics at her beginnings, and she did improve, so there is no reason she could not keep improving.
I'm not sure anyone (not even the authors) know exactly why that is.
I doubt this problem will remain though. Leela was absolutely terrible at tactics at her beginnings, and she did improve, so there is no reason she could not keep improving.
@Kingscrusher-YouTube
I can confirm this, leela still makes a lot or errors. But this is (probably) simply because She has not played enough games, as @grondilu said.
i another thread someone said that 25 million more games are needed in order to reach the strength of Alpha Zero. my guess is, it needs 100 million games, maybe 300 millon, to really not blunder any more at all. Leela is currently at 7 million+ games. (THAT, @ everybody, IS WHY WE NEED YOUR SUPPORT!! NOW!!! :-) )
If Leela still blunders regulary at 30 million+ games regulary we will need to discuss what to do then. I am actually expecting that this first try will fail. Like all complex code projects it is not uncommon that the first version lands in the trash. See eg. the pawn promotion bug mentioned by @BigGreenShrek in #10 - But of course i hope i am wrong :-)
Further, there are lots of other improvements thinkable, like eg. opening books compiled from earlier AIs to speed up the learning process, and endgame tablebase access.
Once Leela wins the computer chess world championships regulary, everybody will jump into this new technology and we will have our AI Fruits as well.
I can confirm this, leela still makes a lot or errors. But this is (probably) simply because She has not played enough games, as @grondilu said.
i another thread someone said that 25 million more games are needed in order to reach the strength of Alpha Zero. my guess is, it needs 100 million games, maybe 300 millon, to really not blunder any more at all. Leela is currently at 7 million+ games. (THAT, @ everybody, IS WHY WE NEED YOUR SUPPORT!! NOW!!! :-) )
If Leela still blunders regulary at 30 million+ games regulary we will need to discuss what to do then. I am actually expecting that this first try will fail. Like all complex code projects it is not uncommon that the first version lands in the trash. See eg. the pawn promotion bug mentioned by @BigGreenShrek in #10 - But of course i hope i am wrong :-)
Further, there are lots of other improvements thinkable, like eg. opening books compiled from earlier AIs to speed up the learning process, and endgame tablebase access.
Once Leela wins the computer chess world championships regulary, everybody will jump into this new technology and we will have our AI Fruits as well.
the pawn promotion wasnt a bug
just leela didnt learn to promote pawns yet back then why?
because before it had very few possibilities where it could promote pawns and see the benefits or disadvantages by promoting the pawns differently so it made stupid promotions because it didnt know witch promotion option is good.
just leela didnt learn to promote pawns yet back then why?
because before it had very few possibilities where it could promote pawns and see the benefits or disadvantages by promoting the pawns differently so it made stupid promotions because it didnt know witch promotion option is good.
@user589544 #17
If we define a bug to be unexpected behaviour in the software, then this is a bug, no? but yes, it is not a syntactical or semantical programming error.
@BigGreenShrek #1
Questions:
* how to stop the software after it has been started?
* how much internet traffic is created?
(for people with limited internet access)
Does this work the same as with running client.exe? or does the data get transferred directly from Google servers to lczero.org/?
* will it use Processing power on my computer or does it all run on Google Servers? Aka, can i run my engine on my comp meanwhile for analysis?
* Can i use this even if i do not have a GPU on my computer?
If we define a bug to be unexpected behaviour in the software, then this is a bug, no? but yes, it is not a syntactical or semantical programming error.
@BigGreenShrek #1
Questions:
* how to stop the software after it has been started?
* how much internet traffic is created?
(for people with limited internet access)
Does this work the same as with running client.exe? or does the data get transferred directly from Google servers to lczero.org/?
* will it use Processing power on my computer or does it all run on Google Servers? Aka, can i run my engine on my comp meanwhile for analysis?
* Can i use this even if i do not have a GPU on my computer?
@user589544 In both links in the OP, information is sent from Google to lczero. Your internet connection should not matter (except of course in the initial stages when you're setting it up).
The Colab script will stop after 12 hours, or sooner. The Compute script will not stop until your credits run out. Instructions for stopping it earlier are given in the guide I linked.
Since both the Colab and Compute scripts run on Google servers, they do not use any processing power from your computer, and they do not require you to have a GPU.
The Colab script will stop after 12 hours, or sooner. The Compute script will not stop until your credits run out. Instructions for stopping it earlier are given in the guide I linked.
Since both the Colab and Compute scripts run on Google servers, they do not use any processing power from your computer, and they do not require you to have a GPU.
Regarding the tactical weakness shown, it's likely we need a combination of more training (to identify promising candidate moves -- right now she misses very simple sacrifices, for example) and more layers in the neural network (to handle complex multi-ply tactics).
This topic has been archived and can no longer be replied to.