lichess.org
Donate

OMFG! STOCKFISH NOOO! ( AlphaZero vs Stockfish ) Thoughts

Why some people like AlphaZero !? It does not make any sense. When the games played between Kasparov and DeepBlue, it made sense, because the task was to get the most out of the available resources. To admire the fact that the weaker at the moment the algorithm has benefited from the stronger one only due serious superiority in the gland. This is nonsense.
Yes you can admire the general level of the game. But, perhaps, everyone can do such a trick at home. Putting a weaker program on a powerful computer, and a stronger program on a weak one, but that no one does, because it's pretty stupid.
@no_bullet_thanks

That's just it. Cars ARE better at covering distances quickly than humans.

My point is that the whole talk of "fairness" only really makes sense if you can actually control all the variables except what you want to test.

If you can't, then it's silly to complain of a particular test as being unfair.

In matches between, say, SF/H/K or other such traditional engines, it makes sense to talk of fairness, because you can control for everything but the engine itself.

You can have them run on the same machine, with ponder off, give them the same hash size, have them use the same opening book, use the same number of threads, use the same time control, etc.

Then everything else is equal, and you're just comparing the engines themselves. If someone did such a test and had one engine running on 2 threads and the other engine on 4, or one engine had a different book, or some other such difference, then it would make sense to say the conditions were not fair.

When you can't control for some factor, like SF being written to run on one kind of hardware and AlphaZero on another, with the two types of hardware not being directly comparable. this no longer makes sense (and even less so when you're talking about humans vs computers).

In the case of a race, the same thing applies. Humans don't transport themselves by rolling on the ground, and cars don't use legs. There's no making that "equal" or "fair".

Nowhere in the paper does Google claim "our approach is vastly superior to SF's approach, independent of hardware or anything else, and we've completed a carefully controlled test that shows this".

All this hullabaloo over "fairness" is misplaced and distracts from the accomplishment, which is showing that an approach previously thought inapt for chess can actually work.

I share your enthusiasm about neural networks.

However, lets make clear that the test was not scientific. Their Algorithm outperformed Stockfish because it had the better conditions.

Still it is a very exciting experiment and shows that machine learning has a great future.
Well, if we're going to sling about things like "unscientific", I'll have to ask for the evidence you have for SF's having inferior conditions to A0 in the test :)
Well, if we're going to sling about things like "unscientific", I'll have to ask for the evidence you have for Alpha zero playing better than Stockfish in the test :)
@no_bullet_thanks I didn't accuse anything of being "unscientific". You did, so bouncing that back doesn't work so well :)

The score of the 1300 games they played at A0's full strength is the evidence I submit for A0's having played better.

Now back to the original question :)
@no_bullet_thanks

So, there are a lot of different claims there, some true, some not, and many irrelevant to the claim you're trying to support.

Which claims from that thread are you citing as your evidence?

Also, keep in mind, the claim you're trying to support is that the conditions of the match favored A0 over SF to such a degree that the conditions explain A0's scoring more than SF.

This is different than simply saying that SF could be configured to play more strongly, which is nearly always true at a minimum by adding/upgrading hardware (so, for example, saying that SF would play more strongly with more hash, even when true, does not support the claim that the conditions were biased against SF, much less go anywhere close to supporting the claim that it accounts for all of A0's score above SF).

This topic has been archived and can no longer be replied to.