Maybe it is transferable in some way.
Alpha Zero was done by 13 people. They also treated go and shogi. So about 4 people should suffice for chess. Or 1 person with 4 times more time.
The self learning was done by 24 h of self learning on massive hardware. You could also do the self learning in a longer time with lesser hardware.
Essential is the 700,000 batches of each 4096 games against itself. This is the essence of the paper.
You could also start from Stockfish which is open source.
Apply sponteneous mutations e.g. 64 to the existing Stockfish
and let the mutants play against each other and against the parent. Let the strongest survive and kill all others. Now apply mutations again. This allows to self evolve Stockfish along the lines of Alpha Zero.
Computing time to run the play against itself remains essential in this method.
I have actually done some investigation into neural networks ( very preliminary one I admit ).
Currently I have a Javascript program that can recognize MNIST hand written digits with 97% accuracy.
What I already know is that perceptron is 1950's idea. My network works with sigmoid neurons. Sigmoid neurons were the mainstream until recently because they are continuous, smooth, normalized and easy to handle mathematically.
However it turned out that rectified linear units ( ReLU ) learn faster in many cases.
This is just a technical note, the type of neurons is not the cardinal issue.
The problem is how to encode a chess position in a meaningful way using neurons. I have not the slightest clue how to do it, but i'm defininetly interested in any ideas. I suspect that this will take quite a number of neurons and layers and it will be very expensive computationally.
It seems unlikely to create a high-quality neural network for PCs because of large volumes. On github, you can find neural networks that do not achieve results above 1400 elo. In addition, the process of self-study is long. So far, no other options still not seen, except for Stockfish + calculated auxiliary tables for PCs at the moment.
Also, it's far from the fact that a well-functioning neural network can occupy a small space on the hard drive. In addition, the Deep Mind went to the AlphaZero for more than two years. And also it is not known how many gigabytes it takes in its entirety.
Nevertheless, it is possible that someone will succeed.
But at the present moment there is already a program Giraffe.
For success, need a program that is stronger.
It just occured to me that if you could teach a neural network engine evaluations that wold make a neural engine potentially stronger than the engine which provided the evaluations. This is because the engine has to perform a search, which is expensive, while a single feed forward on a neural network is cheap.
I do not know, I can not say for sure, but I do not believe in neural networks for PCs at the present moment. But I think that interesting variations and strong working program can get.
Only the training is expensive. Once you trained a network it could easily run on a PC. Essentially you can estimate the performance requirement like this:
Let's say you have N neurons in each layer and you have K layers. Then you have NxN matrices for weights. Each feed forward takes time on the order of multiplying NxN matrices with N size column vectors, so you need on the order of K*N*N multiplications.
A PC operates on the order of GFLOP ( GFLOP = 10^9 floating point operations / second ). So if you have 10 layers of 10000 neurons, which is quite a network, then a feed forward is on the order of 1s ( 10 * 10^4 * 10^4 = 10^9 ).
Hello,
thank you all for the answers.
So, I started developing this with Java.
I used sigmoid neurons connected as a simple perceptron with back-propagation for training.
The minimum needed number of neurons to encode the position is (6 + 6 + 1) * 64 = 832 neurons (6 white pieces + 6 black + 1 empty). I also, added extra neurons to encode possible moves for each piece on each square (this also encodes en-passant, castling possibilities, promotion types, etc). This made everything to about 4k neurons.
There is one output neuron witch should return -1 for a certain loss and +1 for a win.
For the training strategy, I just let the engine play stockfish (at first I trained it by playing itself, but I thought playing stockfish will accelerate the learning process). If it wins, I train each position (from move 0 to last move) of the engine with +1 and -1 if it loses. I do the same with stockfish's positions.
The issue that I am having now is that I didn't find any decent multi-threaded neural network library in java.
So, the training is stuck on synchronization into the point that I am learning with 1 thread at max.
I trained the engine with 1000 games, and it still plays really bad in my opinion. I need more processing power to train it with more games. Or I will have to look for a decent multi-threaded library.
There is already an effort to write an opensource clone of Alphagozero, known as leelazero. They estimate that training will take around 1700 years on a good OpenCl machine. They are currently writing a distributed training system.
Playing stockfish for training sounds more like the approach taken by the original Alphago, which took about 100 times longer to train.
You also seem to go for a version without MCTS. If the results for alphagozero are generalizable here, you should expect to lose around 2000 Elo points from doing that, as explained in the AlphaGoZero paper.
Make sure that you cripple Stockfish when you play it. Say remove it’s opening book and put it on a Apple Pie processor. That’s how you do it.
Hello,
@VictualSquid do you have a link to that distributed training system? sounds interesting...
yes, I am not using MCTS. For now, I just select the best move with no deep search at all (i.e. the engine tries to play all possible moves and sees which position the AI prefers).
I thought about using MCTS after that I get (relatively) satisfying results.
I don't see the point of removing stockfish's opening book. The engine will play stockfish and occasionally introduces random moves. I suppose, after playing enough games it can get better than stockfish. And if so, at that point I would switch it to play itself.
But, again I am far from that for now.