hello i am intersted in chess programming. Has anyone experimented with making an engine using purely deep learning methods and reinforcement learning and not just standard alpha beta pruning?
You mean like Leela Chess zero, and Alpha Zero. I woudl say as those two exist. And source code for the first is open so you can study it
@petri999 i know the engines you are suggesting. Alpha zero is googles deepmind project so something like that is out of the question. As for leela it is considered to be a large project using also a large amount of computational sources. Of all the bots here on lichess made by individual players is there anything related to neural networks?
@radiohead7 I don't know what you are looking for but I'm interested in chess programming too and I want to create my own chess website. I know someone who wrote a chess program with a friend and I like it.
I have almost finished my chess GUI from ground I have started few days ago...
The primary idea was to make an interface with Stockfish, but also I would make my own "bruteforce" engine. The second is to experiment with NN and make my own engine based on it.
NN is in general unpredictable and need to supervised during learning process and actual play, depending of which type of NN is used, number of neurons etc, as can easy become unstable... I have a plan as well to make one NN based engine for a research. I had some studies and succees in a past with creating and using backprop NN with OCR.
The primary idea was to make an interface with Stockfish, but also I would make my own "bruteforce" engine. The second is to experiment with NN and make my own engine based on it.
NN is in general unpredictable and need to supervised during learning process and actual play, depending of which type of NN is used, number of neurons etc, as can easy become unstable... I have a plan as well to make one NN based engine for a research. I had some studies and succees in a past with creating and using backprop NN with OCR.
@fastlearner50 are you reffering to supervised learning for training the neural network with the dataset of games there are online or to reinforcement learning with selfplay? as far as i am concerned such an engine is only made by deepmind (alpha zero) and the team that made leela chess. Do you think it is generaly possible or it dosent worth the time for the expected results?
I do really think it is not worthed with the current hardware we all have on desktop. Calulation power needed to adjust NN would be enormous, even with ideally balanced number of neorons with good enough number of games in order to accompiish all in nowadays multicore processors we all use. That is the reason Alpha Zero will probably not be available for public until we all do not have cheap supercomputer on the desktop...
In my experience, self-learning with self-play without suprevising is practically impossible to achieve. Leela chess engine is worth to study, at least to get familiar with all aspects need to be used for NN based engine.
In my experience, self-learning with self-play without suprevising is practically impossible to achieve. Leela chess engine is worth to study, at least to get familiar with all aspects need to be used for NN based engine.
@radiohead7 Take a look at the giraffe engine. It was the first one to use neural networks.
www.chessprogramming.org/Giraffe
You will find Matthey Lai's publication their.
www.chessprogramming.org/Giraffe
You will find Matthey Lai's publication their.
Warning: very subjective viewpoint, trying to extract some principles from ML that applies to many NN approaches: (WIP, any two cent from any perspective is welcome). BTW chessprogamming.de is a good place to base your quick learning from, from the little I experienced. They use correct terminology. I like to use less terminology to aim at what I think is essential.
The art is in the training set sampling or definition (batch or RL). As for GO the surprise from Deep-mind zero approaches was that there was a bunch of far from perfect games to train from in order to start learning play closer to ideal competitive gaming.
On the other hand, chess engines may have neglected machine learning approach for a long time, even when they could have been ripe for experiment. The zero news, may have kick-started all sort of neural nets approaches (and training set and algo designs) by showing the approach had promises and was able to find many games that the classical engines could not compute.
Obviously the A/B engines through the human designers also learn from the self-learning engine, they adjust the heuristics in pruning, modulating different aspect of evaluation, probably mutating and selecting those to cover the blind-spots that the zero engine games against them uncovered. (this is my very personal view, very arguable, even by myself maybe in a few months, but it looks like a good narrative to me now).
how representative is the training set of the environment where the engine will be tested (or playing). Was it a good covering of chess space (how coarse, how far from perfect does it need to be in order to learn how to be better), do we need all the random ones that big computers power produced, or can we extract lessons from studying and measuring those already produced training sets (I am talking of the leela experiments and all the open source and open training data descendants, some of which are bots here on lichess). Then perhaps we could learn how to design balanced training sets that teach good move and bad moves, perhaps with an economy on their numbers, if we can find landmarks and metrics to gauge the quality of the training set, that it is not concentrated in such a small part of chess space, that when tested slightly off that part, the engine has no clue what to do. Generalization is a key concept in machine learning and in RL it is also a question, although the sequential sampling may make it less obvious than in batch training.
2 paragraphs taken out, in case this is already too much, and not clear. I can develop, but maybe questions are better. hopefully somebody will find some ideas in that. I may post more of my stories later. depending on how many rotten tomatoes land.
thank you for reading. I am trying to tune my stories, and English is not my first language. iterative point of view.
The art is in the training set sampling or definition (batch or RL). As for GO the surprise from Deep-mind zero approaches was that there was a bunch of far from perfect games to train from in order to start learning play closer to ideal competitive gaming.
On the other hand, chess engines may have neglected machine learning approach for a long time, even when they could have been ripe for experiment. The zero news, may have kick-started all sort of neural nets approaches (and training set and algo designs) by showing the approach had promises and was able to find many games that the classical engines could not compute.
Obviously the A/B engines through the human designers also learn from the self-learning engine, they adjust the heuristics in pruning, modulating different aspect of evaluation, probably mutating and selecting those to cover the blind-spots that the zero engine games against them uncovered. (this is my very personal view, very arguable, even by myself maybe in a few months, but it looks like a good narrative to me now).
how representative is the training set of the environment where the engine will be tested (or playing). Was it a good covering of chess space (how coarse, how far from perfect does it need to be in order to learn how to be better), do we need all the random ones that big computers power produced, or can we extract lessons from studying and measuring those already produced training sets (I am talking of the leela experiments and all the open source and open training data descendants, some of which are bots here on lichess). Then perhaps we could learn how to design balanced training sets that teach good move and bad moves, perhaps with an economy on their numbers, if we can find landmarks and metrics to gauge the quality of the training set, that it is not concentrated in such a small part of chess space, that when tested slightly off that part, the engine has no clue what to do. Generalization is a key concept in machine learning and in RL it is also a question, although the sequential sampling may make it less obvious than in batch training.
2 paragraphs taken out, in case this is already too much, and not clear. I can develop, but maybe questions are better. hopefully somebody will find some ideas in that. I may post more of my stories later. depending on how many rotten tomatoes land.
thank you for reading. I am trying to tune my stories, and English is not my first language. iterative point of view.
This topic has been archived and can no longer be replied to.