@RichardRahl said in #8:
> You are right, I am not using FEN positions as input. Indeed I calculate "magic" numbers i.e features.
Just to make more precise what i meant by magic number. I did not mean the signal from chess input, but the parameters in your algebraic or functional formulation. Also, since we are on a open source platform, I think, keeping your encoding secret is not really making this discussion as interesting as it could. At least you could give some list of the features (the set does not have to be optimal, but at least give us a scope of your experiments dependent or controlable variables).
> These features are intended to be understandable by humans and rather simple. Otherwise the NN can not coach us.
> The goal is to make evaluations explainable. It is not to beat stockfish.
That is where I find your thread appealing, and why I mean that you are not alone in wanting that. I would call that an engine as an actual tool for post-game analysis by a human. Not one that is optimized to beat a series of similar engines via some finite set of tournament formats with rigid contraints on hardware and competition factors, all of which are far from human play and experience.
However, I would put some "bémol" on your association with correct evaluation and depths of evaluation. The SF design is built around a fixed material counting (1,3,3,5,9, ?) (?=W,D,L), for the actual position information static evaluations. Not all nodes explored are being evaluated with full position information (the static evaluation that gives scores). So keep in mind that there is a bias toward only looking for material conversions at some remote depth. But in chess we really care not about those (do we?), we care about the final conversion of any advantage, not only material. What is the material value of a mate? That can get complicated, but even when making that complicated, this discontinuity problem persists (just with many branches mixing types of things to convert or not).
> Actually I do not only compute the features from FEN positions, but also from the whole game history.
Well, that is interesting. and actually informative. I would like to know more. Are you of the open source, open data, and open science philosophy? I am.
> Since you are curious about what features I calculate, here are a few of the most important ones (in total I have over 100 features)
OOOPS. sorry, I am impulsive. I should have read further... . But this fits with not overwhelming the reader, well done (I am bad at that....). Apologies.
> (with important I mean if the values are changed slightly the NN will predict different results)
>
> ["last move by"] // Can be 1 or 0 (white or black)
> ["Half-Open C File {black}"]
> ["{history} {castle} 0-0 {white}"]
> ["P == p"] // Means equal count of Pawns
> ["Open D File"]
> ["Fianchetto King Side {white}"]
> ["King Side Pawn Majority {black}"]
> ["{history} {castle} 0-0 {black}"] // Gives the NN information if black casteled in the past
> ["Half-Open E File {white}"]
> ["Open C File"]
> ["Half-Open D File {white}"]
> ["b == n"] // Means Black has an equal count of Bishops & Knights
> ...
>
> the more useful features the better. Later the user will only be shown the relevant features, that matter in the given position.
the crux and where my inbox ramble matters. Be careful how you tangle all your features, from a quantitative point of view. That is also where the "magic" numbers problem arises (not the features, or the input values transmitted to the NN first layer, but the formulation parameter values). Do you do any tuning? Or discussion of those parameters associated to how you feature become ordered quantitative input of Neural Nets (because that is their "language" they only work with quantities, be it the trivial binary order if obligatory (not much to tell as a function of 0,1).
> I am still on my way inplementing new features, there are many features that stockfish uses as well. www.chessprogramming.org/Evaluation
> And there are features that I come up with on my own / put together from the internet.
SF has already been labeling some code variables with human feature names. But we can't make heads or tails of those, because of the magic numbers accumulating since SF1. So I advise toward being careful in your feature set. Throwing all of them there sure will make for more fitting neural net over the training set, but watchout for the testing set .
The more parameter you put into your formulations of features (whether fixed or tuneed) the less discrimination of which label is a factor do you get. Neural Nets like a0 lc0, and NNue, gave up on that to have maximal fitting&predict ability for their Training/Testing partitioning of sample database (that is where i need one more word that does not add confusion).
Motherload Database = lichess (of some category constraint set or all of it)
???? Database = correct size and "spread" subset of motherload from which cross-validation (best way to do it) or simple random partitions (means disjoint), are taken to form the training set and the disjoint testing set.
Succesful training means successful generalization (if you aim at SF as target, then you need to predict stockfish over input not part of training). This is a double (or more) optimisation problem. Best training AND Best testing under the objective function. That is why besides the target vector, one often finds what is called a regularization term in the loss function (or whatever chess engines schools like to call it).
So now I had insisted on target definition. And you new statement about history of positions as part of input process remains as a further question.
Back to the X of the Y in NN(X(game)) approaches Y(X(Game)) on the dataset (with cross-valiadation measure as criterion of fit). i.e. what matters is not how you fit the data, but how you predict outside of the data. (hence the test partition). Sorry for the repeats, I am hoping it helps.
There is no ASCII (at least not on my keyboard with some contorted non-touch typing dance of the fingers) for Converge to, approaches, of fits... math where are you?
Since you don't have a mathematical formulation, perhaps sharing your code as last resort could help some of us make one high level model for you, that would allow to see your overall information flow, beneficial for you, and for me trying to explain the warning about it matters what you put and how you put it, in the salad of features. I disagree with the "more" the better.
Even without engines, having a flat set of features is what makes human theory look shaky and only good for hindsight (which I don't believer it is). Because experienced players have already done the feature salad pruning intuitively and only mention the winner in their annotations when going into positional arguments (some prefer to also forget the words and only keep their intuition as secret to be used in their own games only....;)
