Tofiks 1.3.0. is looking good!

What's new?

Plenty! Not long ago I started working in a new position with a lot of responsibilities and a heavy workload. I was naturally worried that my chess hobby will take a hit and be pushed further back on the list of my responsibilities. However, while being tired from work where I do software development for many hours I find it even more important to dedicate time for myself. And in the last year that's always been working on my engine. Also chatting with technical minded people at work always keeps my own gears spinning at a higher speed and I come up with ideas and extra motivation to work on Tofiks.

New looks

Tofiks is programmed in the Go langauge which has this Gopher mascot. So I commissioned a painting to reflect Tofiks in that style and here is the result:

I think it looks absolutely adorable and compared to the old AI Generated picture this guy seems like he knows what he is doing. Which is fair given how much it has grown ever since.

New home

I previously hosted Tofiks on Google Cloud. While my trial was on-going and I had free Google trial money to spare I did not realize how expensive it would be to host it there. But with my trial ending and real charges racking up I had to make a smarter decision and moved it to Hetzner. It seems very similar hardware at a fraction of the price. And I mean fraction - I will be saving more than 90% on monthly bills. I do not mean to say that Google is expensive but evidently they offer very professional services high availability and true large scale enterprise level features. They are often worth it but not in my case.

Making new friends at CCRL

Tofiks has been noticed by the CCRL - Computer Chess Rating List. It's a group of dedicated people who take it upon themselves to search out engines and test their capabilities against each other. Some members became aware of Tofiks and included it in some tests where it plays against other engines. Currently there are 699 engines on the CCRL blitz rating and Tofiks-1.2.0 takes spot #544. Not geat, not terrible! But that just fills me with dedication to work harder so it can move up the ranks. Having passion for something is great. Having passion with a goal in front of you is a recipe for success!

Why threefold repetition?

Threefold repetition makes very little logical sense. Why three? Why not two, five or ten? Let's think about it for a moment. Let us assume perfect play or at least perfect play with some constraints where still both players play the very best moves to their abilities. If we start at a position X then play some moves x_1, x_2, ..., x_n and we arrive back at X, there really should be no logical reason to play anything else than the same moves- those were the best moves then. Nothing has changed and those are the best moves now. So it is only reasonable that position X will repeat again. ...and again and again... So twofold repetition makes just as much sense if not more all things being equal and logical.
I think threefold repetition is a very human rule - it would be too easy to accidentally stumble into the same position twice and the game abruptly ending in a draw and the players being robbed of a match. Computers should be less error prone and noticing a position repeating twice it will naturally lead to repeating thrice.

If this sounds like incoherent rambling or me trying to re-invent the rules of chess, please be patient. I have a point. I was extending the test suite of my engine and adding different self-tests that the engine could perform to ensure that it behaves well. I added some tests where it would count the possible number of positions after playing a few moves and then compare them against known results on the internet. I made it solve some simple mates and made sure it gave the right answer. This has helped me a lot in development because I can make changes and have a sanity check button where I can see that it still performs all the baseline things an engine should be able to when working correctly.

I added a threefold repetition check - I sourced some of Tofik's games which ended in threefold repetition draw and asked it to verify that he actually understands that it's a draw. All worked fine! Then I gave Tofiks a position where he's one move away from being checkmated but there is a possibility to deliver endless checks forcing a threefold repetition. It saw the moves to avoid getting mated but evaluated the position as losing nevertheless.

It could not see the threefold repetition from a distance. Why? It was able to look many moves ahead but could not see the draw right in front? After a lot of debugging it became apparent that the issue was transposition tables.

Transposition tables store evaluation scores already done by the engine's search algorithm. So if we ever encounter the same position via transposition we can reuse the result in some cases. After the first check the engine evaluated the position as losing because it was down material. So when it gave the second check it looked no further since it had already seen that position. So it seems the engine was never able to repeat the same position three times to determine that it is a draw. Downgrading from threefold repetition to twofold repetition seemed to solve the problem. It still played the same moves but now it did so with the correct evaluation. I am not sure if the change affected its playing ability much but having wrong evaluation in your search seemed wrong. Maybe there are other ways to mitigate the error but the side-effect of twofold repetition is that it is also less computationally expensive as threefold repetition.

What's in a bit?

Recently I had the time to reflect some more on my code and I realized that I am using 32bit integers to store scores returned by the search and 16bits to encode the moves. And it did not seem right. For reasons that might be too advanced the sum of those two present a pretty had limit. So I could not increase one without decreasing the other. 16bit integers allow values in the range of −32,768 to 32,767. This is more than enough to have evaluation in centipaws with enough granularity to express any nuance one might imagine. This allowed me to free 16bits that could be used for storing moves. Given the extra bits in the move I was able to store some extra information and context to help the engine with some aspects. The most important ones are - encode in the move what piece is being moved, so it is easier to tell whether e4e5 is moving a pawn, king, rook or queen, without the need to look up what piece is stored on that square. Sounds trivial but when that needs to be done millions of times each second any optimization becomes huge. Also I also encode a flag indicating whether a move is a capture - again trivial but can save some computations needed to check if the destination square is occupied. Similarly I store information about the move being castling, en passant capture, a double pawn push. All to make it easier for the engine to update the state of the board with least computation.

While the change initially seemed to slow down the move generation since we now need to do more work adding all this extra information it ended up paying off in the end because during play when moving pieces it actually made the job easier giving a net positive performance increase. Not much but it's a start and I have a feeling there is more to be found.

Results

So what is all this work worth? I again played a round robin tournament between Tofiks 1.0, 1.1, 1.2 and 1.3. Each playing 200 games against every other engine. Totaling 1200 games of 10+0.5 and 64Mb of cache each.

Rank Name        Elo     +/-   Games    Wins  Losses   Draws   Points
1 tofiks-1.3     182      27     600     378      89     133    444.5
2 tofiks-1.2     108      25     600     315     134     151    390.5
3 tofiks-1.1     -28      24     600     202     250     148    276.0
4 tofiks-1.0    -304      35     600      57     479      64     89.0

For three weeks worth of work I am happy to see a relative gain around 70~80 Elo. It's hard work. It seems like every next Elo gain comes with so much more effort than the one before - I am sure players who play and improve themselves can relate to that.

What's next?

I actually can almost copy the same ideas of improvement from my blog about Tofiks 1.2.0 update. However, the change of types of the moves and score give me more flexibility - I still have 10 spare bits in my move data type that can be used for clever ideas. With more tests and more bugs fixed I also feel more brave and ready taking up the Texel tuning challenge - I feel there could be huge gains to be made. Watching Tofiks play I feel the mid game is its weakest area - it often places pieces poorly allows enemy outpost pieces, ignores King safety. I hope you will hear from me again very soon!

Try it!

As usual I encourage you to try out playing my bot @likeawizard-bot and you can find the source code and binaries on Tofik's GitHub.