lichess.org
Donate

Improvement suggestions

So there are a couple of things that I think could be improved. I wonder what the other BOTs opinions on those are.

The first one that I have mentioned a couple of times is the Armageddon game. Why doesn't White get time odds to compensate for the drawing odds? That could either be done through a fixed time advantage, or the players could bid for who gives White more time and still plays Black. Time advantages on lichess can e.g. be done with the "give opponent +15 seconds" feature. I think that would be a lot more fair considering the high draw rate in top bot games which otherwise put White at a huge disadvantage.

Another one is the rather strange time controls. I have never in any human or bot tournament seen a time control with more seconds increment than minutes base time. Also I have seen a lot of strange time management choices in bots considering that rather strange time control.
I would suggest something like e.g. 25'+25'' instead and possibly 10'+10'' and something like 2'+2'' later which seems much more natural. (of course other time controls are possible too) This is not as big of a deal, just a minor change I would like to see. @Ajile mentioned he doesn't want to have games be decided by time trouble however with todays hardware even a few seconds of increment are loads of time.

Another idea I had, more as open suggestion, was to possibly split the tournament into a section for established engines like Stockfish, Leela, Komodo etc. and one section for new not as strong engines that the users wrote themselves. Should make for fewer lopsided matches of Stockfish demolishing some weak engines. Though the current system in that regard isn't too bad either. (one could even consider some kind of multi-stage tournament like TCEC where one can qualify for the next higher stage)

What are your thoughts?
1 and 2 I agree with. I do agree with Ajile that the time trouble might be an issue depending on how bad your network connection is (keeping your Move Overhead high means that engine will have less time to think/search).

3. Splitting the tournament seems like a lot of work for @Ajile by himself but if more people are able to help out it could be possible. Currently, I don't see this as a big issue but if more BOTs join in the next tournament it would be a problem.

1) If I understood the rules well, in stage 5 of the match the winner is the first to win a game; stage 6 only happens if there are 4 draws in stage 5, therefore, 1-1 as final result (not 2-2, as stated by the rules). And in this last stage, the winner is the white player only if he wins the game.

Sounds good to me. I just don't understand why in stage 6 the game has to be casual (as stated by rules).

2) I don't see any strange time in present clocks. Their estimates are:

(15' 45'') ==> 90m
(5' 15'') ==> 30m
(1' 3'') ==> 6m

Maybe the clocks are unusual, but I don't see any problems here whose solution will improve the championship. They are the "championship clocks", and I have no problem with that.

3) Splitting the engines into divisions seems to me to result in a improvement for the championship. More than one engine can be a winner, without obscuring the honors of the absolute winner (obviously situated in the highest category). But, as @LeelaChess has said, all work falls on @Ajile...
1) I forgot to mention what I think could be done if stage 5 and 6 is considered hard after 8 games in which we see equivalent engines. Time odds seem not such practical to me, and I believe we must put an end on the match at this point. So, I think that in stage 5 all 4 games could simply be played, and, in the case of a 2-2 result, stage 6 would be as it is (or only with the difference that the color would be randomly assigned by the tournament director).
Thank you all for the suggestions. Stage 5 was added after the 1st Lichess Bots Championship so the last stage will happen less frequently (it only happens after 4 more draws, so 2 points for white and 2 for black). Stage 6 is beneficial for the second mentioned player to make sure that the match is decided. This is done without changing what the players start with (pieces and time), only how the result is counted changes, to make sure that the changes don't force any players to play worse than they did in stage 5 (they can choose to play differently/risky, but they aren't forced to).

It would be great if all players could play 15+45 rated games (as often with white as with black) until there was a winner, but that would take too long, so the time limit decreases during the stages of the match (even though that isn't ideal, I think it's better than giving opponents different time controls to determine the strongest player).

At stage 6 both players are basically of equal strength (or very small differences), so a coin flip could also be used, but I prefer to let winners be decided by games. The last game is beneficial for the second mentioned player and that could lead to the first mentioned player taking bigger risks, so the game is casual instead of rated (in case someone cares about the possible rating loss in that game).

If the first mentioned player (player Xbot) and the second mentioned player (player Ybot) are the best players of the tournament and of equal strength, then the second mentioned player (Ybot) will probably win in stage 6, but the first mentioned player (Xbot) will win the lower bracket (Challonge calls it the Losers Bracket) and play as the second mentioned player against the former second mentioned player (Ybot) in the Finals. Xbot will then win in stage 6 (like Ybot did before) and win the finals, but that will be the first loss of Ybot, so Ybot will get a rematch against Xbot (53 in the bracket). If they then also draw all of their games, then Ybot will win the tournament. Ybot is then a bit lucky, but only after drawing against Xbot in 13*3=39 games (at that point they both know they're equal).

I'm not against changing the rules in a future Lichess Bots Championship (I won't change them during the tournament because that wouldn't be fair), I just wanted to explain why the rules are the way they are (they aren't perfect, but some thought has gone into them). I would encourage others to create bot tournaments with other rules (I would like it if there were more bot tournaments and different kinds of bot tournaments).

Maybe participants can also create the perfect bot tournament for themselves. Anyone can start a bots tournament where the participants choose their own rules for their matches (only the opponent of the match has to agree). They can write their own rules or use existing rules like the ones used by the 2nd Lichess Bots Championship, Chess World Cup 2017, or TCEC Season 14. The participants just put the rules they use in the tournament forum and the opponent replies to the post that they agree with the rules. As a participant you can look for your own opponents that will agree with your rules. Both players also post links to the games in the tournament forum so the Judges can decide if the matches were played by the rules of the match players. The Judges are the participants of the tournament who have won at least one game in a bot tournament (so every Judge has to have played a fair game at least once). They also decide when there's a disagreement between players and they decide which match will be the final (some bots will keep on winning, they will have to win against each other to become the tournament winner). The Judges decide by voting in the tournament forum within (for example) 1 week when there's something they have to decide on. To make the matches of the tournament more interesting (a bot can't keep challenging the same weaker bots to get higher in the tournament), bots can only challenge other bots that have at least 1 less win in the tournament than themselves and at most 1 more win than themselves. You can choose to only challenge new or established engines or there could be several of these tournaments (one for each division). Participants have to choose their rules and opponents themselves and Judges aren't forced to do anything (it's a right, not an obligation), so nobody has to do much to make this happen (just start a bots tournament team).
I have also told Ajile about the 3rd improvement but there are not many self-made engines and they should send the code to Ajile to verify that they are not cheating. The armageddon I think should be like the WCC 5+3 for white and 4+3 for black. The time controls are strange but I don't know if the should be changed because for example I lost game 3 against @sugarbot-raspi3 because every once in a while my internet restarts for an unknown reason and it happened during the game so I lost (my engines was always playing a move at near 5 secs and then went to 20 secs. Link to the game: lichess.org/XKLMpCPYeEag).
I don't think that higher increment helps with lag spikes or similar things because most engines time managements will have them use more time when playing with increment (which makes sense because you get some time back every move). So if your connection goes down the higher reserve in less increment games should counter balance the less buffer per move through the increment.
@Hoplites in armageddon it makes sense to be casual since there for one side a draw is the same as a loss so that side has to go all out for win rather than going for objectively best moves.
Not sure what you mean with championship clocks, I never in my life have seen any serious tournament with more increment seconds than starting minutes. It's not a big deal of course but I do feel like less increment and more starting time to be a more natural time control.
The reason why I feel the no time odds Armageddon is a bit problematic is because at the highest level the match is probably go that distance. Once we get to top hardware SF or Leela most games will be drawn so I think in the final rounds we will see lots of these coin flips. But of course a tournament with so few games will be decided by luck anyway so it's not that much of a difference. (even though in my opinion an easily avoidable one)

As for self written engines, I also thought there were few but actually I already have seen quite a few in this tournament. I think it would be feasible to have such a division, also agreed with the sending Ajile the source code.

This topic has been archived and can no longer be replied to.