Final knockout tournament clarification
Algorhythm-sxv opened this issue · comments
Hi, I noticed that the FAQ mentions that
Some percentage of the top bots will then be promoted to a second knock-out tournament
Is a knock-out format really appropriate? My concern is that a knock-out tournament cannot determine any placement except the top bot (i.e. you get no information about the second-best bot, since the second-best bot can be paired with the best bot and get knocked out early).
I feel that being able to definitively rank the top engines, as well as quantify strength differences between them would provide more interesting material for a video as well as more satisfying results for the competitors.
As one possible alternative to a knock-out tournament, TCEC (the premier computer chess competition) uses a multiple double-round-robin format, where each engine will play N openings from both sides against all other engines. The drawback of this particular format is that the number of games grows quite quickly with the number of competitors. An N-times double round robin with 32 engines would require nearly 2000*N games to complete. Depending on the number of cores available for the tournament this may not be feasible.
Other suggestions and discussions are welcome!
I totally agree that round robin would be the ideal format, but I am concerned about how long it would take to run. Will make a final decision once I'm able to get a sense of the number of engines that are in contention for the top spot. I am open to hearing any suggestions though.
I'm sure you would have people lining up to donate compute, and the chess programming community have some group servers that may help mitigate the bias from private hosting