[EM] Round robin tournament statistics

[Kristofer Munsterhjelm]

Forest,

You surely know statistics better than I do, so let me ask something 
I've been wondering about for some time. It even somewhat ties into the 
subject you've been discussing.

Say you want to find out who's the best player (team, etc) in a round 
robin tournament. However, arranging matches is expensive, mainly in 
time. So you want to pair two players (teams, etc) against each other 
just enough to be able to decide who is best.

How would you do this?

It seems you could decide upon a confidence level and then have a given 
pair stop playing once you're confident that one of the players in 
question beats the other player. The level would then be picked so that 
one is reasonably sure that all pairwise contests "point the right way" 
(have the right winners). That would be conservative, since methods 
don't necessarily use all the information of every contest.

It gets more difficult when one takes ties into account, though. For 
most games, no pair is exactly tied in the long run, but one could 
imagine a game where if both players cooperate, there's always a tie 
(such as two players in chess agreeing to always do a grandmaster draw, 
based on tit-for-tat reasoning). Then a long run of ties would in itself 
be significant: it means that neither player is (or chooses to be) any 
better than the other. Just eliminating ties from consideration, as you 
did in the winner calculation, wouldn't work because it could take a 
really long time before a non-tie result is granted.