Where the Bodies are Buried

I recently revised the A drops on my 48 double elimination bracket so that the byes would be more evenly distributed when the bracket was run with only 40 entrants.

To validate the change, I’ve run extensive simulations, and reported the results in a form I haven’t used for a while: the analyzed bracket. It was an interesting exercise, and one that shows how errors of this kind can easily go undetected. The analyzed bracket does, indeed, show the problem, but it’s subtle enough that it would be easy to overlook if you didn’t know where it might lurk.

Here are analyzed lower brackets for a 40 DE run on a 48 bracket, before (4840DElosersOld) and after (4840DElosersNew) the change. The tourney was run with luck = 1 and an elite threshold of 0, with payouts of 50, 30, and 20 for the top three places.

The two brackets look much alike. I’d hoped to find a statistically significant fairness (C) penalty for the old bracket, but because the problem appears in a part of the bracket where expectations are low, the effect can’t be seen in fairness (C). It would be possible to detect it with a payout schedule generous enough to reward those who exit the tourney in round H, but such payout schedules are uncommon enough that I’ve never seen one in a real tourney.

To find the problem without an exotic payout schedule, you’d have to know to look carefully at round H, because that’s where the bodies are buried. Fairness (b:H) for the original bracket, with the bad drops, is 34.3, but only 24.8 for the revised bracket. The round H skill levels top quadrant are [0.771, 0.822, 0.582, 0.822, 0.583, 0.771] for the bracket with the bad drops, and [0.663, 0.824, 0.719, 0.822, 0.720, 0.663] for the revised bracket (and the other quadrants are very similar). Only with a good deal of experience looking at better brackets are you likely to recognize that distribution as pathological.

The lesson here, I think, is the importance of listening sympathetically to player’s complaints. Nine times out of ten, there’s little substance in them, but the tenth time, they may be pointing out a problem that’s otherwise easy to overlook.

In this case, the complaints were about a perceived maldistribution of byes. My original bracket was (as far as I know) no worse than anyone else’s on this point (and significantly better in other respects). I know from experience that the denizens of round H in such brackets are often in an ornery mood. So I initially found it easy to assume that there wasn’t much validity to the objection. But I was wrong. And now that I know that, my brackets will be better.