Dividing the Pie, Redux

Some time ago, in Dividing the Pie I wrote that my new fairness (C) statistic was not going to be useful for testing the relative fairness of various payout schemes. This seems to be another matter on which I was just wrong.

The argument was that the way the statistic worked was to reward a scheme for getting the most money to the best players, so that if the design had any tendency at all to select the best player for the number one prize, it would always tend, on average, to improve fairness (C) if you took money away from the lower places, which would be occupied on average by lesser players, and added it to the top prize.

This seemed an unsatisfying result to me, and it led me to speculate about some other measure that I might develop that would also observe the common-sense notion that it would be fairer, perhaps in a fairness (B) sense, to spread the money around a little.

It turns out that that’s not needful. Fairness (C) alone can be used to guide the division of a prize fund without necessarily recommending a winner-take-all payout scheme.

In this post, I’ll use an iterative process to determine an optimal payout scheme for the 13-player, unshifted bracket that I’ve been discussing for the last few posts.

The starting point for this was the observation that the design tended to award the third prize to a stronger player, on average, than won the second prize. So, perhaps it would be good to alter the present payout scheme, which divides the prize pool 50%, 30%, and 20% to the champion, the runner up, and the consolation winner, respectively. If the consolation winner is a better player, on average, than the runner up, perhaps it would be fairer to take some of the money that goes to the runner up and give it to the consolation winner instead.

I was wary about what sort of shifts to allow, as I thought that any iterative process would, if unrestrained, simply push all of the money into the prize for the champion. But then I noticed that some of my new payout schemes were returning fairness (C) statistics that were better than I got for a winner-take-all payout. So I decided to run an experiment based on iterative adjustments to see what happened.

This could take a very long time if I didn’t constrain the results somewhat, so I decided to limit my analysis to payouts that paid no more than the three players who are currently paid (the champion, the runner up, and the consolation winner), and which paid each of these places some multiple of 5% of the prize fund.

Here’s the protocol I used. I started with the current payout scheme, 50/30/20, and ran a long simulation, with a million trials to determine the fairness (C) measure:

50/30/20: 53.71

Then I did another six simulations, with all six possibilities for taking 5% from one of the places and giving it to one of the other two:

45/35/20: 54.05
45/30/25: 51.36
55/25/20: 53.30
50/25/25: 50.96
55/30/15: 56.16
50/35/15: 56.42

Million-trial simulations take about an hour to run, so I did only 50,000 trials for each of the six transfer possibilities. But even with those short runs, most of the differences were statistically significant, and I thought the trends were clear. Giving an extra 5% to the consolation winner is an improvement, and it’s better to take it from the runner-up’s prize rather than the champion’s prize. Doing the opposite was the worst option.

So, I took the 50/25/25 payout, and ran a full million-trial simulation so that the new benchmark would be a solid one. The longer run yielded a slightly worse fairness (C) number.

50/25/25: 51.14

I then repeated this process for four more rounds, until no swap improved fairness (C) and further. Here is the sequence of best allocations:

50/30/20: 53.71
50/25/25: 51.14
45/25/30: 50.44
40/30/30: 49.16
35/30/35: 48.46.

My initial thought that this sort of process would devolve, eventually, into a winner-take-all scheme was exactly wrong. It appears that the movement is in the direction of more even distributions, not more uneven ones.

It’s not clear that this exercise is really very useful. Most tourney directors would, I suspect, find a 35/30/35 payout scheme unappealing. But before discarding the idea of fairness testing for payout schemes, I think we should try it on some other designs–perhaps something more (or, possible, less) interesting will happen.