[Election-Methods] Second run of multiwinner proportionality test

[Kristofer Munsterhjelm]

Hello all,

I've rewritten my program that tests the proportionality of PR methods 
by assigning binary issue profiles to voters and candidates and 
comparing the council's proportion of candidates in favor of each issue 
with the proportions of the people.

There were some bugs in my previous version. For one, I incorrectly 
implemented IRV so that it got a higher score than should be the case. 
The new version now puts that which is to IRV as SNTV is to Plurality as 
one of the best methods.

The full results (of those better than the average randomly chosen 
assembly) are:

0.176552    QPQ(div 0.1, multiround)
0.176552    QPQ(div 0.1, sequential)
0.191093    QPQ(div Sainte-L, sequential)
0.209409    QPQ(div Sainte-L, multiround)
0.230898    STV
0.248373    Maj[Eliminate-Plurality]
0.259064    QPQ(div D'Hondt, sequential)
0.26736     Meek STV
0.280724    QPQ(div D'Hondt, multiround)
0.314229    Maj[Plurality]
0.318016    Maj[AVGEliminate-Plurality]
0.358127    Maj[Eliminate-Heisman Trophy]
0.362992    ReweightA[Heisman Trophy]
0.391753    Maj[AVGEliminate-Heisman Trophy]
0.391753    Maj[Heisman Trophy]
0.393261    -- Random candidates --

That's 23 rounds, RMSE, normalized for each round so that 0 is best of 
ten thousand and 1 worst of ten thousand random assemblies. The other 
end (of those most majoritarian) has:

0.70004     Maj[Borda]
0.705089    Maj[AVGEliminate-Borda]
0.709886    Maj[Cardinal-20(norm)]
0.718258    Maj[ER-QLTD]
0.722618    Maj[Cardinal-20]
0.731794    Maj[ER-Bucklin]
0.750181    Maj[Eliminate-VoteForAgainst]
0.758258    Maj[Schulze(wv)]
0.761436    Maj[AVGEliminate-Antiplurality]
0.8394      Maj[Eliminate-Antiplurality]

Some notes on the terminology: Eliminate-X is loser elimination. 
AVGEliminate-X is like Carey's "Q method", only generalized: it 
eliminates all of those with worse than average scores. Maj[X] is the 
simple porting of single-winner X to a multiwinner system, where one 
just picks the n (for a council of n) highest ranked in the social 
ordering. "Heisman Trophy" is the positional system 2, 1, 0, 0, .... 0. 
VoteForAgainst is 1, 0, ..., -1. Antiplurality is 1, 1, 1, ..., 0. 
ReweightA[X] is like RRV, only with positional scores (of positional 
method X) instead of range scores.


The really strange thing here is that my method seems to have a 
substantial small-state (small-party) bias. For instance, QPQ with a 
divisor of 0.1 is scored much better than QPQ with a divisor of 0.5 
(Webster/Sainte-Lague) or with 1 (D'Hondt).

I don't know why that happens, as it's not obvious from the idea 
(generate hidden binary issue profiles, generate ranks of all candidates 
  based on Hamming distance to each candidate, run through election 
method, compare proportions of TRUEs in the issue profiles of the 
assembly to the proportions among the people). Could it be something 
related to the assumption that people vote sincerely? Or is the variety 
of positions so large that in order to get a lower score, it's better to 
elect the one that supports your opinion than one who would deprive you 
of the opinion while smoothing out all other opinions a little bit? But 
if so, then using RMSE to measure party proportionality in multiparty 
states would be flawed, and someone would have written about it.

Other odd results: Ordinary STV scores better than Meek STV (Meek is 
usually considered better) and single-round QPQ scores better than 
multi-round QPQ (where the latter is usually considered better). Some 
methods that fail Droop proportionality score better than ones that pass 
it: namely, IRV (Elimination-Plurality) scores better than Meek STV.

Perhaps the better PR rules do, well, better most of the time, but there 
are some instances where they do much worse. To detect that reliably, 
I'll have to add Pareto-domination tests or median (instead of/in 
addition to average) scores.

It may also be tha 23 rounds is far from enough, but I've run some of 
these longer (up to 500 rounds) and the general position isn't that far 
off. Some times, IRV even gets ahead of STV.

(Here's an example with only QPQ being tested, with 76 rounds:

  0.223125    QPQ(div D'Hondt, sequential)
  0.231456    QPQ(div D'Hondt, multiround)
  0.153442    QPQ(div Sainte-L, sequential)
  0.16047     QPQ(div Sainte-L, multiround)
  0.146262    QPQ(div 0.1, sequential)
  0.146262    QPQ(div 0.1, multiround)

  The small-party bias still seems to hold. And here's a 423-round test 
with Meek and ordinary STV, and IRV:

  0.203673    Maj[Eliminate-Plurality]
  0.208361    STV
  0.220629    Meek STV.)