[EM] Simulations on method similarity 2

Thu May 6 00:10:51 PDT 2010

Hello,

Here's a dump of the results for a new "scenario 7": of the nine ballot
types, only four of them will be used, chosen randomly. (This means
on average between 2-3 of the ballot types will specify a second
preference.) This is a decent simulation of most contrived scenarios, I
think.

One note: There was a bug in my SPST code making it seem more like DSC
and less like FPP than it is. It didn't make that much difference. It's
fixed now...

Each number, again, is percentage of scenarios where two methods agree
among scenarios where there was some disagreement somewhere.

For example, you can see below that FPP and Approval almost always
disagreed (10%). FPP usually agrees with DSC, VFA, and SPST (92%+).

Approval doesn't have close friends in this scenario, but its usual match
Bucklin is still the closest one (49%).

IRV most closely matches margins (85%), then VFA, DSC, WV, C//A, MMPO,
SPST, and CdlA (80%+). Approval is the furthest (18%).

The Condorcet methods, MMPO, and CdlA stick together (90%+). Generally 
IRV, Bucklin, DAC, MAP have some proximity after that.

Bucklin is closest to DAC, C//A, CdlA (85-88%), then MAP and WV (80-81%).

Positional analysis favors Bucklin and DAC (73-74%).

fpp 0  10.7  78.3  68.6  69  73.3  69.6  62.6  59.5  92.7  73.5  94.1  59.5
 92.9  66.9
appr 10.7  0  18.2  26.6  28.4  27.4  34.9  48.1  49.4  15.8  36.6  13.7  30  16.7  33.7
irv 78.3  18.2  0  82.9  83.2  85.5  83  70  63  83.7  75.9  84.2  69.4  82.5  81.4
mmpo 68.6  26.6  82.9  0  97.4  93.4  91  78.4  66.4  74.9  79.3  72.7  84.9  75  92.8
schwv 69  28.4  83.2  97.4  0  94.6  93.5  80.3  67.4  75.1  81.1  72.9  85
 75.9  94.7
schm 73.3  27.4  85.5  93.4  94.6  0  92.4  79.2  69.7  79.4  80.3  77.2  81.2  80.3  91
ca 69.6  34.9  83  91  93.5  92.4  0  86.7  70.6  75.6  83.5  73.4  83.7  76.5  96.3
buck 62.6  48.1  70  78.4  80.3  79.2  86.7  0  74.2  67.6  88.5  65.6  81.8  68.5  85.5
pos 59.5  49.4  63  66.4  67.4  69.7  70.6  74.2  0  64.2  73.5  62.4  62.4
 64.9  67.8
dsc 92.7  15.8  83.7  74.9  75.1  79.4  75.6  67.6  64.2  0  79  97.7  64.6
 98.5  71.9
dac 73.5  36.6  75.9  79.3  81.1  80.3  83.5  88.5  73.5  79  0  76.9  74.7
 79.4  81.7
vfa 94.1  13.7  84.2  72.7  72.9  77.2  73.4  65.6  62.4  97.7  76.9  0  62.5  96.8  69.8
map 59.5  30  69.4  84.9  85  81.2  83.7  81.8  62.4  64.6  74.7  62.5  0
 65.5  86
spst 92.9  16.7  82.5  75  75.9  80.3  76.5  68.5  64.9  98.5  79.4  96.8  65.5  0  72.8
cdla 66.9  33.7  81.4  92.8  94.7  91  96.3  85.5  67.8  71.9  81.7  69.8  86  72.8  0

Below are criteria: Condorcet, MD, Plurality, and election of a candidate
with a majority-strength defeat respectively. They're percentages, and
higher is worse.

For Condorcet, the best non-Condorcet methods are CdlA and MMPO. IRV is
easily next. The worst is Approval (largely due to the nature of the
scenario, I would think), then positional, then FPP and methods close to
FPP.

For MD, some of the methods are perfect (Approval, MMPO, WV, C//A, 
Bucklin, DAC, CdlA). After that the best are MAP, margins, positional,
and IRV. FPP is by far the worst.

For Plurality, most methods are perfect. The ones that fail are MAP (4.8%),
MMPO (1.8%), and margins (0.5%).

For electing candidates who have a majority loss, the best are WV and MMPO
(4.5%), followed by CdlA, MAP, C//A, and margins. Approval is
the worst, followed by FPP, positional, VFA, DSC, SPST, IRV, DAC, and
Bucklin in that order.

fpp 20.962  18.716  0  32.956
appr 65.066  0  0  65.438
irv 6.758  5.972  0  17.506
mmpo 1.46  0  1.828  4.5
schwv 0  0  0  4.5
schm 0  2.65  .51  8.506
ca 0  0  0  7.932
buck 13.222  0  0  13.594
pos 24.874  3.1539  0  30.818
dsc 15.182  12.294  0  26.516
dac 13.892  0  0  15.544
vfa 17.118  12.8599  0  28.452
map 11.118  .648  4.81  7.128
spst 14.284  12.294  0  26.278
cdla 1.24  0  0  6.562

Then here are the most patterns in the 50k trials:

ABAAAAAAAAAAAAA 19052     ABAAAAAABAAAAAA 6652     AAAAAAAAAAAABAA 2185
ABBBBBBBBABABAB 1639     ABABBBBBBABABAB 1419     ABBBBBBBBBBBBBB 1405
ABAAAAABBABAAAA 1025     AABBBBBAAAAABAB 1021     ABBBBBBBBAAABAB 911
ABAAAAABBABABAA 748     ABABBBBBBAAABAB 692     ABAAAAABAABAAAA 584
ABAAAAABAAAABAA 582     ABAAAAABBAAAAAA 558     AABAAAAAAAAAAAA 557
ABABBBBBAAAABAB 511     AAABBAAAAAAABAB 442     ABAAAAABAAAAAAA 440
ABCBBBBBBBBCBBB 431     ABAAAAABAABABAA 386     ABBBBBBBAAAABAB 365
ABAAAABBBAAABAB 343     ABAAAAABBAAABAA 341     ABAAAABBAAAABAB 323
ABBBBBBBBABACAB 293

In plain English:
19052 Approval was all by itself.
6652 Approval and positional were alone.
2185 MAP was alone.
1639 FPP, DSC, VFA, and SPST were alone.
1419 Those four were joined by IRV.
1405 FPP was alone.
1025 Approval, Bucklin, positional, and DAC were alone.
1021 The Condorcet methods, MMPO, IRV, CdlA, and MAP disagreed with 
FPP, Approval, Bucklin, positional, DSC, DAC, VFA, and SPST.
911 FPP, DSC, DAC, VFA, and SPST were alone.
748 Approval, Bucklin, positional, DAC, and MAP were alone.
692 FPP, IRV, DSC, DAC, VFA, SPST against the others.
584 Approval, Bucklin, and DAC alone.
582 Approval, Bucklin, and MAP alone.
558 Approval, Bucklin, and positional alone.
557 IRV is alone (!).
511 FPP, IRV, positional, DSC, DAC, VFA, SPST against the others.
442 MMPO, WV, MAP, and CdlA alone.
440 Approval and positional alone.
431 FPP is alone. IRV and VFA have a second answer. Everyone else has a
third.
386 Approval, Bucklin, DAC, MAP alone.
365 FPP, positional, DSC, DAC, VFA, SPST alone.
343 Approval, C//A, Bucklin, positional, MAP, and CdlA against the others.
341 Approval, Bucklin, positional, and MAP alone.
323 Approval, C//A, Bucklin, MAP, and CdlA alone.
293 MAP is alone. FPP, DSC, VFA, SPST have a second answer. Everyone else
has a third.
(this covers 86% of the scenarios)

Here's an example where IRV stands alone:

47 A>C
33 B>A
2 B>C
18 C>B

IRV is the only method that elects B. Is this bad? This kind of scenario
is actually often used to illustrate burial in Condorcet methods: You
assume that the A>C voters are insincere and actually prefer B to C. In
most methods where this doesn't work, it's because A already wins without
having to change A>B to A>C. IRV is the only method here where the
strategy doesn't work because B wins both before and after.

Here's a scenario where FPP is alone. Every column is a voter; the first
row is the first preference and the second row is the second, if there
is one. (The quantities are rounded so it's possible something appears
to be a tie here, but it really wasn't.) FPP picks A; anything else picks
B.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCC
...................BBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCBBBBBBBBBBBBB

Here's one where the Condorcet methods, MMPO, MAP, and CdlA are alone.
They pick C and everything else picks A.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCC
CCCCCC........................CCCCCCCCCCCCCCCCCAAAAAAAAAAAAAAAA

Here's one where only FPP picks A; IRV and VFA pick B; and everything
else picks C.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCC.CCCCCCCCCCCCCCCCCCCCBBBBBBBBBBBB

Here's one where only MMPO and WV pick B. FPP, IRV, DSC, VFA, SPST pick
C, and everything else picks A.

AAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCC
..........AAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBAAAAAAAAAAAAAAAAAAAAA

An odd one where positional recommends C; Approval, Bucklin, and DAC
pick A; and everything else picks B:

AAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCC
...................AAAAAAAAAAAAACCCCCCCCCC......................

A very divided one: FPP, IRV, DSC, VFA, SPST elect C. Approval, C//A,
Bucklin, DAC, and CdlA pick B. MMPO, WV, margins, positional, and MAP
pick A.

AAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCC
BBBBBBBBBBBBBBBBBBBBBB.............BBBBBBBBBBAAAAAAAAAAAAAAAAAA

Here's one where the methods that fail Plurality do so, and elect B.
Otherwise FPP-like methods plus IRV and positional pick A and the rest
pick C. Note that there is a very tiny B>A faction that you can't see 
here.

AAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCC
..........................CCCCCCCCCCCCCCCCCCCCCCC..............

Here MMPO is alone picking B. Approval, WV, C//A, Bucklin, DAC, MAP,
and CdlA pick C. FPP, IRV, margins, positional, DSC, VFA, SPST pick A.
(Actually I think MMPO is a B-C tie, like the next one...)

AAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCCCC
.................................CCCCCCCCCCCCCCCCC.............

A strange one: IRV, DSC, and VFA pick B. FPP and positional favor
A. Everyone else picks C. MMPO is a BC tie. I find these ones strange
because the "FPP-like methods" (FPP, DSC, VFA, SPST) are split up among
all three choices.

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCC
CCCCCCCCCCC...................CCCCCCCCCCCCCCCCCCCCCCBBBBBBBBBBB

Another one IRV is alone on, in picking A. The FPP-like methods pick B
and the others all pick C.

AAAAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBCCCCCCCCCCC
CCCCCCCCCCCCCCC..........CCCCCCCCCCCCCCCCCCCCCCCCCCCAAAAAAAAAAA

Here's one where IRV and positional are alone in picking A! The FPP-like
methods and margins pick C, and all others pick B. Note that there is
a tiny C>A faction you can't see.

AAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCCCCCCC
.....................AAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBBBBBBBBBBB

Here positional is the only method that wants to elect C. Approval says
B, and everyone else picks A (majority favorite).

AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB.......

DSC and SPST are alone in picking A. FPP, IRV, and VFA pick C, and
everything else picks B:

AAAAAAAAAAAAAAAAAAAAAAABBBBBBBBBBBBBBBBBBCCCCCCCCCCCCCCCCCCCCCCC
BBBBBBBBBBBBBBBBBBBBBBBAAAAAACCCCCCCCCCCC.......................

That's it for now.

Kevin Venzke