<div dir="ltr">Please consider merging your code with Warren D. Smith's election simulator available at <a href="https://www.rangevoting.org/IEVS/IEVS.c">https://www.rangevoting.org/IEVS/IEVS.c</a></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Jan 4, 2021 at 12:15 AM VoteFair <<a href="mailto:electionmethods@votefair.org">electionmethods@votefair.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">I've written a C++ program that generates randomized ballots, feeds <br>
these ballots to a separate program that calculates a winner according <br>
to various vote-counting methods, compares the results, and calculates <br>
failure/success rates.<br>
<br>
The program does two kinds of tests:<br>
<br>
* IIA: Tests successes/failures according to the Independence of <br>
Irrelevant Alternatives (IIA) criterion. Specifically it calculates <br>
which candidate would win with all the candidates, and then it removes <br>
each of the non-winning candidates, one by one, to test whether a <br>
different candidate would win. If any of the comparisons yield a <br>
different result -- such as a 6-candidate contest giving a different <br>
winner compared to the 7-candidate contest that uses the same ballots <br>
(with one candidate omitted in the 6-candidate contest) -- then that's <br>
counted as one failure.<br>
<br>
* "Agree/Disagree": Tests how often one counting method yields a winner <br>
who is the same as, or different from, the winner according to another <br>
vote-counting method.<br>
<br>
ASSUMPTIONS/CONDITIONS:<br>
<br>
The randomized ballots assume that the voters are expressing their <br>
sincere preferences, without any tactical voting, and without ranking or <br>
rating two candidates at the same preference level.<br>
<br>
For both tests, when a tie occurs for the winner of either test case, <br>
the tied case is ignored. This means ties do not count as a failure or <br>
a success.<br>
<br>
For my purposes I've used 4,000 randomized cases per test, and each test <br>
uses 17 ballots. Unless otherwise specified, the test (or full test in <br>
the case of IIA) uses 7 candidates.<br>
<br>
IIA RESULTS:<br>
<br>
The Independence of Irrelevant Alternatives (IIA) success rate of the <br>
Condorcet-Kemeny method is 79%, which is a failure rate of 21%.<br>
<br>
The IIA success rates of the following methods are all about 1% or 2%, <br>
which is a failure rate of 99% or 98%:<br>
<br>
* IRV: Instant Runoff Voting<br>
<br>
* IPE: Instant Pairwise Elimination (described in ElectoWiki)<br>
<br>
* IRMPL: Instant Runoff Minus Pairwise Loser, which uses PLE (see below) <br>
as a safety net under IRV.<br>
<br>
* STAR: Score Then Automatic Runoff<br>
<br>
Another method, PLE, which is an abbreviation for Pairwise Loser <br>
Elimination, has a calculated 100% success rate, which is zero failures. <br>
This method successively eliminates the Condorcet loser, one round at <br>
a time, and stops with a tie when it encounters a Condorcet <br>
(rock-paper-scissors-like) cycle. This perfect success rate occurs <br>
because tied cases are ignored, which leaves only cases that have no <br>
cycles at any level, which means the method finds the Condorcet winner <br>
in both the 7-candidate case and the 6-candidate case.<br>
<br>
Conclusion: Methods that eliminate one candidate at a time frequently <br>
fail the Independence of Irrelevant Alternatives (IIA) criterion. In <br>
contrast, the Condorcet-Kemeny method, which uses all the pairwise <br>
counts (not just the biggest or smallest pairwise counts or differences <br>
between pairwise counts), yields a dramatically better IIA success rate.<br>
<br>
Important: In real elections the success rates would be higher -- there <br>
would be fewer failures -- because real elections have meaningful <br>
differences between candidates. Remember that this software randomizes <br>
the ballots without any bias. This means that almost all the test cases <br>
are "semi-balanced" or "sitting on the fence" (or maybe "finding the <br>
highest sand dune rather than finding the highest mountain") kinds of cases.<br>
<br>
Advocates of STAR voting may claim these numbers are not meaningful for <br>
STAR voting because STAR voting uses Score ballots, not ranked ballots. <br>
(On Score ballots the gap between preference levels is significant.) <br>
Yet fans of STAR voting also claim that its use of a top-two runoff <br>
discourages tactical voting, particularly the tactic of favoring the use <br>
of high and low preference levels, and avoiding the use of middle <br>
preference levels. Keeping in mind that these tests assume the voters <br>
are voting sincerely, I believe these two claims are contradictory. <br>
(Feedback on this or any other part of this message is welcome.)<br>
<br>
AGREE/DISAGREE RESULTS:<br>
<br>
Below are the results from the "Agree/Disagree" test, which in my tests <br>
compare the Condorcet-Kemeny winner with the winner from each of the <br>
indicated vote-counting methods. Specifically the "agree" percentages <br>
refer to matches with the Condorcet-Kemeny winner, and the "disagree" <br>
percentages apply when the method identifies a different winner. (By <br>
definition the Condorcet-Kemeny method would yield 100% agreement.)<br>
<br>
About the "ties" numbers specified in parentheses: They are counts out <br>
of 4,000 cases, not percentages. These tied cases are not counted in <br>
either the success or failure percentages.<br>
<br>
Note that when there are only two candidates, all the methods always agree.<br>
<br>
number of candidates: 2<br>
IPE agree/disagree: 100% 0% (0 ties)<br>
IRMPL agree/disagree: 100.0% 0% (0 ties)<br>
STAR agree/disagree: 100.0% 0% (0 ties)<br>
IRV agree/disagree: 100.0% 0% (0 ties)<br>
PLE agree/disagree: 100.0% 0% (0 ties)<br>
<br>
number of candidates: 3<br>
IPE agree/disagree: 95.1% 4.8% (0 ties)<br>
IRMPL agree/disagree: 95.7% 4.2% (0 ties)<br>
STAR agree/disagree: 95.2% 4.7% (296 ties)<br>
IRV agree/disagree: 93.0% 6.9% (643 ties)<br>
PLE agree/disagree: 100.0% 0% (286 ties)<br>
<br>
number of candidates: 4<br>
IPE agree/disagree: 92.5% 7.4% (59 ties)<br>
IRMPL agree/disagree: 91.3% 8.6% (9 ties)<br>
STAR agree/disagree: 94.8% 5.1% (440 ties)<br>
IRV agree/disagree: 84.0% 15.9% (1582 ties)<br>
PLE agree/disagree: 100.0% 0% (943 ties)<br>
<br>
number of candidates: 5<br>
IPE agree/disagree: 92.3% 7.6% (103 ties)<br>
IRMPL agree/disagree: 88.9% 11.0% (14 ties)<br>
STAR agree/disagree: 93.6% 6.3% (435 ties)<br>
IRV agree/disagree: 77.7% 22.2% (2485 ties)<br>
PLE agree/disagree: 100.0% 0% (1724 ties)<br>
<br>
number of candidates: 6<br>
IPE agree/disagree: 90.6% 9.3% (172 ties)<br>
IRMPL agree/disagree: 84.9% 15.0% (27 ties)<br>
STAR agree/disagree: 91.4% 8.5% (420 ties)<br>
IRV agree/disagree: 69.7% 30.2% (3203 ties)<br>
PLE agree/disagree: 100.0% 0% (2513 ties)<br>
<br>
number of candidates: 7<br>
IPE agree/disagree: 88.7% 11.2% (219 ties)<br>
IRMPL agree/disagree: 81.6% 18.3% (67 ties)<br>
STAR agree/disagree: 89.4% 10.5% (441 ties)<br>
IRV agree/disagree: 59.5% 40.4% (3517 ties)<br>
PLE agree/disagree: 100.0% 0% (3063 ties)<br>
<br>
As the number of candidates increases, the methods more often disagree <br>
with the Condorcet-Kemeny method. So the bottom numbers, where there <br>
are 7 candidates, are the most revealing.<br>
<br>
The bottom numbers show that IRV -- Instant Runoff Voting -- is the <br>
worst of these methods. It agrees in about 60 percent of the non-tie <br>
cases. The other three methods -- IPE, IRMPL, and STAR -- have similar <br>
success rates of about 80 or 90 percent.<br>
<br>
Of course this result -- that IRV is not a good vote-counting method -- <br>
is not surprising. Yet it's nice to have numeric confirmation.<br>
<br>
LINKS:<br>
<br>
Here are links to the two programs used in these tests:<br>
<br>
<a href="https://github.com/cpsolver/VoteFair-ranking-cpp/blob/master/generate_random_ballots.cpp" rel="noreferrer" target="_blank">https://github.com/cpsolver/VoteFair-ranking-cpp/blob/master/generate_random_ballots.cpp</a><br>
<br>
<a href="https://github.com/cpsolver/VoteFair-ranking-cpp/blob/master/votefair_ranking.cpp" rel="noreferrer" target="_blank">https://github.com/cpsolver/VoteFair-ranking-cpp/blob/master/votefair_ranking.cpp</a><br>
<br>
GOAL:<br>
<br>
My hope is that this software takes us a step closer to yielding numbers <br>
to better characterize HOW OFTEN each vote-counting method passes or <br>
fails each of the "fairness" criteria, the ones that are currently <br>
flagged as "yes" or "no" in this comparison table:<br>
<br>
<a href="https://en.wikipedia.org/wiki/Comparison_of_electoral_systems#Compliance_of_selected_single-winner_methods" rel="noreferrer" target="_blank">https://en.wikipedia.org/wiki/Comparison_of_electoral_systems#Compliance_of_selected_single-winner_methods</a><br>
<br>
I realize the numbers calculated by my software are not suitable as <br>
estimates for real-life elections -- because randomized ballots and <br>
randomized candidates do not match real-life elections. Yet these <br>
calculated numbers provide a peek at ways to compare methods more <br>
meaningfully than just flagging methods as pass-or-fail.<br>
<br>
THANKS:<br>
<br>
If you find any software bugs, please tell me, either here or on GitHub.<br>
<br>
Feedback is welcome. That's why I've posted these results here.<br>
<br>
Richard Fobes<br>
----<br>
Election-Methods mailing list - see <a href="https://electorama.com/em" rel="noreferrer" target="_blank">https://electorama.com/em</a> for list info<br>
</blockquote></div>