[EM] Elections methods performance criterion [was Arrow's axioms]

[Ken Johnson]

>Date: Fri, 5 Mar 2004 17:51:15 -0800 (PST)
>From: Forest Simmons <fsimmons at pcc.edu>
>...
>
>We can use a less fuzzy concept than "social utility" to be measured by CR
>ballots.  If we take the CR values to be probabilities of adequate
>representation, then the sum over all ballots of the CR for a candidate is
>the expected number of voters that will be adequately represented by the
>candidate.  We might better call this expectation "social representation"
>rather than "social utility."
>
>  
>
This interpretation is convenient because it automatically puts 
individuals' CRs on a common scale of 0 to 1, deftly sidestepping the 
problem of "interpersonal comparison of utilities" that Steve Eppley 
mentioned in his post. However you define it, I think the cardinal 
rating concept provides a useful basis for defining the objective of 
election systems (i.e., maximize "social utility" or "social 
representation"), even if CR is not itself a practical voting method. 
This definition provides a quantifiable performance standard by which 
the merits of alternative voting systems can be characterized and compared.

Cardinal ratings are, in my view, a more useful definitional standard 
than rank-based standards, because of their greater expressive power. 
The ratings imply a transitive rank ordering, and ratings distinguish 
degrees of preference (strong or weak). In addition, there is a more 
direct connection between CRs and the fundamental concept of social 
utility or representation.

Although it may not be possible, in practice, to ascertain voters' 
sincere CRs, one could predict the outcome of any particular voting 
method given some presumed sincere CR profile and could quantify the 
outcome's average CR over a statistical ensemble of all possible CR 
profiles. I ran some simple numerical simulations to test out this idea. 
Results are tabulated below. (Note: There is also a separate 
election-methods thread relating to simulations that Kevin Venzke has 
been doing. I'm not sure how this relates to his results, but my model 
is probably much more rudimentary.)

For each test case I assumed a certain number of voters and candidates 
and I ran a Monte Carlo simulation of 100,000 elections with random 
(uniformly distributed) CRs. Following is a list of the election methods 
that I tested:

(1) Sincere CR, "SiCR". This is not a practically realizable method 
because voters will strategize, but it is the standard by which the 
other methods are evaluated.
(2) Strategic CR, "StCR". This only represents the most trivial voting 
strategy: exaggerate. Individuals' CRs are linearly scaled so that the 
lowest and highest values are at 0 and 1.
(3) Sincere Approval Voting, "SiAV". Sincere CRs are rounded to 0 or 1. 
Voters can potentially approve all candidates or approve none.
(4) Strategic Approval Voting, "StAV". Same as SiAV, except applied to 
exaggerated CRs. In other words, the approve/disapprove CR cutoff level 
is the average of the highest and lowest sincere CRs.
(5) Majority. This is a Condorcet method with "Smith/MinMax" cycle 
resolution. If there is no Condorcet winner, consideration is limited to 
the candidates who win the most two-way contests, and the one who 
suffers the least-worst defeat against any other candidate is selected. 
(This may not be the best representative of Condorcet methods, I just 
picked it for testing purposes.)
(6) Plurality.
(7) Random.

All ties are resolved by random choice.

Trial runs are tabulated below. The "SiCR" row represents the SiCR 
winner's CR, averaged over all voters. The "mean" is an average, and the 
"range" is the minimum and maximum, over 100,000 simulated elections. 
The remaining rows represent the amount by which the CR values decrease, 
relative to SiCR, for each method (e.g. "SiCR-StCR" represents the mean 
and range of the difference between the SiCR winner's and the StCR 
winner's average CRs). A small value in these rows is good - it 
indicates that the method's performance is close to the SiCR standard.

As expected, the relative performance of Plurality is noticeably 
degraded when there are more than two candidates. With three candidates, 
Approval appears to have slightly better performance than Majority. When 
there are ten candidates, Majority performs better. This is probably due 
to the large number of ways 10 candidates can be ranked (3628800 
possible strict preference rankings, plus indifference rankings - this 
compares to 1024 possible approval ratings). Of course, the "crisis of 
choice" created by this huge number of ranking options may, to some 
extent, be detrimental. If voters cannot discern their preferences on a 
sufficiently fine scale, they may vote indifference between many 
candidates, or may express many preferences randomly; and this might 
somewhat dilute the theoretical performance advantage of Majority when 
there are many candidates.

Is Sincere CR a useful performance metric for evaluating election 
methods, and are there other potentially better comparison criteria that 
people have used?

Ken Johnson


num_voter=10
num_candidate=2
SiCR:             mean 0.551700332840196     range [0.229040232984872 
0.863446203564651]
SiCR-StCR:        mean 0.0104330518092656    range [0 0.311396666442433]
SiCR-SiAV:        mean 0.00758379245390728   range [0 0.234348255440491]
SiCR-StAV:        mean 0.0105441802090586    range [0 0.311396666442433]
SiCR-Majority:    mean 0.0105302529155023    range [0 0.281387064795814]
SiCR-Plurality:   mean 0.0105613367781385    range [0 0.268875109478149]
SiCR-Random:      mean 0.0517924800564136    range [0 0.567096745384255]
 
num_voter=10
num_candidate=3
SiCR:             mean 0.577627811423126     range [0.300739113037274 
0.891234944164729]
SiCR-StCR:        mean 0.00643898594501008   range [0 0.208600214629151]
SiCR-SiAV:        mean 0.0112306660540103    range [0 0.2492314143065]
SiCR-StAV:        mean 0.010234125996482     range [0 0.261924538759141]
SiCR-Majority:    mean 0.0118451864266665    range [0 0.268048027870865]
SiCR-Plurality:   mean 0.0175346182631857    range [0 0.316315966603937]
SiCR-Random:      mean 0.076927965299296     range [0 0.569005124951544]
 
num_voter=100
num_candidate=3
SiCR:             mean 0.524253354378763     range [0.441475239831965 
0.631821479584611]
SiCR-StCR:        mean 0.00212956123814931   range [0 0.068479744242475]
SiCR-SiAV:        mean 0.0033425211375264    range [0 0.0803792567385083]
SiCR-StAV:        mean 0.00304235085706283   range [0 0.0730502008335385]
SiCR-Majority:    mean 0.00394114964985563   range [0 0.0877621552632714]
SiCR-Plurality:   mean 0.00588016175577818   range [0 0.121080952692736]
SiCR-Random:      mean 0.0242776145281066    range [0 0.183426720826094]
 
num_voter=100
num_candidate=10
SiCR:             mean 0.544034472376144     range [0.481335227230615 
0.633085126414266]
SiCR-StCR:        mean 0.000404771738675042  range [0 0.0215636099063402]
SiCR-SiAV:        mean 0.00611560766074788   range [0 0.0818549382408317]
SiCR-StAV:        mean 0.00579899505919091   range [0 0.0811189374850716]
SiCR-Majority:    mean 0.0041345604560462    range [0 0.0694586480805467]
SiCR-Plurality:   mean 0.020864388384224     range [0 0.155759903041651]
SiCR-Random:      mean 0.044254682627219     range [0 0.20199827990111]