Ratings as a Standard was: Re: [EM] FWD: Borda Count by Paul Dumais
Blake Cretney
bcretney at postmark.net
Thu May 6 11:39:17 PDT 1999
Bart Ingles wrote:
> In the following ratings example:
>
> Candidates
> group votes A B C
> I 50 100 95 0
> II 50 5 100 0
>
> By arguing that ratings are irrelevant, you are saying that the above
> example is an exact tie between A and B. To me this is not even a close
> contest.
That is our point of disagreement. More on this below, but let me
point out that if not for candidate C, this would have been an exact
tie in ratings as well.
>
> Suppose groups I and II each have a list of 20 issues that they consider
> equally important, and that candidates A and B are otherwise equal in
> education, honesty, etc. Group I agrees with candidate A on all 20
> issues, and with B on only 19 of the 20. Group II only agrees with
> candidate A on one of the 20. It should be clear that we are not
> measuring levels of certainty, but levels of agreement.
>
> If you want to inject levels of uncertainty into this argument, you
> could suppose that the voters were potentially wrong, or wrong about the
> candidates, on (say) two of the 20 issues. Then the ratings would be
> accurate to within +/- 10 points. If you clip the ratings at 100, that
> would mean that the correct group I score for A could be as low as 90,
> and for B as high as 100. In this were to happen, then 50% of the
> votes would be absolutely wrong under a ranked system. A rated system
> could treat group I's A/B vote as less significant, thereby minimizing
> this error.
Remember how I defined a "wrong" or "right" choice:
> > > > If you look at the issue from my perspective, the problem is that
> > > > people often will make wrong decisions, but are at least slightly
more
> > > > likely to make right ones. So, when a person says that
> > > > A>B
> > > > you can use this as evidence that A is in fact better than B. Note
> > > > that the assumption here is that people are attempting to find the
> > > > best candidate from a global perspective, but may get the answer
> > > > wrong, due to self-interest or ignorance.
That is, I am defining a correct choice as the true best choice, not
merely an accurate representation of the individual voter's thinking.
Since what we want to find is the best candidate, this makes sense.
But if we look at the more narrowly defined kind of wrong vote that
your example is interested in, I still think it is handled well by
ranking. I do not agree with the contention that "50% of the votes
would be absolutely wrong". Unless error is playing favourites, it
will serve to elevate A further over B at least as often as it
elevates B over A. Since more of a change is necessary to make B over
A then to keep A over B, the result will be simply to reduce the
margin of A's victory over B. This is desirable if the difference
between the two is so shaky, and tends to replicate some of the effect
you hope to get from Ratings.
To give a more familiar example of a similar phenomenon, if a poll is
published rating candidate A and B 10% apart, and the poll is rated as
+-10% 19 times out of 20, this does not mean that A and B have an
equal chance of being in the lead. Similarly, just because the A and
B ratings fall within your accuracy threshold does not mean that
either ranking is equally likely.
>
> If you don't believe in ratings, then presumably you wouldn't allow
> equal rankings (an oxymoron?), since you would in effect be allowing
> voters to make ratings decisions (i.e. how high does B have to be rated
> to deserve equal ranking with A? 67 points? 75?)
I see equal rankings as useful for voters who don't want to make a
decision between two candidates, because they have limited
information. Often elections will have a large number of fringe
candidates, and it is easier for voters if they don't have to rank
them all.
I don't see an equal ranking as an expression of "these candidates
are more similar than they are different" or anything like that. It
is purely a convenience. In other words, any difference in
preference, no matter how small, can justify different rankings.
>
> If you want to argue that group I's rating for B cannot be accurate to
> within 10 points, and that we can only say that it is between 0 and 100,
> then how could the voters possibly handle a ranking situation with 50
> candidates? In order to do so, they would have to resolve differences
> between candidates to an equivalent of 5 points, on average.
Whether or not people can translate their opinions about candidates
into precise ratings is not my point. Remember that I define the
correctness of a vote by how close it is to the best outcome, not by
how close it is to the voter's personal opinion.
Although I would expect some relation between voter's ratings and
actual suitability, I wouldn't place any upper bounds on the error
possible by a voter. For example, in a two candidate race.
A B
I 100 0
II 0 100
If candidate A is the best candidate, then group II is way off. If
candidate B is best, group I is way off. Any attempt to say, voters
will always be accurate within 10 points is meaningless. This is why
I am particularly concerned that Ratings based methods, even without
strategy, are unduly affected by voters who overestimate the
importance of single issues or overestimate the difference between
similar candidates.
Also, I don't think most voters could rank 50 candidates with each
ranking being meaningful. Of course, they wouldn't have to if only
some of them are viable., I don't think most voters could rank 50
candidates with each ranking being meaningful. Of course, they
wouldn't have to if only some of them are viable.
> > > I don't see how you can improve accuracy by discarding that
> > > information.
> > >
> > > On the other hand, I am not that caught up with Average Ratings.
Median
> > > Ratings would answer your objection, and still make my point. Of
> > > course, I don't advocate either as election methods, but only use them
> > > as tools to quantify sincere-rated examples.
> >
> > Average ratings matches well with our concepts of finding the
> > candidate with the most support and finding the candidate to maximize
> > [normalized] utility. So, I understood why you would view it as a
> > standard, and I felt it was necessary to criticize it.
> >
> > Median Ratings, however, is just another method. Why should I care
> > if one method is closer to it than another?
>
> Medians are a natural way of evaluating rated examples, since a
> candidate with the highest median rating is by definition the candidate
> rated higher than all others *by a majority of voters*.
If I understand you correctly, you are referring to the fact that if
a single candidate is given the highest rating by a majority of
voters, that candidate will win. Many methods (including plurality)
also give a victory to a majority favourite, so although this is true
of Median Ratings, it hardly defines it.
> It would be an
> good method on its own, if not for a serious strategy weakness. Since
> strategy is not an issue with sincere examples, I believe it is useful
> as a standard, especially when used as a cross-check along Average
> ratings.
Consider the example you mentioned above:
> In the following ratings example:
>
> Candidates
> group votes A B C
> I 50 100 95 0
> II 50 5 100 0
> By arguing that ratings are irrelevant, you are saying that the above
> example is an exact tie between A and B. To me this is not even a close
> contest.
You claim that this is "not even a close contest," but a single voter
could decide this election using Medians.
An additional voter expressing the preference
A B C
100 99 0
would result in an election for A, but
A B C
99 100 0
would result in an election for B.
So, obviously Medians considers these to be very close. I don't see
how you can view Medians as a standard by which to judge other methods
if its results seem so fundamentally irrational to you.
> > > > The "satisfaction maximization" method by which you are interpreting
> > > > the results is as follows. Each person states their satisfaction
with
> > > > each potential result on a 0 to 100 scale. From this point of view
> > > > the only incorrect vote is one that does not accurately describe the
> > > > voters perceived self-interest. This perspective assumes people will
> > > > not vote altruistically.
> > >
> > > Not at all. A voter is just as likely to have the good of the
community
> > > in mind when rating candidates as when ranking. I suppose an outcome
> > > beneficial to the community would provide most voters some
satisfaction,
> > > though, even if not based on self-interest.
> > >
> > > Just to be clear, my scale wasn't based on personal satisfaction, but
on
> > > the individual's assessment of suitability for the office, where 100 is
> > > completely adequate, and 0 is completely useless.
> >
> > Consider the following situation. Candidates A, B, and C are running
> > for office. Their ratings, based purely on self-interest are
> > group votes A B C
> > I 60 100 90 0
> > II 40 90 100 0
> >
> > A 96
> > B 94
> > C 0
> >
> > So, the winner is A. From the Average-Ratings point of view this is
> > also the candidate that provides the greatest average satisfaction.
> >
> > Now, if the group I voters decide to vote based on community
> > interests instead of self-interest, they would use these average
> > ratings as their individual ratings, normalizing them of course. So
> > we get
> >
> > A 96 * 100/96=100
> > B 94 * 100/96=97.9 approx 98
> > C 0 * 100/96=0
> >
> > So, if group I is voting with community interest the votes are
> >
> > group votes A B C
> > I 60 100 98 0
> > II 40 90 100 0
> >
> > A 96
> > B 98.8
> > C 0
> >
> > So, B is the winner. So, group I not only hurt themselves, but hurt
> > the community as a whole by their community interest. I think it's
> > clear from this that Ratings assumes people vote for their
> > self-interest, or it won't work properly.
>
> I think you're off on several counts here. First, you cannot
> mathematically transform selfish votes into altruistic votes simply by
> averaging them together. Second, even if an altruistic vote were the
> same as an average vote, your example would not prove that the original
> votes were selfish -- the original votes could have all been altruistic
> and yielded the same result.
I don't need to prove that the original votes were selfish. This was
assumed. I started by presenting a selfish rating for each candidate,
and pointed out that if everyone voted this way the method would
maximize total satisfaction, or normalized utility. Then I showed
that if some voters actually viewed maximizing total satisfaction as
their goal (as opposed to maximizing their personal satisfaction), the
result would be that they would get neither.
I think that I am on pretty solid ground with my averaging process,
under two assumptions:
1. That group II's perception of their selfish interest is the same
as group I's perception of group II's selfish interest.
2. That utility and normalized (rated from 0 to 100) utility are the
same in the example. Of course, I can just assume they are for this
example, but an argument could be raised if this is rarely the case,
and that therefore my example is atypical.
If this is the case than group II's "altruistic" vote is designed to
maximize utility.
In reality, however, you don't have to agree with my precise method
of averaging. It is obvious that if group I wants to think of the
community interest, this has to be some combination of group I and
group II's selfish interests, because together they are the community.
It is also clear from my example that this means making their votes
more like that of group II, and increasing the likelihood that B will
be elected.
---
Blake Cretney
More information about the Election-Methods
mailing list