Ratings as a Standard

[Bart Ingles]

Blake Cretney wrote:
> 
> Bart Ingles wrote:
> > In the following ratings example:
> >
> >                   Candidates
> > group votes   A     B     C
> >   I    50      100    95    0
> >  II    50        5   100    0
> >
> > By arguing that ratings are irrelevant, you are saying that the above
> > example is an exact tie between A and B.  To me this is not even a close
> > contest.
> 
> That is our point of disagreement.  More on this below, but let me
> point out that if not for candidate C, this would have been an exact
> tie in ratings as well.

Only if you normalize the ratings.  Since the ratings are intended to
describe a scenario that could be handled differently by different
election methods, it is probably better to leave the numbers alone as
much as is practical, and let the actual methods decide how to handle
the situation.  For example, the group I voters might be more likely to
abstain under some methods.

If you want to start with unlimited absolute ratings, then any such
"Raw" ratings above 100 or below 0 should be handled by compressing or
clipping the values, so that all such values are at or near 100 (or 0),
without disturbing middle ratings.  This should allow the rated
scenarios to predict outcomes under various actual methods, since the
degree to which a candidate is "overqualified" could be considered
irrelevant.

As for the absence of candidate C in the above example, I don't believe
that comparing isolated pairs of candidates is necessarily meaningful. 
For example, if C had stayed out of the race, how do you know that C'
wouldn't have run in his place?  The A vs. B race, with B "normalized"
to zero, would just be a hypothetical construct that presents a
(potentially) distorted view of the voters preferences.


> > Suppose groups I and II each have a list of 20 issues that they consider
> > equally important, and that candidates A and B are otherwise equal in
> > education, honesty, etc.  Group I agrees with candidate A on all 20
> > issues, and with B on only 19 of the 20.  Group II only agrees with
> > candidate A on one of the 20.  It should be clear that we are not
> > measuring levels of certainty, but levels of agreement.
> >
> > If you want to inject levels of uncertainty into this argument, you
> > could suppose that the voters were potentially wrong, or wrong about the
> > candidates, on (say) two of the 20 issues.  Then the ratings would be
> > accurate to within +/- 10 points.  If you clip the ratings at 100, that
> > would mean that the correct group I score for A could be as low as 90,
> > and for B as high as 100.   In this were to happen, then 50% of the
> > votes would be absolutely wrong under a ranked system.  A rated system
> > could treat group I's A/B vote as less significant, thereby minimizing
> > this error.
> 
> Remember how I defined a "wrong" or "right" choice:
> > > > > If you look at the issue from my perspective, the problem is that
> > > > > people often will make wrong decisions, but are at least slightly
> more
> > > > > likely to make right ones.  So, when a person says that
> > > > > A>B
> > > > > you can use this as evidence that A is in fact better than B.  Note
> > > > > that the assumption here is that people are attempting to find the
> > > > > best candidate from a global perspective, but may get the answer
> > > > > wrong, due to self-interest or ignorance.
> 
> That is, I am defining a correct choice as the true best choice, not
> merely an accurate representation of the individual voter's thinking.
> Since what we want to find is the best candidate, this makes sense.

That's fine, if you have an oracle to tell you who the best candidate
is.  The point of democracy is to let the voters decide who is the
best.  Why wouldn't you want an accurate representation of the voters'
thinking?


> But if we look at the more narrowly defined kind of wrong vote that
> your example is interested in, I still think it is handled well by
> ranking.  I do not agree with the contention that "50% of the votes
> would be absolutely wrong".  Unless error is playing favourites, it
> will serve to elevate A further over B at least as often as it
> elevates B over A.  Since more of a change is necessary to make B over
> A then to keep A over B, the result will be simply to reduce the
> margin of A's victory over B.  This is desirable if the difference
> between the two is so shaky, and tends to replicate some of the effect
> you hope to get from Ratings.

Not all errors are random, especially within a single election.  The
voters who rate A over B could all be using the same bad information,
for example.  You could say that the errors are random across many
elections, of course, so that the likelihood that such an error would
have caused a reversal of position would always be less than 50%, but
would approach 50% when A and B approach equality in the ratings.  Only
errors that caused a reversal of position would change the outcome of
the election (at least in the example).

> To give a more familiar example of a similar phenomenon, if a poll is
> published rating candidate A and B 10% apart, and the poll is rated as
> +-10% 19 times out of 20, this does not mean that A and B have an
> equal chance of being in the lead.  Similarly, just because the A and
> B ratings fall within your accuracy threshold does not mean that
> either ranking is equally likely.
> 
> >
> > If you don't believe in ratings, then presumably you wouldn't allow
> > equal rankings (an oxymoron?), since you would in effect be allowing
> > voters to make ratings decisions (i.e. how high does B have to be rated
> > to deserve equal ranking with A?  67 points?  75?)
> 
> I see equal rankings as useful for voters who don't want to make a
> decision between two candidates, because they have limited
> information.  Often elections will have a large number of fringe
> candidates, and it is easier for voters if they don't have to rank
> them all.
> 
> I don't see an equal ranking as an expression of "these candidates
> are more similar than they are different" or anything like that.  It
> is purely a convenience.  In other words, any difference in
> preference, no matter how small, can justify different rankings.

But when a scenario with nearly equally-rated candidates A and B is
transposed onto an election using rankings, the question of whether or
not A is ranked over, under, or equal to B will depend entirely on
strategy incentives and on how closely rated the two are.  If the two
are close enough, strategy will be more important to the voter than true
ranking.  Better to provide some incentive for equal ranking in such
cases, than to encourage coin-flipping or to allow strategy to take
priority.

> > If you want to argue that group I's rating for B cannot be accurate to
> > within 10 points, and that we can only say that it is between 0 and 100,
> > then how could the voters possibly handle a ranking situation with 50
> > candidates?  In order to do so, they would have to resolve differences
> > between candidates to an equivalent of 5 points, on average.
> 
> Whether or not people can translate their opinions about candidates
> into precise ratings is not my point.  Remember that I define the
> correctness of a vote by how close it is to the best outcome, not by
> how close it is to the voter's personal opinion.

Yes, but you don't show how overriding the voters' opinions would yield
a better candidate.


> Although I would expect some relation between voter's ratings and
> actual suitability, I wouldn't place any upper bounds on the error
> possible by a voter.  For example, in a two candidate race.
>      A    B
> I   100   0
> II  0     100
> 
> If candidate A is the best candidate, then group II is way off.  If
> candidate B is best, group I is way off.  Any attempt to say, voters
> will always be accurate within 10 points is meaningless.  This is why
> I am particularly concerned that Ratings based methods, even without
> strategy, are unduly affected by voters who overestimate the
> importance of single issues or overestimate the difference between
> similar candidates.
> 
> Also, I don't think most voters could rank 50 candidates with each
> ranking being meaningful.  Of course, they wouldn't have to if only
> some of them are viable., I don't think most voters could rank 50
> candidates with each ranking being meaningful.  Of course, they
> wouldn't have to if only some of them are viable.
> 
> > > > I don't see how you can improve accuracy by discarding that
> > > > information.
> > > >
> > > > On the other hand, I am not that caught up with Average Ratings.
> Median
> > > > Ratings would answer your objection, and still make my point.  Of
> > > > course, I don't advocate either as election methods, but only use them
> > > > as tools to quantify sincere-rated examples.
> > >
> > > Average ratings matches well with our concepts of finding the
> > > candidate with the most support and finding the candidate to maximize
> > > [normalized] utility.  So, I understood why you would view it as a
> > > standard, and I felt it was necessary to criticize it.
> > >
> > > Median Ratings, however, is just another method.  Why should I care
> > > if one method is closer to it than another?
> >
> > Medians are a natural way of evaluating rated examples, since a
> > candidate with the highest median rating is by definition the candidate
> > rated higher than all others *by a majority of voters*.
> 
> If I understand you correctly, you are referring to the fact that if
> a single candidate is given the highest rating by a majority of
> voters, that candidate will win.  Many methods (including plurality)
> also give a victory to a majority favourite, so although this is true
> of Median Ratings, it hardly defines it.
> 
> > It would be an
> > good method on its own, if not for a serious strategy weakness.  Since
> > strategy is not an issue with sincere examples, I believe it is useful
> > as a standard, especially when used as a cross-check along Average
> > ratings.
> 
> Consider the example you mentioned above:
> > In the following ratings example:
> >
> >                   Candidates
> > group votes   A     B     C
> >   I    50      100    95    0
> >  II    50        5   100    0
> > By arguing that ratings are irrelevant, you are saying that the above
> > example is an exact tie between A and B.  To me this is not even a close
> > contest.
> 
> You claim that this is "not even a close contest," but a single voter
> could decide this election using Medians.
> 
> An additional voter expressing the preference
>         A     B    C
>         100   99   0
> would result in an election for A, but
>         A     B    C
>         99    100  0
> would result in an election for B.
> 
> So, obviously Medians considers these to be very close.  I don't see
> how you can view Medians as a standard by which to judge other methods
> if its results seem so fundamentally irrational to you.

It doesn't follow ratings changes as well as Averages, but seems to come
closer than ranked methods.  If you view majority rule as having
absolute priority over ratings, then Medians follows ratings about as
close as is possible.

> 
> > > > > The "satisfaction maximization" method by which you are interpreting
> > > > > the results is as follows.  Each person states their satisfaction
> with
> > > > > each potential result on a 0 to 100 scale.  From this point of view
> > > > > the only incorrect vote is one that does not accurately describe the
> > > > > voters perceived self-interest.  This perspective assumes people will
> > > > > not vote altruistically.
> > > >
> > > > Not at all.  A voter is just as likely to have the good of the
> community
> > > > in mind when rating candidates as when ranking.  I suppose an outcome
> > > > beneficial to the community would provide most voters some
> satisfaction,
> > > > though, even if not based on self-interest.
> > > >
> > > > Just to be clear, my scale wasn't based on personal satisfaction, but
> on
> > > > the individual's assessment of suitability for the office, where 100 is
> > > > completely adequate, and 0 is completely useless.
> > >
> > > Consider the following situation.  Candidates A, B, and C are running
> > > for office.  Their ratings, based purely on self-interest are
> > > group  votes     A    B    C
> > > I      60      100   90    0
> > > II     40       90  100    0
> > >
> > > A 96
> > > B 94
> > > C 0
> > >
> > > So, the winner is A.  From the Average-Ratings point of view this is
> > > also the candidate that provides the greatest average satisfaction.
> > >
> > > Now, if the group I voters decide to vote based on community
> > > interests instead of self-interest, they would use these average
> > > ratings as their individual ratings, normalizing them of course.  So
> > > we get
> > >
> > > A 96 * 100/96=100
> > > B 94 * 100/96=97.9 approx 98
> > > C 0 * 100/96=0
> > >
> > > So, if group I is voting with community interest the votes are
> > >
> > > group  votes     A    B    C
> > > I      60      100   98    0
> > > II     40       90  100    0
> > >
> > > A 96
> > > B 98.8
> > > C 0
> > >
> > > So, B is the winner.  So, group I not only hurt themselves, but hurt
> > > the community as a whole by their community interest.  I think it's
> > > clear from this that Ratings assumes people vote for their
> > > self-interest, or it won't work properly.
> >
> > I think you're off on several counts here.  First, you cannot
> > mathematically transform selfish votes into altruistic votes simply by
> > averaging them together.  Second, even if an altruistic vote were the
> > same as an average vote, your example would not prove that the original
> > votes were selfish -- the original votes could have all been altruistic
> > and yielded the same result.
> 
> I don't need to prove that the original votes were selfish.  This was
> assumed.  I started by presenting a selfish rating for each candidate,

My mistake, I thought you were trying to show that ratings always
assumed selfish votes.

> and pointed out that if everyone voted this way the method would
> maximize total satisfaction, or normalized utility.  Then I showed
> that if some voters actually viewed maximizing total satisfaction as
> their goal (as opposed to maximizing their personal satisfaction), the
> result would be that they would get neither.
> 
> I think that I am on pretty solid ground with my averaging process,
> under two assumptions:
> 1.  That group II's perception of their selfish interest is the same
> as group I's perception of group II's selfish interest.
> 2.  That utility and normalized (rated from 0 to 100) utility are the
> same in the example.  Of course, I can just assume they are for this
> example, but an argument could be raised if this is rarely the case,
> and that therefore my example is atypical.
> 
> If this is the case than group II's "altruistic" vote is designed to
> maximize utility.
> 
> In reality, however, you don't have to agree with my precise method
> of averaging.  It is obvious that if group I wants to think of the
> community interest, this has to be some combination of group I and
> group II's selfish interests, because together they are the community.

But if group II's selfish interests are harmful to the community, while
group I's are not, then group I would benefit the community more by
voting "selfishly". 

Bart Ingles


>  It is also clear from my example that this means making their votes
> more like that of group II, and increasing the likelihood that B will
> be elected.
> 
> ---
> Blake Cretney