[EM] Ratings as a standard

[Bart Ingles]

> > Blake Cretney wrote:
> > >
> > > Bart Ingles wrote:
> > >
> > > > Nearly a year ago, Blake objected to my use of average ratings as a
> > > > standard for comparing methods, partly on the grounds that average (or
> > > > total) ratings would give more weight to extremist voters.
> > > >
> > > > The discussion assumed we were using sincere ratings in hypothetical
> > > > examples only, and that the ratings were on an absolute scale.  There
> > > > was no suggestion that the ratings standard itself be used as a voting
> > > > method.
> > > >
> > > > At the time I had suggested some ideas for ratings-based standards
> where
> > > > the effect of extreme votes could be limited.  It now occurs to me that
> > > > this is entirely unnecessary, and that extremist voting is not a
> > > > problem.
> > > >
> > > > All you need to do is stipulate that comparisons using average ratings
> > > > (i.e. social utilities) are valid so long as the actual methods you are
> > > > comparing don't give undue influence to extremist voters.  You simply
> > > > compare the methods with the understanding that no method would or
> > > > should yield the highest possible rating in all situations.

[BC]
> > > My problem with this, is that I do not accept the average ratings
> > > standard.  Since it is fairly intuitive, I felt I should give some
> > > justification for why I reject it.  The best reason for rejecting a
> > > standard, it seems to me, is if it can be shown to require an absurd
> > > conclusion, in some cases.
> > >
> > > The average ratings standard seems to do this, for examples like the
> > > following:
> > > 1  voter   A 500 B 0
> > > 40 voters  A 5   B 9
> > > The average ratings standard says that A should win, but I tend to
> > > doubt this.  It seems more reasonable to conclude that the single
> > > voter is being unreasonable.
> > >
> > > Of course, you could avoid this conclusion by coming up with a new
> > > standard that balanced average ratings with some kind of extremist
> > > avoidance.  But such a standard would no longer be intuitive, and
> > > there would be no real reason for accepting it.  Such a standard might
> > > restrict average ratings from falling into obvious absurdity, but I
> > > would suspect that its conclusions would still be incorrect, just not
> > > taken to the logical extreme.

[BI]
> > My point was that you don't need to come up with a more complex
> > standard.  You only need to require that any actual methods to be
> > compared are not subject to extreme voting.

[BC]
> Here's the way I see the discussion so far.  You came out with a
> theory, which was that if you get a sincere absolute rating from each
> voter on each candidate, than the proper winner would be the candidate
> with the highest total rating.  I was skeptical, and suggested a

[BI]
That was never my "theory".  I never claimed that the winner should
necessarily be the candidate with the highest total rating.  My focus
was at the opposite end of the scale -- that an election whose winner
has an extremely low rating, when other candidates have much higher
ratings, is absurd and indicative of a breakdown in the election method
used.

I should extend that and say that such an outcome could also be a result
of a bad nominating process.  The two-candidate example you gave is
obviously the latter.


[BC]
> situation in which I felt this was clearly not the case.  You seem to
> agree.  My conclusion is that the theory has been disproven.  The
> truth or falsity of a theory shouldn't depend on how you want to use
> it.  If the theory predicts one thing, and we agree that it has
> predicted falsely, we have to reject, or at least amend the theory.
> We can't just say, "the theory is true, but in that case I wouldn't
> use it."
> 
> So, I await either an explanation of why absolute ratings is right in
> this situation
> 
> > > 1  voter   A 500 B 0
> > > 40 voters  A 5   B 9
> 
> or a new theory to improve on absolute ratings.

[BI]
And I have already done so.  My initial examples assumed a 0-100 scale,
and that all voters had favorites rated 100, and least favorites rated
0.  Given that subset of possible scenarios, I showed that it is
possible for a Condorcet winner with an average utility of nearly zero
to defeat a near first-choice majority candidate with an average utility
of over 50.  I argued that this outcome was absurd.  Rather than defend
the outcome in that example, you raised examples which were not in the
range of scenarios I was considering.

Again, here is the kind of example I was concerned with:

     Voter utility
     100 99.9 99.8                                    0.2 0.1 0.0
     ---------------------/\/--      --/\/-----------------------
499   A                                                      B  C
3     B  A                                                      C
498   C                                                      B  A

Condorcet Winner:  B
Avg. utility:      A=50.1997   B=0.3997
    
It seems to me that your other examples are irrelevant.  If your
hypothesis is that the Condorcet winner is always desirable, and I give
an example like the one above, your choices are to either explain why B
is a better candidate than A _in_this_example_ or else reject your
hypothesis.

If you contend that the outcome is a result of extremist voting, then
which voters are the extremists?  Since only three voters strongly favor
B, they would seem to be the most suspect.  Does this mean that
Condorcet favors the extreme voters in this case?  If not, then since
they also favor A, how could the 499 A voters be considered extremist?


> > With this requirement, all
> > reasonable methods will score equally with the above example -- the
> > winner will always score 8.78 out of a possible 500 (assuming the
> > maximum range is 0-500).

[BC]
> Why should there be a maximum range?  Presumably you could always
> come across someone who felt even more strongly than allowed by the
> bounds given.

[BI]
Because utility rating systems, such as Von Neumann-Morgenstern
utilities, use fixed ranges.  Because an open-ended, emotion-based
"feeling" scale is not what I was advocating, and has nothing to do with
my point.

> > Of course, this is an odd example where the vast majority of voters are
> > almost completely indifferent.  It also seems to imply a very bad
> > nominating process, since the majority of voters don't like either
> > candidate.

[BC]
> Maybe they're saving the upper range for some kind of truly great,
> heroic candidate.  It may mean that they're just optimistic.
> Presumably the lone voter feels he has found such a candidate.

[BI]
In order for the 40 "reasonable" voters to reserve their top rating for
this heroic potential candidate, the potential candidate must exist.  If
he exists, but is not in the race, there must be something terribly
wrong with the nominating process.  If he doesn't exist, then the voters
must not be reasonable after all.