[EM] Ratings as a standard

[Blake Cretney]

Bart Ingles wrote:

> Blake Cretney wrote:
> > 
> > Bart Ingles wrote:
> > 
> > > Nearly a year ago, Blake objected to my use of average ratings as a
> > > standard for comparing methods, partly on the grounds that average (or
> > > total) ratings would give more weight to extremist voters.
> > >
> > > The discussion assumed we were using sincere ratings in hypothetical
> > > examples only, and that the ratings were on an absolute scale.  There
> > > was no suggestion that the ratings standard itself be used as a voting
> > > method.
> > >
> > > At the time I had suggested some ideas for ratings-based standards
where
> > > the effect of extreme votes could be limited.  It now occurs to me that
> > > this is entirely unnecessary, and that extremist voting is not a
> > > problem.
> > >
> > > All you need to do is stipulate that comparisons using average ratings
> > > (i.e. social utilities) are valid so long as the actual methods you are
> > > comparing don't give undue influence to extremist voters.  You simply
> > > compare the methods with the understanding that no method would or
> > > should yield the highest possible rating in all situations.
> > 
> > My problem with this, is that I do not accept the average ratings
> > standard.  Since it is fairly intuitive, I felt I should give some
> > justification for why I reject it.  The best reason for rejecting a
> > standard, it seems to me, is if it can be shown to require an absurd
> > conclusion, in some cases.
> > 
> > The average ratings standard seems to do this, for examples like the
> > following:
> > 1  voter   A 500 B 0
> > 40 voters  A 5   B 9
> > The average ratings standard says that A should win, but I tend to
> > doubt this.  It seems more reasonable to conclude that the single
> > voter is being unreasonable.
> > 
> > Of course, you could avoid this conclusion by coming up with a new
> > standard that balanced average ratings with some kind of extremist
> > avoidance.  But such a standard would no longer be intuitive, and
> > there would be no real reason for accepting it.  Such a standard might
> > restrict average ratings from falling into obvious absurdity, but I
> > would suspect that its conclusions would still be incorrect, just not
> > taken to the logical extreme.
> 
> My point was that you don't need to come up with a more complex
> standard.  You only need to require that any actual methods to be
> compared are not subject to extreme voting.  

Here's the way I see the discussion so far.  You came out with a
theory, which was that if you get a sincere absolute rating from each
voter on each candidate, than the proper winner would be the candidate
with the highest total rating.  I was skeptical, and suggested a
situation in which I felt this was clearly not the case.  You seem to
agree.  My conclusion is that the theory has been disproven.  The
truth or falsity of a theory shouldn't depend on how you want to use
it.  If the theory predicts one thing, and we agree that it has
predicted falsely, we have to reject, or at least amend the theory. 
We can't just say, "the theory is true, but in that case I wouldn't
use it."

So, I await either an explanation of why absolute ratings is right in
this situation

> > 1  voter   A 500 B 0
> > 40 voters  A 5   B 9

or a new theory to improve on absolute ratings.

> With this requirement, all
> reasonable methods will score equally with the above example -- the
> winner will always score 8.78 out of a possible 500 (assuming the
> maximum range is 0-500).

Why should there be a maximum range?  Presumably you could always
come across someone who felt even more strongly than allowed by the
bounds given.

> Of course, this is an odd example where the vast majority of voters are
> almost completely indifferent.  It also seems to imply a very bad
> nominating process, since the majority of voters don't like either
> candidate.

Maybe they're saving the upper range for some kind of truly great,
heroic candidate.  It may mean that they're just optimistic. 
Presumably the lone voter feels he has found such a candidate.

---
Blake Cretney