[EM] Bart's "Median Rating" method?

[Steve Eppley]

Bart wrote:
> Steve Eppley wrote:
> > I think the definition of Bart's Median rating method needs
> > clarification, since Bart's claim about highest median rating
> > and majority appears dubious.  Here's an example to illustrate
> > the problem:
> > 
> >    voter 1:  A=95, B=65
> >    voter 2:  A=85, B=60
> >    voter 3:  A=50, B=20
> >    voter 4:  A=40, B= 0
> >    voter 5:  A=45, B=55
> > 
> >    There is a majority (80%) who rank A ahead of B.
> > 
> >    Average rating for A = 63
> >    Average rating for B = 40
> > 
> >    Median rating for A  = 50?
> >    Median rating for B  = 55?
> > 
> > If I've interpreted correctly how Bart intends it to be
> > tallied, B is the candidate with the highest "median rating."
> > But I wouldn't agree that B is rated higher than A by a
> > majority of voters.
> 
> Correction:  A majority rate the Median winner (B) higher
> than any majority rates the opponent (A).

There are two problems with that overly terse reply:

1. It appears false.

How do we calculate how "a majority rate an alternative"?  
Since Bart hasn't bothered to explain how he intends this to 
be calculated, I'll guess that it's the average of the ratings 
of the alternative, considering just the members of the given 
majority. 

In the example above, if we select for "Median Winner" B 
the majority which maximizes its average rating of B, 
this majority consists of voters 1, 2, and 5.  
This majority rates B:   avg(65,60,55) = 60

Contrary to Bart's claim, there is a majority (voters 1, 2, 
and 3) who rate A higher:
   avg(95,85,50) = 76.67


2. Since Bart didn't object to the way I tallied his 
Median Rating method in the example above, I can assume 
that I understand how Median Rating is tallied.  So I can 
reach a conclusion regarding another of Bart's claims.

In the recent conversation between Bart and Blake about 
voters' ratings of candidates, Bart wrote:
   -snip-
   >>> Medians are a natural way of evaluating rated examples,
   >>> since a candidate with the highest median rating is by
   >>> definition the candidate rated higher than all others 
   >>> *by a majority of voters*.    
   -snip-  

We can now conclude that Medians are not a natural way to 
evaluate rated examples, any more than it would be natural for 
humans to walk on all fours.  Median Rating is a terrible way 
to evaluate rated examples, as the example above shows.  
Here's an example which may make this even clearer:

   voter 1:  A=99, B=51
   voter 2:  A=99, B=51
   voter 3:  A=50, B=51
   voter 4:  A=49, B= 0
   voter 5:  A=49, B= 0

   One voter (#3) has a tiny preference for B.
   All the other voters "strongly" prefer A more than B.  

Median Rating suggests that B is best, which suggests that 
Median Rating should not be used to evaluate ratings.


---Steve     (Steve Eppley    seppley at alumni.caltech.edu)