[EM] "designing to the benchmark", strategy hurts

[Brian Olson]

There has been a pattern in the computer industry of some group  
defining a benchmark to measure the performance of various computers,  
and then the next generation of computers does amazingly well on  
popular benchmarks. There have even been pretty obvious cases where  
some feature was added to a computer that had very little real world  
usefulness but sure made the benchmarks awesome.

I relate this little cautionary tale because of the latest round of  
claims that simulation results prove method X is the best ever. Of  
course, I did the same thing with my sims ( http://bolson.org/voting/ 
sim.html ) way back 4-5 years ago. I designed a simulator that could  
measure the social utility of election results, and naturally the  
best result came from the election method which just summed up  
voter's personal-utility-votes and picked the overall best. That's an  
awful lot like ideal range voting. And indeed it's great and  
expressive and better than Condorcet _when everyone is honest_.


http://www.rangevoting.org/StratHonMix.html
It's measuring the wrong thing. Isn't that performance curve about  
the same as without any strategic players? Good methods get good  
answers, even in the face of adversity, ok, BUT

Do the strategic voters make out unfairly well vs the honest voters?

(in sims I ran, they do)

THAT is the problem with strategy vulnerable election methods, like  
raw rating summation or WDS's "range voting".

That's what I designed IRNR to solve, squash the power of approval- 
ized ratings ballots and make casting honest differences between  
choices interesting again. It turns out to have some inconsistencies  
and oddities, but I still think it's pretty good and maybe fixable.

I'll have to take a closer look at Hay. I personally absolutely  
insist on deterministic election methods, so I got turned off to Hay  
when I read that it was non-deterministic and had a deterministic  
lesser sibling; but from the discussion it sounds like there's  
something to it. Maybe its transform is what I was looking for in IRNR.


Brian Olson
http://bolson.org/