[EM] "designing to the benchmark", strategy hurts

[Abd ul-Rahman Lomax]

At 02:43 AM 2/9/2007, Brian Olson wrote:
>There has been a pattern in the computer industry of some group
>defining a benchmark to measure the performance of various computers,
>and then the next generation of computers does amazingly well on
>popular benchmarks. There have even been pretty obvious cases where
>some feature was added to a computer that had very little real world
>usefulness but sure made the benchmarks awesome.

Right. However, what's happening here is a little different.

It's true that methods can be designed to satisfy election criteria. 
However, to continue the analogy, what if a computer maker attempted 
to design a computer to provide the fastest calculations *overall*? 
We'd expect that computer -- if the mfr was successful -- to perform 
well on general benchmarks.

And they do. You can game the system *a little*.

>I relate this little cautionary tale because of the latest round of
>claims that simulation results prove method X is the best ever.

In this case, it is not at all suprising, because, as Mr. Olson will 
note, Range is, essentially, the benchmark. Range was designed to 
maximize social utility, whereas the other methods were designed to 
satisfy criteria that were *presumed* to be associated with benefit 
to society. Range, in a sense, is designed to satisfy the utility 
benchmark quite directly. It uses the utility benchmark to choose a winner!

What's interesting here, though, is we now have some measure of *how 
much* it is better. And under what conditions it is better. How much 
is Range N, with N>2, better than Approval. How much is Range 100 
better than Range 10?

>  Of
>course, I did the same thing with my sims ( http://bolson.org/voting/
>sim.html ) way back 4-5 years ago. I designed a simulator that could
>measure the social utility of election results, and naturally the
>best result came from the election method which just summed up
>voter's personal-utility-votes and picked the overall best. That's an
>awful lot like ideal range voting. And indeed it's great and
>expressive and better than Condorcet _when everyone is honest_.

Right. And this has to be understood. Range is ideal with honest voters.

Now, what happens when voters aren't honest? We have a lot of 
*theory* about this, most of it rather abstracted from any kind of 
real-world measurement. Simulations are also abstracted, to a degree, 
but should correlate with real-world performance much better than 
determining what "criteria" methods satisfy. A criterion may seem 
reasonable but may be utterly inapplicable in the real world. If 
parties simply don't raise clones, perhaps because of the expense 
involved, a method failing ICC might not be deficient *in practice* 
at all. Under other circumstances, ICC might be quite important.

>http://www.rangevoting.org/StratHonMix.html
>It's measuring the wrong thing. Isn't that performance curve about
>the same as without any strategic players? Good methods get good
>answers, even in the face of adversity, ok, BUT
>
>Do the strategic voters make out unfairly well vs the honest voters?

Again, this is another version of the standard objection to Range. If 
strategic voters under Range reverse preferences, they gain no 
advantage by it. They *may* gain an advantage by voting Approval 
style, but I would expect that advantage to be small, and, in 
particular, the "harm" done to less melodramatic voters is, 
practically by definition, small.

I use the pizza election example. A group of people must buy one kind 
of pizza. They hold an Approval election. A majority prefer Pizza A, 
in fact, but they also find B acceptable, and they so vote. A 
minority prefer B and detest A, and they so vote. B, of course, wins 
under Approval.

The B voters gained, the A voters lost (compared to sincere 
preferences, which, to express, we should really use Range.) But it 
would not be appropriate to say that the A voters "made out unfairly 
well." They consented. Only if the B voters were being deceptive, 
they actually had only a small preference for B, but just wanted to 
get their own way, could we reasonably state that there was some 
unfair advantage taken.

Even then, though, most functional groups would still say, "If you 
want it that much, fine. I'm okay with B, if I wasn't, I wouldn't 
have approved it."

Where voters are truly exaggerating, voting the extremes when, in 
fact, their preference is relatively small, they are taking quite a 
risk. The rest of us aren't harmed nearly as much as they are.

>(in sims I ran, they do)

*How much* is the question.