[EM] Condorcet How? Abd

[Abd ul-Rahman Lomax]

At 12:56 PM 4/11/2010, Kevin Venzke wrote:
>--- En date de : Dim 11.4.10, Abd ul-Rahman Lomax
><abd at lomaxdesign.com> a écrit :
>>>In a given election yes, it is easy to miss the mark. But in
>>>general,
>>>aiming for the median voter is the most reliable. (That is
>>>assuming you don't know utilities, which I'm really not sure you
>>>showed how to find.) To see this, you assume utility is based on
>>>issue space distance, and that the voters aren't distributed
>>>unevenly.
>>
>>I didn't show how to find utilities, I only showed various
>>possibilities consistent with the votes.

Actually, I did follow an algorithm. I'd say it's reasonable. No claim
that it is definitive. Obviously, how people will vote may depend on
how the votes will be counted. But less than we might think

>Yes, and my response is what can I possibly do with that? You used one
>method that was rather Borda-like in character. One can't evaluate
>methods
>using a Borda-like criterion or you'll end up advocating something
>Borda-like.

The algorithm was not "Borda-like" because it did not prohibit equal
ranking bottom. It's true that I did not use equal utility top,
because there was no information to justify it.

>>To study voting system performance, I'm saying, one must
>>*start* from utilities, not from preference order without
>>preference strength information. Voter behavior is not
>>predictable without preference strength information.
>>Strategy, in general, doesn't make sense without an
>>understanding of preference strength.
>
>We sort of have been doing this when Juho questions the story behind
>my
>scenarios.

Yes. But the long-time use of voting systems criteria that depend
solely on preference order is a hard habit to break. And the whole
discussion of "strategy" has been often off the point. The use of
"strategy" (as being about the idea that people will bullet vote to gain some
advantage, which is assumed to be improper) neglects that a bullet
vote reflects a strong preference. The voter wants their favorite to
win, strongly enough that the voter suppresses 
the expression of remaining preferences.

IRV encourages this lower expression, because it 
doesn't require a true majority, but it 
"eliminates" candidates, hence lower preference 
votes will only be counted if your more preferred candidate is eliminated.

In other words, in a non-LNH method like Bucklin, 
A>B means that you actually are willing to elect 
B, enough to risk the possibility that B beats 
your favorite, A. With higher preference 
strength, you will vote only for A. Whereas with 
IRV, A>B (>C) tells us nothing about the 
preference strength of A>B, nor, in fact, about 
the strength of B>C. It only tells us that some preference exists.

>>>Thus when you have a situation where every voter
>>>chimed in on some question, and they didn't do that for any other
>>>question, you shouldexpect (on average) a utility problem when the
>>>outcome goes against the majority opinion.
>>
>>I'll agree that this is the "norm." However, it can go
>>drastically wrong.
>>
>>How can we detect the exceptions?
>
>Right, that's the question.

In order to detect the exceptions, and use that 
information, we must risk some level of error in 
that. Let's be clear about something.

Suppose a set of voters has, for a complete 
candidate set, internal absolute utilities. We 
can imagine a set of voters all with the same 
resources financially. They have, however, 
various degrees of interest in politics and who 
wins elections. The utility is the "value" of the 
election of a candidate to the voter, and may be 
negative. Assuming that all voters have equal 
resources, the positive value is what the voter 
would pay to be assured that the candidate would 
win, and a negative value is what the voter would 
find adequate as compensation if a disapproved candidate wins.

Can we agree that the ideal winner of an election 
is the one which would maximize the sum of 
utilities of the voters? Note that these are 
*not* normalized utilities, and that if there 
were a Clarke tax or the like, collecting from 
those who assigned positive values and paying 
this to those with negative values, such that the 
benefit of the election is equalized across all 
voters, the best candidate would represent 
maximized value for *all* voters, not just those 
whose preferred candidates won.

These utilities, except in certain narrow 
situations, cannot be directly determined. We can 
only infer them, to some degree, by voter 
behavior in elections and, as well, with respect 
to campaign donations and responses to polls. But 
it is not necessary to know them in order to use, 
in simulations, such a social preference profile 
to predict voter behavior under various voting 
systems, and to compare performance.

Okay?

This is roughly Warren Smith's approach, of 
course, though details may certainly differ.

The Majority Criterion require majority 
preferences to prevail, at a point where the 
majority may not have sufficient information to 
choose the social utility winner, assuming that 
the majority would want to do so. In fully 
discussed deliberative election, a serious 
"utility error" may be detected and avoided 
through two means: abstention by voters with low 
preference strength, and, as well, a posible 
deliberate choice by voters to please those with 
stronger preferences, i.e., socially cooperative 
behavior. In top-two runoff, if the primary fails 
to forward the social utility winner to the 
runoff, and if the error is large, and if 
write-ins are allowed in the runoff (which makes 
it closer to deliberative process), a write-in 
might well win. If the runoff voting method is 
such that a write-in candidacy doesn't need to 
create a spoiler effect, this can be facilitated.

I consider it an unresolved question, how common 
it would be in real elections that picking a 
majority preference will cause utility error. 
(Bayesian regret in Warren's work.) But I'm sure 
it happens. How much damage is done, I don't 
know. But it could be significant. And the 
scenario we have been working on does show this.

>>Sure, the majority criterion and the condorcet criterion
>>are usually a sign of good performance, but it is obvious
>>that exceptions exist, and we should not denigrate a voting
>>system if it, under an exception condition, it violates the
>>criteria!
>
>I wouldn't, no. But I would presumably have some model that explains
>why violation of the criterion worked.

Sure. Now, let's assume that the votes given as 
the election scenario were sincere, and reflected 
sincere preferences. Shouldn't we start there?

 From the classic study of preference, it has 
been assumed that a bullet vote is strategic, not 
sincere. It is assumed that if the voter votes 
just for A, instead of, say, A>B, the voter does 
have a preference between B and C, and is 
suppressing that to gain advantage. However, that 
is not a realistic assumption.

Suppose a voter is only familiar with A and 
"approves" of A, will be pleased to see A 
election, and has no opinion about either B or C. 
The voter, then, assigns no value to the election 
of B or C, but does assign value to the election 
of A. The bullet vote for A is a perfect 
expression of the voter's preferences. Suppose we 
shift this to a situation where the voter does 
have some preference between B and C, but it is 
small. If C is elected the voter will emigrate, 
if B is elected, the voter will merely make sure 
that there is enough money to buy a ticket, and 
that the voter can emigrate quickly! In IRV, 
rationally, the voter should rank B. But in 
systems that are collecting utility information, 
the utility of B is small and speculative. (It's 
negative in the system I proposed).

In a full-on Range voting system, with high 
resolution, the voter would indeed show a 
preference for B. But only in the second 
scenario. In the first the value for B is zero. 
The voter, by bullet voting, is totally 
abstaining from all other pairwise elections. 
(I'll neglect average range, which I consider 
politically foolish as a proposal, requiring 
judgement about something where we have only air to build on.)

I took the proposed votes, and inferred from them 
what votes would be equivalent expressions of 
preference in, not Borda, but Bucklin, 
specifically 3-rank Bucklin-ER, and I've come to 
the (tentative) conclusion that rational votes in 
Bucklin-ER are actually sincere range votes, 
particularly if a majority is required or there 
is further process. For 3-rank Bucklin ER, the 
votes would be Range 4, with possible ratings of 
0, 2, 3, 4, and midrange represents approval 
cutoff. That's why rating 1 is missing, it is a disapproved rating.

(In a more sophisticated Bucklin system, rating 1 
would be allowed, and could be used to determine 
ballot configuration in a runoff. This would 
encourage, a little, bumping up, to a rating of 
1, an only-disapproved-a-little candidate.)

>>I was just pointing out that the outcome you claimed was
>>obviously bad wasn't. It might be that, on average, this
>>outcome would be poorer than the other,
>
>Yes, I'm afraid that's what I call "bad." If I didn't call this
>"bad" I would also have to be pretty undecided about the resolution of most
>two-candidate FPP elections.

Actually, if the Bayesian regret was low, using 
"bad" would be hyperbole. Two-candidate elections 
are not the kind of scenario considered here. I 
do not believe that choosing the majority 
preference is an error in two-candidate 
elections, on average, and the incidence may be 
rare. We don't have a two-candidate election, the 
campaign was not over two candidates, it was over 
three, and it was very close. These are 
conditions where, in fact, utility error may be 
much more common, if we require the Condorcet 
criterion. The election does not fail the 
majority criterion. If we require a majority, the 
election simply fails. Now, take results like 
that and require a runoff between A and B. Who 
will win? Most voting systems students will 
assume that B will win, based on the votes in the 
first election. However, we know that in 
one-third of runoff elections, and these were not 
as close as this election, there is a "comeback." 
That means that in two-thirds there is not. 
Usually, the plurality winner goes on to win the 
election. Note that it only takes a very small 
amount of shift among the B and C voters to allow 
A to win, perhaps some lower turnout (which tests 
preference strength). In a real runoff with 
initial votes as described, A will almost certainly win.

We know, from the bullet voting, that the 
preference of the A voters is strong, it is less 
likely to shift in a direct campaign between A and B.

And, as I hope I show, the simplest analysis of 
likely normalized utility profiles shows that A 
is, indeed, the range winner (Range 4). That is the center, not the extremes.

This kind of thinking leads me to conclude that 
plurality is a better voting system than we often 
think. We dislike plurality because of the 
breakdowns, not because of the normal function!

>>but it was not a truly bad outcome,
>>under reasonable assumptions of likely
>>utility, the first utility scenario I gave, which used Range
>>2 utilities, i.e., normalized and rounded off so as to make
>>all the votes sincere and sensible. The bullet voters then
>>had equal bottom utilities for the other candidates, and
>>those who ranked had stepped utilities. Simple. And showing
>>that A was, indeed (with these assumptions, which seem
>>middle-of-the-road to me), the utility maximizer, by a
>>fairly good margin!
>
>This was the Borda-like thing I mentioned above.

Borda is Range with what is, in the end, a 
bizarre assumption. Use a Borda ballot and stop 
discarding "illegal votes," but count them 
rationally, i.e., allow equal ranking and 
therefore empty ranks, and keep the rank values 
the same, and it is a Range ballot. Since the 
only difference between Borda and Range is this, 
the system I used was Range, not Borda.

>>You can make a contrary assumption, that the A voters were
>>"strategic." That they "really" would be happy with B. I'm
>>assuming, instead, that their votes would be sincere. And
>>likewise the votes of the other voters.
>>
>>Look, A *almost* has a majority in first preference. I'm
>>very suspicious of claims that an election outcome is
>>"terrible" if it depends on some close-shave majority that
>>failed.
>
>You are really missing my complaint then. According to your stepped
>utility analysis C voters don't like B that much at all. If they know
>that the method interprets such votes that way, then it is really bad
>to vote sincerely for C.

We don't know how much they like B. We know that 
they like B more than A, that's all. So I 
inferred the middle assumption, that their 
utility for B was midway between that for A and 
C. That gives equal wiggle room in both directions.

What you are doing now, Kevin, is criticizing the 
original votes, which were the assumption. And, 
it seems, you are criticizing as well, possible 
(unstated) ballot limitations. Clearly, the 
ballot allows equal ranking bottom, but it seems 
you assume it does not allow equal ranking top. I 
assumed, instead, that an expressed preference 
had real value, was not merely forced by the 
ballot. I gave it a middle value, neither 
extreme. Could you suggest an analysis more likely to be accurate?

>I initially read your last paragraph with disbelief. In my
>interpretation,
>C and his votes are just noise. The task of the election method is
>to pick
>the right candidate between A and B, just as it would be in FPP
>(where C
>would probably have died off pre-election). To be unable to do this is
>quite useless in my view.

If that is the task of the election method, why 
is C on the ballot? Suppose that C is *not* on 
the ballot, suppose that C was a write-in. If 
that's true, then C would very likely be the utility winner. Not A or B.

What is the "right candidate" between A and B and 
why are we limited to that? I agree, it is likely 
that the right candidate is either A or B, but it 
is not at all impossible that it is C. With my 
middle-of-the road assumptions, I came up with A 
as the utility maximizer, and B and C not that 
far apart from each other. From pure preference, 
sure, it looks like C is noise. If, in fact, to 
the C voters, B and C are almost equally 
prefered, C is indeed noise, and the "sincere 
vote" for C is indeed strategically foolish. 
Indeed, if B and C were to agree on who runs, and 
it would probably be B, then all the campaigning 
would be dedicated to B instead of split, and the 
likely result would be an increase in B support, and it's already close.

But from the voting preferences given and 
reasonable assumptions about the normalized 
utility profiles behind them, A is the best 
winner! So criticizing a voting system because it 
gives A the victory with those votes is 
backwards. I would claim that, ideally, no 
election would ever terminate the process with 
other than a manifest condorcet winner, this 
would always go back to the voters, as is normal 
with deliberative election process, which never 
elects without a majority of the votes supporting 
the winner. But there is no condorcet winner in 
the election being studied, because, while B 
beats A 51:49, C beats B 27:24, and A beats C 49:46.

So, since there was no majority for A, this 
election, ideally, would go back. For practical reasons, it might not.

>--- En date de : Dim 11.4.10, Abd ul-Rahman Lomax
><abd at lomaxdesign.com> a écrit :
>>>This should rather say, if I proposed utilities behind
>>the scenario, I
>>>could make those utilities say anything I wanted.
>>
>>I pointed out some extremes, which reveal as the ideal
>>winner A, B, or C. In other words, you are apparently
>>agreeing with me.
>
>Yes.
>
>>However, I believe that I showed that a
>>middle-of-the road assumption about underlying utilities,
>>with stated assumptions that were not designed to make it
>>turn out some particular way, A could indeed be the best
>>winner.
>
>Yes you did.

Here, you accept it, but I want to make sure that 
was not accidental. I wrote "could be." I'll make that stronger. "Likely is."


>>(I did not set out to "prove" that A was the best winner,
>>but rather just to attempt to infer utilities from the
>>voting patterns, which didn't allow me to assume equal
>>ranking except at the bottom).
>>
>>The matter hinges on the A voters, who are, after all,
>>almost a majority. Why did none of them rank B or C? The
>>only reasonable assumption is that they have strong
>>preference, and that's what ices it.
>
>I'm happy to say that A voters have a strong preference, but why
>should only the A voters get to benefit from this? Are you saying the other
>voters don't have a strong preference against A?

No, they do, and they are given full credit 
("benefit") for it. The problem is that they 
don't explicitly agree on strong preference *for* B or C.

I was not analyzing the votes from the point of 
view of strategy in the voting system used, and I 
don't really know what that system is. I was 
assuming, in fact, sincere votes, votes that 
express preferences accurately. If the 51% did, 
in fact, have strong preference for both B and C 
over A, they therefore had weak preference 
between B and C. (That is the restriction that 
comes from normalization, the assumption that all 
voters have the same range of preferences. One person, one vote.)

If they have weak preference between B and C, in 
Range 4, they would rationally vote max rating 
for both. But you have 5% of voters who vote 
contrary to that, because they bullet vote for B. 
The linchpin voters are the C voters. If the 
method is plurality, their strategy is obvious, 
if they want to improve the outcome. But what if 
some of them, say, anything over 1% of total 
voters, have weak preference for B over A? They 
will prefer to express their clear preference for 
C to betraying their favorite, and that is real 
voter behavior. It was Nader's message in 2000, 
and, obviously, many voters bought it.

In IRV, their strategy is also obvious. IRV works 
with an election like this, unless. Unless enough 
C voters abstain from ranking B. Some will. And 
there you go, IRV is likely, still, to give this 
to A! Only a Condorcet method will give it to B, narrowly.

>>This is the classic
>>reason to violate the Condorcet or Majority criteria: a
>>strong preference of a minority, particularly when the
>>margin is thin.
>>
>>If, in fact, B and C were true clones, with only minor
>>preference between them, the assumption of a significant
>>reduction of utility between them (which is the other factor
>>that lowers the rating for B and C) would fail.
>>
>>If the method allowed equal ranking, we'd see that in the
>>votes, and B might win. The A votes would be the same, the B
>>bullet voters would be the same, but the other B and C
>>voters would equal rank B and C. Because of the B bullet
>>voters, B would win by a small majority.
>>
>>So my result for A could be an artifact of the voting
>>system not allowing equal ranking. I used Range 2, which
>>doesn't give a lot of room for "creative interpretation."
>>That was much easier with Range 10, as I showed. With Range
>>2, there wasn't any other reasonable way to interpret the
>>votes.
>
>It's possible that with equal ranking it would be different, but if we
>are not going to ask a method to behave unless voters use equal
>ranking, I guess we could just use Approval.

Approval is Not Bad, but this would be the 
problem: what about the C voters? Approval would 
not allow them to express their preference for C, 
and if it is strong, some of them will not express it, and A will win.

So, in fact, Bucklin.

>49 A
>5 B
>19 B>C
>27 C>B

B wins, assuming these are the votes. Bucklin is an approval method.

However, the election scenario is unrealistic, 
because we have almost half of the voters voting 
monolithically, and then the other half does so 
as well. For these voting patterns to arise, the 
election must be highly partisan. Non-partisan 
voters do not vote like this. In a real election, 
some of the non-partisan voters who prefer A 
would also approve, at lower rank, of B or C. And vice-versa.

For the rest of this post, I will repeat the 
utility analysis, which is Range 4, with the 
approval cutoff being 2. A rating of 2 represents 
indifference between the election of the candidate and the expected outcome.

49 A was analyzed as A=4, B and C are 0. This was 
overly pessimistic. If I'm going to be "central," 
I should rate them as 1, i.e., the middle 
disapproval rating. This is an average, some 
voters might rate B or C as zero, some as "almost 2".

5 B likewise.

19 B>C This is analyzed as B= 4, C = 3. This is 
accurate, because the actual voter rating would 
be in the range of 2 to 4, so I'll assume that 
the mean is 3. A, however, should be considered 
to be 1, for the same reason as with the A voters with regard to B and C.

27 C>B likewise.

So, the new analysis:

                 A       B       C
49 A            4       1       1
5 B             1       4       1
19 B>C          1       4       3
27 C>B          1       3       4

totals          247     157     170

Notice this phenomenon: the social preference 
order matches the first preference order. This, I 
suspect, is very common. This is why plurality voting is a decent method!

What would happen if a majority were required? 
Suppose that the runoff is between the top two 
candidates, using an analysis like what I just 
did, which assumes that preferences are spread 
such that they average as I indicated. The runoff 
would be between A and C, not A and B, and that 
is probably the best. After all, C beats B pairwise.

Dhillon and Mertens showed that Rational 
Utilitarianism, which is tantamount to Range 
Voting with von Neumann-Morganstern utilities, is 
the unique system that satisfies all of a set of 
Arrovian criteria modified to allow consideration 
of equal ranking and preference strength 
expression, and, here, the assumption is that 
normalized utilities are not modified by 
probabilities, i.e., they are zero-knowledge. 
They are raw utilities, then, normalized to the 
range of 0-4, and averaged across the factions.

The average votes end up (average out) as if each 
faction voted in a Range 3 election, and it is 
possible that I could or should redo the analysis 
stated that way. In this re-analysis, a bullet 
vote is equivalent to ratings of 3, 0, 0, and a 
ranking of two candidates is equivalent to 3, 2, 0.