[EM] IRV vs Plurality (back to the pile count controversy)

[Abd ul-Rahman Lomax]

At 05:17 PM 1/21/2010, robert bristow-johnson wrote:
>On Jan 21, 2010, at 4:26 PM, Abd ul-Rahman Lomax wrote:
>
>>But ... it raises some security issues. And with central counting
>>there are other issues. This is a red herring, because we are
>>talking about precinct summability, and when the number of
>>candidates is very small, precinct summability isn't relevant,
>>because the raw ballot data may be transmitted.
>
>no, the problem is that the raw ballot data may be the only practical
>information to transmitt if the number of candidates is *large*, not
>very small.  when the number of candidates is very small, then it
>makes sense to transmit the tallies for piles because the number of
>piles, which are precinct summable, is manageable.

This is correct. I actually stated it oppositely, slip of the pen, so 
to speak. It's still a red herring, because the topic is precinct 
summability and the general use of precinct sums. The only precinct 
sum that can be used with IRV is the relevant-ballot-pattern summary, 
which becomes extremely large very rapidly. Forget about it with 
manual ballots and more than a quite small number of candidates. 
Remember, as well, that preferential voting, like top-two runoff, 
encourages lots of candidates to run, since they can do so with 
relative safety and get a payoff: some first rank votes that show 
support. They can turn that into cash in the next election when they 
are seeking the office again, or in other ways.

>>So, back to the real question: is precinct summability an important
>>practical criterion to be applied to voting systems?
>
>i would ask instead if precinct summability is important for
>security?  i believe that it is.

Good. So do I. Or was that a slip?

>   and i believe that it is perfectly
>practical when the number of *credible* candidates is small.  doesn't
>matter what the voting system is.  IRV, or whatever.

Yes. But how small? Don't use the bogus numbers that aren't at all 
realistic given real-world election rules, and since we are talking 
about the U.S., there must be accomodation for every ballot candidate 
that gets any votes at all in the precinct, plus a write-in provision 
at a minimum, and God help the election officials if there are a 
*lot* of write-in candidates, with the sum being more than enough to 
alter the elimination sequence for the remaining candidate. 
Write-ins, in all the actual election reported counts, are counted as 
a category and then dropped when the total for all of them was 
insufficient to do other then batch-eliminate them, possibly with 
other candidates as I've seen. I.e., one assumes the simplest case, 
that all the write-in votes are for the same candidate.

Now, if it turns out that the write-ins are relevant, suppose that we 
set up some rule to lump all candidates with only one vote and report 
all the others explicitly. But the problem rapidly gets hairy. One 
has to report another candidate as relevant in addition to all the write-ins.

For voting system security issues, one must be able to count the 
votes manually, as part of an audit. I'm sure that Kathy could 
explain audit process, but, again, it gets very hairy rapidly with 
IRV, because vote samples aren't enough, given the sensitivity of the 
method to many small differences in vote patterns. What is actually 
being done? Only ballot images, with machines that collect and report 
them, in toto, from the precinct. In other words, the only solution 
in actual usage that doesn't involve toting all the ballots to a 
central location involves reporting ballot images. But this is 
precisely a system that is quite vulnerable to hacking and some very 
real voting security issues. If there are no paper ballots or at 
least bulletproof paper records that the voter personally verified, 
it's impossible to verify that there were no shenanigans. Precinct 
summable methods are not nearly as sensitive to manipulation as are 
IRV totals, it appears. It can only take a very small shift in voting 
patterns to shift an IRV result, under some conditions, and this 
isn't merely a very close election in terms of overall support for a 
candidate, it gets down to exact preference order and how it 
interacts with elimination sequence, which is determined sometimes at 
many places in the election process. And, note: if it's ballot 
images, these images don't include, generally, the actual write-in 
votes. If it turns out that write-ins need to be counted, only manual 
counting can do it, the name was hand written on a record, if I'm correct.


>>>   for 3 candidates, that number is 9.
>>
>>Okay, three candidates, A, B, C, the ballot possibilities are, to
>>be complete, much more than 9. I'll assume that write-ins are
>>illegal and void the ballot. Some of the possibilities are legally
>>equivalent to others, and in actual IRV ballot imaging, they are
>>collapsed and reported the same, to the displeasure of voting
>>security people who do want to know the "error rate," which
>>includes overvoting and exact overvoting patterns. So-called ballot
>>images are not, generally. They are processed data reducing a
>>ballot to legally equivalent votes. The reduced set is this:
>>
>>A
>>B
>>C
>>A>B
>>A>C
>>B>A
>>B>C
>>C>A
>>C>B
>>
>>Note that this assumes a 2-rank ballot.
>
>no, it can be a 3-rank ballot where the voter declines to rate their
>last choice.  "3rd choice" is left unmarked.

I meant something a little different. I address the possibility of a 
3-rank ballot in the next section. The basic issue here is whether or 
not the third rank is irrelevant or not. If it's irrelevant, I claim, 
it's not really a three-rank ballot, it's got two relevant ranks and 
one that means nothing. Why was it even there?


>>It also assumes that majority vote isn't important.
>
>bullshit.  it (the number of consequential ballot permutations) has
>nothing to do with it (whether or not majority vote is important).

This is, in fact, serious ignorance. Bullshit, properly used, allows 
things to grow. Consider where the growth lies here.

If a majority is required, there is a difference in meaning between 
B>C>A and B>C. I will assume the counting method described by 
Robert's Rules of Order for preferential voting. 3 candidates

Situation with truncated B vote:
35 A>B
34 B>C
31 C

C eliminated, votes become

35 A>B
34 B

Majority basis is 100. 51 votes are required to win. No majority, B 
eliminated. I would guess that Robert doesn't consider this step 
because he is used to thinking of plurality IRV, no majority 
required, and the counting can stop with the last two in that case. A 
would win.

35 A>B. A is plurality winner, no majority, election fails. Who would 
be the runoff candidates? Under Robert's Rules, the question is 
unanswerable and undeterminable from the first round results. It's a 
new election. Under top two runoff rules, the rules were not designed 
for a preferential ballot, but I'd suggest considering *every IRV 
vote* as an approval, then pick the top two in that. This would pick 
B and C. Note that A, who would win under standard IRV rules, doesn't 
even make it into the runoff, using the rule I suggest.

Now, consider the election with the full ranking from the B voters, 
of B>C>A, a vote which Robert was prepared to assume as equivalent to 
B>C. When B is eliminated, the total vote for A becomes 69, a solid majority.

Note that IRV, used to find a majority winner, violates 
later-no-harm, because with a repeated election, even a top two 
runoff, as long as B would be included, B might have won, but with 
the election completing for A, B loses.

Bullshit? I think not. I followed the vote-counting description in 
Robert's Rules, by the book. It does not stop eliminations with two 
candidates, it continues to seek a majority and eliminates until 
there are no more eliminations to do.

>>If it's important, as it would be in an IRV election under Robert's
>>Rules, we have some more possibilities. They are all the three-rank
>>permutations.
>>
>>A>B>C
>>A>C>B
>>B>A>C
>>B>C>A
>>C>A>B
>>C>B>A
>>
>>Each of these is equivalent, for the purposes of finding a
>>plurality winner, to a two-candidate combination.
>
>it's equivalent for the purposes of IRV or Condorcet or *any* method
>that relies solely on the relative rank of candidates.  those 6
>markings are equivalent to the corresponding 6 above.

Note that I stated the condition, which makes those voting patterns 
relevant. Pay attention, Robert, there is far more here than you imagine.



>>>   if you or
>>>Kathy say it's 15, then you're wrong (and it's your slip that's
>>>showing).
>>
>>Well, I won't speak about Kathy, but in terms of practical
>>elections in the U.S., she's right. You did not state enough
>>information to establish your reduced count, ...
>
>yes i did state enough information.  may i remind you?  i said that
>there is *no* consequential difference in these two marked ballots
>(in the case of N=3).  there is no consequential difference between a
>ballot marked A>B to one marked A>B>C .  there is no election
>scenario, whether it's IRV, Condorcet, Borda or any other method
>using ranked ballots that will count those two ballots differently.
>there is no need to separate the A>B and A>B>C into two piles.

I just stated an example election, following stated rules, and in 
actual usage, if anyone is following Robert's Rules for preferential 
voting, and I suspect that there are, at least, some student 
organizations which have done just that. They  simply adopted 
Robert's Rules description by reference to the 10th edition.

Robert, at what point do you say, "Ooops!"

>>>   for N candidates, the number of piles necessary, P(N) is
>>>
>>>            N-1
>>>     P(N) = SUM{ N!/n! }
>>>            n=1
>>>
>>>not
>>>
>>>            N-1
>>>     P(N) = SUM{ N!/n! }
>>>            n=0
>>>
>>>which is appears to be the formula you and Kathy continue to insist
>>>is correct.
>>
>>Which it is under some conditions and yours is correct under some
>>conditions. I assume. I haven't checked them because it's more work
>>than I can put in now.
>
>want me to spell it out.

No. Pay attention. I accepted your formulas as correct. I stated that 
I haven't checked the math. Because it's irrelevant. Those formulas 
will be useful if we want to examine larger elections, but the 
question is which formula to follow, and there is an unstated 
assumption, which is that a limited three-candidate election actually 
could exist with relevant frequency. In practice, it must be a 
two-candidate election, and often IRV isn't used if there are only 
two candidates on the ballot. Depends.

>   it's a simple application of combinatorial
>analysis, what is the first chapter of my introductory probability
>textbook (of a course i took more than 3 decades ago).

Shame on you, taking so long to learn so little. There still is time, 
I assume. All it takes is some humility and some willingness to 
recognize that sometimes other people, you might not expect, turn out 
to know more than you, at least on some point. I write everything I 
write on this list knowing that when I venture into certain areas, 
there are people here who know much more, and when they contradict 
me, my first response is to *very* carefully consider what they 
write. I'll still challenge it sometimes, but because of other 
considerations. In particular, if I'm not completely satisfied, I 
assume that there will be others also not satisfied, and asking 
continued questions or raising continued challenges can help benefit 
the overall readership, those who are interested in the particular discussion.

Not everyone reads my stuff, and it is not written for everyone.

>   you're doing
>it already for the specific case of 3 candidates A, B, and C.  if you
>want to look it up, look for language that says something like: "how
>many unique ways can a group of n items be selected from a pool of N
>items when the order of selection is relevant?"

I never took a statistics class, though I studied combinatorial math, 
probably in the 8th grade (where I was doing it outside of school). 
However, a former lover-wife (complicated story) had to take a 
statistics class, had a textbook, and I tutored her. She got an A and 
went on to finish her degree and is in private practice. You are 
making serious assumptions about the need to explain the math, to 
people who understand the math at least as well as you. I don't 
remember the formulas, but I could derive them when I need them, and 
I often prefer to do that instead of looking them up.

>   and the answer to
>that is N!/n! .  but there is one more fact that you need to toss
>in.  and that fact is that all candidates unmarked or unranked are
>tied for last place.  if there is only one candidate left unmarked,
>we know how all N candidates are ranked, including the unmarked
>candidate.

Again, you are making a simplified assumption that only applies to a 
limited case. I covered your assertion, specifying it and detailing 
where it was accurate and where it was not.

><everything else between is deleted without comment>
>
>>A vote of A>B>C, is that the same as A>B? Robert assumes, yes. But
>>what about write-ins? A>B>C is equivalent to A>B>C>W.
>
>that's not 3 candidates.  that's four.  you just changed the
>premise.  that's an official logical fallacy.  a form of "straw man".

This assumes that we are only talking about abstractions and not real 
elections. The fallacy here is a common one (an overspecification of 
the topic). Note that Robert used language that indicated a broader 
context than his example actually works with.

Are there only three candidates that receive votes, or are there only 
three *credible* candidates that receive votes? I covered this, 
Robert has ignored it. His conditions imply that there are more 
candidates. How does an individual precinct decide which candidates 
are "credible"?

He's correct. That's not three candidates, that's four. I.e., three 
"credible candidates" to use his original language, plus the implied 
possibility of additional "not-credible" candidates, which he never 
bothered to define. But that's the crux of the problem!

His assumption breaks down in at least two ways: first, it doesn't 
apply if there are *any* other candidates on the ballot (unless some 
very particular conditions are satisfied, or, say, one of the 
"candidates" is "All Other." And this only works if "All Other" is 
the first to be eliminated.) The second breakdown is if a majority is 
required. Robert's response to this? "Bullshit." Now, again, when is 
it time to say "Ooops!" Robert, it goes downhill from here unless you 
do a little self-examination. Your call, of course. I'm not your 
shrink, nor your judge, just someone who is being a different kind of 
friend than you might be accustomed to.

>if the write-ins are insignificant (usually the case) we can sweep
>them all into a single insignificant candidate and we have 4
>candidates and 40 piles.

That's right. And if no majority is required, this is enough. 4 
piles. Tell me, how many ranks on the ballot? If write-ins are 
allowed, for full ranking, three explicit ranks are needed, by 
assuming no-ranking as equal-ranking bottom and no majority requirement.

And do think about what happens if it should occur that all the 
write-ins together have more first preference votes than the lowest 
vote-getter of the three on the ballot. How many more piles would be 
needed? It depends on the voting patterns, perhaps, and some 
simplifying assumptions might be made, but it boils down to 
significant delay. And remember this: the canvassing method cannot 
depend on how many candidates there are and it must function even if 
there are enough write-in votes to alter the results. Remember, as 
well, that preferential voting encourages increased candidacy counts, 
for reasons I described. That's how San Francisco gets 23 candidates 
on the ballot for a single office. They were using top two runoff, 
which has *less* effect, I'd predict, on encouraging minor 
candidacies, because it encourages more sincere first preference 
votes, so those Write-ins will have the write-in, generally, in first 
preference....

>   but we'll see that even if all write-ins
>(with different names) are lumped into a single person, that the
>number of ballots with candidate W marked higher than last, will have
>lower totals than any counted with W marked last.  but that is an
>*assumption* that the aggregate write-ins are insignificant (i've
>voted for 32 years and have never seen a governmental election that
>was otherwise).  then we're back to 3 credible candidates and 9 piles
>that need be reported for public consumption (so we can keep them
>election officials honest).
>
>>>not that i am a defender of IRV.  but, you haven't laid a hand on it
>>>regarding "precinct summability".
>>
>>I didn't use my hand. I used my head.
>
>and your head fails.

Darn. Maybe I should use a hammer instead. I knew there was something wrong....


>>>so, before pointing out that someone's slip is showing, it might be
>>>safer to adjust where one's own fig leaf is hanging.
>>
>>I threw the fig leaf away years ago. What you see is what you get.
>>Take it or leave it.
>
>i am picking and choosing.  lest i get sucked into "argumentum
>verbosium" and i've wised up to that technique.

Cool. Keep it up and also start wising up to the substance. Now, 
suppose I think you are full of shit and you think I'm full of shit. 
How could we come to consensus? Any suggestions? You first.