[EM] advocacy by means of exit polls

[RLSuter at aol.com]

In a message dated 8/29/06, Michael Pooles writes:

>> It makes no sense in this case to define "wrong" as anything other
>> than a wrong count of how people actually voted. Therefore, the
>> official results of an election are just as capable of being wrong
>> as exit poll results.
>
> Sure, but in practice, the count that actually matters is the one that
> is called "official" (perhaps after a court or political contest).
> That is the count that third-party polls try to predict, and the one
> that they are usually judged against.

Matters to whom and for what purposes? Have you been paying
attention to the controversies surrounding the multiple problems
with the increasing use in the U.S. of electronic voting machines
produced by poorly regulated private companies? Are you aware
of recent studies showing the extremely easy hackability of some
machines? Are you aware of the large discrepancies between
different states and even different counties in the same states
regarding how elections are administered? If you were living in
Ohio in 2004, as I was, and observing the extreme politicization
of the election process by the Republican secretary of state
who was overseeing the process, and if you had paid attention
to the investigations by independent researchers in the months
after the election showing more than enough evidence of error
and fraud to warrant skepticism of the accuracy of the results
officially certified by the same secretary of state, who arbitrarily
delayed a recount until after the legal deadline for official
certification had passed (for many of the Ohio details, see
http://www.freepress.org/departments/display/19/2005), your
thinking about what "actually matters" surely would be very
different.

>> As for your claim that the margin of discrepancy is "closely
>> comparable between exit polling and phone polling," that is
>> such an improbable sounding claim that you are going to have
>> to provide some pretty convincing documentation to convince
>> me of it. Exit polling is widely understood as much more accurate
>> than phone polling.
>  
> There are many widely understood things that are wrong or misleading.
>
> http://www.mysterypollster.com/main/2004/12/what_about_thos.html
> discusses why US presidential election national exit polls are weaker
> than some other exit polls.  This entry discusses sources of
> systematic error in exit polling:
> http://www.mysterypollster.com/main/2004/12/what_is_the_sam.html

This is an interesting discussion, but it proves nothing about the
2004 election. Also, it's dated December 19, 2004, long before
much of the in-depth research and discussion about election
problems was conducted. All but one of the other web pages
you list are from shortly after the election and therefore don't
reflect the extensive research and analysis done since then.

> http://www.pbs.org/newshour/bb/politics/july-dec04/exitpolls_11-05.html
> is an interview with the co-director of the NEP that mentions other
> sources of error in the 2004 US exit polls (search for "margins").

This interview took place three days after the election, long before
the interviewee, Warren Mitofsky, who led the exit polling effort,
had time to do an in-depth analysis of the polling results. Since
then, he has issued a report that is far from conclusive and leaves
many important questions unanswered and many reasons to be
skeptical of the answers he gave.

> Perhaps most tellingly, http://www.exit-poll.net/faq.html#a15 says
> that the margin of error for the US national exit poll is +/- 3%, and
> for individual states it is +/- 4%.

No, it's not telling at all in light of the fact that nearly every discrepancy
favored Bush and other Republicans. No attempts to explain that fact
have been satisfactory. Furthermore, some of the discrepancies were
double these margins of error or more. This page, by the way, is from
Mitofsky's own organization.

> Most telephone polls are structured to +/- 3% error margins even
> within a state.  For example,
> http://www.quinnipiac.edu/x11379.xml?ReleaseID=492 claims just over 3%
> error margin for each state, and considering the poll's undecided
> voters, the official results from the three states are within those
> margins.
>
> http://www.harrisinteractive.com/harris_poll/index.asp?PID=516 cites
> online and phone polls with error margins at or better than 3%.  The
> phone polls in that and other Harris Interactive polls are close to
> the official results, but the online polls had systematic bias in John
> Kerry's favor.

You can call it "systematic bias" only if you assume that the officially
reported results were all correct, including results in states like Ohio
and Florida with highly politicized election administrations. In light of
all the known problems with U.S. elections in 2004, that is just not
a reasonable thing to assume.

> It is clear that some polls (like the German national exit polls cited
> by Mark Blumenthal) are superior to both the exit and phone polls done
> in the US, but there are also a lot of inaccurate exit polls out there
> -- my original post mentioned a few non-US cases.  Quality differences
> seem much more a result of sample sizes or other processing strengths
> than of the poll location.

Superior only because the U.S. election system is much larger
and much less uniform that than of Germany, which makes the problem
of designing and carrying out exit polls much more difficult and
costly.

In any case, the essential questions of how accurate officially
reported results of the 2004 U.S. presidential election were and
whether or not the exit polls shed any light on their accuracy
have not been answered conclusively, and there are many reasons
to think that the official results for many states were wrong and
don't come close to accurately reflecting how people actually voted.
The only people with understandable reasons to be confident are
Republicans who are happy with the official results and Democrats
and media people who don't want to believe that their early, poorly
investigated conclusions about the election might have been
seriously mistaken.

-Ralph Suter