[EM] Amateur peer-reviewed "journal" for voting methods, criteria, and compliances?

[Jameson Quinn]

My suggestion is not to provide a forum for opinion, but for proven facts.
This would be in addition to the existing space at Democracy Chronicles; it
would not supplant it. As to Ossipoff's suggestion that this material is
already easy to publish in existing academic journals: I would suggest that
the fact that the general knowledge on this list of systems, criteria, and
compliances is far beyond the published literature *prima facie* refutes
that. In fact, I'd turn Ossipoff's own  logic back at him: if you're not
interested in publishing in a peer-reviewed forum, then you have many
non-peer-reviewed fora available, and this proposal has nothing to do with
you.

Jameson

2012/9/28 Michael Ossipoff <email9648742 at gmail.com>

> To: Jameson and all who received his message:
>
> Whose head-up-the-a** idea was that?
>
> Certainly not.
>
> Who decides who your "peers" are? Who chooses them?
>
> At Democracy Chronicles, there is free and open discussion. If you
> don't agree with something said in an article, then there is a
> comments space available, in which you can express your disagreement
> and tell your reasons for disagreeing. ...and the author of the
> article always has the opportunity to reply to your comments.
>
> The readers can decide which argument is more convincing.
>
> That's called "free and open discussion".
>
> What Jameson proposes is something quite different. Among some small
> group of people, and one of them, unilaterally, can block the
> readership's access to an article.
>
> There is already a system of such journals--the academic and
> professional journals. I invite Jameson to participate in those
> journals by submitting papers to them.
>
> Several of us have used the Democracy Chronicles comment space to
> express disagreement with articles there.
>
> Richard Fobes posted in the comment space to exprress his disagreement
> with my article about ICT and the ICT poll at Democracy Chronicles.
> But maybe Richard doesn't like a forum in which both sides can be
> heard in free and open discussion.
>
> I, too, have used the Democracy Chronicles comment space to disagree
> with an article there.
>
> Jameson, if you disagree with one or more of my articles, then I
> invite you to express your disagreement in the comments space.
>
> Mike Ossipoff
>
>
>
>
>
>
> On Fri, Sep 28, 2012 at 2:04 PM, Jameson Quinn <jameson.quinn at gmail.com>
> wrote:
> > One subdomain of voting methods in which the peer-reviewed academic
> > literature is decidedly behind the amateur enthusiast community (that's
> us)
> > is in its coverage of different methods, criteria, and compliances. This
> lag
> > is unfortunate for several reasons. For one thing, it affects whether
> known
> > facts can be covered on wikipedia. For another, though some academics
> > clearly are aware of the amateur knowledge, insofar as others aren't, it
> > leads to needless misunderstandings and/or duplication of effort.
> >
> > I believe we can fix this by creating a peer reviewed open-access journal
> > which combines the strengths of the academic and enthusiast communities.
> The
> > purview of such a journal should be strictly limited to what is
> > mathematically expressed and/or provable, so that sufficiently meticulous
> > amateurs can be considered as qualified peer reviewers (when accompanied
> by
> > at least one experienced reviewer).
> >
> > This would take significant work to start and sustain. I'm volunteering
> to
> > do up to around half the work, but in practice that means I need at least
> > two or three others to step forward as volunteers before we can seriously
> > get started. Here's what I think we need to do:
> >
> > Managing editors
> >
> > These would be the people who would be ultimately responsible for
> > everything. However, their role qua¹ managing editors would be more to
> > shepherd things along; they may or may not also take the role of
> reviewers,
> > authors, etc. I think a group of 3-5 managing editors would be
> sufficient to
> > get things done without burnout. Among the managing editors would need
> to be
> > at least one with a relevant postgraduate degree (for instance
> mathematics,
> > statistics, economics, or political science).
> >
> > Site
> >
> > I think this would work best if it were a sub-branch of some credible
> > existing site. That way, any existing credibility would be inherited. My
> > first suggestion would be Adrian Tawfik's "Democracy Chronicles". I'd
> also
> > be happy to discuss it if any existing organizations (hint, hint) wanted
> to
> > lend their name and/or site.
> >
> > Hosting, software, etc.
> >
> > I suggest that the journal should be run MediaWiki software, the same
> > software Wikipedia runs on. However, all "main space" articles should be
> > protected from changes by all but a limited group of editors. This would
> > allow freewheeling discussion on "talk" pages, but keep actual
> "published"
> > content in a stable, citeable form.
> >
> > Purview
> >
> > I think that this should focus on only four kinds of articles: system
> > definitions (or equivalent re-formulations to help with proofs); criteria
> > definitions; inter-criteria implications and equivalencies; and
> > system-criteria (non)compliance proofs/counterexamples. Initially, only
> > single-winner systems and criteria would be considered, although that
> could
> > change later. Systems and criteria would not be considered "published"
> > without a certain level of "coverage" in terms of (non)compliance proofs.
> > Thus, each individual "article" would frequently (though not always) be
> > under a page in length. This short length and limited purview would
> > establish an important differentiator between this effort and existing
> > journals.
> >
> > Peer reviewers
> >
> > We'd need to have a broad, balanced group of peer reviewers. Reviewers
> would
> > NOT be required to have any specific degrees, but WOULD be required to
> > demonstrate a clear knowledge of the norms of mathematical proof. I'd
> think
> > that 12-20 reviewers is a reasonable goal. I would expect that around
> 2/3 of
> > these reviewers would be capable "amateurs"; I hope we can get
> participation
> > from enough professional academics to constitute at least 1/3 of the
> > reviewers. (I already have several ideas of whom I'd ask, though I'd also
> > expect the other managing editors to help with this.)
> >
> > Peer review and other article life-cycle issues
> >
> > There would be a clear naming scheme to distinguish the various article
> > types. Articles would initially be developed in the main namespace in
> > unlocked form. This would allow any wiki user to help or comment.
> > (Obviously, spammers and other troublemakers would need to be banned.)
> >
> > When an article was considered ready for peer review, its main author
> would
> > tag it as such, and it would be protected from further editing (though
> the
> > associated talk page would still be open for comments).
> >
> > Peer reviewers would have a chance to volunteer using one-shot
> pseudonymous
> > accounts (for which the identities would be secretly verified by any
> > managing editor). Thus, authors would NOT be anonymous, but reviewers
> WOULD.
> > Reviewers would be encouraged to volunteer if they have any serious
> negative
> > concerns on an article, even if they do not wish to fully review all
> aspects
> > of that article.
> >
> > Each article would need at least 3 reviewers, at least 1 of whom must
> have
> > prior experience with the peer review system at this or any other
> journal.
> > If that requirement isn't met by volunteers, the managing editor would
> > attempt to assign reviewers until it was.
> >
> > A review would consist of any number of suggestions, along with a
> > determination of "acceptable as is", "acceptable with minor revisions",
> > "potentially acceptable with major revisions", or "unacceptable".
> Because of
> > the sharply limited scope of each individual article, it is expected that
> > "acceptable as is" would not be as negligibly rare as it is in most peer
> > review. Reviewers who gave one of the top two determinations would be
> > expected to have carefully reviewed the entire article; those who didn't,
> > wouldn't.
> >
> > Once an article had all its reviews, its author would be given
> permission to
> > revise if necessary. When they tag it as "revisions done", those
> permissions
> > would be revoked, and reviewers would have a chance to raise (or lower)
> > their determination. This process could iterate if necessary.
> >
> > In order to be considered "published", an article would need to be
> > unanimously graded "acceptable". At that point, it would be permanently
> > locked, though the talk page would remain open.
> >
> > ....
> >
> > Supposing we have 20 "core" systems and 20 "core" criteria which are to
> be
> > (dis)proven for each of the "core" systems. That's a substantial total:
> 400
> > proofs. However, perhaps half of those will already exist in the
> published
> > literature, and perhaps half of the remainder will be utterly trivial
> (such
> > as well-known counterexamples). That leaves about 100 compliances that
> would
> > need careful review. With 15 reviewers, that's about 20 "core"
> compliances
> > each for review. Including non-"core" systems and criteria, I'd expect
> that
> > to increase to around 30. If the average reviewer handled 1 a month
> (plus 1
> > of the "trivial" cases), that workload would take around 3 years to burn
> > through, with each month's "issue" containing around 5 serious and 5
> trivial
> > compliances. I think that that's roughly doable, if we put our minds to
> it.
> >
> > Also, note that once we got the ball rolling and showed we were doing a
> good
> > job, we could attempt to get an existing professional society to "adopt"
> the
> > journal. If this were successful it would massively increase our
> credibility
> > and ability to attract new peer reviewers and authors.
> >
> > ....
> >
> > The above gives a basic idea of what it would take. Obviously, there's a
> lot
> > of minor and not-so-minor details still to work out. But I hope that this
> > message is enough to get the ball rolling.
> >
> > So: please respond. Any comments, suggestions, or questions? Do you think
> > you could be a managing editor or peer-reviewer? Is there someone else
> you
> > think should be in on this conversation?
> >
> > Jameson
> >
> > ¹ "qua" in this case means approximately "as, per se"
> >
> > ----
> > Election-Methods mailing list - see http://electorama.com/em for list
> info
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.electorama.com/pipermail/election-methods-electorama.com/attachments/20120928/8f680d7c/attachment-0004.htm>