[EM] Amateur peer-reviewed "journal" for voting methods, criteria, and compliances?

[Michael Ossipoff]

To: Jameson and all who received his message:

Whose head-up-the-a** idea was that?

Certainly not.

Who decides who your "peers" are? Who chooses them?

At Democracy Chronicles, there is free and open discussion. If you
don't agree with something said in an article, then there is a
comments space available, in which you can express your disagreement
and tell your reasons for disagreeing. ...and the author of the
article always has the opportunity to reply to your comments.

The readers can decide which argument is more convincing.

That's called "free and open discussion".

What Jameson proposes is something quite different. Among some small
group of people, and one of them, unilaterally, can block the
readership's access to an article.

There is already a system of such journals--the academic and
professional journals. I invite Jameson to participate in those
journals by submitting papers to them.

Several of us have used the Democracy Chronicles comment space to
express disagreement with articles there.

Richard Fobes posted in the comment space to exprress his disagreement
with my article about ICT and the ICT poll at Democracy Chronicles.
But maybe Richard doesn't like a forum in which both sides can be
heard in free and open discussion.

I, too, have used the Democracy Chronicles comment space to disagree
with an article there.

Jameson, if you disagree with one or more of my articles, then I
invite you to express your disagreement in the comments space.

Mike Ossipoff






On Fri, Sep 28, 2012 at 2:04 PM, Jameson Quinn <jameson.quinn at gmail.com> wrote:
> One subdomain of voting methods in which the peer-reviewed academic
> literature is decidedly behind the amateur enthusiast community (that's us)
> is in its coverage of different methods, criteria, and compliances. This lag
> is unfortunate for several reasons. For one thing, it affects whether known
> facts can be covered on wikipedia. For another, though some academics
> clearly are aware of the amateur knowledge, insofar as others aren't, it
> leads to needless misunderstandings and/or duplication of effort.
>
> I believe we can fix this by creating a peer reviewed open-access journal
> which combines the strengths of the academic and enthusiast communities. The
> purview of such a journal should be strictly limited to what is
> mathematically expressed and/or provable, so that sufficiently meticulous
> amateurs can be considered as qualified peer reviewers (when accompanied by
> at least one experienced reviewer).
>
> This would take significant work to start and sustain. I'm volunteering to
> do up to around half the work, but in practice that means I need at least
> two or three others to step forward as volunteers before we can seriously
> get started. Here's what I think we need to do:
>
> Managing editors
>
> These would be the people who would be ultimately responsible for
> everything. However, their role qua¹ managing editors would be more to
> shepherd things along; they may or may not also take the role of reviewers,
> authors, etc. I think a group of 3-5 managing editors would be sufficient to
> get things done without burnout. Among the managing editors would need to be
> at least one with a relevant postgraduate degree (for instance mathematics,
> statistics, economics, or political science).
>
> Site
>
> I think this would work best if it were a sub-branch of some credible
> existing site. That way, any existing credibility would be inherited. My
> first suggestion would be Adrian Tawfik's "Democracy Chronicles". I'd also
> be happy to discuss it if any existing organizations (hint, hint) wanted to
> lend their name and/or site.
>
> Hosting, software, etc.
>
> I suggest that the journal should be run MediaWiki software, the same
> software Wikipedia runs on. However, all "main space" articles should be
> protected from changes by all but a limited group of editors. This would
> allow freewheeling discussion on "talk" pages, but keep actual "published"
> content in a stable, citeable form.
>
> Purview
>
> I think that this should focus on only four kinds of articles: system
> definitions (or equivalent re-formulations to help with proofs); criteria
> definitions; inter-criteria implications and equivalencies; and
> system-criteria (non)compliance proofs/counterexamples. Initially, only
> single-winner systems and criteria would be considered, although that could
> change later. Systems and criteria would not be considered "published"
> without a certain level of "coverage" in terms of (non)compliance proofs.
> Thus, each individual "article" would frequently (though not always) be
> under a page in length. This short length and limited purview would
> establish an important differentiator between this effort and existing
> journals.
>
> Peer reviewers
>
> We'd need to have a broad, balanced group of peer reviewers. Reviewers would
> NOT be required to have any specific degrees, but WOULD be required to
> demonstrate a clear knowledge of the norms of mathematical proof. I'd think
> that 12-20 reviewers is a reasonable goal. I would expect that around 2/3 of
> these reviewers would be capable "amateurs"; I hope we can get participation
> from enough professional academics to constitute at least 1/3 of the
> reviewers. (I already have several ideas of whom I'd ask, though I'd also
> expect the other managing editors to help with this.)
>
> Peer review and other article life-cycle issues
>
> There would be a clear naming scheme to distinguish the various article
> types. Articles would initially be developed in the main namespace in
> unlocked form. This would allow any wiki user to help or comment.
> (Obviously, spammers and other troublemakers would need to be banned.)
>
> When an article was considered ready for peer review, its main author would
> tag it as such, and it would be protected from further editing (though the
> associated talk page would still be open for comments).
>
> Peer reviewers would have a chance to volunteer using one-shot pseudonymous
> accounts (for which the identities would be secretly verified by any
> managing editor). Thus, authors would NOT be anonymous, but reviewers WOULD.
> Reviewers would be encouraged to volunteer if they have any serious negative
> concerns on an article, even if they do not wish to fully review all aspects
> of that article.
>
> Each article would need at least 3 reviewers, at least 1 of whom must have
> prior experience with the peer review system at this or any other journal.
> If that requirement isn't met by volunteers, the managing editor would
> attempt to assign reviewers until it was.
>
> A review would consist of any number of suggestions, along with a
> determination of "acceptable as is", "acceptable with minor revisions",
> "potentially acceptable with major revisions", or "unacceptable". Because of
> the sharply limited scope of each individual article, it is expected that
> "acceptable as is" would not be as negligibly rare as it is in most peer
> review. Reviewers who gave one of the top two determinations would be
> expected to have carefully reviewed the entire article; those who didn't,
> wouldn't.
>
> Once an article had all its reviews, its author would be given permission to
> revise if necessary. When they tag it as "revisions done", those permissions
> would be revoked, and reviewers would have a chance to raise (or lower)
> their determination. This process could iterate if necessary.
>
> In order to be considered "published", an article would need to be
> unanimously graded "acceptable". At that point, it would be permanently
> locked, though the talk page would remain open.
>
> ....
>
> Supposing we have 20 "core" systems and 20 "core" criteria which are to be
> (dis)proven for each of the "core" systems. That's a substantial total: 400
> proofs. However, perhaps half of those will already exist in the published
> literature, and perhaps half of the remainder will be utterly trivial (such
> as well-known counterexamples). That leaves about 100 compliances that would
> need careful review. With 15 reviewers, that's about 20 "core" compliances
> each for review. Including non-"core" systems and criteria, I'd expect that
> to increase to around 30. If the average reviewer handled 1 a month (plus 1
> of the "trivial" cases), that workload would take around 3 years to burn
> through, with each month's "issue" containing around 5 serious and 5 trivial
> compliances. I think that that's roughly doable, if we put our minds to it.
>
> Also, note that once we got the ball rolling and showed we were doing a good
> job, we could attempt to get an existing professional society to "adopt" the
> journal. If this were successful it would massively increase our credibility
> and ability to attract new peer reviewers and authors.
>
> ....
>
> The above gives a basic idea of what it would take. Obviously, there's a lot
> of minor and not-so-minor details still to work out. But I hope that this
> message is enough to get the ball rolling.
>
> So: please respond. Any comments, suggestions, or questions? Do you think
> you could be a managing editor or peer-reviewer? Is there someone else you
> think should be in on this conversation?
>
> Jameson
>
> ¹ "qua" in this case means approximately "as, per se"
>
> ----
> Election-Methods mailing list - see http://electorama.com/em for list info
>