[EM] Proportionality: the small bias effect seems to be real for Harmonic

[Kristofer Munsterhjelm]

On 2024-09-23 19:56, Toby Pereira wrote:
> Interesting as always, Kristofer. A couple of things:
> 
> My intuition is that the most balanced result for 2 candidates is at 33 
> and 67 rather than 25 and 75. 25 and 75 seems to suggest you're just 
> splitting the electorate into two and finding the best candidate for 
> each half, rather than finding the best 2 candidates for the entire 
> electorate. Given that harmonic voting works on the scores voters give 
> to all elected candidates rather than simply their best candidate, I 
> would suggest that the Sainte-Laguë delta giving the 33/67 result is 
> what I would consider to be the correct behaviour. I would expect 
> methods that assign voters to a single candidate to be 75/25.

I can see that argument. The question is really what proportionality means.

We have to determine if 33/67 being more desirable than 25/75 is because 
33/67 is more proportional *as such*, or if we're deliberately giving up 
some raw proportionality to get an assembly that's better adjusted as a 
whole (more able to work as a unified body, related to the "utility" 
measure).

If it's the former, then we'd need to find a better definition of what 
we mean by proportional. Whatever it is should use hidden information 
(something depending on opinion space) so that the 1D normal reaches 
optimal proportionality by 33/67 instead of 25/75. Just saying "what 
harmonic/psi/Phragmen says is good is good" blinds us to the possibility 
that there may exist (yet undiscovered) methods that do what the 
yardstick does, but better.

If it's the latter, then that means that what we're balancing is the 
ability of the method to evenly cover opinion space with its winners, 
and the ability of the winning assembly to work in a way similar to a 
high-quality single winner.

That this balance isn't "all proportionality at all costs" makes sense. 
Party list PR methods with Plurality ballots can't do that balance well, 
even when they're proportionally unbiased, and thus exhibits other 
problems: an assembly elected by Sainte-Laguë may still have kingmaker 
problems, something that would be improved by having a "centrist center 
of gravity" to break the ties instead. But D'Hondt can't do that because 
it doesn't know who the centrists are, since all it's got are first 
preference votes.

As you point out, it's not that strange for Harmonic to deviate from the 
Monrovian proportionality concept, since it does, after all, take 
ratings of all winners by all voters into account. What I'm saying is 
that we should try to find out what the ideal of proportionality ought 
to be. Or find some of the different aspects of the elephant, if we 
can't characterize it exactly.

> Also, I think using a the normal distribution rather than a uniform one 
> complicates matters, even if it is more realistic. The normal and 
> uniform will have the mean and median the same, but the percentiles 
> won't be the same as what I call the "permeantiles" (percentile 
> equivalents when using the mean). So while I would expect the 
> Sainte-Laguë delta to give 33/67 for a uniform distribution, I'm not 
> sure I'd necessarily expect this result for the normal distribution, 
> although your results suggest it does give this anyway. This is because 
> harmonic voting works on scores rather than ranks, so I wouldn't expect 
> it to particularly follow the percentile data. So your results are a bit 
> of a surprise in that respect.

I'm not quite sure I get what you're saying. By permeantile, do you mean 
something related to the expected value of a truncated distribution, the 
way the mean is the expected value of the full distribution?

I agree that Harmonic doesn't try to place itself at a given percentile 
for the normal, since it hasn't been designed to do so. It is 
surprising, though, that my first proportionality measure (opinion space 
proportionality) seems to prefer deltas that lead to 25-75 to ones that 
lead to 33-67.

If there is a Monroe component to this measure, then I would expect 
Monroe itself to show up as being very proportional by that measure. I 
should check that - only problem is that implementing the assignment 
algorithm is kind of a hassle.

> While the percentiles and "permeantiles"* would clearly match for a
> uniform distribution, I was assuming that they definitely wouldn't match
> for a non-uniform distribution. That might be true anyway but I'm not
> entirely sure. It might be that for a normal distribution they would
> also match, which would remove the tension I was discussing in my last
> post. In any case, it would be interesting to see results for a uniform
> distribution.

I did some quick integral evaluations, so take this with a bit of salt, 
but it seems like the uniform distribution percentiles are the same as 
the normal.[1]

Suppose we have a uniform between -1 and 1. the pdf is 1/2. Each voter 
rates a candidate equal to the negative of their distance (I'm skipping 
the constant of 20 this time as it shouldn't matter.) As before, the 
winners' positions are at x_1 and -x_1. Then, if my calculations are 
correct, the Harmonic score for given delta and x_1 is:

(-2 * x_1^2 * delta - 2 * x_1^2 - 2 * x_1 - 2 * delta - 1)/(2 * delta * 
(delta + 1))

The optimum is achieved when x_1 = -1/(2(delta+1)), which might look a 
bit familiar.

The quantile is (2 * delta + 1) / (4 * delta + 4), which *definitely* 
looks familiar.

For different values of delta:
	delta = 0	x_1 = -1/2	25th percentile
	delta = 0.5	x_1 = -1/3	33rd percentile
	delta = 1	x_1 = -1/4	37.5th percentile.

> Also, in the case where delta approaches zero, resulting in only each
> voter's favourite winner counting towards the quality function, the 
> results in general wouldn't look at all proportional, but I'm not
> sure it really matters because the 25/75 result that you wanted was
> likely an intuition that you could be persuaded out of!

Oh, delta=0 definitely isn't proportional in every case. Monroe proper 
has a constraint that each candidate can only be assigned to a certain 
number of voters, so that each candidate represents the same number of 
voters. On symmetric distributions like the normal and uniform with 
symmetrically positioned candidates, you get that anyway, but I would 
imagine that Harmonic with delta=0 would be disproportional if the 
distribution is skewed: some winners would be backed by more voters than 
others. Meanwhile, Monroe's constraint would keep that from happening, 
even for a Monroe-type idea of proportionality.

To put it differently, Harmonic delta=0's results would be more like 
Chamberlin-Courant but with the weights discarded after election. E.g. 
for something like

100: A (10) > B (9) > C (0)
   1: C (10) > B (1) > A (0)

the intuitively proportional result for two winners is {A, B}, but 
Harmonic with delta=0 would choose {A, C}. Monroe would pick {A, B} but 
Chamberlin-Courant would do something like: A (weight 100/101), C 
(weight 1/101).

In this sense delta=0 is akin to at-large IRV and LPV0+.

This also shows that methods that are on-average proportional (for some 
model) may still fail quite badly outside that model, or even some of 
the time in-model. That's a weakness of VSE-type analysis. We could 
possibly get some information about how often that happens (still 
in-model) by reporting something that is to the standard deviation what 
the VSE is to the mean.

-km

[1] A better mathematician might be able to determine if this is true of 
every symmetric distribution. Such a strategy would probably use that 
the pdfs of distributions that are symmetric about 0 are odd, and then 
something with voters on the far right canceling out voters on the far 
left. But turning that into an actual proof would still be quite a lot 
of work.