[EM] Voter satisfaction measure in a general case?

[Juho Laatu]

I think you can't really have any good rating style satisfaction measures in methods that measure rankings only.

> On 14 Oct 2017, at 11:33, Magosányi Árpád <m4gw4s at gmail.com> wrote:
> 
> Hi,
> 
> I am looking for established methods to measure voter satisfaction for Condorcet votes, where the result is available, but no data on the utility function.
> 
> I understand that it can be measured by assigning an utility function to each voter, and see how close the result came to that utility function. However I could not find anything on how to construct that utility function.
> 
> My thoughts:
>  - I would use a "satisfaction index"
>  - The best choice have 100% satisfaction index (I am okay with disregarding the fact that even that might not be 100%)

It seems you want to measure normalised satisfaction, not absolute satisfaction. I mean that even if my true preferences would be a=52 b=51, you will count my preferences as a=100, b=0. That's ok, although it doesn't reflect voter satisfaction accurately (to me all candidates were about equal).

>  - If there is a dummy choice (e.g. "I disagree with anything I rank below this"), than that one is 0%, and the worst ranked is -100%

Here you actually have "data on the utility function". You ask voters to tell where their preferences change from "ok" to "not ok". You might get quite sincere results. But that applies also to asking voters to rate all the candidates in range 100% to -100%. Rankings can then be derived from those ratings.

The dummy choice might however be used for some other reason (not just to collect ratings). One could for example have an election that would be inconclusive if the winner loses to the dummy choice (acceptability limit). In that situation you would get the (0%) ratings sort of free, without really asking for ratings :-) .

>  - If there is no dummy choice, then the worst ranked is 0%
> 
> Now what should the function look like for the choices in between?
> - Linear (as it is easy and middle ground)?
> - Logarithmic/exponential (as choices might mostly near good solutions or mostly bad)?
> - Gaussian (as we are talking about Bayesian regret)?
> - Result of some clever computation based on vote statistics?

I'll try to address the problem of measuring in between candidate satisfaction measures using some simple examples.

49: a=100, b=99, c=0
02: b=100, a=1, c=0
49: c=100, b=99, a=0

It is not possible to see from these rankings that voters like b very much.

The ratings could be as well:
49: a=100, b=1, c=0
02: b=100, a=1, c=0
49: c=100, b=1, a=0

We might have also some problems with clones.

50: a1 > a2 > a3 > a4 > b
50: b > a1 > a2 > a3 > a4

Most functions will give a1 a high score although b voters might dislike a1 about as much as a1 voters dislike b.

In summary, if you want to estimate satisfaction based on the given rankings (and maybe the acceptability threshold), you must make some assumptions like having an even distribution of candidates (in the voter opinion space). Collecting ratings (and deriving rankings from them) would be one step simpler, but not be problem free either.

If I'd assume even distribution of candidates, maybe the linear approach (out of the ones that you proposed) would be sufficient. Or maybe it is typical that voters like their own candidate and hate most of the others. In order to find out what this function looks like (it could be quite different in different voting societies), one could make a small query and ask people how they sincerely rate different candidates. One could approximate the function (curve between favourite 1000% and least favourite 0%) based on those answers.

BR, Juho