[EM] When and how can we speak of "individual utility" and "social utility"?

[Jobst Heitzig]

Hello folks!

Attention: this is quite a long posting that deals with the fundamental 
question of when and how it is possible to define the terms 
"individual utility" and "social utility" in a meaningful way.

In particular, I try to make clear when and when not the sum of
individual utility values can be considered a measure of social
utility.

The details are not too complicated, though, so you need no deep 
understanding of mathematics or philosophy to follow the reasoning,
I hope.

This is the outline:

1. Preliminaries

2. Individual utilities
 2.1 Non-Archimedean individual utilities
 2.2 Partially ordered non-Archimedean individual utilities

3. Social utilities
 3.1 The comparison problem
 3.2 Normative justifications
 3.3 "The greatest good for the greatest number"
 3.4 Basic axioms for social utilities
 3.5 Deriving "social utility = mean individual utility"
 3.6 Why this is still wrong!

Have fun!
Jobst


1. PRELIMINARIES
================

Throughout, we assume that we are faced with a fixed set of finitely 
many "options"  x,y,z,...  (e.g. candidates in an election).

We are interested whether an individual (e.g. a voter) and whether
society as a whole can assign meaningful "utility" values to all
options. In the econometric literature, such utility functions from
the (binary) preferences the individual has about certain lotteries...

Lotteries:
A lottery  a  is a random procedure that results in exactly one of the
given options, where each option  x  has a certain known probability 
a(x)  between 0 and 1. The probabilities the lottery assigns to all
given options sum up to one:  sum_x a(x) = 1.  Those options  x  that
have a positive probability  a(x)>0  are called the "possible outcomes" 
of lottery  a.

Lottery composition:
Lotteries can be "composed" in the following way. For lotteries  a,b  
and a real number  p  between 0 and 1, let  (p?a:b)  designate the 
composed lottery that applies lottery  a  with probability  p  and 
lottery  b  with probability  1-p. 
For example, when lottery  a  results in x or y with equal probability, 
and lottery  b  results in z with certainty, then lottery  (1/3?a:b)  
results in x, y or z with probabilities 1/6, 1/6 and 2/3, respectively. 
Note that, by this definition,  (1?a:b)=a,  (0?a:b)=b,  (p?a:a)=a,  
and  (p?a:b)=(1-p?b:a).


2. INDIVIDUAL UTILITIES
=======================

Now let us first try to construct utility values for some fixed individual 
by presenting that individual with some pairs of lotteries  a,b  and 
asking her whether she considers  a  "at least as desirable" as  b...

Weak preference relation:
Let  aRb  designate the fact that the individual considers the lottery  a  
"at least as desirable" as the lottery  b.  This defines a binary relation  
R  on the set of all possible lotteries with the given options as possible 
outcomes.  R  is called the "weak preference relation".


Assume we realize that all answers of our individual conform to the 
following five conditions:

(Tot) Totality of R:
      For all a,b, either  aRb  or  bRa  or both.

(Trans) Transitivity of R:
        If  aRb  and  bRc  then  aRc.

(Comp) Invariance of R under lottery composition:
       If  aRb  then  (p?a:c)R(p?b:c)  for all p,c.

(Decomp) Invariance of R under lottery decomposition:
         If  (p?a:c)R(p?b:c)  then  aRb.

(Archi) The Archimedean property:
        For all a,b,c: if  bRc,  there is some p>0 such that  (p?a:b)Rc


If this all is the case, one can mathematically show that there is
indeed a "utility function"  u  which assigns to each lottery  a  
a real number  u(a)  with the following three properties:

(Mon) Strict monotonicity:
      u(a) >= u(b)  if and only if  aRb.

(Exp) Lottery utilities are expected utilities:
      u(p?a:b) = p*u(a) + (1-p)*u(b).

(Uni) Uniqueness up to transformations of scale:
      For every function  v  that satisfies (Mon) and (Exp),
      there are real numbers  r,s  such that  v = r + s*u.

These are essentially the properties economists usually take for
granted when speaking about individual "utility".

However, they only hold when the individual's preferences respect all
of the above five conditions. But what happens when some of them are 
not true for some individual?


2.1 Non-Archimedean individual utilities
----------------------------------------

Let us assume that only the last of the five, the Archimedean property
(Archi), is not true. This can easily happen when, e.g., 

   a = your only child is shot dead,
   b = you receive 1 cent,
   c = nothing happens.

If (Archi) would be true, there would have to be a lottery in which
your child is shot dead with some positive probability  p,  in which you
receive 1 cent otherwise, and which lottery you prefer to nothing
happening. It's obvious that we cannot expect every rational
individual to have such preferences. (In my personal view, I expect
*no* rational individual to have such preferences!)

Fortunately, (Archi) is the least essential of the five conditions for
constructing a utility function  u.  Without (Archi), the above
mathematical result still holds with only one minor difference: the
utility values  u(a)  may not all be elements of the set of standard
real numbers but may be numbers of a more general kind. (For
mathematicians:  u  is a function from the set of lotteries 
into some finite-dimensional totally-ordered real vector space.)

More precisely, the utility values  u(a)  may be numbers that contain
"infinitesimal" components. These so-called "non-standard real numbers" 
can be written in the form 

   r0 + r1*eps + r2*eps² + r3*eps³ + ... + rk*eps^k,

where  eps  is some "infinitesimal unit" which is larger than zero but
smaller than every positive standard real number.

Just like standard real numbers, such numbers with infinitesimal components 
can be compared, added and averaged with probabilities as weights.
Only dividing by such a number is a bit more complicated, but it is
unclear what "dividing something by a utility" should mean anyway.
Almost all the usual "laws" of arithmetics hold with such numbers.

In the above example, your utilities could turn out to be this:

   u(a) = -9999999999
   u(b) = 1*eps
   u(c) = 0

These values are consistent with your preferences since there is no
positive probability  p  such that  p*u(a)+(1-p)*u(b)>0.  This is
because probabilities are standard real numbers. That is, these utility 
values correctly tell you to prefer the sincere outcome  c  to any 
lottery that might kill your child and otherwise gives you a cent only. 
(It has never been cheaper to save your child's life than for just 1 cent! 
Non-Archimedean utilities make it possible!)


2.2 Partially ordered non-Archimedean individual utilities
----------------------------------------------------------

Let's now assume that another of the five conditions turns out to be
false for some individual, namely the totality assumption (Tot): For
some pair of lotteries  a,b,  our individual may refuse to find either 
a  at least as good as  b  or  b  at least as good as  a.  In fact, our
individual may point out that she is undecided between  a  and  b 
because from one point of view  a  may seem strictly preferable, but
from a different point of view  b  may seem strictly preferable. Also,
she would refuse to agree that then  a  and  b  should be considered
"equally desirable" since both her points of view tell her they are not!

We may suggest to her to decide first which of her two points of view
is more important in this situation. But this is just a question of
preference on a higher level: preference between points of view. Our
individual may again tell us that she considers neither point of view
"more important". When she considers both points of view "equally
important" or is even completely undecided about them, this helps
nothing to clarify the question about  a  and  b.

But for such an individual, for whose preferences only the conditions 
(Trans), (Comp), and (Decomp) seem to be true, we can still find a 
"utility function"  u  that satisfies the properties (Mon) and (Exp)! 
Only this time, the utility values  u(a)  may be numbers of a still more 
general kind: They can still be compared, added and averaged with
probabilities as weights, but comparisons may now result in the answer 
"neither larger nor smaller nor equal". In other words, the comparison 
relation "<=" is still transitive but no longer total. (Mathematically 
speaking,  u  is a function from the set of lotteries into some 
partially-ordered real vector space.)

Formally, such utility values can always be represented by vectors of
non-standard real numbers, where a vector  (r1,r2,...,rk)  is considered 
"less or equal" than another vector  (s1,s2,...,sk)  if and only if  
r1<=s1 and r2<=s2 and ... and rk<=sk.
(Mathematically speaking, the vector space is equipped with the 
"product" partial order.)

For example, assume that there are 5 options v,w,x,y,z and the
individual has only the following preferences:

  xRx,
  xRv, xRy,
  vRv,
  vRw, yRy,
  wRw,
  wRz, yRz,
  zRz.

In a so-called Hasse diagram, these preferences look like this 
(use a monospaced font for reading this):

    x
   / \
  v   \
  |    y
  w   /
   \ /
    z

These preferences are not total, since neither is  y  preferred to either
v  or  w,  nor  v  or  w  to  y,  nor is  y  considered equivalent to  
v  or  w.  The following vectors of real numbers represent a utility 
function that is compatible with these preferences:

   u(x) = (2,2)
   u(v) = (2,1)
   u(y) = (0,2)
   u(w) = (1,0)
   u(z) = (0,0)


As we see, there are no serious problems with finding a meaningful 
individual utility function as long as (Trans), (Comp), and (Decomp) 
are true for the individual. There will be serious problems, however, 
when one of these three conditions turns out to be false, and we will 
face such a problem when trying to define social instead of individual 
utility...


3. SOCIAL UTILITIES
===================

So far, we were only concerned with a single individual and her 
preferences and utility function. However, when discussing election 
methods, we frequently stumble upon something termed "social utility" 
of a candidate.

On this mailing list, this is often just used as a synonym for the sum 
(or mean) of all the utilities the individual voters assign to a candidate. 
But even in this simplistic usage of the term, we run into an immediate 
problem: So far, our discussion above only justifies the assumption that 
for each individual voter, there is something which could meaningfully be 
called the "(individual) utility function" of that voter and which is 
uniquely determined by that voter's preferences only up to scale 
transformations.


3.1 The comparison problem
--------------------------

Let's designate voter  i's  utility function by  ui.  While the scale 
ambivalence of each  ui  is irrelevant as long as we look at only one voter 
at a time, it becomes an essential problem as soon as we start to look at 
the utility functions of two or more voters at the same time: As long as  
ui  and  2*ui  are essentially the "same" utility function, it is just not 
*possible* to sum up the utility functions of different voters in a 
meaningful way. Each of the sums  u1+u2,  2*u1+u2,  and  u1+2*u2  can claim 
the same right of being "the sum" of voter 1's and voter 2's utility functions.

So, in order to find a sensible way of summing up utility functions, one 
has first to decide for a common scale for all these functions. This problem 
is known as the "comparison problem" since it can be paraphrased like this: 
Is there a meaningful way to compare utility values from different individuals?

As we cannot solve this profound philosophical problem here, let us for now 
assume that there is indeed some natural common scale for all individual 
utility functions. In other words, let us assume here that 

(Uni+) Scale uniqueness:
       for each voter  i,  a unique utility function  ui  is selected.

Obviously, we *can* then average the individual utility values  ui(x) 
for some option  x  and call the result  t(x).

However, it is not at all clear that this computed number  t(x) 
deserves to be called "social utility" of  x!


3.2 Normative justifications
----------------------------

To justify this, we would have to proceed in a manner analogous to what
we did to derive at individual utilities: We need to relate the number
t(x)  to the "preferences" society has concerning lotteries of options, 
and test whether the conditions (Mon) and (Exp) are indeed true with 
respect to these preferences and with the choice  u=t.  Designating the 
fact that society should consider a lottery  a  "at least as good" as a 
lottery  b  by  aSb,  at least the following condition should turn out to 
be true if we want to call  t  the "social utility function":

(MonT) Strict monotonicity of the function t:
       t(a) >= t(b)  if and only if  aSb.

And in the best case, it also fulfils this:

(ExpT) Social lottery utilities are expected utilities:
       t(p?a:b) = p*t(a) + (1-p)*t(b).


But how can "society's" preferences be studied when there is no single
individual to ask for them? As we know, there is not at all consensus
as to what society's preferences are in any given situation. Therefore, 
it seems we can only approach our task in a normative way: we can only 
try to justify certain properties we can *expect* of "good" society's 
preferences that could be useful in showing the validity of (MonT).
 
Just to say "a society should prefer that options which maximizes the sum 
(or mean) of the individual utilities" will of course be *no* justification, 
since that would just amount to saying "t is the right choice since 
t is the right choice".


3.3 "The greatest good for the greatest number"
-----------------------------------------------

Historically, using the sum is strongly connected with utilitarianism which, 
in essence, claims to want "the greatest good for the greatest number".
One must be very careful not to jump from that goal to the conclusion that 
it can be achieved by taking sums and only sums! For it is not at all clear 
what that vague phrase, as appealing as it may sound, actually means. 
Mathematically speaking, it refers to two target functions ("good" and "number") 
which should be maximized -- at the same time!? 
Usually, when maximizing one thing, the other thing will not in general be 
maximal at the same time. 

Some mutually exclusive interpretations of the above phrase include:

(i) Maximize the number of those voters ("greatest number") that assign the 
highest possible utility value ("greatest good") to the option under 
consideration. 
(This version justifies Plurality voting.)

(ii) Maximize the number of those voters ("greatest number") that assign a 
utility value above some threshold ("greatest good") to the option under 
consideration. 
(This version justifies Approval voting.)

(iii) Maximize the utility value ("greatest good") which at least half of 
the voters ("greatest number") assign to the option under consideration. 
(This version justifies electing the option with the highest median rating.)

(iv) Prefer option  x  to option  y  when a majority ("greatest number") 
assigns a higher utility ("greatest good") to x than to y.
(This version can easily lead to cycles, as we all know.)


3.4 Basic axioms for social utilities
-------------------------------------

But some basic properties of "society's" preferences can probably be 
justified safely with the vague goal "the greatest good for the greatest
number":

(Par) The "Pareto" principle:
      If  ui(x)>=ui(y)  for all i, then  xSy.
      If  ui(x)>=ui(y)  for all i, 
      and  uj(x)>uj(y)  for some j, then  xSy  but not  ySx. 

(Anon) Anonymity: 
       If for each possible utility value  r,  
       the number of voters  i  for which  ui(x)=r  
       equals the number of voters  i  for which  ui(y)=r,  
       then  xSy  and  ySx.

(Neut) Neutrality: 
       If an option  x  is replaced by an option  x'  and every voter 
       assigns the same utility value to  x'  which he has assigned to  x,
       then society should prefer  x'  to exactly the same options it 
       preferred  x  to, and prefer to  x'  exactly the same options it 
       preferred to  x,  and consider  x'  equivalent to exactly the same 
       options it considered  x  equivalent to.

In addition to these conditions that relate society's preferences to 
individual utilities, one can also try to justify properties similar to the 
five properties (Tot), (Trans), (Comp), (Decomp), and (Archi) we started with: 

(STot) Totality of S:
       For all a,b, either  aSb  or  bSa  or both.

(STrans) Transitivity of S:
         If  aSb  and  bSc  then  aSc.

(SComp) Invariance of S under lottery composition:
        If  aSb  then  (p?a:c)S(p?b:c)  for all p,c.

(SDecomp) Invariance of S under lottery decomposition:
          If  (p?a:c)S(p?b:c)  then  aSb.

(SArchi) The Archimedean property:
         For all a,b,c: if  bSc,  there is some p>0 such that  (p?a:b)Sc.


3.5 Deriving "social utility = mean individual utility"
-------------------------------------------------------

Here's the good news: If the above conditions really hold and if there are
"enough" options, we can now prove that taking sums or means of individual
utilities is indeed the right thing. "Enough" options means that for each
possible combination of individual utility values, we find a lottery to 
which the voters indeed assign these individual utility values. 

THEOREM: 
Assume that individual preferences and utilities fulfil (Tot), (Trans), 
(Comp), (Decomp), and (Uni+), that social preferences and utilities fulfil 
(Par), (Anon), (STrans), (SComp), and (SDecomp), and that there are "enough" 
options. Then the function  t  fulfils (MonT).

Proof: 

(i) First recall that (Tot), (Trans), (Comp), (Decomp), and (Uni+) imply that 
there are unique individual utility functions  ui  from the set of lotteries into 
the set of non-standard real numbers such that (Mon) and (Exp) hold. 
Also, note that (STrans), (SComp), and (SDecomp) imply that there is some 
function  u  from the set of lotteries into the set of sequences of non-
standard real numbers such that  u(a) >= u(b)  if and only if  aSb,  and such
that  u(p?a:b) = p*u(a) + (1-p)*u(b)  for all a,b,c,p,  and such that every
other function  u'  with the same properties is of the form  u' = r + s*u
for some r,s.    

(ii) Now we show that (MonT) holds by showing that  t(a0)>=t(b0)  if and only 
if  u(a0)>=u(b0).  Assume that  t(a0)>=t(b0).  Sort the voters into some 
arbitrary  ordering  i1,i2,...,in  where  n  is the number of voters.  Since 
there are "enough" options, we find for each integer  j  between 1 and  n-1  some 
lotteries  aj,bj  such that  uk(aj)=ul(a0),uk(bj)=ul(b0)  for all k and l for 
which k-l=j.  In other words, the individual utility values the voters assign 
to  aj,bj  are just those for  a0,b0,  permuted by moving them  j  voters to 
the right. Because of (Anon), we know that  ajSak  and  bjSbk  for all j,k,
that is, all of the  aj  and all of the  bj  are "socially equivalent".
Not let  a  be the lottery that realizes each of the lotteries  aj  with the
same probability  1/n,  and define  b  similarly as a "mixture" of all the  bj.
Because of (SComp), we have  a0Sa, aSa0, b0Sb, and bSb0.
Because of (Exp), we have  ui(a)=t(a0)  and  ui(b)=t(b0)  for all i.
Because of (Par), this implies  aSb.  Because of (STrans), we finally know that
a0SaSbSbo  implies  a0Sb0.
On the other hand, assume that  a0Sb0  but not  t(a0)>=t(b0).  Because the 
latter two are elements of a totally ordered vector space, this means that  
t(a0)<t(b0),  hence  b0Sb, bSa, aSa0  because of what we just proved. Because
of (STrans),  aSa0Sb0Sb  implies that also  aSb.  Because of (Par), this implies 
that  ui(a)=ui(b)  for all i, hence  t(a0)=t(b0).  But this is a contradiction
to our assumption that not  t(a0)>=t(b0).  Hence  a0Sb0  implies  t(a0)>=t(b0).

Q.E.D.

With similar reasoning, we can also conclude that under the premises of 
the theorem, also (ExpT) holds, and every other function  t'  which fulfils 
(MonT) is of the form  t' = r + s*t.


3.6 Why this is still wrong!
----------------------------

And here's the bad news: Without (SComp) and (SDecomp), the sum or mean individual
utility cannot be shown to be the unique choice of a "social utility function".
Even worse: If the sum or mean is used, (SComp) and (SDecomp) follow! Hence any
evidence against (SComp) and (SDecomp) is in fact immediate evidence against taking
sums.

Why should some seemingly harmless and natural condition (SComp) be wrong?

Here's why: Consider the two options

   a = some unknown stranger gets $1,000,000, the rest gets nothing
   b = some other unknown stranger gets $1,000,000, the rest gets nothing.

Then, in accordance to (Anon), most people will agree that both are socially 
equivalent. But I claim that at the same time most people would consider the
lottery  (1/2?a:b)  to be socially preferable to both  a  and  b,  simply because
it is fairer and fairness is a social good!

But this is in direct conflict with (SComp) which would require that also  (1/2?a:b)
is socially equivalent to  a  and  b.

A similar argument can be used against (SDecomp).

Without (SComp) and (SDecomp), there are many other choices for a "social utility 
function", some of which are in accordance with the requirement that  (1/2?a:b)  
be socially preferable to  a  and  b.  One of these social utility functions,
also called "welfare functions" since they consider fairness a social good and
inequality a social drawback, is the Gini function. 

To recall: The Gini welfare value  g(a)  of a lottery  a  given individual utility 
values  ui(a)  is the weighted sum

   g(a) := u1(a) + 3*u2(a) + 5*u3(a) + 7*u4(a) + ... + (2n-1)*un(a),

where the voters have been ordered so that  u1(a)>=u2(a)>=...>=un(a).


I hope this clarifies some things about "utility".

Yours, Jobst