On the Formal Properties of Weighted Averaging as a Method of Aggregation Author(s): Carl Wagner Source: Synthese, Vol. 62, No. 1, Consensus (Jan., 1985), pp. 97-108 Published by: Springer Stable URL: http://www.jstor.org/stable/20116086 . Accessed: 10/05/2011 13:23 Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unless you have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and you may use content in the JSTOR archive only for your personal, non-commercial use. Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained at . http://www.jstor.org/action/showPublisher?publisherCode=springer. . Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printed page of such transmission. JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Springer is collaborating with JSTOR to digitize, preserve and extend access to Synthese. http://www.jstor.org CARL ON THE FORMAL AVERAGING WAGNER PROPERTIES AS A METHOD OF WEIGHTED OF AGGREGATION I first encountered Keith work on consensus Lehrer's the during summer of 1977,l and was immediately of intrigued by the possibility a formal account of his model of rational group decision developing I adopted as a model for this enterprise the axiomatic method making. was not to discover of social choice and that the theory surprised was as complex of how to aggregate and prob probabilities question lematic as the question of how to aggregate and utilities. preferences The former issue has only recently received the kind of attention which at the latter for several decades, has been directed and is finally as as to to be attested debated, by the essays in beginning vigorously this volume. I am grateful to Barry Loe wer for organizing this forum on in Science Rational Consensus critical essays follow. 1. and Society. My DAVIS replies to the preceding BAIRD I agree with Baird that averages may conceal differences in standard as well as in other distributional a set deviations of of numerical aspects to In certain decisionmaking estimates. situations one is well advised a com to to attention such differences and eschew pay premature In other situations in the form of an average. it is not so clear promise that distributional information is useful. Suppose, for example, that I am a to to decide whether and trying undergo particular surgical procedure consult a group of experts on the probability that the operation will or not to undergo that I must decide whether succeed. Assume this on their based advice. that rule circumstances operation solely (Imagine out delaying in order to give these experts time for further the decision research, and that there is no more qualified group to whom I can turn for advice.) Suppose that they report to me a consensual of probability success equal to 0.6, based on a consensual weighted of their average to use this number initial estimates. It seems to me that I ought in the expected calculating tive of the distributional Synthese 62 (1985) 97-108. ? 1985 by D. Reidel Publishing the operation, irrespec utility of undergoing set of estimates. of initial their aspects 0039-7857/85/0621-0097 Company $01.20 CARL 98 WAGNER in which Baird may reply that the scenarios (a) they all initially agree that 0.6 is the probability of success, (b) their initial opinions are widely are concentrated scattered about 0.6, and (c) their initial opinions in on either side of 0.6 represent two "camps" sorts of very different I agree that one can make situations. about the interesting conjectures state of the art involved in making in question, the prediction depend on seems to me that such of scenarios but it which these obtains, ing are irrelevant to my decision. Perhaps Baird would speculations reply that scenario the knowledge (b) obtained prior to the group's me consensus to to should incline and that I should caution coming in calculations of the initial smallest my employ expected utility in question. Even if the individual respon estimate of the probability received the lowest consensual weight of any of sible for that estimate in the group? This is not caution, but pessimism, the surgeons and is no more warranted than embracing the largest initial estimate of the that in question. probability that scenario (c) obtained suppose prior to the group's Similarly, consensus. to I adopt On what should the prior grounds coming to that of the other? Remember that estimate of one camp in preference of judging the relative expertise I hive no independent means of the two groups. in any case, the distributional relates accord more prominence to information which Baird wishes to the initial set of estimates of the are in question. The crucial issue is not how these estimates probability if the final consensual estimate is. but what, distributed, any, point to our to the in forces individuals method grant weight Nothing if not on the basis opinions of others. How should Imake my decision, consensus of the experts? of an uncoerced If my scheme of utilities is such that a consensual estimated prob success to undergo to in the of 0.6 results my choosing ability equal room I enter of the in somewhat course, may, operating operation, on the prior distribution of different states, depending psychological in question. Imay choose to be encouraged estimates of the probability the probability initially estimated by the fact that one or more surgeons success to be plagued to be 0.8 or allow myself by apprehension to be this probability about the fact that some of them initially estimated case it might be best for me not to be apprised of this 0.4 (in which con since anxiety can have physiological distributional information, are to distractions But such emotional endemic decision sequences). of WEIGHTED AVERAGING 99 Indeed, the normative model which endorses decisionmaking making. on the basis of expected utilities may be viewed in part as an attempt to of such considerations. counteract influence the retrograde Baird, to bringing them to the fore. albeit unwittingly, may be contributing the case in Baird also notes that in, say, a 2-person decision problem, same small weight and the case which each person assigns the other the same in which each person assigns the other the substantial weight both vector is that our (2,2)- His conclusion weight yield the consensual to low from fails differentiate mutual mutual method regard high of our elementary the applicability regard. He is of course assuming so that the weights in question are granted across a hierarchy of model, seems from this perspective, his example evaluative skills. Viewed come at not to than I first less disturbing glance. May considerably accord another's opinion the same weight as my own by different paths? even grudging, acts of concession may, after sequence of measured, an same as of in the result immediate all, cumulate appreciation of view. another's point In both of the foregoing the issue between Baird and examples a we have that is whether differences make ourselves ignored seem to me to which he wishes to emphasize difference. The differences theorists. be data for psychologists rather than decision A 2. ROBERT LADDAGA AND BARRY LOEWER n have assigned subjective i= 1,..., that individuals prob Supposing a sequence to of exhaustive abilities pairwis^ contradictory, Pi(Sj) = we have proposed their l,...,fe, s7, y aggregating propositions a a of form into consensual rule the p by assignment assignments single n (2.1) The p(Sj)= ??=1 WiPiiSj). function propositions (2.2) p is extended by the natural u = sh v to disjunctions and obvious rule v sjr of atomic p(u)=?k=i p(sjk\ and consensual conditional be computed by the rule probabilities for disjunctions u and t are to 100 (2.3) CARL p(u\t) = WAGNER p(uAt)/p(t). to one occur to try alternatively to calculate it might the as a consensual conditional of the average probability weighted p(u \ t) individual conditional and Loe wer note probabilities Pi(u\ t). Laddaga are unconditional that the weights vv?,used to aggregate probabilities, not suitable for this task since, in general, Now (2.4) p(u\t)?t i= \ WiP?u\t) as Laddaga and This is not a proof of some grave defect in our method, seem to think, but just an indication of the fact that the Loewer of conditional involves subtler considerations probabilities aggregation of unconditional than the aggregation The correct way to probabilities. as is well known conditional 1968, (Raiffa, aggregate probabilities, 8, ?11), is given by the weighted average Chapter (2.5) ?7=1 ?iPi(u\t\ where (2.6) ?j=wiPi(t)/i wiPi(t). It is easy to check that (2.5) yields the same result as (2.3), a fact which and Levi (1980) puts to crucial use in his theorem on conditionalization convex sets of probability distributions. in that under aggregation It is obvious by arithmetic averaging, to the atomic propositions dividuals may assign probabilities ..., sk su u and i, of these that some pair of disjunctions, in such a way for each of their assignments, turn out to be independent propositions I have argued for the consensual but not independent assignments. is a bad thing at length (Lehrer and Wagner, that this elsewhere 1983) in the probability is of epistemic significance only if independence at in and that where the initial acts assessment hand, problems problem a are directed to at assigning of of assessment sequence probabilities an formal is after-the-fact atomic propositions, curiosity. independence r that you observe A further example may be in order. Suppose successes r' successes in n' (different) in n trials and I observe trials, WEIGHTED 101 AVERAGING the data generating process agreed by both of us to be a Bernoulli on our data I would select as the density process. Based respective function for the parameter p of this process = r- 1)!, l)!(rc (n xT'^/irl)!xrl(l ?(n, r) with 0<jc<1, and you would select ?(n\ consensual r'). The appropriate density of function here is not a weighted r'), but average ?(n, r) and ?(n\ that we rather j3(n + n\ r+ r'). I cite this example both to emphasize as a method have not endorsed weighted of aggregating averaging to show that and also the functions perfectly unexceptionable density is not function consensual ?(n + n\r+r') independence density 1 success in 2 trials, we For if we each separately observe preserving. on [0,1], as select j3(2,1), the uniform distribution separately of For the uniform the the density function p. density, propositions p e [1/4, 3/4] and p e [5/8, 7/8] are independent. This is not the case for ? = and Loewer the consensual 6x(l x). Are Laddaga density ?(4, 2) one to that this should be disturbed failure to argue prepared by preserve independence? are of course other kinds of assessment where There problems is Such epistemically problems significant. typically independence for exam involve a sequence of independent trials. Suppose, repeated to the 4 possible ple, that you and I are asked to assign probabilities outcomes of flipping a (possibly biased) coin twice. I have information will each that the coin showed heads 3 times in 10 flips and you have information that coin showed heads 8 times in 20 (different) flips. Based on my of HH, HT, the probabilities TH, and sample alone I would estimate as and and based TT, respectively, (0.7)(0.3), (0.3)2, (0.3)(0.7), (0.7)2 on your sample alone, you would estimate these probabilities, respec and (0.6)2. In this situation it (0.6)(0.4), tively, as (0.4)2, (0.4X0.6), our to take a weighted of of the would be ridiculous estimates average a consensual of these events in order to produce dis probabilities and Loewer think that our reason, Laddaga to this sort of thoughtless of any averaging cross we our that It distributions What should doesn't.) path. probability our to is combine the consensual do, clearly, samples, producing 11/30 of heads on any toss, and then invoke independence probability on each toss (on which we agree) to produce of the outcomes the tribution. method (For some us commits consensual assignments (11/30)2, (ll/30)(19/30), (19/30)(ll/30), and 102 CARL WAGNER one hardly notices it in such a simple case, weighted (19/30)2. Although is used here, not to combine our estimates of the probabilities averaging of the events HH, HT, TH, and TT, but to combine our estimates of outcomes of the possible the probabilities {H, T} on any toss. For our our estimates to is of combining samples combining equivalent to the sample sizes with weights these basic probabilities, proportional = + on which we based these estimates: 11/30 and (l/3)(0.3) (2/3)(0.4) (1/3X0.7) + (2/3X0.6) = 19/30. reveals a simple and generally reliable indicator The above example of individuals all the epistemic When significance independence: in order to assign probabilities, make use of independence when i.e., of certain probabilities other prob by multiplying they actually calculate a to be have theoretical commitment assumed abilities, prior they may to incorporate to such independence. Failure such instances of in a in of consensual their individual dis dependence aggregation of tributions would thus ignore an epistemically feature their significant that one We have seen in the above example, however, cases of epistemically of in take account significant our consensus. to within the framework of The dependence approach can a our also shows how method accommodate example easily consensus that some random variable is binomially distributed, contrary to the claim of Laddaga and Loewer. of independence is in One further remark about the preservation of and Loewer make order. Laddaga the strong claim that a method must (even in independence probabilities always preserve aggregating cases where to in it is of no epistemic avoid order significance) to But the of which sequence they employ inconsistency. propositions are bi-level not which do admit argue for this assertion propositions distributions. can easily this fact by however assignments, they may seek to disguise probability I label H & K & after their choice of notation. What is, all, the they = assertion H81K& Since the first two prop P(H & K) P(H)P(K). ositions are amenable only to first order probability and the assignments a to second order third is amenable probability assignment only in higher order probabilities), the entire com (assuming one believes can no be assigned meaningful probability. pound proposition it could easily be if Laddaga and Loewer's Indeed, gambit worked, to show that aggregation methods must preserve modified every feature no matter how esoteric. Thus, common to all individual distributions, for example, nonindependence and nonequality of probabilities assig WEIGHTED AVERAGING 103 to be preserved in consensual dis have ned to two events would in every individual distribution. if these properties obtained tributions the that Lehrer To take an even wilder suppose assigns example, and and I them A B the and 1/5 4/5 assign propositions probabilities that the prob the probabilities 2/5 and 3/5. Let F be the assertion in lowest terms, have a abilities of A and B are rational numbers which, to to and Loewer's denominator 5. According Laddaga equal our set of matrix for the argument, expanded propositions probability A &F, and B& F would be A&F, B&F, A&F 1/5 (L) (W) 2/5 A8iF 0 0 B&F 4/5 3/5 B&F 0 0 assuming that any reasonable method would assign the second and a consensual of zero, consensual fourth propositions prob probability ? abilities for the above four events must take the form p, 0, 1 p, and 0. of F equals 1 and any p not of the form Thus the consensual probability for be with would inconsistent this fact. But this l<fc<5, fc/5, and I based our initial conclusion is absurd. If for example, Lehrer on separate of disjoint examinations samples of 5, he probabilities case A 1 I in in and 2 of the cases, etc., it would be presence detecting to A B and the consensual reasonable assign probabilities eminently Now and 7/10. conclude and Loewer ?111 of their paper with an example Laddaga two of a decision probabilities, problem where, prior to aggregating on course of and afterwards individuals the best action, agree they reply that that prior agreement disagree. To our (correctly) anticipated is shown to be ill founded by the consensual aggregated probabilities, based on weighted averaging probabilities they reply that consensual 3/10 are not rationally compelling. Not ever? Suppose A has seen Si happen 6 times each following dx and d2 and s2 4 times each. B, on the other hand, has seen (in a disjoint sample) Si happen 4 times each following dx their probabilities and d2 and s2 6 times each. They aggregate using the on vector fact that based the consensual (|,^) they have weight in of the This results their each size observed samples. assigning equal outcomes Such consensual Si and s2 equal consensual probability. new basis for calculating strike me as an impeccable probabilities a on If them the utilities. changes prior agreement expected employing 104 best CARL course agreement of action, so be at any price. 3. WAGNER it.We ISAAC are after informed agreement, not LEVI Levi endorses, with some qualifications, cism of our method's failure to preserve consensus as the convex representing and Loewer's criti asserts that He independence. hull of a set of competing Laddaga vectors for example, avoids this problem. that Suppose, I assign the respective 2/9) and (4/9, 2/9,1/9, probabilities a to of exhaustive (1/9, 2/9, 4/9, 2/9) sequence pairwise contradictory, the propositions s1? s2, s3, s4. On each of our assignments propositions u = 5i v s2 and t= s2 v s3 turn out to be independent. If we come to on our t true that condition is and this evidence, agree respective will conditional be and (0,2/3,1/3,0) probability assignments and Levi would that both after before (0,1/3,2/3,0). emphasize to u by either of conditionalization the maximum probability assigned probability Levi and us is 2/3 and the minimum of is 1/3. This is not preservation probability sense in the demanded and Loewer, but, independence by Laddaga of interval estimates under conditionalization rather, preservation relative to an independent item of evidence. (i.e., irrelevant) What Laddaga and Loewer demand is that any consensual assign to ment of unconditional the propositions Su s2, s3y s4 probabilities u and t turn out to be independent. should be such that the propositions To the extent that Levi regards each weighted arithmetic mean of our a as of resolution the conflict" between "potential original assignments our original assignments, of in he is potentially violating preservation sense in of and the Loewer. the Indeed, dependence Laddaga only actual resolution of our conflict which preserves in this independence or for him to sense would be for me to adopt Levi's original distribution no actual To and mine Lehrer be sure, 1983). (see adopt Wagner, of independence of preservation will occur under Levi's if our disagreement is not resolved, scheme but if this is virtue, it is virtue by default. In any case, as I have argued in my reply to Laddaga and Loewer, as our model, of in is independence rarely epistemic significance where, are directed at the probabilities of a set of the initial acts of assessment violation exhaustive the neither Thus, propositions. contradictory, violation of of Levi's model, independence by preservation potential nor its actual violation is cause for concern. by our model pairwise WEIGHTED 105 AVERAGING of consensus is its very positive aspect of Levi's conception area to in of restrain the premature Particularly tendency compromise. to of utilities relative all group decisionmaking, expected calculating in the convex hull of a set of competing the probability distributions seems to me to be a salutary exercise. Of course, distributions these One two courses of action will rarely unanimously rival ways of evaluating establish the superiority of one action over another. beat another in this sense Indeed, one course of action will decisively just when the smallest expected utility attributed by anyone (using his to initial subjective the former the exceeds expec largest probabilities) ted utility attributed by anyone to the latter. In some cases requiring a to adopt one course of action over another to be supported decision in on this way may be excessively But issues like nuclear demanding. (1980) has persuasively safety, as Levi argued, an extreme modesty regarding our probability estimates 4. HANNU is entirely appropriate. NURMI at alleged of Nurmi's is directed of the deficiencies critique as a social choice mechanism. of weighted method He averaging observes first that our method does not always pick the Condorcet or exclude winner the Condorcet loser. Condorcet criteria may be reasonable restrictions on social welfare functions (methods of produc from a profile of individual orderings) since ing a consensual ordering the inputs contain no information about depth of preference. They are on social welfare not reasonable restrictions (methods of functional a from a profile of individual utilities). consensual ordering producing to restrict our advocacy We have been very careful of weighted cases to arithmetic where differences in utilities are inter averaging Most formal (Consensus, p. 118). In such cases, personally comparable results Theorem show that arithmetic (Consensus, 6.9) weighted is the precisely of aggregation. In method averaging appropriate are it is if that differences inter clear, however, formally already to take them into then it is not unreasonable personally comparable, account. The aforementioned theorem strengthens this to the assertion that they must be taken into account. Thus, A's much greater pref erence for x over y can justifiably defeat the more modest preferences of B and C for y over x, even when A, B, and C all receive equal weight. 106 CARL WAGNER next that our method observes violates the weak axiom of as well as path-independence. The examples used preference this involve of von Neumann-Morgenstern the aggregation indicates. But differences in such utilities utilities, as Nurmi forthrightly are not interpersonally and we specifically forbid the comparable, Nurmi revealed to show of such utilities by weighted arithmetic aggregation except averaging, case where one person in the dictatorial receives all the weight (Consensus, pp. 119-120). Nurmi observes further that, under our method, it is possible that x is for a given assignment ranked highest of utilities, but loses first place for another assignment of utilities which differs from the first only in that one individual assigns x an even higher utility than he did before. are used in the two This is only possible, of course, if different weights cases. I see nothing remarkable or disturbing about this. Weights make a difference. to do. That's what they're designed In addition, Nurmi remarks that, under our method, two groups may separately rank x highest, but jcmay lose first place when the groups are even though no one changes his The amalgamated, utility assignments. for this lies in a change in the weights. Again, weights make explanation a difference. In Nurmi's individual 1, who prefers jci, reduces example his own weight to individual 4, who prefers and assigns the difference individual 3, who prefers xu reduces his own weight x2. Similarly, and to individual 2, who prefers x2. Individuals 2 and assigns the difference 4 do not, however, is it surprising or "inconsistent," reciprocate. Why as Nurmi would have it, that x2 might now be ranked first? to the foregoing In addition as a social of our method criticisms choice mechanism, Nurmi identifies what he takes to be a defect in the of arriving at consensual weights general method by iterated weighted of weight matrices. His i.e., by repeated multiplication averaging, observation is that a modest shift in the weights assigned by a single can produce a substantial individual as shift in consensual weights, illustrated by the matrices " "0.50 0.05 0.05 0.00 0.05 0.00 0.45 0.05 0.00 0.00 0.00 0.85 0.50 0.05 0.05 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.50 0.05 0.10 0.00 0.40 0.80 0.40 0.00 0.10 0.10 0.50 0.45 107 AVERAGING WEIGHTED that its fifth row is is identical with Wx except and W2, which consensual 0.05. The corresponding 0.05 0.80 0.00 0.00 are vectors weight 0.10 d= [0.078 0.371 0.041 0.054 0.371 0.085] [0.108 0.514 0.057 0.032 0.257 0.032]. and C2 = an and Nurmi is right in demanding example interesting to is two-fold. the The First, proper way explanation explanation. terms in of than is rather differences ratios, (see compare weights of how apparently for examples substantial Consensus, pp. 131-132, no change in the consensual weight shifts in a weight matrix produce vector because ratios of off-diagonal elements are unchanged; also the to at of it is the p. 134). Second, top important keep in mind example of our elementary the applicability that Nurmi is assuming model, This is an at different levels are precluded. where different weight matrices Thus in the manner he is not simply shifting a first order weight matrix as well. (See also in this indicated above, but all higher order matrices connection my reply to Baird.) Combining the above remarks, I would as follows: describe Nurmi's "Individual doubles the #5 example to which he and halves the the which #1 he assigns weight weight an to in of of each infinite matrices. #6 assigns hierarchy weight Individual #1, on the other hand, assigns nearly half of his available at all to #5. Conversely, to #2 at all levels, and no weight weight to #5 at all individual #6 assigns nearly half of his available weight levels, and no weight at all to #2. The result is that individual #5, who a consensual now received equal to that of #2, weight previously receives only half of the consensual weight received by #2." This does at all levels, the not strike me as being counter-intuitive. If I double, no to someone I for which who has me, and halve respect give weight the weight which I give to someone with substantial respect for me, is it that I find my position undermined? any wonder 5. FREDERICK SCHMITT Schmitt objects to the fact that we do not allow an individual to receive on different different weights in probability competing propositions an assessment From the observation that individual may be problems. 108 CARL WAGNER or than about another, he less informed about one proposition are allowed. that "variant" should be But probabilities infers weights a to a set of mutually not assigned in exhaustive exclusive, propositions a one. acts. The is of isolated unified In to order sequence enterprise assess the probability that Native Dancer will win the race, we need to formal will be. A result know who his competitors (Consensus, this of intuition the possibility Theorem 6.4) supports by granting to different different assigned probabilities by propositions aggregating that this apparent flexibility is functions (Axiom IA) but then showing a set of because consensual (of probabilities largely precluded, must sum to one. Schmitt is exer exhaustive exclusive, propositions) more cised to the point of exclamation ("But the rejection of this assumption is of a piece with the rejection of invariance! Consensual probabilities not sum to one ...") by the fact that this basic property must obviously of probabilities is used in the proof. The reasons for his agitation escape me. NOTE 1 at the Center a participant for Advanced our work by providing I was at the Lehrer supported time on Freedom and Causality which Sciences, Study in the Behavioral me with a Fellowship 1978-79. during in the Institute directed by subsequently REFERENCES and Wagner, Dordrecht-Boston. Keith Lehrer, Reidel, Keith Lehrer, dependence Isaac: Levi, 1981, Rational Consensus in Science Carl: and Wagner, 1983, 'Probability Amalgamation to Laddaga', Issue: A Reply 55, 339-346. Synthese MIT Press, Cambridge, 1980, The Enterprise of Knowledge, Raiffa, Howard: Wagner, Notre Dame Carl: Department Carl: 1968, Decision 1984, 'Aggregating Journal of Formal of Mathematics University of Tennessee Knoxville, U.S.A. TN 37996-1300 Analysis, Addison-Wesley, Probabilities: Subjective Logic 25, 233-240. and Society, and the Mass. Mass. Reading, Some Limitative Theorems', In
© Copyright 2025 Paperzz