How Majority Rule Undermines Collective Truth-Finding Jan Lorenz www.janlo.de Department of Psychology and Methods, Focus Area Diversity Jacobs University Bremen Symposium “Exploring Ignorance” Graduate School of Decision Sciences Konstanz May 20, 2016 582287 research-article2015 RAP0010.1177/2053168015582287Research & PoliticsLorenz Research Article Majoritarian democracy undermines truth-finding in deliberative committees Research and Politics April-June 2015: 1–10 © The Author(s) 2015 DOI: 10.1177/2053168015582287 rap.sagepub.com Jan Lorenz1, Heiko Rauhut2 and Bernhard Kittel3 Abstract The aggregated judgments of many usually outperform individual estimates of vaguely known facts. Communication among individuals may, however, undermine this wisdom-of-crowd effect because it makes judgments mutually dependent. Deliberative democratic theory, on the other hand, suggests that communication promotes correct decisions. We investigate this puzzle about the positive and negative consequences of consensus formation on the wisdom of crowds using experimental methods. Subjects in small deliberative committees had to communicate and thereafter judge vaguely known facts. We varied the agreement rules in groups and compared the groups’ change of performance from initial to final estimates. Interestingly, groups’ performance worsened on average when they had to reach a majority decision. Groups came on average closer to the truth if they had to decide unanimously or if they did not have any restrictions to reach agreement. The low performance under majority rule is robust against different knowledge questions, group sizes and communication types. The majority rule may be worst because it makes people too focused to reach a majority so that valuable minority opinions are disregarded or not even voiced. This implies that majoritarian democracy may be less suitable for truth-finding than less or more restrictive quorum rules. Keywords collective judgment, experimental political science, voting, deliberation, wisdom of crowds Jan Lorenz, Heiko Rauhut, and Bernhard Kittel (2015b). Majoritarian Democracy Undermines Truth-Finding in Deliberative Committees. Research & Politics (2) (2), 1–10 Various aggregation methods Majoritarian of individual beliefs are Jan Lorenz, Heiko Rauhut, and Bernhard Kittel (2015a). Experimental Data: Democracy undermines Introduction used in such committees. For example, the preparation of truth-finding in deliberative committees. UNF:5:LLNmZiLTHF9hnvl7dHVG+A== Dataverse Network Understanding the interplay between collective decisions the IPCC reports has recently been describedHarvard as a “bazaar” collective intelligence is as important for small expert of studies where actors “bargain” about the expected rise of [Distributor] V3 [Version]and committees as it is for large democratic societies. The design 2 / 28 the sea level (Bojanowski, 2011), and pooling of expert of discussion rules and quorums critically shape the way assessments has been used for truth-finding (Bamber and groups find agreement and converge towards correct deci- Aspinall, 2013). Such often chaotic political deliberation sions. Advisory boards and expert committees are increas- processes needs to be better informed by investigations ingly part of political decision making. Such committees about the interplay of collective decisions and collective make decisions on issues ranging from expected economic growth and tax estimates to estimates on the sea level in 2050 and 2100 (task of the “Intergovernmental Panel on 1Jacobs University, 2024/2-1 (Jan Lorenz) Focus Area Diversity and Bremen International Climate Change” (IPCC)). By definition ofProject their task,LO comSchool for Collective Social Sciences (BIGSSS), Bremen, Germany Opinion Dynamics and Decisions: mittee members share the same preference of correctly 2Universität Zürich, Soziologisches Institut, Zürich, Switzerland Procedures, Behavior and Systems Dynamics assessing the state of the world. Disagreement is limited to 3Universität Wien, Department of Economic Sociology, Wien, Austria different beliefs about this state. Because of the presupposi- http://janlo.de Wisdom-of-Crowd Field Experiment1 Question (N=1,226 responses) How many lots will be sold at the end of the festival? 1 3 / 28 Bremen, Viertel-Fest 2008, Tombola “Haste mal ’nen Euro” http://janlo.de/wp/2010/06/22/die-weisheit-der-bremer http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics The Wisdom of the Crowd Situation A question about an unknown fact of metric nature and estimates from many individuals. Wisdom-of-Crowdab Effect The aggregate of estimates is close to the true value although most are far away from. Conclusion If you know and believe in the WoC-Effect: Ask many independently and take the median! (... and the law of large numbers ensures collective intelligence.) a F. Galton (1907b). Vox populi. Nature 75 (1949), 450–451. J. Surowiecki (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies, and nations. Doubleday Books. b 4 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Aggregation by the median Galton: The median is “democratic”. “One Vote, one Value!”2 The mean is not good, because “it gives voting power to the cranks proportional to their crankiness”. Mean: 2 5 / 28 Median: F. Galton (1907a). One vote, one value. Nature 75 (1948), 414. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Aggregation by the median Galton: The median is “democratic”. “One Vote, one Value!”2 The mean is not good, because “it gives voting power to the cranks proportional to their crankiness”. Mean: 2 5 / 28 Median: F. Galton (1907a). One vote, one value. Nature 75 (1948), 414. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics The spatial model of democratic decision Assumptions I Possible policies are on a 1d scale I Voters have single-peaked preferences I Always 2 proposals compete for a majority of votes I New proposals can be set up and voted against the status quo Single-peaked utility functions Duncan Black (1948). On the Rationale of Group Decision-making. Journal of Political Economy 56 (1), 23 Median Voter Theorem I 6 / 28 The median voter is unbeatable in pairwise polls. http://janlo.de Distribution of voters ideal points Anthony Downs (1957). An Economic Theory of Democracy. Harper & Row Publishers, New York Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Theoretical Conclusion Premise 1: The median of many judgments is closest to correct Premise 2: Majoritarian Democracy elicits the median as outcome. ⇒ Majoritarian Democracy elicits the wisdom of the crowd! 7 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Theoretical Conclusion Premise 1: The median of many judgments is closest to correct Premise 2: Majoritarian Democracy elicits the median as outcome. ⇒ Majoritarian Democracy elicits the wisdom of the crowd! Practical Question: Is it a good mechanism for expert committees? Experimental evidence today: Not in deliberative groups. 7 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Application: Expert committees I Expert committees shall I I I Example: Intergovernmental Panel on Climate Change I I I 3 4 8 / 28 find the best possible predictions deliver a common opinion Shall predict sea level 2050 and 2100. Described as “bazaar” where future sea level is “negotiated”.3 Discusses its internal report and decision structures.4 Klimarat feilscht um Daten zum Meeresspiegel-Anstieg, Axel Bojanowski, Spiegel Online, 14.07.2011 Experten drängen auf Umbau des Weltklimarats, Axel Bojanowski, Spiegel Online, 30.08.2010 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Practical questions on expert committees If you want to establish an expert committee 9 / 28 I Should the experts discuss which each other? Experimental results: No clear answer, it can reduce but also increase bias I Should you force the committee to decide on one judgment? Experimental results: Not necessarily, but if you do, better not majority rule! http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Wisdom-of-Crowd under Communication and Collective Decision Laboratory Experiment Group task in experimental lab: I Factual Questions. Example: What percentage of the World’s population lives in Africa? I Incentivize correct answers Deviation Payment ≤ 1.5% e4 ≤ 3% e2 ≤ 6% e1 ≤ 12% e 0.5 larger e0 I 10 / 28 −0.12 −0.06 0 Strong incentive to come closer to the truth! http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics 0.06 0.12 Wisdom-of-Crowd under Communication and Collective Decision Laboratory Experiment Group task in experimental lab: I Factual Questions. Example: What percentage of the World’s population lives in Africa? I I Incentivize correct answers Deviation Payment ≤ 1.5% e4 ≤ 3% e2 ≤ 6% e1 ≤ 12% e 0.5 larger e0 I 10 / 28 15 % −0.12 −0.06 0 Strong incentive to come closer to the truth! http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics 0.06 0.12 Laboratory Experiment I Invited for 6 sessions with 24 participants each I In each session: 9 rounds with 9 questions I In each round: two groups of 9, two groups of 3 members I In each group: Communication interface a. Communication by numbers b. Chat communication I After communication: Decision rules 1. Individual rule: payoff based on individual decision 2. Majority rule: payoff for all based on majority decision (one with 21) (from a pool of 19 questions) No payoff for groups where a majority is not reached. 3. Unanimity rule: payoff for all based on majority decision No payoff for groups where consensus is not reached. 11 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Chat interface all-to-all, free text 12 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Number interface (“My current estimate”) 13 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Example: Majority rule Turnout EU parliament DK 1984 (32.6%) 75 b 50 c a private 0 14 / 28 http://janlo.de 0 e0 d decision 25 Possible motivation a state private opinion b try to pull group up c decide for blue’s opinion d agree for majority’s sake e4 Estimate (%) 100 1 2 3 deliberation time (min) Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Example: Individual rule Mass of aluminium in earth crust (8.1%) 75 50 25 e 0.5 Estimate (%) 100 e4 15 / 28 http://janlo.de 0 1 2 deliberation time (min) Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics decision private 0 Example: Unanimity rule Turnout Bundestag DE 2009 (70.9%) 75 50 e4 e0 Estimate (%) 100 25 16 / 28 http://janlo.de 0 1 deliberation time (min) decision private 0 Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Example: Majority rule Water on earth surface (70.8%) 75 e 0.5 e4 Estimate (%) 100 50 25 17 / 28 http://janlo.de 0 1 2 deliberation time (min) Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics decision private 0 Data on four questions Q6: Initial Estimate Votes USA at World Bank 60 80 100 0 20 40 60 80 100 20 40 20 40 60 80 100 100 12 0 20 40 60 80 100 40 60 80 100 0 20 40 60 80 100 0 60 80 100 20 40 60 80 100 20 40 60 80 100 40 60 80 100 40 60 80 100 0 20 40 60 80 100 10 20 0 10 20 20 Q8: Decision Unanimity 0 0 0 Q6: Decision Unanimity 0 4 8 15 0 0 100 Q8: Decision Majority 30 Q4: Decision Unanimity 80 0 40 0 0 20 Q2: Decision Unanimity 20 Q6: Decision Majority 10 20 30 15 0 0 0 Q4: Decision Majority 60 25 80 40 0 10 60 10 20 40 0 0 5 20 20 Q8: Decision Individual 6 15 0 0 Q2: Decision Majority 0 Q6: Decision Individual 15 30 0 0 Q4: Decision Individual 20 40 10 20 0 10 20 0 0 20 0 Q2: Decision Individual Q8: Initial Estimate Use of own oil Saudi Arabia 10 20 Q4: Initial Estimate Workers in agriculture 50 Q2: Initial Estimate Water on earth surface 0 20 40 60 80 100 0 20 40 60 80 100 Legend: Black vertical lines = Truth Shades of Gray = Payment Zones Shades of Colors: Half Tone = Number Communication, Full Tone = Chat 18 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics 0.6 ● ● 0.3 0.0 ● −0.3 Individual Majority Unanimity change in group certainty (1−6) change in group performance (EUR) Change in Performance and Confidence 0.6 0.5 ● 0.4 0.3 ● 0.2 ● 0.1 0.0 Individual Majority Error bars are adjusted 95% confidence intervals. 19 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Unanimity 0.8 ● 0.4 ● ●● ● ● 0.0 ● −0.4 ●● −0.8 all groups small groups (3 individuals) large groups (9 individuals) Individual Majority Unanimity change in group performance (EUR) change in group performance (EUR) Impact of Group Size and Mode of Communication 0.8 0.4 ● ●● ● ● ● 0.0 ● ● ● −0.4 −0.8 all groups chat communication communication by numbers Individual Majority Unanimity Error bars are adjusted 95% confidence intervals. I I 20 / 28 Larger groups make effect slightly more drastic, esp. individual-rule groups perform better. Unanimity-rule (and majority-rule) groups seem to benefit from chat communication instead of numbers only. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics change in group performance (EUR) Impact of the Question 2 ● 1 ● ● ● ● ● 0 ● ● ● ● I 21 / 28 ● ● ● ● ● ● ● ● ● ● ● ● ● −1 ● ● ● ● ● ● ● ● ● ● Q1 Share of people in Asia/Americas/Africa Q2 Water on earth surface Q3 Mass aluminium/silicium/iron in earth crust Q4 Workers in agriculture Q5 Turnout EU parliament DK/UK/FR 1984 Q6 Votes USA at World Bank Q7 Water content tomato/onion/cucumber Q8 Use of own oil Saudi Arabia Q9 Turnout Bundestag DE 1998/2005/2009 ● ● Individual I ● ● Majority Unanimity Worse performance under majority rule is a robust pattern. The question seems to determine to a large extend if typically groups worsen or improve. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Robustness check: Final performance 4 1.5 ● ● 1.0 ● 0.5 3 ● ● 2 1 Individual I Majority Unanimity ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 0 Individual ● ● ● ● ● ● ● 0.0 22 / 28 group performance (EUR) group performance (EUR) 2.0 Majority ● ● ● ● ● ● ● ● ● ● Q1 Share of people in Asia/Americas/Africa Q2 Water on earth surface Q3 Mass aluminium/silicium/iron in earth crust Q4 Workers in agriculture Q5 Turnout EU parliament DK/UK/FR 1984 Q6 Votes USA at World Bank Q7 Water content tomato/onion/cucumber Q8 Use of own oil Saudi Arabia Q9 Turnout Bundestag DE 1998/2005/2009 ● ● ● Unanimity Worse performance under majority rule is a robust pattern. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Robustness check: Final distance to truth 18 30 16 25 ● ● distance to truth distance to truth ● 14 12 10 20 15 10 ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● 5 ● ● ● ● 8 23 / 28 ● ● ● ● ● ● ● ● ● ● Q1 Share of people in Asia/Americas/Africa Q2 Water on earth surface Q3 Mass aluminium/silicium/iron in earth crust Q4 Workers in agriculture Q5 Turnout EU parliament DK/UK/FR 1984 Q6 Votes USA at World Bank Q7 Water content tomato/onion/cucumber Q8 Use of own oil Saudi Arabia Q9 Turnout Bundestag DE 1998/2005/2009 ● ● ● 0 Individual I ● ● Majority Unanimity Individual Majority Unanimity Worse performance under majority rule is a robust pattern. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Possible explanation In our experiment: 24 / 28 I Epistemic motivation under majority rule may be lower, because minority opinions can be overruled and thus ignored. I At the same time strong prosocial motivation to agree “for the sake of majority”. I Fact: 3/4 of all majority groups ended up in full agreement. http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Conclusion Majority rule seems to be a bad mechanism for groups with epistemic tasks to elicit collective intelligence because • Majority decisions were often worse than the median of initial estimates. Communication made it worse! • Unanimity rule and no quorum rule often enabled groups to find decisions better than the median of initial estimates. Communication made it better! • Moreover, majority rule triggers the most increase of confidence in the quality of the decision. 25 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics 582287 research-article2015 RAP0010.1177/2053168015582287Research & PoliticsLorenz Research Article Majoritarian democracy undermines truth-finding in deliberative committees Research and Politics April-June 2015: 1–10 © The Author(s) 2015 DOI: 10.1177/2053168015582287 rap.sagepub.com Jan Lorenz1, Heiko Rauhut2 and Bernhard Kittel3 Abstract The aggregated judgments of many usually outperform individual estimates of vaguely known facts. Communication among individuals may, however, undermine this wisdom-of-crowd effect because it makes judgments mutually dependent. Deliberative democratic theory, on the other hand, suggests that communication promotes correct decisions. We investigate this puzzle about the positive and negative consequences of consensus formation on the wisdom of crowds using experimental methods. Subjects in small deliberative committees had to communicate and thereafter judge vaguely known facts. We varied the agreement rules in groups and compared the groups’ change of performance from initial to final estimates. Interestingly, groups’ performance worsened on average when they had to reach a majority decision. Groups came on average closer to the truth if they had to decide unanimously or if they did not have any restrictions to reach agreement. The low performance under majority rule is robust against different knowledge questions, group sizes and communication types. The majority rule may be worst because it makes people too focused to reach a majority so that valuable minority opinions are disregarded or not even voiced. This implies that majoritarian democracy may be less suitable for truth-finding than less or more restrictive quorum rules. Keywords judgment, experimental science, voting, deliberation, wisdom (2015b). of crowds Jan Lorenz, Heikocollective Rauhut, andpolitical Bernhard Kittel Majoritarian Democracy Undermines Truth-Finding in Deliberative Committees. Research Various aggregation methods of individual beliefs are Introduction & Politics (2) (2), 1–10 used in such committees. For example, the preparation of Understanding the interplay between collective decisions the IPCC reports has recently been described as a “bazaar” collective intelligence is as important for small expert of studies where actors “bargain” about the expected rise of Jan Lorenz, Heikoand Rauhut, and Bernhard Kittel (2015a). Experimental Data: committees as it is for large democratic societies. The design the sea level (Bojanowski, 2011), and pooling of expert of discussion rules and quorums critically shape the way assessments has been used for truth-finding (Bamber and groups find agreement and converge towards correct deci- Aspinall, 2013). Such often Majoritarian Democracy undermines truth-finding in deliberative chaotic political deliberation committees. sions. Advisory boards and expert committees are increas- processes needs to be better informed by investigations ingly part of political decision making. Such committees about the interplay of collective decisions and collective UNF:5:LLNmZiLTHF9hnvl7dHVG+A== Harvard Dataverse Network make decisions on issues ranging from expected economic growth and tax estimates to estimates on the sea level in 2050 and 2100 (task of the “Intergovernmental Panel on [Distributor] V3 [Version] Climate Change” (IPCC)). By definition of their task, com1Jacobs University, Focus Area Diversity and Bremen International School for Social Sciences (BIGSSS), Bremen, Germany 26 / 28 mittee members share the same preference of correctly 2Universität Zürich, Soziologisches Institut, Zürich, Switzerland assessing the state of the world. Disagreement is limited to 3Universität Wien, Department of Economic Sociology, Wien, Austria different beliefs about this state. Because of the presupposiProject LO 2024/2-1 (Jan Lorenz) Corresponding author: tion that a correct answer exists, the government and the Jacobs University Bremen, Bremen International Graduate School of Opinion Dynamics and Collective Decisions: public often expect experts to deliver consensual answers on Social Sciences (BIGSSS), Campus Ring 1, 28759 Bremen, Germany. Procedures, Behavior and Systems Dynamics which policy decisions and investments can be based. Email: [email protected] http://janlo.de Backup: Average Performance in numbers over time e 0.66 1 e 1.03 1 2 3 4 5 6 7 8 9 10 deliberation period Measure of group performance: Payment for the median initial/intermediate estimate and the payment for the median final decision. 27 / 28 http://janlo.de Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics decision private 0.5 1 2 3 4 5 6 7 8 9 10 deliberation period decision decision 1 2 3 4 5 6 7 8 9 10 deliberation period 1.3 e 0.85 Average group performance (e ) e 1.04 1 0.5 private 0.5 Unanimity 1.3 private 1 Average group performance (e ) Majority e 1.28 e 0.84 Average group performance (e ) Individual 1.3 Preliminary: Content analysis of chat protocol statements # Statement Informational Procedure Resignation Relative Uncertainty Total Uncertainty Relative Certainty Total Certainty Ask for Argument Proposal Rejection Do What Majority Does 28 / 28 http://janlo.de Individual 1287 12.2% 4.3% 0.2% 8.7% 4.1% 0.5% 0.1% 3.8% 1.2% 0.1% Majority 1631 8.6% 5.3% 1.0% 4.0% 2.3% 0.6% 0.7% 3.0% 2.1% 0.4% Project LO 2024/2-1 (Jan Lorenz) Opinion Dynamics and Collective Decisions: Procedures, Behavior and Systems Dynamics Unanimity 1685 5.6% 4.5% 0.4% 4.0% 2.4% 0.7% 0.2% 1.4% 1.2% 0.6%
© Copyright 2026 Paperzz