House of Commons
Scottish Affairs Committee
The Referendum on
Separation for
Scotland:
Do you agree this is a
biased question?
Eighth Report of Session 2010–12
Report, together with formal minutes
Ordered by the House of Commons
to be printed 26 April 2012
HC 1942
Published on 8 May 2012
by authority of the House of Commons
London: The Stationery Office Limited
£0.00
The Scottish Affairs Committee
The Scottish Affairs Committee is appointed by the House of Commons to
examine the expenditure, administration, and policy of the Scotland Office
(including (i) relations with the Scottish Parliament and (ii) administration and
expenditure of the offices of the Advocate General for Scotland (but excluding
individual cases and advice given within government by the Advocate General)).
Current membership
Mr Ian Davidson MP (Labour/Co-op, Glasgow South West) (Chair)
Fiona Bruce MP (Conservative, Congleton)
Mike Freer MP (Conservative, Finchley and Golders Green)
Jim McGovern MP (Labour, Dundee West)
Iain McKenzie MP (Labour, Inverclyde)
David Mowat MP (Conservative, Warrington South)
Pamela Nash MP (Labour, Airdrie and Shotts)
Simon Reevell MP (Conservative, Dewsbury)
Mr Alan Reid MP (Liberal Democrat, Argyll and Bute)
Lindsay Roy MP (Labour, Glenrothes)
Dr Eilidh Whiteford MP (Scottish National Party, Banff and Buchan)
The following members were also members of the committee during the
parliament:
Cathy Jamieson MP (Labour/Co-op, Kilmarnock and Loudoun)
Mark Menzies MP (Conservative, Fylde)
Graeme Morrice MP (Labour, Livingston)
Fiona O’Donnell MP (Labour, East Lothian)
Julian Smith MP (Conservative, Skipton and Ripon)
Powers
The committee is one of the departmental select committees, the powers of
which are set out in House of Commons Standing Orders, principally in SO No
152. These are available on the internet via www.parliament.uk.
Publication
The Reports and evidence of the Committee are published by The Stationery
Office by Order of the House. All publications of the Committee (including press
notices) are on the internet at www.parliament.uk/scotaffcom. A list of Reports
of the Committee in the present Parliament is at the back of this volume.
The Reports of the Committee, the formal minutes relating to that report, oral
evidence taken and some or all written evidence are available in a printed
volume.
Additional written evidence may be published on the internet only.
Committee staff
The current staff of the Committee are Mr Eliot Wilson (Clerk), Duma Langton
(Inquiry Manager), James Bowman (Senior Committee Assistant), Gabrielle Hill
(Committee Assistant) and Ravi Abhayaratne (Committee Support Assistant).
Contacts
All correspondence should be addressed to the Clerk of the Scottish Affairs
Committee, House of Commons, 7 Millbank, London SW1P 3JA. The telephone
number for general enquiries is 020 7219 6123; the Committee’s email address is
[email protected]
The Referendum on separation for Scotland: Do you agree this is a biased question? 1
Contents
Report
1 Summary
3 Introduction
5 5 5 The Inquiry
Background
2 Page
The wording of a referendum question
The principles behind a referendum question
First draft question proposed by the Scottish Government
Second draft question proposed by the Scottish Government
The consequences of bias
How can a fair question be drafted?
Testing proposed questions
8 8 8 9 10 12 14 Conclusions and recommendations
18 Annex 1
19 Annex 2
20 Formal Minutes
22 Formal Minutes
22 List of Reports from the Committee during the current Parliament
23 The Referendum on separation for Scotland: Do you agree this is a biased question? 3
Summary
In this Report, we consider whether the wording of the referendum question proposed by
the Scottish Government is biased, and what should be done to ensure that a fair and clear
question is put to the Scottish electorate.
Based on the evidence, we have no choice but to conclude that the question:
Do you agree that Scotland should be an independent country?
is biased. Experts told us that this was a leading question, biased towards a ‘yes’ answer.
This is important because wording can affect the result of a referendum. This referendum
must be, and be seen to be, fair and acceptable to those on both sides of the argument. It is
essential that the result commands wide acceptance.
It is now widely agreed that the Electoral Commission is the appropriate body to regulate a
referendum and ensure that the wording of the question is clear and fair. This means an
exhaustive process of testing of proposed words and how the options are put to the voters.
This will take some time, and therefore it must be started soon.
It need not wait until a referendum Bill is drafted.
Debate needs to move on to the policies and issues of separation.
Accordingly, we recommend that all parties approach the Electoral Commission now, so
that proposed question wordings can be tested.
The best course is for the parties to agree jointly what question wordings should be put to
the Commission.
If a consensus cannot be reached, those parties who can agree should approach the
Commission together.
Our objective is that all can accept the advice of the Commission on a clear and neutral
question, which can be incorporated into the legislative process from the very beginning.
The Referendum on separation for Scotland: Do you agree this is a biased question?
1
5
Introduction
The Inquiry
1. On 12 October 2011, we launched two separate inquiries on the proposed Referendum
on Separation for Scotland. One focused on the many unanswered questions which
would need to be addressed if the voters of Scotland were to make an informed choice
when they were invited to vote in a referendum. As a result, on 15 February 2012, we
published a Report entitled The Referendum on Separation for Scotland: Unanswered
Questions, which aimed to outline the major policy areas which would need to be debated
in advance of a referendum on separation.
2. The second inquiry was concerned with the process and mechanics by which such a
referendum would be organised and conducted. It is an ongoing inquiry, but this Report
is the first to arise from it, and it focuses on one narrow but very important aspect of the
referendum: the wording of the question. We are issuing the Report now because of the
remarkable unanimity of the evidence we received on this point, and because we
recommend that early progress should now be made on it.
3. Although we are issuing a Report now on the wording of the question in a referendum,
we continue to inquire into the process and mechanics of such a referendum. We have
taken a good deal of oral and written evidence, and will continue to do so. There are a
number of complex issues to be resolved, and we will return to them in later Reports.
4. We are grateful to all of those who have given evidence, both written and oral, to our
inquiry. We also pay tribute to the excellent work of our specialist adviser, Professor Jim
Gallagher.
Background
5. The Scottish National Party’s manifesto for the 2007 Scottish Parliament elections,
Manifesto 2007, promised the “publication of a White Paper, encompassing a Bill,
detailing the concept of Scottish independence in the modern world as part of
preparations for offering Scots the opportunity to decide on independence in a
6
The Referendum on separation for Scotland: Do you agree this is a biased question?
referendum”. In February 2010, the Scottish Government published a draft Bill to achieve
separation, entitled Scotland’s Future: Draft Referendum (Scotland) Bill Consultation
Paper. This document contained three proposals: the first would devolve all matters to
the Scottish Parliament except “defence and foreign affairs; financial regulation,
monetary policy and the currency”; the second would implement the recommendations
of the Calman Commission, to vary income tax and introduce a limited power to borrow
money; while the third proposal was for full separation from the UK. The consultation
contained a proposed question which was based on extending the Scottish Parliament’s
powers to allow it to legislate for separation. The referendum would be overseen by a
commission set up specifically for the purpose.
6. In its manifesto for the 2011 Scottish Parliament elections, Re-elect, the SNP pledged to
bring forward a Referendum Bill during the lifetime of the Parliament. In January 2012,
the Scottish Government issued a consultation document, Your Scotland, Your
Referendum.1 It set out “the Scottish Government’s proposals for the question to be asked
and the rules governing the campaign and the vote”, and included a draft ballot paper,
with a new question: “Do you agree that Scotland should be an independent country?”, a
substantially different question from that proposed in the draft Bill of 2010. The deadline
for the consultation is 11 May 2012.
7. In February 2012, the UK Government published its own consultation paper entitled
Scotland’s constitutional future: A consultation on facilitating a legal, fair and decisive
referendum on whether Scotland should leave the United Kingdom.2 This was intended to:
set out options by which the UK and Scottish Governments can remove the legal
barriers to holding a referendum while ensuring that its question, conduct and
outcome are fair to those on each side of the debate and to all Scottish voters.
The consultation paper contained the UK Government’s views on the potential process of a
referendum on separation, especially its view that the Scottish Parliament did not have the
1
The Scottish Government, Your Scotland, Your Referendum, January 2012
2
Scotland Office, Scotland’s constitutional future: A consultation on facilitating a legal, fair and decisive referendum
on whether Scotland should leave the United Kingdom, Cm 8203 January 2012
The Referendum on separation for Scotland: Do you agree this is a biased question?
7
power to legislate for such a referendum, and that, if it did, the matter would inevitably end
up in the courts where the legislation was likely to be struck down.
8. The consultation set out nine specific questions, one of which was on what question (or
questions) a referendum on separation should ask. The deadline for responses to the
consultation was 9 March 2012. A summary of the responses was published in April 2012.3
3
Scotland Office, Scotland’s Constitutional future, Responses to the consultation, Cm 8326, April 2012
8
The Referendum on separation for Scotland: Do you agree this is a biased question?
2 The wording of a referendum question
The principles behind a referendum question
9. If there is to be a referendum on separation, the wording of the question is of
fundamental importance. In the view of the Electoral Commission the ideal was:
A question that was easy to understand, to the point and unambiguous. The question
[should avoid] leading the voter in one direction or another.4
Ipsos MORI in its written evidence said:
Our view is that the referendum question must satisfy two criteria, it must be clear so
that voters know exactly what they are being asked and it must be fair so that the
results of the referendum are undisputed.5
It added:
We know from years of experience, backed up by academic research, that the
wording of an opinion poll question can affect the answers we get.6
10. Ruth Stevenson set out for the committee the professional standards of question design
outlined in good practice guidance provided by the Market Research Society. She proposed
that any question should be “short, straightforward and clear [...] balanced so as not to lead
voters towards a particular response [... and] presented in a manner that does not
introduce bias”.7 These principles of clarity and balance are obviously correct and must
apply to any question put to the Scottish people in a referendum on separation.
First draft question proposed by the Scottish Government
11. The Scottish Ministers have twice proposed a specific question about separation to put
to the voters. In 2010, the Scottish Government proposed a question on which it asserted it
was legally possible for the Scottish Parliament to legislate. It explained:
Scottish Parliament Legislation must conform to the provisions of the Scotland Act
1998. The Scotland Act has in-built flexibility so that the Scottish Parliament's
powers can be extended over time. The Scottish Parliament has a role in such
4
Q 717 (The Referendum on Separation for Scotland, oral and written evidence, HC 1608)
5
Ev 216
6
ibid.
7
Ev 217
The Referendum on separation for Scotland: Do you agree this is a biased question?
9
processes […] It is therefore legitimate for a referendum held under an Act of the
Scottish Parliament to ask the people questions related to an extension of its powers
insofar as this is within the framework of the Scotland Act.8
It therefore proposed the following question:
The Scottish Government proposes that, in addition to the extension of the powers
and responsibilities of the Scottish Parliament set out in Proposal 1, the Parliament’s
powers should also be extended to enable independence to be achieved.
YES I AGREE
NO I DO NOT AGREE9
12. This was, the Scottish Government acknowledged, drafted with the aim of satisfying
the legislative constraints under which it operated. (The evidence we have heard strongly
suggests that even this contrived formulation does not succeed in being legally competent,
but the Committee will deal with that more fully in a subsequent Report.) So it is not
surprising that it does not score highly on tests of intelligibility: it is hardly simple, to the
point, or unambiguous. It is not even absolutely clear that, if it gained majority support, the
consequence would be separation: “independence” would merely be “enabled”, the exact
meaning of which is unclear. Therefore this convoluted question wholly fails to meet the
standards which should be expected of a question which determines a nation’s future. It is
neither simple nor clear.
Second draft question proposed by the Scottish Government
13. The Scottish Ministers have now proposed a different question. In the consultation
paper which they produced in response to the UK government’s offer to remove the
problem of lack of legislative competence, the Scottish Government put forward a new
formulation:
Do you agree that Scotland should be an independent country?
This question is undoubtedly shorter and simpler than their first attempt. But in evidence
to our inquiry it was uniformly characterised as unfair. Our witnesses agreed that they saw
this as a leading question. The eminent pollster Peter Kellner told us that “As a pollster, I
8
Scotland’s Future: Draft Referendum (Scotland) Bill Consultation Paper, p. 16
9
ibid., p. 20
10
The Referendum on separation for Scotland: Do you agree this is a biased question?
try to avoid attitudinal questions to which the answers are yes or no [...] I would say as a
pollster, “Do you agree or disagree?” You offer both options.”10
14. Matthew Elliott, of the NOtoAV campaign, put it simply. “If you asked a pollster to ask
that sort of question, they would say you were trying to rig the poll. It is a leading
question.”11 Other professionals agreed. Martin Boon, of ICM Research, remarked that
“The question presented beginning “Do you agree” fundamentally cannot be balanced if it
excludes the words “or disagree”. That is my primary starting point and fundamental
objection to it.”12 Andrew Hawkins, the Chairman of ComRes, said that “In the way it is
construed, I do agree it is a biased [question] because it does not present any of the
alternatives.”13
15. This is an elementary point in question design. In Annex 1, we give an example of a
recent GCSE Statistics question in 16-year-olds were asked why the question “Do you agree
that canteen food is value for money?” was biased.
16. Based on the evidence we have received, we have no choice but to conclude that the
question currently proposed by the Scottish Government is biased in that it tends to
lead the respondent towards the answer “yes”.
The consequences of bias
17. Martin Boon of ICM Research told us that the wording of the question could have a
significant effect on the outcome:
You may have seen some public opinion research conducted and commissioned by
Lord Ashcroft, which asked three questions. The first one [...] is, “Do you agree that
Scotland should become an independent country?”, which was yes 41 and no 59.
Then he added the word “disagree” to the statement, and it moved from 41/59 to
39/61. He then presented entirely that kind of option, which was, “Should Scotland
become an independent country or should it remain part of the United Kingdom?,
10
Q 258
11
Q 469
12
Q 816
13
Q 891
The Referendum on separation for Scotland: Do you agree this is a biased question?
11
and the numbers moved to 33/67 [...] these words will move the Scottish public one
way or the other.14
The research to which Mr Boon referred surveyed 3,090 Scottish adults. The full results are
reproduced in Annex 2. If the wording of the question can change opinion from 41/59 to
33/67, it is clear that the wording is of vital importance.
18. A number of our witnesses pointed out that a referendum was not the same as an
opinion poll. Mr Kellner, for example, said that a biased question might not matter as
much at the end of a referendum campaign when the voters had heard all the arguments.15
Other witnesses made the same point: question bias mattered a great deal in surveys as
respondents answered a lot of questions quickly, but they would have more time to reflect
and consider their view in a referendum.
19. This, however, misses the central point. A referendum which will decide the future of
the country must be, and be seen to be, scrupulously fair. There must be no scope
afterwards for either side to claim that the process was flawed. This was accepted by our
witnesses. For example, Ipsos MORI said “It must be seen to be fair so that the losing side
in the ballot does not try to discredit or challenge the result on the basis of the question
wording.”16
20. There must no bias whatsoever. Therefore, the process for determining the question
should be designed to ensure that is so. Unfortunately, the proposals by the Scottish
Ministers are inconsistent with that aim. They are biased towards separation. It is also clear
that, as well as setting the question, the Scottish Government wants to set the timing and
rules of the referendum. Its consultation paper says:
It is for the Scottish Government to propose to the Scottish Parliament the timing
and terms of the referendum and the rules under which it is to be conducted. The
Scottish Parliament should decide these matters.17
14
Q 829
15
Q 258
16
Ev 217
17
Your Scotland, your Referendum, p. 5
12
The Referendum on separation for Scotland: Do you agree this is a biased question?
21. This runs wholly contrary to the evidence given to us by the distinguished
constitutionalist, Professor Vernon Bogdanor, who noted that “The players should not also
be the referee”.18 Other witnesses agreed. Starting the process by proposing first a contrived
question, and then secondly an obviously leading one certainly does not suggest that an
SNP administration can be trusted to play the role of referee in this process.
How can a fair question be drafted?
22. The Electoral Commission has a statutory responsibility for regulating referendums in
the UK, and has, since its creation, overseen three referendums: on the establishment of a
Regional Assembly in the North East of England, on increasing the powers of the Welsh
assembly, and on changing the voting system used for electing the House of Commons. In
the UK Government’s consultation paper, it is proposed that the Electoral Commission
should also oversee this referendum, and it seems that the Scottish Government now seems
to be willing to amend its initial plans to set up a special, one-off, Commission appointed
by it and reporting to the Scottish Parliament alone to regulate the separation referendum.
We welcome this movement because otherwise it would have been seen as a participant
wishing to set the rules of the game as well as being a player and would clearly have been
wholly unacceptable.
23. Our witnesses thought that the Electoral Commission was the appropriate body to
regulate the wording of a referendum question. Martin Boon of ICM Research told us that
“We have the Electoral Commission in this country and it is an independent Executive
body”19, while Nigel Smith concurred that the Commission should “comment on” any
proposed question.20 Mark Diffley of Ipsos MORI believed that the Electoral Commission
“will have a key role”.21 The Commission itself told us that “clarity to us means that it is an
independent source of advice. We would like to be that source.”22
18
Q 353
19
Q 835
20
Q 836
21
Q 898
22
Q 737
The Referendum on separation for Scotland: Do you agree this is a biased question?
13
24. There was consensus among our witnesses that the advice of the Electoral Commission
should be accepted. As Willie Sullivan of the Electoral Reform Society said:
If the Electoral Commission went back to the Scottish Parliament with the question
that it had recommended and tested and the Scottish Parliament tried to change it,
organisations like ours and others—hopefully, everybody—would make quite a fuss
about it and say they had no confidence in it.23
Professor Stephen Tierney, of the University of Edinburgh, added that “the Electoral
Commission has a lawful role in this regard and I would imagine a question that they
thought unintelligible would clearly be open to judicial review”.24 This position was shared
by other witnesses.
25. None of our witnesses thought that the SNP’s plan was acceptable and all agreed
that the Electoral Commission, which is experienced and neutral, was the appropriate
body to ensure that this referendum is as fair as possible. The Electoral Commission
itself is prepared to do so. We recommend that the Electoral Commission should have
that responsibility.
26. This task will include dealing with the wording of the question: the Commission
presently has a statutory duty of testing proposed questions for intelligibility. This will
extend to the words used and the structure of the question. John McCormick, of the
Commission, explained:
What we mean by clarity and intelligibility. It is that the voters get the result that they
want, so therefore they understand the question and its implications […] that is part
of the question testing process […] we are looking for clarity about outcomes and
clarity about the question, and an unbiased question.25
27. All our witnesses agreed that thorough testing of the question was of great importance.
Ipsos MORI advised that “Our view is that there also needs to be a significant testing of
different question options before a final decision can be made”.26 They added that such
testing could be done in a number of ways. With proper polling, they would “have a better
23
Q 465
24
Q 234
25
Q 718
26
Ev 216
14
The Referendum on separation for Scotland: Do you agree this is a biased question?
idea of whether a question is fair once different options have been tested rigorously and
analysed against differently-worded questions, with robust sample sizes”.27
28. In addition, it was felt that “cognitive testing” of questions would also be essential.
Cognitive testing is an approach we use to assess how questions are understood and
answered by respondents […] this would be extremely helpful in providing evidence
from voters about the wording of different question options.28
As the Commission explained to us, such tests can sometimes produce surprising results:
voters do not always take the meaning which legislators believe or hope that they will from
the words which have been proposed. Andrew Scallan explained that “The research
demonstrates that each word has to be considered in each question to make sure that
people understand it”.29
29. No question should be placed before the electorate unless the Electoral Commission
is completely satisfied as to the intelligibility and fairness of the wording. The
Commission should conduct a very thorough programme of testing of a number of
different question options before coming to a view on the wording which is clearest and
fairest.
30. Professor John Curtice of the University of Strathclyde added:
Certainly, whatever is initially proposed, we need to ensure that the Electoral
Commission has the right not just simply to test a single proposition but indeed is
able to test what it regards as reasonable alternatives.30
Testing proposed questions
31. Testing the question must be done carefully and thoroughly, and consider the full
range of options so as to choose the best. That will take time. Mr McCormick said:
We are recommending that we have twenty weeks to test. That is based on the
experience last time of testing and re-testing and going round the circuit again. This
is based on the experience of the single question referendums last year.31
27
Ev 217
28
ibid.
29
Q 750
30
Q 316
The Referendum on separation for Scotland: Do you agree this is a biased question?
15
Mark Diffley thought the timescale would be “three or six months [...] you would need to
go through a process where you cognitively tested the question with enough people”, while
Andrew Hawkins, of ComRes, felt that “two to three months” would be an appropriate
length of time.32
32. We accept that the process of testing should not be skimped or rushed; but we also
have concerns that the period of uncertainty before this referendum is to be held is already
too long, even on the basis of the timetable proposed by the UK Government, which would
allow it to take place in 2013. The evidence from business on the effects of this uncertainty
was clear. For example, Martin Wolf, of the Financial Times, said “Uncertainty is itself
costly [...] for this reason, it would seem to me to be very much in the interests of all
concerned to minimise the period of uncertainty”.33 It is therefore a matter of some
concern to us that there is no sign at present of any work on testing possible questions.
33. The Electoral Commission’s role is advisory and not decisive. It can only work with
questions which are proposed to it, and it was made clear to us in evidence by the
Commission that it was not for it to initiate the drafting of a neutral, unbiased question: it
would simply respond to what is proposed.
34. Dr Matt Qvortrup, of Cranfield University, made the point that “What is important is
that the question is seen to be fair by both sides of the argument”. He went on to give an
example:
In Montenegro when they voted for independence there, they had a representative
from the yes side, a representative from the no side and then a neutral person – I
believe a Supreme Court judge – to sit between them. They then came up with a
question they could all agree on.34
35. In our view, it is of great importance that the Electoral Commission is able to
consider and test a question agreed by all concerned, failing which alternative wordings
which have been proposed both by those who support separation and by those who do
31
Q 716
32
Q 905
33
Q 410
34
Q 629
16
The Referendum on separation for Scotland: Do you agree this is a biased question?
not. In that way, the Commission will be able to recommend on the basis of a rigorous
and objective process of testing a question that both sides of the argument will be able
to agree to.
36. The Political Parties, Elections and Referendums Act 2000 (PPERA) gives powers to
the Electoral Commission which would enable this to happen. This could wait until a
section 30 Order is passed, and the Scottish Government is considering draft legislation. If
the Electoral Commission were given powers under that legislation of the same sort that it
has under Westminster legislation, it could then test a proposed question and advise on it.
There is, however, no reason for the necessary work to be delayed as long as that, and two
very strong arguments why it should not.
37. The first is that debate on a separation referendum needs to move as swiftly as possible
from questions of procedure and drafting, like the wording of the question, to the
substantive choice which faces the Scottish electorate: whether or not to leave the United
Kingdom. Given that it is widely agreed that the Electoral Commission should play the key
role in setting the question, there is absolutely no reason for them not to start on that as
soon as possible. Indeed, there is every reason for them to get ahead. They themselves
recommended that they should be allowed 20 weeks to do this work thoroughly, and some
of our witnesses thought that rather longer than that might be needed. It is important that
the electorate knows as soon as possible the question that it will be asked.
38. There is a second reason for getting ahead with this work now. We agree with our
witnesses that the question must be, and be seen to be, a fair one. If the result of the
referendum is to command wide acceptance, then the question must be agreed in advance
by both sides of the argument. An obvious way to do that would be for those on both sides
of the argument to have access to the fullest possible advice from the Electoral Commission
before the legislative process begins in Westminster or Holyrood. That way, once their
advice has been accepted, their neutral view will be incorporated in the legislation from the
very beginning, removing a potential source of conflict, and reassuring the electorate that
this aspect of the process is wholly unbiased.
The Referendum on separation for Scotland: Do you agree this is a biased question?
17
39. The Commission made clear to us that it was not its role to propose questions, but
rather to test the questions proposed by Governments or others. We understand this, but
that puts the onus on others to place before the Commission questions which it can test for
clarity and fairness, and then advise which best meets the needs of a separation
referendum. The Commission should test, as they rightly argued, how every word in a
proposed question is understood; this should include the words “separation”,
“independence”, “state”, “country”, “nation”, “remain” and “become”, and other relevant
words. It should also test the structure of the question paper, including whether it is better
to have a question which solicits the answer “yes” or “no”, or a formulation that invites the
voter to say “I agree” or “I disagree” and if so to what sort of statement.
40. There is a mechanism for achieving this. The Electoral Commission does not have to
wait for draft legislation before offering advice. Section 10 of PPERA provides that:
The Commission may, at the request of any relevant body, provide the body with
advice and assistance as respects any matter in which the Commission have skill and
experience.
A relevant body includes the UK or Scottish Governments. The Commission may also
provide advice to “registered [political] parties”. It is therefore entirely possible for either or
both Governments or the political parties involved to seek the advice of the Commission.
They should do so on an agreed selection of possible questions, ideally jointly.
41. The Scottish Government’s proposed question (or some variant of it, if it accepts
the criticisms we have made of how it is presently drafted) and a question proposed by
the opponents of separation could therefore be referred to the Commission for advice.
The best way of achieving that would be for the political parties represented in the
Scottish Parliament to agree jointly on a question, or, if they cannot, a range of
questions, on which the Commission’s advice should be taken. If a consensus cannot be
reached, then an individual party or group of parties should submit a question, or
range of questions. Both sides should agree in advance to accept the Commission’s
advice in order to remove from dispute the wording of the question and allow the
people of Scotland to focus on the important issues of substance involved in the SNP’s
proposals for separation.
18
The Referendum on separation for Scotland: Do you agree this is a biased question?
Conclusions and recommendations
The principles behind a referendum question
1.
These principles of clarity and balance are obviously correct and must apply to any
question put to the Scottish people in a referendum on separation. (Paragraph 10)
Second draft question proposed by the Scottish Government
2.
Based on the evidence we have received, we have no choice but to conclude that the
question currently proposed by the Scottish Government is biased in that it tends to
lead the respondent towards the answer “yes”. (Paragraph 16)
How can a fair question be drafted?
3.
None of our witnesses thought that the SNP’s plan was acceptable and all agreed that
the Electoral Commission, which is experienced and neutral, was the appropriate
body to ensure that this referendum is as fair as possible. The Electoral Commission
itself is prepared to do so. We recommend that the Electoral Commission should
have that responsibility. (Paragraph 25)
4.
No question should be placed before the electorate unless the Electoral Commission
is completely satisfied as to the intelligibility and fairness of the wording. The
Commission should conduct a very thorough programme of testing of a number of
different question options before coming to a view on the wording which is clearest
and fairest. (Paragraph 29)
Testing proposed questions
5.
In our view, it is of great importance that the Electoral Commission is able to
consider and test a question agreed by all concerned, failing which alternative
wordings which have been proposed both by those who support separation and by
those who do not. In that way, the Commission will be able to recommend on the
basis of a rigorous and objective process of testing a question that both sides of the
argument will be able to agree to. (Paragraph 35)
6.
The Scottish Government’s proposed question (or some variant of it, if it accepts the
criticisms we have made of how it is presently drafted) and a question proposed by
the opponents of separation could therefore be referred to the Commission for
advice. The best way of achieving that would be for the political parties represented
in the Scottish Parliament to agree jointly on a question, or, if they cannot, a range of
questions, on which the Commission’s advice should be taken. If a consensus cannot
be reached, then an individual party or group of parties should submit a question, or
range of questions. Both sides should agree in advance to accept the Commission’s
advice in order to remove from dispute the wording of the question and allow the
people of Scotland to focus on the important issues of substance involved in the
SNP’s proposals for separation. (Paragraph 41)
The Referendum on separation for Scotland: Do you agree this is a biased question?
19
Annex 1
A specimen GCSE Statistics (Higher Tier) paper produced by Edexcel in 2003 included the
following question:
8. Mary is carrying out an investigation into the cost of food at her college canteen.
She asks people in the queue for canteen food
“Do you agree that canteen food is value for money?”
(a) Why is her sample of people likely to be biased?
............................................................................................................................................…
.......................................................................................................................................... (1)
(b) Why is her question biased?
…....................................................................................................................................... (1)
(c) Suggest two reasons for her to carry out a pilot survey.
(i).................................................................................................................................…….
(ii)....................................................................................................................……… (2)
For another investigation Mary selects a sample of 30 students from the 720 students at her
college.
(d) Describe how she would select a simple random sample.
.............................................................................................................................................
...................……………………..........................................................................................
…………........................................................................................................................... (2)35
35
Edexcel GCSE Statistics (Higher Tier) Paper 1H (1389), 2003
20
The Referendum on separation for Scotland: Do you agree this is a biased question?
Annex 2
Lord Ashcroft commissioned a survey on the effect of the wording of referendum
questions, for which the fieldwork was carried out between 26 and 31 January 2012. The
results are below.
Question: Do you agree that Scotland should be an independent country?
Total
Male
Female Age
18-24
Age
25-39
Age
40-59
Age
60+
Social
Grade
ABC1
Social
Grade
C2DE
Yes
(%)
41
46
35
42
39
43
38
37
44
No
(%)
59
54
65
58
61
57
62
63
56
Question: Do you agree or disagree that Scotland should be an independent country?
Total
Male
Female Age
18-24
Age
25-39
Age
40-59
Age
60+
Social
Grade
ABC1
Social
Grade
C2DE
39
38
39
35
36
45
34
37
40
Disagree 61
(%)
62
61
65
64
55
66
63
60
Agree
(%)
Question: Should Scotland become an independent country, or should it remain part of the
United Kingdom?
Total
Male
Female Age
Age
Age
Age
18-24 25-39 40-50 60+
Social
Grade
ABC1
Social
Grade
C2DE
The Referendum on separation for Scotland: Do you agree this is a biased question?
Independent 33
(%)
35
31
28
33
37
30
29
37
UK (%)
65
69
72
67
63
70
71
63
67
36
36
www.lordashcroftpolls.com
21
22
The Referendum on separation for Scotland: Do you agree this is a biased question?
Formal Minutes
Wednesday 25 April 2012
Members present:
Mr Ian Davidson, in the Chair
Fiona Bruce
Jim McGovern
Iain McKenzie
David Mowat
Pamela Nash
Simon Reevell
Mr Alan Reid
Lindsay Roy
Draft Report (The Referendum on Separation for Scotland: Do you agree this is a biased question?), proposed
by the Chair, brought up and read.
Ordered, That the draft Report be read a second time, paragraph by paragraph.
Paragraphs 1 to 41 read and agreed to.
Summary agreed to.
Resolved, That the Report be the Eighth Report of the Committee to the House.
Ordered, That the Chair make the Report to the House.
Ordered, That embargoed copies of the Report be made available, in accordance with the provisions of
Standing Order No. 134.
[Adjourned till Wednesday 16 May 2012 at 2.00 p.m
The Referendum on separation for Scotland: Do you agree this is a biased question?
23
List of Reports from the Committee during
the current Parliament
The reference number of the Government’s response to each Report is printed in brackets after the
HC printing number.
Session 2010–12
First Report
Postal Services in Scotland
HC 669 (HC 884)
Second Report
Video Games Industry in Scotland
Third Report
UK Border Agency and Glasgow City Council
HC 733
Fourth Report
The Scotland Bill
HC 775
Fifth Report
Student Immigration System in Scotland
Sixth Report
The Referendum on Separation for Scotland:
Unanswered Question
HC 1806
Seventh Report
The Crown Estate in Scotland
HC 1117
HC 500 (Cm 8067)
HC 912 (Cm 8192)