Endeavour Fund
Smart Ideas and Research Programmes
Assessor Guidelines for Research
Programmes
2017 Endeavour Round
May 2017
The Ministry of Business, Innovation and Employment reserves the right
to withdraw or amend, at any time, this document or any part of it.
All persons, entities, techniques and ideas contained in this document are fictitious and
are for training purposes only. Any resemblance to real persons (whether living or dead),
entities, techniques or ideas is unintentional and purely coincidental.
Contents
Welcome .................................................................................................................................................................. 1
How does this document help you? ........................................................................................................................ 2
PART A: ALL ASSESSORS ........................................................................................................................................... 3
What you need to know – at a glance ..................................................................................................................... 4
The 2017 Process ..................................................................................................................................................... 6
Roles and responsibilities ........................................................................................................................................ 8
Assessors ......................................................................................................................................................... 8
Lead Assessors ................................................................................................................................................ 9
MBIE ................................................................................................................................................................ 8
Assessor training ............................................................................................................................................. 9
The Application and Assessment Process .............................................................................................................. 10
Research Programmes Application Process ...................................................................................................... 10
Key steps of the assessment process ................................................................................................................ 10
Assessing and scoring proposals ....................................................................................................................... 11
Assessment comments...................................................................................................................................... 11
An introduction to the Scoring Guide ............................................................................................................... 12
Structure and content of the scoring guide .................................................................................................. 12
Consistency ................................................................................................................................................... 12
Weighing up ‘attributes’ of varying merit ......................................................................................................... 13
Other scoring tips .............................................................................................................................................. 13
Your obligations ..................................................................................................................................................... 14
Conflicts of interest ........................................................................................................................................... 14
Direct............................................................................................................................................................. 14
Indirect .......................................................................................................................................................... 14
Managing conflicts ........................................................................................................................................ 14
Confidentiality ................................................................................................................................................... 14
Communications with MBIE .............................................................................................................................. 15
Official Information Act 1982 ............................................................................................................................ 15
Using the IMS portal .......................................................................................................................................... 15
PART B: EXCELLENCE ASSESSORS........................................................................................................................... 16
Assessing Excellence .............................................................................................................................................. 17
Risk .................................................................................................................................................................... 18
Area of research ................................................................................................................................................ 18
Interpreting Criteria, Sub-criteria and Attributes .................................................................................................. 20
Scoring Guide for Research Programmes - Excellence .......................................................................................... 24
PART C: IMPACT ASSESSORS .................................................................................................................................. 33
Assessing Impact.................................................................................................................................................... 34
Benefit to New Zealand ..................................................................................................................................... 35
Implementation Pathway(s) .............................................................................................................................. 35
Interpreting Criteria, Sub-Criteria and Attributes.................................................................................................. 36
Scoring guide for Research Programmes - Impact ................................................................................................ 39
Appendix A............................................................................................................................................................. 46
Investment Signals for 2016-19......................................................................................................................... 46
Appendix B ............................................................................................................................................................. 48
Dimensions of Impact........................................................................................................................................ 48
Appendix C ............................................................................................................................................................. 49
Some guidance to help you strengthen your approach to Vision Mātauranga ................................................ 49
Appendix D ............................................................................................................................................................ 50
Assessing Vision Mātauranga Relevance ........................................................................................................... 50
Welcome
Welcome to the MBIE 2017 Endeavour Round and thank you for agreeing to contribute your time
and experience to assess research proposals submitted to it.
If you have assessed proposals for us before, then a special welcome back. Please note that while
we have not made any substantive changes to the policy settings for Research Programmes, there
are some differences between assessing Research Programmes in 2016 and Research Programmes in
2017. Please familiarise yourself with this new approach and what it means for you.
MBIE will invest in high-calibre Research Programmes that deliver excellence and impact. Research
Programmes supports ambitious, excellent and well defined research ideas that, collectively, have
credible and high potential to positively transform New Zealand’s future in areas of growth or critical
need.
Research Programmes proposals will be relatively large in scale and duration, and they will propose
stretchy, fit-for-purpose science with well-managed scientific or technical risks or a high degree of
innovation. Your role as an Assessor is critical to identifying which of the many research proposals
submitted to us really sing with excellent science and the potential to deliver credible transformative
impact for New Zealand. Your contribution will help shape the recommendation to our Science
Board about which Research Programme proposals should be funded.
The experience and expertise that you bring to the assessment process is greatly appreciated.
Excellent science and the impact it can deliver relies on strong peer review and constructive critique.
Thank you for your time and support of MBIE’s science investment processes.
Dr Prue Williams
General Manager
Science Investments
1
How does this document help you?
This document has three parts so that you can focus on information relevant to your assessments
and avoid unnecessary reading.
What should you read?
 Part A - All assessors
 Part B - Excellence assessors
 Part C – Impact assessors
If you are assessing both excellence and impact, please read Parts B and C together. We will let you
know in the lead up to proposals being assigned what we would like you to assess. Note that these
two aspects are assessed at different stages in the process, so that you will only be assessing
excellence or impact at any one time.
We strongly recommend that you read this document before you begin assessing proposals. You
may also want to read other relevant documents, particularly those set out in Figure 1 which will
provide additional context. These are, or will be provided on the MBIE website, and you will be
advised of the links as soon as possible.
Figure 1: Document Map
An important note:
Any change in the process will be notified via MBIE’s Alert e-newsletter for the Endeavour Round. If
you do not already receive science investment Alerts you can sign up to receive them here.
2
PART A: ALL
ASSESSORS
3
What you need to know – at a glance
Your role
Scope of the
fund
As an assessor:
 only assess and comment on what you are asked to assess and comment on. If
you are assessing impact, do not comment or re-litigate the assessment of
excellence. If you are assessing excellence, do not comment on impact.
 your assessment score and comments must align with the scoring guides in Parts
B and C of these guidelines.
The Endeavour Fund invests in excellent research with high potential for impact in
areas of future value, growth or critical need for New Zealand. The Fund can invest in
research that is targeted basic research as well as research that is more applied.
That said, the Government is looking to focus a greater proportion of investment in
the Endeavour Fund in research with a potentially high, but longer-term
transformative impact.
The Fund invests in research that has the potential for impact in economic,
environment, and/or social outcomes, as outlined in the investment signals for this
fund (refer Appendix A)
Judgement,
horizons of
research and
state of the
sector(s)
The breadth of the Endeavour Fund’s scope across the research horizons means that
you will have to apply judgement when assessing science excellence and impact,
relative to the stage of research and the area of impact. The three horizons can be
thought of as: generating new ideas, developing emerging ideas, and leveraging
proven ideas. More information on horizons is available in the National Statement of
Science Investments, p31.
For example, in the generate new ideas horizon, research might pose higher levels of
risk than more developed research and have a less well defined or developed
implementation pathway that, refers to next users rather than end users.
By comparison in the leverage proven ideas horizon, research might have a welldefined implementation pathway with proven engagement with end users.
In undeveloped sectors e.g. a transformative new technology for which few or no end
users currently exist, then planning for the development of new implementation
pathways is appropriate.
If assessing excellence, you might want to ask:
 Is the size of risk, and plans to mitigate that risk, commensurate with the stage of
research and state of the sector?
 Is the proposed research excellent and fit for purpose for the proposed benefit
impact sought?
If assessing impact, you might want to ask:
 Has the applicant described a vision of how this opportunity can be
transformative for New Zealand in the future?
 Is the proposed implementation pathway credible relative to the proposed stage
of research, and the state of the sector?
4
Judgment and
areas of impact
The breadth of the Endeavour Fund’s scope across economic, environment, and social
outcomes means that you will have to apply judgement when assessing benefit and
implementation pathways.
For example, the potential benefits associated with research in the high-value
manufacturing space, may be substantive in size but be narrowly focused because its
implementation pathway involves commercialisation via a small number of
companies.
By comparison, research in the environmental or social space may have broad
benefits for New Zealand that is implemented via engagement with public policy
makers and regulators.
You might want to ask:
 Are the scale and breadth of proposed benefits credible given the area of impact?
 Is the proposed implementation pathway credible for this particular area of
impact?
 Has the applicant described how the new science or technology has the potential
to have breadth of application across multiple sectors, industries, or user groups
in the future?
Refer to Appendix B for more detail of the dimensions of impact framework for the
wider science systems.
Vision
Mātauranga in
proposals
Consider if the applicant has identified:
 opportunities, needs, requirements, contributions or innovations from Māori
knowledge, people or resources
 relevant, specific Māori interests (collectives, businesses and communities)
 a line of sight from research design to delivery of outcomes
 appropriate and relevant elements that should be integrated throughout the
proposal.
Relevant
changes this
year
The Gazette Notice has been updated and clarifies that those proposals with a
primary focus on health outcomes are not eligible to apply, although a proposal
where health is a secondary focus is eligible.
5
The 2017 Process
What are the
key dates for
applicants?
These key dates are indicative and may be subject to change.
What do I do if
I think I have a
conflict of
interest?
If you think you have a direct or indirect conflict of interest with any proposal that is
assigned to you, please notify us as quickly as possible by:
 emailing [email protected] to advise us of the circumstance leading to the
conflict of interest and we will assess whether the conflict can be managed, or
 declining the proposal in the Investment Management System (IMS portal).
Research Programmes
Call for Proposals and Application Guidelines
Late September 2016
available
Early October 2016
Roadshow sessions
12 noon, 8 February 2017 Closing date for registrations
12 noon, 9 March 2017
Closing date for proposals
April 2017
Proposals assessed
May 2017
Science Board decisions on Excellence
June 2017
Applicants notified of Science Board decisions
July 2017
Proposals assessed
August 2017
Science Board final decisions
September 2017
Applicants notified of Science Board decisions
1 October 2017
Contracts begin
If we consider the conflict of interest is not manageable we will reassign the proposal
to another assessor.
The quality of
your comment
is critical
Your comments must:
 align with the reference statements and criteria in the Scoring Guide
 reflect your score and give guidance and the rationale for your assessment.
Your scores and comments will be used to inform Science Board decisions and may
also be the basis of feedback for applicants. It is important that your comment:
 supports your score and fairly reflects the assessment
 is accurate, professional, and honest.
Therefore:
 do not comment on the skill or expertise of fellow assessors
 do not comment on the score and comments of fellow assessors (unless you are a
lead assessor preparing summary comments, who may wish to reflect individual
comments from other assessors) do not comment on impact if you are assessing
excellence
 do not comment on excellence if you are assessing impact.
6
Why is
information
security vital?
We cannot overly stress the importance of information security and confidentiality.
Do not discuss or communicate any aspect of the assessment you are working on with
the applicant or anyone outside the process, as you may undermine the integrity of
the process.
Should anyone contact you regarding a proposal or your assessment, please contact
us immediately at [email protected].
Only use the project note function in the IMS portal to communicate with the lead
assessor or other assessors. For security reasons, do not use your own personal
communication channels to discuss proposals or communicate with other assessors.
What do I do if
I have any
questions?
Send any queries to us by email at assessors@[email protected]. Include your telephone
contact details and time of availability if your query is urgent. Including the subject of
your enquiry in the subject line of your e-mail will enable us to pass your question on
to the right person quickly and provide a swift response.
What other
supporting
documents are
available to
me?







National Statement of Science Investment 2015-2025 (the NSSI) outlines the
government’s 10-year vision for New Zealand’s science system.
Endeavour Fund Investment Plan 2016-2019 (the Investment Plan) outlines the
government’s strategy for its science investment for the next three years.
New Zealand Gazette Notice, Number 2016-go4875, August 2016 (the Gazette
Notice) sets the criteria the Minister for Science and Innovation requires the
Science Board to use in making funding decisions for the Fund.
Vision Mātauranga policy outlines the government’s policy framework that aims
to unlock the innovation potential of Māori knowledge, resources and people to
assist New Zealanders to create a better future.
International Science for New Zealand: Strategic Action Plan 2017-2027 provides
guidance about government’s expectations of the contribution of international
partnerships and collaborations to New Zealand science (due for release early
2017).
Applicant Guidelines for Completing a Proposal explain the purpose of each
section of the application form and suggest information applicants may wish to
include in a proposal.
Portal Guidelines for Assessors explain how to use the IMS portal to read
proposals and record assessment details.
7
Roles and responsibilities
Assessors
Your role is to assess:
 only what has been assigned to you. You will be able to view the entire proposal but you will
only be able to score what you have been asked to score (excellence and/or impact)
 the proposal against the scoring guide by assigning a score and providing a constructive
comment that justifies your score
 each proposal, and score it relative to the scoring guide NOT relative to other proposals
 by following the scoring guidelines. Do not assess or comment on impact if you an excellence
assessor and do not assess or comment on an excellence assessment if you are an impact
assessor.
Your role does not include:
 weighing up the relative merits of different proposals
 balancing the portfolio of investment.
You also need to enter the assessment scores and comments in the IMS portal. In doing this, we
need you to:
 declare situations of direct or indirect conflict of interest
 respect the confidentiality of the proposals that you are assessing
 undertake your assessments in accordance with the guidance in these guidelines.
For excellence only you will rate the ‘scientific or technical risk’ of the proposed research. Using the
descriptors below, the characterisation should be consistent with the summary you have developed.
Additional information is provided in the scoring guide.
Table 1: Assessment of scientific or technical risk
Descriptor
What is the unmitigated scientific or
technical risk?
What is the mitigated scientific or
technical risk?
Do the proposed additional benefits
justify the proposed mitigated risk?
High
Medium
Low
High
Medium
Low
Yes
No
For Vision Mātauranga ensure that in your scoring decisions due weight is given to any contribution
made by the proposal to Vision Mātauranga. Your comments on the criterion you are addressing
should include reference to Vision Mātauranga. More information on Vision Mātauranga is available
in the scoring guides and in Appendices C and D.
MBIE is also interested overall in how well Vision Mātauranga has been addressed. The table below
provides a guide to help MBIE understand how well applicants have addressed Vision Mātauranga
and whether you agree with them or not.
8
Table 2: Classifications of Vision Mātauranga – Excellence and Impact
Key questions
1) Is Vision Mātauranga, in your
opinion, relevant to the
proposal?
2) Does the applicant recognise
that Vision Mātauranga is
relevant to the proposal?
3) If Vision Mātauranga is, in
your opinion, relevant to the
proposal, how well has the
applicant addressed this?
Assessor comment
Yes
No
Yes
No
Very well
Moderately well
Not well
Not Relevant
Where there is a divergence of view between
your and the applicant’s view (i.e. you consider
it is relevant but the applicant does not or vice
versa) to whether or not Vision Mātauranga
applies, you must give reasons for this.
If Vision Mātauranga is not relevant to the
proposal and this is consistent with the
applicant’s view (i.e. the answer to questions 1
and 2 is “No”, then the only comment you need
to make is “Vision Mātauranga is not relevant to
that proposal”.
Lead Assessors
If you are also identified as a Lead Assessor, you will
 assess the proposals assigned to you
 read all individual assessments for a proposal (when scoring is complete)
 prepare a summary of the individual assessments, including those related to both scientific or
technical risk and Vision Mātauranga set out above. Your summary should synthesise different
assessors’ views and reflect any significant differences.
MBIE
Our role is to support the assessment process and assessors by providing you with information and
guidance on administrative matters, the assessment process and the interpretation of guidance and
other documents. It is not appropriate for us to become involved in the assessment of specific
proposals.
Assessor training
You will be provided with material to help you navigate through assessments closer to the time. We
will contact you when the training material is available.
9
The Application and Assessment Process
Research Programmes Application Process
The Research Programmes investment mechanism uses a two-stage assessment process.
First, proposals are assessed for science excellence only. Based on the results of the assessment, the
Science Board will determine which proposals are of sufficient quality, based on the excellence
criterion, to proceed to an impact assessment.
MBIE will advise all applicants of the Science Board’s decision on which proposals will proceed to the
impact assessment stage and those that are declined.
Proposals that proceed will be assessed for impact and the results of the impact assessment will be
combined with the excellence assessment, to provide an overall assessment for each proposal. For
the impact-assessment stage only, assessors will assess proposals before they meet as a panel where
they will review/discuss their individual scores and comments.
Figure 2: Research Programmes Application Process
Key steps of the assessment process
The process in Figure 3 and Figure 4 illustrate, at a high level, the steps in each stage.
Figure 3: Assessment of Research Programmes Science Excellence
Step 1:
Independent assessment of
Excellence by individual assessors –
score and comment for each
excellence sub-criterion.
Step 2:
Summary comment by lead
assessor.
Step 3:
Assessors review summary
comment - lead assessor edits if
appropriate.
Step 4:
Proposals ranked on excellence
scores and assessment comments.
10
Figure 4: Assessment of Research Programmes Impact
Step 1:
Independent assessment of Impact by
individual assessors – score and
comment for each impact subcriterion.
Step 2:
Summary comment by lead assessor.
Step 3:
Proposals reviewed by Impact Panel.
An agreed score is determined by the
Panel. If appropriate, panel modifies
summary assessment comment.
Assessing and scoring proposals
When assessing a proposal you will need to consider whether, and how:
 the proposal is delivering excellence or impact (depending on your assigned role)
 the proposal responds to the signals set out in the Investment Signals (Appendix A) and
Dimensions of Impact (Appendix B).
Do this by considering the proposal against the excellence and impact criteria that are described in
Table 4 and Table 7. More detail on what you need to consider can be found in the scoring guides.
Assessment comments
Below are some simple guidelines for you to work to in drafting your assessment comment.
When reading the proposal it is useful if you can highlight the following questions in your scoring
and commentary:
 What were the strengths and highlights of the proposed research?
 What were the deficiencies or weaknesses of the proposed research?
 Were there any concerns, issues around the proposed research?
You should assess the proposal and address:
 Only the criterion / criteria you have been asked to assess, please check you have addressed
sub-criteria and their attributes
 Include comments on whether or not Vision Mātauranga opportunities have been identified and
followed through, and if not whether they should have been. Guidance on Vision Mātauranga
can be found in Appendix D.
11
Your comments should reflect your score, the rationale for your assessment, and guidelines.
 Ensure that your comments align with your scores, so they are mutually supportive, and explain
why the particular score was given.
 Ensure, for the sub-criteria of excellence, you comment on ‘risk’ and ‘innovation’.
 Identify the specific reasons for a deficiency, particularly where the proposal has a score of 4 or
less.
Word limits are in place for the length of your commentary – see Table 3 below.
Table 3: Comment lengths
Assessor/individual comments
Lead Assessor Summary Comment
Research Programmes Proposals
140 words
280 words
An introduction to the Scoring Guide
Detailed guidance on how to carry out assessments has been incorporated in the Research
Programmes Scoring Guide. For excellence see Part B and for impact see Part C.
The purpose of the Scoring Guide is to assist assessors in making judgements on scores, and to
promote consistency in assessment between assessors by providing common:
 interpretations of terms
 understanding of how much merit or quality a proposal should have to warrant each score.
Structure and content of the scoring guide
Scoring of proposals follows a scale of 1 to 7, with 1 being low quality and 7 high quality.
The Scoring Guide shows
 scores on the left-hand column
 descriptions of the assessment criteria and sub-criteria in the columns to the right
 within each sub-criterion there are a number of ‘attributes’, drawn from the text in the Gazette
Notice, which need to be considered in scoring a proposal.
Each level of score contains a set of reference statements on how much merit or quality a proposal
should possess for each attribute, in order to warrant a particular score.
As Vision Mātauranga is assessed within each attribute where relevant, for each score there is also a
separate indication of the Vision Mātauranga practice you might see at that score.
The contributing attributes of each sub-criteria on which the guide is based are listed in the section
titled Interpreting Criteria, Sub-criteria and Attributes in Parts B (for excellence) and C (for impact).
Consistency
Your score should align with the reference statements in the Scoring Guide. This will assist to
achieve consistency in your scoring across the proposals that you assess and in your assessments
relative to other assessors.
12
Weighing up ‘attributes’ of varying merit
The Scoring Guide shows attributes of equivalent merit or quality are grouped together against
particular scores. For example, to achieve a score of 7, every attribute would be considered
outstanding.
Individual attributes will often merit different scores. In this situation you need to consider the
balance across all attributes in determining the overall score. You may need to consider whether
there are some attributes that should particularly influence the score or should have a dominant
influence. For example, a serious flaw in one attribute would justify a low overall score, even if
some attributes justified higher scores.
Other scoring tips
Ensure your scores and comments reflect and align with the scoring guide. Use key words and
reference statements.
Only assess information presented in the proposal. Applicants are obliged to present the relevant
information. If information in the proposal has obvious gaps, reflect that in your scoring and record
any significant issues in your comments.
Be wary of ‘drift’. It is not unusual for scoring to change as you gain experience with the assessment
process. If it has drifted you may want to review your scores.
13
Your obligations
Conflicts of interest
MBIE takes the issue of conflict of interest very seriously. We follow a rigorous process to maintain
the credibility of the Science Board’s funding decisions and to assure applicants that their proposals
are given fair and reasonable assessment.
Conflicts of interest may occur on two different levels, as outlined below:
Direct
Examples of this could be where an assessor:
 is directly involved with the proposal (as a participant, manager, mentor or partner)
 has a close personal relationship with the applicants, for example family
 is a collaborator or is in some way involved with the applicant’s research programme.
Indirect
Examples of this could be where an assessor:
 is employed by an organisation involved in the proposal but is not involved directly in the
applicant’s research programme
 has a personal and/or professional relationship with one of the applicants, for example, an
acquaintance
 has an involvement with a proposal that is in direct competition with a proposal being assessed
or where the impacts proposed by a proposal under discussion may compete with an assessor’s
personal business interests.
Managing conflicts
Where there is a clear and direct conflict you must declare the conflict of interest to MBIE and not
proceed with the assessment of the proposal. We will reassign the proposal to another assessor.
You must also declare all indirect or unclear conflicts in relation to any allocated proposal to MBIE. If
we agree that the conflict is significant, we will reassign that proposal to another assessor. You
should advise all direct or indirect conflicts to MBIE by emailing [email protected].
Confidentiality
As mentioned on page 7, we cannot overly stress the importance of information security and
confidentiality.
Please ensure the safe-keeping of all proposals and related documents (for example, workbooks and
notes) and destroy all documentation after the assessment process is complete.
You should not discuss any aspect of the assessment process with parties other than those involved
in the assessment process. Under no circumstances should you discuss a proposal or potential
proposal with any applicant. If an applicant contacts you, please direct them to MBIE at
[email protected] and inform MBIE of the contact.
You will be required to indicate your agreement to the confidentiality terms and conditions when
you first log into to the IMS portal.
14
Communications with MBIE
Assessors should always communicate with MBIE via the email address [email protected] in
the first instance. This enables MBIE to ensure that questions are answered by the right person and
that the responses are copied to other assessors if appropriate. Please do not phone or email
specific individuals in MBIE.
Official Information Act 1982
MBIE is required to protect and release information in accordance with the Official Information Act
1982. Proposals, assessments and other information that is held by MBIE, including online content
stored in the IMS portal, is subject to the provisions of the Act and may be required to be released in
full or in part if requested.
Using the IMS portal
Portal Guidelines for Assessors will be available on MBIE’s webpage soon. We will also send an
email to you advising they are available.
15
PART B:
EXCELLENCE
ASSESSORS
16
Assessing Excellence
Your assessment of Excellence in Research Programme proposals must be based on the
assessment criteria which are taken from the Gazette Notice, and translated in the scoring guide.
Points to note:
Excellence should be considered in the context of:
 Research horizons: early stage research may pose higher scientific or technical risk than later
stage research. Both approaches are valid.
 Areas of research: excellent research should be appropriate to the relevant discipline.
Excellence includes two sub-criteria: Science and Team. Beneath each sub-criteria sit attributes
(Figure 5). Further details of the attributes can be found in Tables 5 and 6, pages 20-23.
All proposals must explain whether or not the proposed research is relevant to the Vision
Mātauranga policy. How they might do this is discussed in Appendix C and Appendix D.
Table 4: Criteria – Research Programmes Excellence
Excellence
i. Science (weighted 25%)
Research should be well-designed, well-performed, and leverage additional value from wider
research. Assessment must have particular regard to whether the proposed research, science or
technology or related activities:
a. progress and disseminate new knowledge;
b. possess scientific risk, technical risk, or innovative approaches;
c. are well-positioned in the domestic and international research context; and
d. have a well-managed research plan and credible approach to risk management.
ii. Team (weighted 25%)
The proposed team should have the demonstrated mix of complementary skills, knowledge and
resources to deliver the proposed research, science or technology or related activities and to
manage risk.
17
Figure 5: Layers of criteria, sub-criteria and attributes – Research Programmes Excellence
Criteria
Excellence
Sub-criteria
Attributes
Team
Science
Progress and
disseminate new
knowledge
Possess scientific risk,
technical risk or
innovative
approaches
Well positioned in
the domestic and
international
research context
Well managed
research plan and
credible approach
to risk management
Skills, knowledge
and resources
Risk
We are looking to invest in research which has scientific or technical risk or stretch because it may
have the potential to deliver greater value or impact for New Zealand than results of less risky
research.
You need to consider the:


level of scientific and technical risk and the rationale for the research approach. That is,
does the potential proposed benefit justify the level of risk?
plans to ensure that scientific and technical risk, along with the programme risk, is well
managed.
There could be a large number of possible combinations to consider in deciding overall scores for
assessment criteria. For practical reasons the scoring guide only gives some of the possible
combinations. You should be flexible in deciding what combination applies to the proposal you are
assessing, and how that might affect the score.
For example, you might score proposed research with high technical risk that is well managed quite
highly, yet score another piece of proposed research with a similar level of technical risk quite lowly
because the risk does not appear to be well managed.
Similarly, you might score proposed research with high technical risk with the potential to deliver
sizable benefits to New Zealand quite highly, yet score another piece of proposed research with a
similar level of technical risk quite lowly because the level of proposed benefit does not justify that
risk.
Area of research
Each discipline of science should be assessed within the context of what excellent science is
appropriate for that discipline
For example, the nature of the scientific or technical risk or innovation for social science is likely to
be different from that for high value manufacturing, or environmental research.
18
The key issue is whether the research is fit for purpose for the proposed research goals and benefits,
and is the degree of risk (or stretch) or innovation well managed and providing additional value.
19
Interpreting Criteria, Sub-criteria and Attributes
Table 5: Research Programmes Excellence, Sub-criteria Science
Criteria a: Excellence
Sub-criteria i: Science
“...Research should be well-designed, well performed and leverage additional value from wider research. Assessment must have particular regard to whether the
proposed research, science or related activities:”
Attribute
…progress and disseminate new
knowledge
Explanation
‘Dissemination’ means making the research results available for potential end users (or next users), so that impacts or benefits
can be achieved. The means of dissemination may vary according to the situation and should not be confined to publication in
the peer reviewed scientific literature.
…possess scientific risk, technical risk,
or innovative approaches
The Investment Plan encourages applicants to take risks, or embody ‘stretch’ or significant innovation in their proposals, in order
to increase the potential benefits from the proposed research. The overall outcome should be fit for purpose research that
meets the research goals.
The risks need to be managed through risk mitigation strategies and/or contingency plans, and the level of risk resulting needs to
be considered against the potential additional benefit. ‘Additional’ means benefits that would not be possible if the risks were
not taken. For Research Programmes, a balanced approach should be taken so that the potential additional value is at least
sufficient to justify the risk.
Risk and additional benefit need to be considered together but they can exist in many combinations. The reference statements in
the scoring guide provide examples of where some combinations should sit in the scoring range. Assessors need to use their
judgement in deciding where other combinations might most appropriately sit. Note the scoring guide does not contain all
possible scenarios of risk and additional benefit. You should use your judgement in assessing how the particular combination in
the application affects the score - within the constraints of the scoring guide.
The concepts involved are:
‘Scientific and technical risk’ has its source in the scientific basis of a proposal. Scientific and technical risk may include the
assumption of a scientific hypothesis which steps beyond that which can be confidently asserted based on current knowledge
and scientific principles; or the application of experimental or other scientific techniques in a context which is unproven or
speculative.
20
Criteria a: Excellence
Sub-criteria i: Science
“...Research should be well-designed, well performed and leverage additional value from wider research. Assessment must have particular regard to whether the
proposed research, science or related activities:”
Attribute
Explanation
‘Technical risk’ has its source in new technology which needs to be developed to enable the research to be done, and the
availability and suitability of experimental or supporting facilities and equipment.
Table 5.1 below explains the different types of risk used in the scoring guide.
‘Innovative’ means bringing in new methods or ideas, or a new combination of existing methods and ideas. Degrees of
innovation range from minor innovations in existing processes or techniques to the implementation of completely new processes
or techniques that are a significant advance from the status quo. Innovation can also include the application of existing processes
or techniques in new or unexpected areas.
Incorporates the extent to which the attribute contributes, as appropriate, to the achievement of the objectives of Vision
Mātauranga policy.
Table 5.1: Assessment of scientific or technical risk
Descriptor
What is the unmitigated scientific or
technical risk?
What is the mitigated scientific or
technical risk?
Do the proposed additional benefits
justify the proposed mitigated risk?
High
Medium
Low
High
Medium
Low
Yes
No
j
…are well-positioned in the domestic
and international research context
‘Well positioned’ means that the proposed research takes account of existing knowledge and research, either by avoiding
redundancy or overlap, or using existing knowledge/research as a platform for achieving more significant advances in knowledge
than would otherwise be the case.
Links with key related science activities (often funded separately) are described and are complementary or synergistic.
International links provide leverage and additional value.
21
Criteria a: Excellence
Sub-criteria i: Science
“...Research should be well-designed, well performed and leverage additional value from wider research. Assessment must have particular regard to whether the
proposed research, science or related activities:”
Attribute
…have a well-managed research plan
and credible approach to risk
management
Explanation
A ‘credible research’ plan is one which contains all of the expected elements and sets them out in a way which is scientifically and
managerially competent and can be effectively implemented. The expected elements include the research methodology, the
research design and proposed outputs, a risk management plan, and provision for access to and use of the facilities and
equipment for carrying out the research.
Extent to which the proposed research is appropriate to its stage of research, proposed benefit or is fit for purpose.
‘Credible’ could also consider whether the excellence of proposed research is fit for purpose in relation to its research goals.
22
Table 6: Research Programmes Excellence, Sub-criteria Team
Criteria a : Excellence
Sub-criteria ii: Team
“…The proposed team should have the demonstrated mix of complementary skills, knowledge and resources to deliver the proposed research, science or technology
related activities and to manage risk:”
Attribute
The extent to which the proposed
team has the demonstrated mix and
level of complementary skills to deliver
the proposed activities and to manage
risk.
Explanation
Both the mix and the level of skill are important, e.g. a team may have the appropriate mix of scientific, technical, engagement
and dissemination skills but there may be important variation in experience in applying those skills, and in the level of contribution
proposed from experienced researchers.
Track record means the demonstrated ability to deliver on the promised research performance in proposals. Key researchers
should participate to the extent required for them to have significant impact and influence on the delivery of the project.
If the team as a whole has a track record in previous research projects then it may be possible to consider their collective track
record. If the proposed combination of researchers is new, then you will have to consider the track records of key individuals.
If there is no track record information available, provided or able to be reasonably inferred; e.g. from publications record,
Assessors should regard that negatively, i.e. mark the team score down.
23
Scoring Guide for Research Programmes - Excellence
The following guide contains these references:
Māori capitals - this term is used to refer to the collective material and non-material assets held by Māori. They include:
 Human capital - knowledge and knowledge systems, traits, talents etc which people and populations have
 Natural capital - land, water bodies, living things, taonga species, the environment
 Social capital - the social structures (such as whanau, hapu, iwi, urban Māori structures, community), norms (such as tikanga
Māori) and values in daily life (such as trust, manaaki, indigeneity)
 Financial capital - economic wealth, financial assets
 Physical capital - equipment and physical infrastructure e.g. digital networks, vessels, housing stock
 Large natural group - usually an iwi (tribe) or a cluster of hapu (sub-tribes) with a significant population, and a large distinctive claim area.
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
1 (Low
quality)
Negligible
No relevant content
or connections
Inadequate
Sub-criterion ii
Team
(weighted 25%)
Progress and disseminate new knowledge
There is no new knowledge created and no consideration has
been given to disseminating the research results.
Skills mix
Both the mix and level of skills in the team are inadequate in all
respects.
Scientific risk , technical risk OR innovative approaches
Scientific and/or technical risk is negligible or low, OR the
application is not innovative.
Track record
The team, or key individuals in the team if the team is new, have a
track record of failing to deliver research results.
None/not
Are well positioned in the domestic and international
research context
Application ignores major aligned or related research or
information. The linkage to the relevant research landscape is
missing.
Have a well-managed research plan and credible approach to
risk management
The research plan is inadequate and does not contain any of
the expected elements. Appropriate scientific and technical
risk management is missing. The proposed research is not fit
24
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
Sub-criterion ii
Team
(weighted 25%)
for purpose in any respect.
2
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
Low
The research proposed does not unlock any Māori innovation.
Vision Mātauranga elements are negligible or not present in
the research plan design.
The team has inadequate capability to engage with Māori
stakeholders or their interests. Their track record is negligible.
Progress and disseminate new knowledge
There is limited new knowledge created which does no more
than amplify or further explain what is already known. The
means of dissemination proposed are not well matched to the
type of knowledge involved.
Skills mix
The team lacks most but not all of the skills and the level of skills
required to deliver the research, related activities and manage risks.
Scientific risk, technical risk OR innovative approaches
Scientific and/or technical risk is low and so is the additional
value that could be potentially achieved, or the application
has little innovation
Track record
Track record information is insufficient or lacks positive examples to
give confidence that the research will be satisfactorily delivered.
Little
Limited
Insufficient
Lacking
Poor credibility
Are well positioned in the domestic and international
research context
Little recognition of major aligned or related research. The
linkage to the relevant research landscape is inadequate.
Have a well-managed research plan and credible approach to
risk management
The research plan contains some of the expected elements,
but the information provided is insufficient for the plan to be
plausible. Scientific or technical risk management is poor. The
proposed research is fit for purpose in very few respects.
25
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
3
Sub-criterion ii
Team
(weighted 25%)
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
Low to moderate
The research proposed has insufficient Vision Mātauranga
elements in its design and delivers little relevant innovation
value to Māori.
Some
Most
Doubtful
effectiveness
Some aspects
inadequate
Progress and disseminate new knowledge
The new knowledge created, is incremental in character, ie is
no more than a logical extension of what is already known.
The means of dissemination proposed are workable but
unlikely to be very effective.
Scientific risk , technical risk OR innovative approaches
Scientific and/or technical risk is low but has additional
proposed benefit that could be achieved, or contains some
elements of innovation but no approaches that are
completely new.
Significant gaps
Are well positioned in the domestic and international
research context
There is some recognition of related research or existing
information, but this has only partly influenced the research
design. The linkage to the relevant research landscape has
significant gaps.
Have a well-managed research plan and credible approach to
risk management
The research plan contains most of the expected elements,
but the details are limited and some elements are
26
The team lacks the level of capability or capacity required to
navigate and engage with Māori stakeholders and their interests for
the science proposed. Little or poor track record with Māori capitals
relevant to the science proposed.
Skills mix
The team has some significant gaps in the types and levels of skills
required to deliver the research, related activities and manage risks.
Track record
The track record of the team, or of individuals in the team is patchy,
ie mix of good and not so good.
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
Sub-criterion ii
Team
(weighted 25%)
inadequately developed. Scientific and technical risk
management has significant gaps. The proposed research is fit
for purpose in some respects but the connection to purpose is
not well demonstrated.
4
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The research approach is a doubtful match for unlocking
Māori innovation potential through the science proposed.
There are gaps in key Vision Mātauranga elements in the
research plan design which suggests low effectiveness to
deliver new knowledge or intellectual property relevant to
Māori.
The team has some significant gaps in the capability or capacity
required to navigate and engage with Māori stakeholders and their
interests for the science proposed. They have patchy track record
with Māori capitals relevant to the science proposed.
Moderate
Progress and disseminate new knowledge
The knowledge created is new but is largely an extension of
existing knowledge rather than breaking significant new
ground. The means of dissemination proposed are
appropriate but moderately effective.
Skills mix
The research team has a mix and level of skills for carrying out the
work which is generally well matched to those required to carry out
the research, related activities and manage the risks. However,
there are some gaps or deficiencies.
Scientific risk , technical risk OR innovative approaches
Scientific and/or technical risk is moderate and is largely
counterbalanced by the additional proposed benefit that
could be achieved, or there is significant innovation in the
proposal but based more on new applications of existing
approaches than new approaches.
Track record
The team, and/or individuals in the team, have a generally good
record of successfully delivering proposed research.
Significant In some
aspects
Reasonable
Largely appropriate
Largely satisfactory
but with gaps or
deficiencies
Partly
Are well positioned in the domestic and international
research context
There is reasonable recognition of related research and
existing knowledge, and this is leveraged in research design
27
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
Sub-criterion ii
Team
(weighted 25%)
but with some gaps. The linkage to the relevant research
landscape is adequate.
Have a well-managed research plan and credible approach to
risk management
The research plan contains all of the expected elements, but
the level of detail is not of a satisfactory standard in some
cases. Scientific or technical risk management is adequate.
The proposed research is largely fit for purpose but with some
connections that need further explanation.
5
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The research approach is a reasonable match for unlocking
Māori innovation potential through the science proposed.
Largely appropriate Vision Mātauranga elements appear in
the research plan design however there may be some gaps or
deficiencies including the identification and mitigation of
distinct risks. Existing relevant Māori knowledge has been
considered. New knowledge and intellectual property of value
to Māori interests is delivered.
The team has moderate capability and capacity for reasonable
engagement with core Māori stakeholders for the science
proposed. Although there may be some gaps or deficiencies. They
have a largely satisfactory track record with Māori capitals relevant
to the science proposed.
Moderate to high
Progress and disseminate new knowledge
The new knowledge created breaks new ground rather than
being incremental in character, and the means of
dissemination proposes is appropriate and likely to be
effective.
Skills mix
The research team has a mix and level of skills for carrying out the
work which are well matched to those required for carrying out the
type of research, related activities and manage the risks involved.
There are no significant gaps.
Scientific risk , technical risk or innovative approaches
Scientific and/or technical risk is moderate and is more than
counterbalanced by the additional proposed benefit that
28
Track record
The team, and/or individuals in the team, have a consistent record
of successfully delivering proposed research.
More than
significant
Comprehensive
Appropriate
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
No gaps.
New/novel
Meets good
practice standards
Good/effective
Sub-criterion ii
Team
(weighted 25%)
could be achieved, or is significantly innovative and contains
approaches which are new, ie have not been proposed
before.
Are well positioned in the domestic and international
research context
There is comprehensive recognition of related research and
existing knowledge and this is competently leveraged in
research design. The linkage to the relevant research
landscape is well demonstrated,
Have a well-managed research plan and credible approach to
risk management
The research plan contains all of the expected elements,
meets expected standards of good practice throughout, and
should be delivered as stated. Scientific or technical risk
management is appropriate and credible. The proposed
research is fully fit for purpose.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The research approach is a very good match for unlocking
Māori innovation potential through the science proposed.
Credible Vision Mātauranga elements are support research
plan design. Existing relevant Māori knowledge has been
incorporated. Distinct risks and sensitivities are well mapped
and mitigated. New knowledge and intellectual property of
strategic value to Māori interests is delivered. Some likelihood
of attracting international attention.
29
The team has a moderate to high level of capability and capacity to
navigate and engage effectively with Māori stakeholders, networks,
interest groups in various settings relevant to the science proposed.
They have a strong track record of engaging effectively with Māori
capitals relevant to the science proposed.
Score
6
Key words
Very high/ highly
Excellent
Insightful
Widely significant
Exceeds best
practice Very
good/effective
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
Sub-criterion ii
Team
(weighted 25%)
Progress and disseminate new knowledge
The new knowledge created, is potentially significant beyond
the immediate field of research, and the means of
dissemination proposed are excellent and likely to be very
effective
Skills mix
The research team has a mix and level of skills for carrying out the
work which is excellently matched to those required for effectively
carrying out the type of research, related activities and managing
risks involved. This is a very good team.
Scientific risk , technical risk or innovative approaches
Scientific and/or technical risk is high and is counterbalanced
by the additional proposed benefit that could be achieved, or
is highly innovative with ideas and/or approaches which are
markedly new and likely to attract attention from other
researchers and potential end users
Track record
The team, and/or individuals in the team, have an excellent record
of successfully delivering quality research, related activities and
managing risks.
Markedly
Comprehensive
Very large
Very credible
Are well positioned in the domestic and international
research context
There is comprehensive and insightful recognition of related
research and existing information and this is extremely well
leveraged in research design. The linkage to the relevant
research landscape is comprehensive.
Significant
Have a well-managed research plan and credible approach to
risk management
The research plan contains all of the expected elements, at a
level of competence and detail which exceeds expected
standards of good practice, and gives confidence that the plan
will be delivered as stated. Scientific or technical risk
management is likely to be effective. The proposed research is
fully fit for purpose.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
30
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
7 (High
quality)
Sub-criterion ii
Team
(weighted 25%)
Indications for
identifying Vision
Mātauranga at this
scoring level
The research approach is a very good match for unlocking
Māori innovation potential though the science proposed.
Very credible and comprehensive Vision Mātauranga
elements are support and enhance to the research plan.
Existing relevant Māori knowledge has been fully
incorporated and helped shape the research. Distinct risks
and sensitivities are very well mapped and mitigated. New
knowledge and intellectual property of significant strategic
value to Māori interests is delivered. Very likely to attract
international attention.
The team have high level of capability and capacity to navigate
Māori stakeholders, networks, interest groups and their
expectations relevant to the science proposed. They have a very
strong track record of high value engagement with Māori capitals
relevant to the science proposed.
Very High
Progress and disseminate new knowledge
The new knowledge created, is potentially of international
significance and likely to attract interest accordingly, and the
means for dissemination proposed are wholly appropriate and
likely to be extremely effective.
Skills mix
The mix and level of skills in the team are exceptionally well
matched to achieving the excellent delivery of the research, related
activities and managing risks. This is an outstanding team.
Scientific risk , technical risk OR innovative approaches
Scientific and/or technical risk is high but this is substantially
exceeded by the additional proposed benefit that could be
achieved, or is outstandingly innovative with new approaches
that are ground breaking and are likely to attract international
attention.
Track record
The team has an international reputation for delivering excellent
research, related activities and managing risks. The team stands
comparison with the best performing teams elsewhere.
Excellent
Outstanding
Exemplary
Wholly appropriate
Very/extremely
large
Wholly credible
Exceeds best
practice standards.
Internationally
significant
Are well positioned in the domestic and international
research context
The recognition of related research and existing information is
of outstanding calibre, and is carried through into a research
design which takes full advantage of the opportunities for
leverage thus provided. The linkage to the relevant research
landscape is outstanding.
31
Score
Key words
Reference statements for attributes
Criteria a. Excellence
Sub-criterion i
Science
(weighted 25%)
Sub-criterion ii
Team
(weighted 25%)
Have a well-managed research plan and credible approach to
risk management
The research plan not only contains all of the expected
elements but the approach and detail are exemplary and give
high confidence that the research will be delivered excellently
and as planned. Scientific or technical risk management is
very likely to be effective. The proposed research is fully fit for
purpose.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion attributes
above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The research approach is the best match for unlocking Māori
innovation potential in regard to the science proposed.
Wholly credible Vision Mātauranga elements are fundamental
to the research plan design to a very high standard. Existing
relevant Māori knowledge is central to the research. Distinct
risks and sensitivities are very well mapped and mitigated.
New knowledge and intellectual property of very significant
strategic value to Māori interests is delivered on a large scale.
Extremely likely to attract international attention.
32
The team has outstanding capability and capacity to expertly
navigate significant and diverse Māori stakeholders, networks,
interest groups and their expectations relevant to the science
proposed. They have an extremely strong track record of high
value engagement with Māori capitals relevant to the science
proposed.
PART C: IMPACT
ASSESSORS
33
Assessing Impact
Your assessment of Impact in Research Programmes proposals must be based on the assessment criteria which
are taken from the Gazette Notice, and translated in the scoring guide.
Impact includes the sub-criteria Benefit to New Zealand and Implementation Pathway(s). Beneath each sub-criteria
sit attributes (Figure 6). Further details of the attributes can be found in Tables 8 and 9, pages 36-38.
All proposals must explain whether or not the research is relevant to the Vision Mātauranga policy. How they might
do this is discussed in Appendix C and Appendix D.
Table 7: Criteria – Research Programmes Impact
Impact
i. Benefit to New Zealand (weighted 25%)
Research should have direct and indirect benefits or effect on individuals, communities or society as a whole,
including broad benefits to New Zealand’s economic, social, human or natural capital. Assessment must have
particular regard to:
a. the credibility of the need for, scale and extent of potential benefits from the proposed research, science or
technology or related activities; and
b. the relevance and additional value they deliver to New Zealand.
ii. Implementation Pathway(s) (weighted 25%)
The credibility of implementation pathway(s) to deliver public benefits to New Zealand, not limited to a single firm or
end user, and the strength of relationships with relevant end users, beneficiaries, or stakeholders.
Figure 6: Layers of Criterion, sub-criteria and attributes – Research Programmes Impact
34
Benefit to New Zealand
A description of the benefits we are looking to invest in are included in the investment signals in the MBIE
Contestable Research Fund Investment Plan (Appendix A).
Your overall assessment of benefit should be driven by the combined size, breadth/extent, and credibility of the
benefits.
Credibility includes whether or not there are potential end-users, next users, or beneficiaries, and the quality of the
estimates of benefit. In assessing these aspects, the relevance and additional value provided by proposals also
needs to be considered.
You should also consider the breadth/extent of the proposed benefits, which may include aspects that go beyond
the direct benefits associated with the output of the research. In addition to those referred to in Appendix B these
can include:
 Benefits across multiple sectors
 Faster uptake of results in multiple areas
 Improved state of the environment
 Potential to scale up regional initiatives to nation-wide implementation
 Consistency of standards or approaches for regulators
 Improved social well-being
 Better use of resources
 Preservation or enhancement of cultural heritage and values
 More efficient processes
 Upskilling industry
 Support for emerging new sectors
 Creation of research platform which has upstream utility for new users
 Job creation e.g. via new start-ups
 Development of a cluster of businesses
 Multinational business attraction to or retention in New Zealand
 Protecting existing markets, or impact on New Zealand’s reputation
 Diversification of the economy.
If a sector is new or emerging, or only affects a narrow range of companies, then a credible vision of how the
research will contribute to building a sector that will transform New Zealand in the future, or how the technology
may affect multiple sectors in the future is needed.
Implementation Pathway(s)
The implementation pathway is expected to be appropriate to the stage of the research.
For example, if the research is more applied, a detailed description of implementation pathways is expected in
proposals and you would expect to see more end-user involvement.
Alternatively, if the research is at an earlier stage of development, then next users would be more relevant, and a
line of sight towards implementation should be visible, but not to the same extent as with more applied research.
In both cases, you there should be some indication that pathways have been given serious thought and that planned
implementation is not limited to a very generic ‘one size fits all’.
35
Interpreting Criteria, Sub-Criteria and Attributes
Table 8: Impact, Sub-criterion Benefit to New Zealand
Criterion a: Impact
Sub-criterion i: Benefit to New Zealand
“Research should have direct and indirect benefits or effect on individuals, communities or society as a whole, including broad benefits to New Zealand’s economic,
social, human or natural capital. Assessment must have particular regard to…”
Attribute
…the credibility of the need for,
scale and extent of potential
benefits from the proposed
research, science or technology
or related activities
Explanation
Scale’ means size, but this should be expressed in a way which sensibly reflects the type of outcome delivered by the research. For
example, an economic development project may be able to express scale in financial terms or degree of penetration of markets etc.
Social and environmental projects may use scalars of a different type, eg the level of impact on or significance for:
 reducing environmental effects
 resolving social problems
 developing more effective policies, etc.
In assessing estimates of scale it is important to apply the principle of additionality. ‘Additionality’ in this context means:
 value over and above that which would be expected to occur anyway through routine research investment by existing,
scientifically competent businesses or user organisations;
 value which exceeds the cost of doing the research (or value for money).
‘Extent’ means the coverage of the benefits, i.e. irrespective of scale whether benefits are concentrated in a narrow area (eg individual
organisations) or are of widespread potential impact (e.g. across sectors). Given a particular scale of impact, proposals that are of
widespread coverage should score more highly than those which are of narrow impact.
It is important to consider wider benefits, which could be quite diverse (see section above).
Both elements of ‘credibility’ are important. The first issue is the extent to which plausible end users or beneficiaries have been
identified (this is a general question as compared to the specific questions about implementation pathway which are addressed under
the Implementation Pathway criterion). The second issue is about the quality of the justification provided for the estimates of benefit.
The supporting analysis needs to be robust, the information on which the analysis is based should be authoritative, ie from credible
sources, and the logic and assumptions used in estimating the benefit(s) should make sense. Note that for earlier stage research, the
benefits may be more speculative or less certain than later stage research.
36
Criterion a: Impact
Sub-criterion i: Benefit to New Zealand
“Research should have direct and indirect benefits or effect on individuals, communities or society as a whole, including broad benefits to New Zealand’s economic,
social, human or natural capital. Assessment must have particular regard to…”
Attribute
…the relevance and additional
value they deliver to New
Zealand
Explanation
Investment signals are set in Appendix A and B
In assessing Impact, Assessors should consider the extent to which proposals will enable:
 potential impact for New Zealand
 more investment in research with higher (impact) risk and longer term horizons to impact
 better leveraging of wider investment and knowledge in New Zealand and overseas
 giving greater effect to Vision Mātauranga.
Areas of strategic importance are set out in Appendix A and B. Alignment with those areas can be taken to also mean relevance to New
Zealand’s current and future needs.
Proposed impacts should be aligned with one or more of the detailed outcome areas;
 Economic
 Environment
 Society; and
 Vision Mātauranga across all 3 of the above objectives.
The Investment Plan states that giving effect to Vision Mātauranga is a priority across all of the objective areas.
37
Table 9: Impact, Sub-criterion Implementation Pathways
Criterion b.: Impact
Sub-criteria ii.: Implementation Pathways
“The credibility of implementation pathway(s) to deliver public benefits to New Zealand, not limited to a single firm or end user, and the strength of relationships with
relevant end users, beneficiaries, or stakeholders.
Attribute
…The extent to which there are
credible implementation
pathway(s) to deliver public
benefits to New Zealand, not
limited to a single firm or end
user, and the strength of
relationships with relevant end
users, beneficiaries, or
stakeholders.
Explanation
The implementation pathway is expected to be appropriate to the stage of the research. If the research is more applied, a detailed
description of implementation pathways is expected in proposals. If the research is at an earlier stage of development, then next users
would be more relevant, and a line of sight towards implementation should be visible, but not to the same extent as with more applied
research.
A ‘credible implementation pathway’ is one which contains sufficient end or next-user specific information to confirm that the analysis
takes account of the characteristics of the end-use area and is not simply a generic, one-size-fits-all description. Appropriate
implementation pathways vary significantly across different sectors and types of research. There needs to be enough detail so that
pathways can be traced, and the role of each participant/end user is clear.
The impact delivery plan needs to contain the information referred to above. The information should be authoritative (derived from
credible and reliable sources), set out in a logical pattern and supported by good quality analysis and explanation.
‘Relevant’ in this context means that the end users etc. need to be credibly linked to the implementation of the projected impacts. If
there is no relevant link then the strength of the relationship is irrelevant and the score should be marked down accordingly.
The strength of the relationships can be measured against a range of parameters which includes the length of time over which the
relationship has been developed, the quality of the relationship (deep seated or superficial for example) and the level of commitment of
the stakeholders/end users/beneficiaries. The level of commitment can be gauged to some extent from the level of user-involvement in
steering the research (e.g. via an advisory group), commitment to specific actions or to providing various types of assistance. In some
cases the provision of co-funding may reflect the level of commitment of end users or stakeholders, in others co-funding may not be a
relevant factor (co-funding is not a requirement for proposals).
The information set out above should be included in the impact delivery plan.
38
Scoring guide for Research Programmes - Impact
The following guide contains these references:
Māori capitals - this term is used to refer to the collective material and non-material assets held by Māori. They include:
 Human capital - knowledge and knowledge systems, traits, talents etc which people and populations have
 Natural capital - land, water bodies, living things, taonga species, the environment
 Social capital - the social structures (such as whanau, hapu, iwi, urban Māori structures, community), norms (such as tikanga
Māori) and values in daily life (such as trust, manaaki, indigeneity)
 Financial capital - economic wealth, financial assets
 Physical capital - equipment and physical infrastructure e.g. digital networks, vessels, housing stock
 Large natural group - usually an iwi (tribe) or a cluster of hapu (sub-tribes) with a significant population, and a large distinctive claim area.
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
1
(Low
quality)
Not
present/credible/
relevant
Negligible
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
Relevance and additional value
Not relevant to areas of strategic importance to New Zealand as
set out in the Investment Plan. Provides no additional value and Is
not aligned with any of the future directions of investment in the
Investment Plan.
Credibility of implementation pathways
There are no credible implementation pathways and no
supporting impact plan.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are negligible and/or are
not credible at any level. Potential benefits are almost completely
captured by a small area of limited impact for New Zealand.
Relationship strengths
Proposed impact relationships are not relevant and/or not
credible.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The benefits of the proposed research are not distinctly relevant
to Māori innovation requirements.
39
The benefits of the proposed research are not distinctly
relevant to Māori innovation requirements.
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
2
Little
Very low
Unsatisfactory
3
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
Relevance and additional value
Marginally relevant and of little potential significance to one or
more areas of strategic importance to New Zealand. There is little
additional value in areas that are aligned with one or more future
directions of investment.
Credibility of implementation pathways
Implementation pathways are set out but are of poor
credibility and the supporting impact plan is not satisfactory.
Credibility, scale and extent of benefits
The potential benefits are very low and not significant within a
sector. The estimates of benefits are not very credible or well
explained, including the links to stakeholders, next users, end
users and beneficiaries able to achieve uptake. Potential benefits
are captured by a small area with limited spread of benefits.
Relationship strengths
The proposed impact relationships are not particularly
relevant and are of a standard that gives no confidence that
commitments based on those relationships will be realised.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
Some
The benefits of the proposed research have a limited amount of
relevance to Māori Innovation requirements.
The implementation pathway has limited or insufficient
arrangements for engaging with core Māori stakeholders who
are likely not to have been fully or correctly identified.
Relevance and additional value
Some relevance but of very limited potential significance to one or
more areas of strategic importance to New Zealand. There is some
additional value in areas that are aligned with one or more future
directions of investment.
Credibility of implementation pathways
The implementation pathways set out are of limited
credibility and the supporting impact plan is incomplete and
not convincing.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are low and are not
significantly spread across a sector. The estimates of benefits are
Relationship strengths
The proposed impact relationships are not convincingly
relevant and are of a standard that gives little confidence that
Limited
Low
Not convincing
40
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
not very credible or well explained, including the links to
stakeholders, next users, end users and beneficiaries able to
achieve uptake. Potential benefits are captured by a moderately
sized area with some spread of benefits.
4
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
commitments based on those relationships will be realised.
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The benefits of the proposed research have low to moderate
relevance to Māori. It’s unclear if any economic and/or social
and/or environmental benefits will be realised by any Māori
interests or if Māori contributions to the science are identified
and valued. The proposal is unconvincing in terms of delivering
new knowledge or intellectual property for Māori.
Relevance and additional value
Largely relevant and of some potential significance to one or more
areas of strategic importance to New Zealand. There is moderate
additional value in areas that are aligned with one or more future
directions of investment.
The implementation pathway has some significant gaps in
terms of Vision Mātauranga elements and/or inadequate
arrangements for engagement with and responding to core
Māori stakeholders who may not to have been fully identified.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are moderate and may
have some impacts within a sector or some impacts across several
sectors. The estimates of benefits have some credibility but the
supporting explanation is not completely convincing, including the
links to stakeholders, next users, end users and beneficiaries able
to achieve uptake. Potential benefits are captured by a diverse
areas with some breadth of spread of benefits.
Relationship strengths
The proposed impact relationships are largely relevant and
are of a standard that gives reasonable confidence that
commitments based on those relationships will be realised, at
least in part.
Some significance
Partly
Moderate
Reasonable
Some gaps/questions
41
Credibility of implementation pathways
The implementation pathways set out are partly credible but
are supported by an impact plan that has gaps and leaves
some key questions unanswered.
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
5
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The benefits of the proposed research are moderately relevant to
Māori achieving their goals and links have been satisfactorily
described. Moderate economic and/or social and/or
environmental benefits are likely to be realised to some Māori
interests. Māori contributions to the science have been identified
and are valued.
The implementation pathway has satisfactory arrangements
for engagement with well identified core Māori stakeholders.
Core Vision Mātauranga elements are well developed and
embedded e.g. satisfactory Māori participation in research is
evident, risks and sensitivities well mitigated, obviously
relevant tikanga Māori has been satisfactorily gauged and
described throughout the pathway, intellectual property
management approaches are capable of responding to
indigenous knowledge management issues/requirements if
raised.
Very
Relevance and additional value
Very relevant and of substantial potential significance to one or
more areas of strategic importance to New Zealand. There is
substantial additional value in areas that are aligned with one or
more future directions of investment.
Credibility of implementation pathways
The implementation pathways set out are credible, fit for
purpose and are supported by an impact plan that is
comprehensive and competent.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are large and will have
significant impacts within a sector or some impacts across several
sectors. The estimates of benefits are credible and well explained,
including the links to stakeholders, next users, end users and
beneficiaries able to achieve uptake. Potential benefits are
captured by many areas with significant spread of benefits.
Relationship strengths
The proposed impact relationships are wholly relevant and
are of a standard that gives good confidence that
commitments based on those relationships will be realised.
Substantial
Credible
Large
Wholly
Good ( confidence)
No gaps or questions
42
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
6
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The benefits of the proposed research are very relevant to Māori
achieving large or significant direct and indirect economic and/or
social and/or environmental benefits and are presented as such.
Māori contributions are valued and are central to the delivery of
benefits.
The implementation pathway is informed by comprehensive
engagement with a comprehensive range of well identified
Māori stakeholders who are participating in and contributing
to the research. A comprehensive range of Vision
Mātauranga elements are well developed and embedded
which are likely to include e.g. risks and sensitivities well
mitigated, the role of tikanga Māori throughout the pathway
has been identified, credible intellectual property agreements
are in place and respond to identified indigenous knowledge
management requirements.
Very substantial
Relevance and additional value
Very relevant and of very substantial potential significance to one
or more areas of strategic importance to New Zealand.
There is very substantial additional value in areas that are aligned
with one or more future directions of investment.
Credibility of implementation pathways
The implementation pathways set out are credible, fit for
purpose and are supported by an impact plan that is
comprehensive and answers all of the key questions.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are very large and will
have substantial impacts within a sector and/or significant impact
across several sectors. The estimates of benefits are very credible
and very thoroughly explained, including the links to stakeholders,
next users, end users and beneficiaries able to achieve uptake.
Potential benefits are captured by diverse areas with very
significant breadth of spread of benefits.
Relationship strengths
The proposed impact relationships are entirely relevant and
are of a high standard, giving full confidence that
commitments based on those relationships will be realised.
Very large
Comprehensive
Entirely
High
Full (confidence)
43
Score
Key words
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
7
(High
quality)
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
Indications for
identifying Vision
Mātauranga at this
scoring level
The benefits of the proposed research are highly relevant to
Māori achieving their strategic aspirations and presented in this
context. Very large or significant direct and indirect economic
and/or social and/or environmental benefits are likely to be
realised across large natural groupings or sectors. Māori
contributions are highly valued and are fundamental to the
delivery of benefits.
The implementation pathway is informed by very credible
engagement & strong value-exchanging relationships with
diverse Māori stakeholders which results in meaningful Māori
participation in, and contributions to, the research.
Appropriate tikanga has been identified throughout the
pathway and responsibilities allocated. Risks and sensitivities
are well mitigated. Intellectual property agreements align
strongly to indigenous knowledge management best practice
approaches.
Extremely (large,
relevant)
Relevance and additional value
Extremely relevant and of potentially transformative significance
to one or more areas of strategic importance to New Zealand.
There is very large additional value in areas that are aligned with
one or more future directions of investment.
Credibility of implementation pathways
The implementation pathways set out are completely
credible, fit for purpose and are supported by an impact plan
that is very comprehensive and of high quality, and is an
exemplar of what should be provided.
Credibility, scale and extent of benefits
The potential direct and indirect benefits are extremely large and
will have impacts that are nationally significant. The estimates of
benefits are highly credible and convincingly explained, including
the links to stakeholders, next users, end users and beneficiaries
able to achieve uptake. Potential benefits are captured across
multiple New Zealand sectors with outstanding spread of benefits.
Relationship strengths
The proposed impact relationships are extremely relevant and
are of the highest standard, giving total confidence that
commitments based on those relationships will be realised at
an exceptional level.
Completely
Highest
(quality/standard)
Total (confidence)
Exemplary
Vision Mātauranga
Assess how satisfactorily the proposal identifies and addresses appropriate Māori-relevant aspects within all sub-criterion
attributes above. The following indicates practices (indicative not absolute) you might see at this level of scoring.
44
Score
Key words
Indications for
identifying Vision
Mātauranga at this
scoring level
Reference statements for attributes
Criteria b: Impact
Sub-criterion i
Benefit to New Zealand
(weighted 25%)
Sub-criterion ii
Implementation Pathway(s)
(weighting 25%)
The benefits of this research are extremely relevant to Māori
achieving their strategic aspirations and presented in this context.
Extremely large or significant and potentially transformative direct
and indirect economic and/or social and/or environmental
benefits are likely to be realised across large natural groupings.
Māori contributions are very highly valued fundamental to the
delivery of benefits.
The implementation pathway is informed by exemplary
engagement with diverse Māori stakeholders which optimises
significant and meaningful Māori participation in, and
contributions to, the research. Appropriate tikanga has been
identified throughout the pathway and responsibilities
allocated. Risks and sensitivities are well mitigated.
Exemplary intellectual property agreements which model
international indigenous knowledge management best
practice are in place.
45
Appendix A
Investment Signals for 2016-19
Source Document: MBIE Contestable Research Fund Investment Plan 2016-19
46
47
Appendix B
Dimensions of Impact
Source Document: MBIE Contestable Research Fund Investment Plan 2016-19
48
Appendix C
Some guidance to help you strengthen your approach to Vision
Mātauranga
Source Document: Guidelines for Completing Proposals 2016 Science Investment Round
The Vision Mātauranga policy (Vision Mātauranga) aims to unlock the science and innovation potential within
Māori knowledge, resources, and people to assist New Zealanders to create a better future. Giving effect to
this policy is a priority across the fund’s economic, environmental and societal objectives; therefore Vision
Mātauranga must be addressed in your proposal.
This guidance is to help you consider Vision Mātauranga in planning your proposal given this should shape how
you develop the proposal, who should be involved, and therefore what content should be included in each of
the application sections. Your proposal should show how you have considered, identified and responded to
opportunities presented by relevant Māori knowledge, resources or people. This could include, for example,
explaining and evidencing:
 the steps you have taken to identify research opportunities relevant to Māori interests (collectives,
businesses and communities),
 which Māori interests will be involved, in what capacity, and the rationale for their selection
 what contributions or innovations you will draw from them and how those contributions are
integrated in the proposal
 how you propose to respond to the distinctive issues, needs and requirements of those Māori
interests
 which of the four Vision Mātauranga outcome benefits outlined in the Investment Plan you propose
to address and how.
Please consider if you have provided sufficient information to show:
 your analysis of Māori needs, opportunities or resources linked to government and other national
strategies, or specific strategies developed by relevant Māori interests
 how your proposal responds to relevant values, histories, relationships, rights, aspirations and
interests held by related Māori interests
 where your research is taking a generic approach, a Māori-centric approach, kaupapa Māori research
approach, or a mix across the proposal, and the rationale for this
 appropriate Māori voices and expertise relevant to the design of the proposal including the
methodology, methods, or applications especially if it includes kaupapa Māori research
 the agreed engagement methods or principles specific to the proposal, especially if you are proposing
work at the interface between knowledge systems
 specific and agreed Māori roles and responsibilities that could include Māori as: researchers, funders,
knowledge contributors, participants, end-users, partners, leaders, advisors, or governance members
 the specific commitments between your team and Māori eg decision-making, ownership of IP
 appropriate use of Māori characterisation
 that your processes, solutions, tools, frameworks and metrics in the implementation pathway are
relevant to Māori world views, knowledge and context
 the international contribution your research will make to the interface between knowledge systems
 how the above is being resourced and supported
 what risks are associated with the above and how they might be managed and mitigated.
If you think Vision Mātauranga is not relevant to your research, you should test this assumption with
independent advisors with relevant strategic Vision Mātauranga experience. You will need to provide evidence
to explain why you consider Vision Mātauranga is not applicable.
49
Appendix D
Assessing Vision Mātauranga Relevance
Vision Mātauranga is the science investment policy that applies to all our contestable funds. It seeks
to achieve benefits to New Zealand from the innovation potential of Māori knowledge, resources
and people. The outcomes being sought through the policy appear in the investment signals section
on pages 18-19 of the Investment Plan (Appendix A). Consistent with the goal of a more flexible and
responsive science system, in time we are looking at increasingly flexible and dynamic approaches
across the research being undertaken to accelerate unlocking Māori innovation potential.
The base proposition in our investment approach is that Māori have an interest in all research unless
the applicants provide a rationale to show that Vision Mātauranga is not applicable. We expect to
see conscious consideration of meaningful and evolving practice in how Māori-held knowledge,
resources and people are involved in research and its uptake to deliver excellence and impact.
A proposal must show how the applicant has considered, identified and responded to opportunities
presented by Māori knowledge, resources and people relevant to the research being undertaken.
The critical word here is ‘relevance’ since the Māori interests in, potential contributions to, and
outcomes sought from each proposal will be different. Vision Mātauranga-relevant content, its
significance in the various components of a proposal, and how you assess its contribution to the
overall scoring within each of the sub-criteria will vary for each proposal.
Vision Mātauranga is assessed within all of the sub-criteria. It is not a separate or additional
assessment.
As you assess proposals it is helpful to keep prompting yourself about the Vision Mātauranga
elements at each point. For example
 when assessing to what extent a proposal recognises related research or existing
information (within Research Proposals, Criteria 1 Science) ask yourself how well related
Māori research or knowledge has been recognised as part of the whole body of related
research or existing information
 when assessing to what extent a proposal has strength of relationships with relevant end
users, beneficiaries and stakeholders (within Research Proposals, the Implementation
Pathway sub-criteria) ask yourself which Māori end users, beneficiaries and stakeholders
have been identified, if relevant engagement approaches have been described, if they look
credible for Māori, if Māori stakeholders have been fully and correctly identified etc.
Ensure that in your scoring decision making you give weight, appropriate to the research approach
being taken, to the quality of Vision Mātauranga-driven elements in the proposal as well. The table
below provides a guide to scoring.
If a proposal does not address the Vision Mātauranga elements that might reasonably be expected
at a particular scoring level, the proposal should not be scored at that level. That is to say, unless the
proposal presents a sound case for not including Vision Mātauranga, reasonable Vision Mātauranga
elements relative to each scoring level must be evident.
50
Understanding what we’ve asked applicants
Applicants should have undertaken an analysis of Māori needs and opportunities, and have their
thinking shaped by that process, before developing the research proposal. In doing so, they can
propose that Vision Mātauranga is not relevant. Reflecting this in a statement such as: ‘Vision
Mātauranga is not applicable’ is insufficient and provides no evidence about the rationale for this
premise. A more credible approach will identify how this premise has been established, for
example:
“Vision Mātauranga is not relevant to our research because there is not a distinctive Māori
need being addressed, or distinctive knowledge base being drawn from. There are Māori
interests in uptake of the research and we have tested options with xxx identified on the
basis of xxx who confirm they could be interested once xxx.”
Where you consider the applicants have NOT provided a credible rationale for discounting Vision
Mātauranga, you should identify that in your comments. If you can see opportunities to address
Māori needs or draw on Māori knowledge to provide greater value to what has been proposed, then
also capture these in your comments. For each criteria, you will need to consider the specific Māorirelevant components within the overall factors considered in each criteria as outlined in the Scoring
Guide.
Understanding relevance
There are two approaches that can assist in determining what Māori-relevant content is important in
a proposal. Firstly, applicants were also asked to consider the material outlined in Appendix C in
developing their proposals. You can look to where this material is reflected and evidenced in the
proposal, using your judgement to identify how important these elements are and how convincingly
they have been addressed. A good proposal will typically reflect clear, specific and relevant Vision
Mātauranga components that enrich the overall approach being taken.
Secondly, research approaches relevant to Māori interests are broad and varied. They can be
characterised as shown in the five columns of Table 10 or can contain elements of each category. If
the research type is not specified, you will get a feel for it from the proposal. The centrality of Māori
knowledge grows as you move from left to right in Table 10 as does the importance of Māori
characterisation, voice, methodology, and methods. Māori expectations from, and investment in,
shaping the research approach and proposal content also generally increases left to right as does the
influence of Māori-defined processes.
However, each research approach serves in different ways and each proposal should therefore
contain those elements of specific Māori-relevance that support the type of research, its focus and
the outcomes it proposes to deliver.
We expect excellence and impact in every proposal. The approaches outlined in Table 10 help to
identify where relevant Vision Mātauranga content is likely to appear in different assessment criteria
and the type of influence it may have. Shallow, unsubstantiated, or misdirected attempts to address
Vision Mātauranga can tend to reflect activity rather than advancement, irrespective of which
category the research can be defined by. As an assessor, you need to be confident that you can
navigate the difference.
What to look for
The validity of the Vision Mātauranga content in a proposal is context specific. It depends on the
issue being addressed, the contribution of Māori-held knowledge, the processes and principles
51
described, and the outcomes sought. Your assessment will involve looking for clear identification,
inclusion and evidence of relevant Māori-held knowledge or resources in each proposal and how
they contribute to its excellence and impact. Applicant responses should be specific, reflect integrity
and be fit for purpose given the focus of the research and its potential impact(s), as should the
broader characteristics of the proposal.
The assessor training will include further consideration of Vision Mātauranga with examples.
These two questions will guide your assessment of Vision Mātauranga in Smart Ideas proposals,
within the excellence assessment criteria:
1. Are there opportunities for Māori to contribute to or benefit from the research (ie, is the Vision
Mātauranga policy relevant to the impact of the proposal?)
2. Does this proposal propose a credible approach to addressing Vision Mātauranga?









THINGS TO CONSIDER WHEN ASSESSING VISION MĀTAURANGA RELEVANCE
Responses that are credible for some research may not be credible for others depending on the
specific research programme and the context of the research being proposed.
Proposals should always contain reference to Vision Mātauranga no matter what the level of
relevance, ie there needs to be sufficient indication that it has been considered, if only to state it is
not relevant and provide rationale to support this assumption.
Proposals are more convincing when they contain credible rationale and specific details. For
example which Māori organisations should be involved and why; what distinctive knowledge or
opportunity is being addressed etc. Generalised and vague references may suggest poorer
understanding of how Vision Mātauranga is to be successfully implemented and, typically, the less
equipped the team will be to deliver.
Do not assume that a researcher, who happens to be of Māori descent, knows about Vision
Mātauranga or that a non-Māori researcher won’t. A strong proposal will specify the kind of skills,
knowledge, and roles of the team members who will progress the Māori relevant aspects of the
proposal and how that capability will be utilised.
Do not accept that just mentioning Māori in a proposal is a credible response to the Vision
Mātauranga policy. Consider whether what is being stated or proposed is credible, sufficient, and
likely to be effective for delivering the full value of the science.
If Māori have been identified as partners, end users or beneficiaries of the research, sufficient detail
must be provided to show appropriate engagement in delivery of the impact.
In considering Vision Mātauranga relevance, intellectual property associated with research involving
indigenous species or other matters is often overlooked. We encourage you to identify how
intellectual property has been addressed in the context of the proposal.
It is highly likely that research involving any aspect of natural resources (ie any proposal addressing
the Fund’s environmental objective) will be relevant to Māori given the role of Māori as
environmental stewards (Kaitiaki) and decision-making partners under New Zealand Law. The extent
and nature of that relevance is a matter of judgement.
We encourage you to provide sufficient detail in your comments to support your assessment of how
well a proposal has addressed Vision Mātauranga. For example…“the proposal appears to have
failed to recognise relevance to Māori and intellectual property implications arising from use of
indigenous species for commercial application” rather than …“Vision Mātauranga is not well
addressed”.
52
Table 10: New Zealand spectrum of research approaches relating to unlocking Māori potential and the likely implications for assessing proposals
Research
defined
categories
Research not specifically
involving and not
specifically relevant to
Māori
Research specifically
relevant to Māori
Characteristics
Research benefits NZ
generally with no additional/
specific impact on Māori and
distinctive Māori
participation has not been
sought.
Research results may
contribute to unlocking the
innovation potential of
Māori and some Māori
participation may
contribute to maximising
the benefit for New
Zealand.
Māori may be involved as
participants, researchers,
investors, stakeholders on
the same basis as their nonMāori counterparts.
Māori may:
 provide advisory input
 provide specific uptake
potential as stakeholders
 be participants.
Research Involving Māori
Research results will specifically
contribute to unlocking the
innovation potential of Māori.
Māori end user involvement is
substantive from ideas inception
to outcome delivery to a credible
pathway shaped through that
involvement.
Māori may:
 provide advisory input or
governance
 be members of the research
team
 be participants or subjects
whose data may be sought and
analysed.
Māori-centred research
Research results contribute specifically to unlocking
the Māori innovation potential or the research
addresses an issue distinct to Māori knowledge,
people or resources.
Māori stakeholder involvement is substantive from
idea inception to outcome delivery or a credible
pathway shaped through that involvement.
The research focus is more likely to meet some
expectations and quality standards set by Māori.
Through a variety of roles, Māori are likely to:
 shape the research design and analysis
 be significant participants
 be involved as senior members of research teams
 direct the research
 open access to taonga (treasures)
 provide substantive knowledge and end user
involvement that may involve access to networks
and communities
 be significant implementation agents
 be significant and direct beneficiaries of new
knowledge and IP.
Kaupapa Māori Research
Research directly contributes to unlocking the
innovation potential of Māori knowledge, people and
resources. It addresses a distinct Māori issue and the
analytical framework may be shaped by a Māori
world view. Māori research methodologies are used.
Māori end user involvement is substantive from idea
inception to outcome delivery or a clear pathway is
planned to specifically unlock Māori innovation
potential.
Core to this approach is:
 Māori voice and characterisation
 Tikanga Māori and Māori values guide and
permeate the research
 research that is very targeted to expectations and
quality standards set by Māori
 creation of new Māori knowledge and IP.
Māori could:
 initiate the proposal, define the research problem
and design the proposal
 identify the outcomes and accountabilities
 lead the research team and may constitute the
entirety of it
 be exclusive participants.
Assessment is on a proposal-by-proposal basis, but specific Māori-relevant content is likely to influence as follows:
Implications for
assessment
No specific focus is expected
on specific Māori issues,
knowledge or resources
although there may be
elements of Māori
participation or investment.
Specific Māori-relevant
content is more likely to be
important in assessing the
impact criteria than the
excellence criteria.
Specific Māori-relevant content is
more likely to influence the
research approach in the Science
sub-criteria and the impact
assessment criteria. It may
influence Team assessment (mix
and skills) but is less likely to be
significant that for Māori-centred
research.
53
Distinctive Māori input is likely to have shaped the
research, may be substantive, and may therefore be
important in assessing both the excellence and impact
criteria.
Distinctive Māori content is likely to be critical in both
the excellence and impact criteria.
Scoring criteria should be considered through science
and relevant Māori perspectives.