Economic Impact Assessment of Nine
clusters of FRDC Completed RD&E
Projects
Project Number: 2011/504
Final Report for Round 2 Evaluations
to
Fisheries Research and Development Corporation
27 August 2015
by
Peter Chudleigh, Andrea Bath, Mark Hibbard and Talia Hardaker
Agtrans Research
AGTRANS RESEARCH
Suite 36, Benson House, 2 Benson Street,
Toowong, Brisbane, Australia
PO Box 385, Toowong Qld 4066
Telephone: (07) 3870 4047 or (07) 3870 9564
Facsimile: (07) 3371 3381
E-mail: [email protected]
Agtrans Research
Page i
Table of Contents
Executive Summary ........................................................................................................... iii
Abbreviations ...................................................................................................................... v
1. Introduction ..................................................................................................................... 6
2. Methods........................................................................................................................... 8
3. Results ........................................................................................................................... 12
4. Discussion ..................................................................................................................... 17
5. Conclusions ................................................................................................................... 23
Acknowledgments............................................................................................................. 24
References ......................................................................................................................... 24
Agtrans Research
Page ii
Executive Summary
The FRDC Five Year Plan (FRDC, 2010) divides its investment into 14 themes. The
Corporation has set an impact assessment objective of evaluating in economic terms all
projects in all themes over the five year period commencing in March 2011. Projects
were defined as those having been completed (or substantially completed) in the five
years prior to 2011.
The first population of projects was defined in January 2011 and projects were placed
into each of the 14 themes. Some of the 14 themes had far more projects than others and
those themes with high numbers of projects were divided into two or more clusters of
projects. This resulted in 25 clusters across the 14 themes.
Evaluation of the first eight clusters was completed in October 2012 and the evaluation
report provided to FRDC. In the second 18 months period (i.e. July 2013 to December
2014) a further nine of the 25 clusters were subjected to impact assessment resulting in
the current summary report. The impact assessment used cost-benefit analysis (CBA) to
estimate investment criteria for each cluster of projects.
The nine clusters evaluated in this second round comprised:
1 cluster from theme 1 (Biosecurity and aquatic animal health),
1 cluster from theme 4 (Ecologically sustainable development),
1 cluster from theme 5 (Governance and regulatory systems),
1 cluster from theme 6 (Resource access and allocation),
4 clusters from theme 7 (Production, growth and profitability), and
1 cluster from theme 8 (Consumers, products and markets).
Each cluster comprised between seven and 50 projects. As the entity for evaluation
reporting was the cluster, costs and benefits for each cluster had to be built up from
information on the individual projects in the cluster. This was achieved largely through
access to the FRDC data base and contact with Principal Investigators of projects,
government agencies and industry personnel.
The value of total funding for each of the nine clusters (FRDC plus other investment)
ranged from $3.7 million to $44.2 million, with a total value for all clusters of $137.4
million (in nominal $ terms). The FRDC nominal investment in the nine clusters analysed
varied for each cluster (32.6% to 62.4% of the cluster total). FRDC contributed 40% of
the total nominal investment across all nine clusters.
The majority of the benefits identified from the nine clusters (202 projects in total) were
economic in nature although significant numbers of environmental and social/community
benefits also were identified. The major beneficiary of the impacts of the nine clusters of
research investment has been the fishing industry (51% of the number of benefits
identified), with 43% of the identified number of benefits being public in nature and 6%
to overseas interests. The results demonstrate the significant spillovers of benefits to the
Agtrans Research
Page iii
public sector from research targeted at the fishing industry. Insignificant spillover
benefits to other Australian industries were identified.
A number of the identified benefits were valued, and investment criteria for each of the
clusters of investment calculated. Benefits were estimated over 30 years from the final
year of investment in the research. Benefits and costs were expressed in 2013/14 dollar
terms, and discounted to 2013/14 using a discount rate of 5%.
The net present values (NPVs) for total investment for the individual clusters ranged
from $6 million to $124 million and the Benefit-Cost Ratios (BCRs) ranged from 1.8:1 to
3.9:1. FRDC investment made up 37.9% of the total investment in present value terms,
and the NPVs for FRDC investment in individual clusters ranged from $4 million to $60
million.
When all nine clusters are aggregated, the BCR for the $266.5 million investment in the
nine clusters (present value terms) was 2.6:1, with the Present Value of Benefits (PVB) of
$684.0 million and an NPV of $417.5 million. For the FRDC investment of $101.0
million (present value terms), the NPV was $170.2 million.
Agtrans Research
Page iv
Abbreviations
BCR
CBA
CRRDC
FRDC
IRR
NPV
PVB
PVC
R&D
YTK
Benefit-Cost Ratio
Cost-Benefit Analysis
Council of Rural Research and Development Corporations
Fisheries Research and Development Corporations
Internal Rate of Return
Net Present Value
Present Value of Benefits
Present Value of Costs
Research and Development
Yellow Tail Kingfish
Agtrans Research
Page v
1. Introduction
The Fisheries Research and Development Corporation (FRDC) required a series of economic
impact assessments to be undertaken, commencing in 2011, on its whole portfolio over a five
year period. The impact assessments were to serve two purposes:
 They will provide a key input into FRDC’s assessment of its program performance
against its Five Year plan and inform its future directions; and
 They will provide FRDC’s input to an overall performance assessment of the Rural
R&D Corporations being compiled by the Council of Rural Research and
Development Corporations (CRRDC).
The FRDC has five programs and within these programs there are 14 themes (FRDC, 2010)
(see Table 1.1). All projects completed in the five years prior to 2011 are to be evaluated over
several years in three rounds. This report presents the second round of evaluations which
includes the evaluation of nine clusters of investment (see Table 1.2). Economic assessment
reports on each of the nine clusters are contained in Appendices 1 to 9 of this report.
Table 1.1: FRDC Programs and Themes
Programs
Program 1: Environment
Program 2: Industry
Program 3: Communities
Program 4: People
development
Program 5: Extension
and adoption
Theme
Number
1
2
3
4
5
6
7
8
9
10
11
12
13
14
Theme Name
Biosecurity and aquatic animal health
Habitat and ecosystem protection
Climate change
Ecologically sustainable development
Governance and regulatory systems
Resource access and allocation
Production, growth and profitability
Consumers, products and markets
Value from aquatic resources
Resilient and supportive communities
Leadership development
Workforce development
Innovation skills
Extension and adoption
Number of
Clusters
3
3
1
3
1
1
4
2
1
1
1
1
2
1
6|Page
Table 1.2: Clusters of Investment Included in Second Round of Evaluations
Appendix Number
1
2
3
4
5
6
7
8
9
Cluster of Investment Evaluated
Theme 1: Biosecurity and aquatic animal health (YTK, abalone
and oysters)
Theme 4: Ecologically sustainable development (Management)
Theme 5: Governance and regulatory systems
Theme 6: Resource access and allocation
Theme 7: Production, growth and profitability (Enhancement,
nutrition and health)
Theme 7: Production, growth and profitability (Genetics/selective
breeding)
Theme 7: Production, growth and profitability (Systems and
production)
Theme 7: Production, growth and profitability (Profitability)
Theme 8: Consumers, products and markets (Consumers/markets,
products and supply chains)
Section 2 of the report provides a brief summary of the methods used in the cluster
definitions, selection and analyses. Section 3 reports both the qualitative and quantitative
results for the evaluations. A discussion of the results is provided in Section 4. Conclusions
are provided in Section 5.
7|Page
2. Methods
2.1 Cluster Definition and Selection
The population of projects completed in the five years prior to 2011 were organised
according to the theme in which they belong. Where there were many projects in a theme,
FRDC identified subgroups of like projects within the theme. Agtrans Research then used
FRDC’s grouping to split large themes into two or more clusters for the purpose of the
evaluations.
Twenty-five clusters were defined and were to be evaluated in three rounds. The first round
included the evaluation of eight clusters. The second round includes the evaluation of nine
clusters and the third round will include the evaluation of the remaining eight clusters. The
population of projects in each theme was updated for the second round assessments. For the
third round, the population of projects within each theme is likely to be updated again to
accommodate projects completed since the last population definition.
The nine clusters of investment evaluated in the second round are provided in Table 1.2.
Table 2.1 provides further information on the each of the nine clusters.
Table 2.1: Nine Clusters of Investment Analysed in Round Two
Cluster Name
No. of
projects
FRDC
Program
Total FRDC
value of
cluster
(nominal $m)
Theme 1: Biosecurity and aquatic
animal health (YTK, abalone and
oysters)
Theme 4: Ecologically sustainable
development (Management)
Theme 5: Governance and
regulatory systems
Theme 6: Resource access and
allocation
Theme 7: Production, growth and
profitability (Enhancement,
nutrition and health)
Theme 7: Production, growth and
profitability (Genetics/selective
breeding)
Theme 7: Production, growth and
profitability (Systems and
production)
Theme 7: Production, growth and
profitability (Profitability)
13
1
2.11
Total value
(FRDC +
other)
(nominal
$m)
4.38
50
1
12.96
26.63
22
2
5.75
15.19
7
2
2.05
3.72
15
2
4.99
12.61
18
2
5.33
17.97
28
2
14.42
44.18
14
2
2.48
3.97
8|Page
Theme 8: Consumers, products
and markets (Consumers/markets,
products and supply chains)
Total
35
2
4.37
8.77
202
-
54.46
137.42
The financial contribution that FRDC made to the nine clusters as a proportion of total
funding for the clusters was 39.6% (in nominal terms). The leverage ratio for FRDC funding
was therefore approximately $1.50 of other funding for every $1 of FRDC funding.
2.2 Individual Analyses
Steps Undertaken
Each cluster of investment in the Round Two evaluations was subject to the following steps
consistent with the guidelines developed and provided by the CRRDC (2007):
1. Information from the original project proposals, final reports, and any progress reports or
other relevant reports and material was assembled. Direct access to the FRDC project
management database was provided by FRDC. This facilitated minimal involvement by
FRDC personnel in locating, copying and sending electronic or hard copy information to
the consultants.
2. An initial description of the project background, objectives, costs, activities and outputs,
outcomes and benefits was drafted. Additional information needs were identified.
3. Telephone and/or email contact was made with Program Managers, Principal
Investigators and/or industry personnel, and if appropriate, the draft or a portion of the
draft sent to that person for perusal and comment, together with specific information
requests.
4. The actual and potential benefits from the investment in each cluster were identified and
described in a triple bottom line context. Some of these benefits were then individually
valued, or grouped together and valued.
5. Information provided included categorisation of beneficiaries, confidence in the analyses,
the relationship to National and Rural Research Priorities and FRDC’s Key Performance
Indicators. Observations and implications for future investment and evaluation were
summarised.
6. Some analyses proceeded through several drafts, both internally within the project team
as well as externally via FRDC personnel and others.
7. Some final drafts were passed by some industry personnel for comment.
The variation in the range of benefits identified from the nine clusters makes it difficult to
generalise with respect to the methods of quantifying the impacts. In a wider context, the
factors that determined the resulting investment criteria included:




The cost of the R&D.
The magnitude of the net benefit per unit of production or unit of natural
resource/environment affected; this net benefit per unit also took into account any
costs of implementation.
The quantity of production or natural resource/environment affected by the R&D, in
turn a function of the size of the target audience or area, and the level of initial and
maximum adoption ultimately expected, and level of adoption in the intervening
years.
The discount rate.
9|Page




The time elapsed between the R&D investment and commencement of the accrual of
benefits.
The time taken from first adoption to maximum adoption.
An attribution factor was applied when the specific project or investment considered
was only one of several pieces of research or activity that had contributed to the
outcome being valued. The attribution factor was sometimes estimated by the
proportion of the total costs contributed by the project being evaluated. However,
where specific costs were not available, an estimate of the percent contribution to
benefits from the original investment was made and applied to the benefits.
The probability of an R&D output, commercialisation etc. occurring was applied
when the research was not complete or when some further investment was required
before the outputs of the research were translated into outcomes and extended to the
industry.
Counterfactual
Defining the ‘without R&D’ scenario or baseline to assist with defining and quantifying
benefits is often one of the more difficult assumptions to make in investment analyses. The
‘without’ scenario in the evaluations is referred to as the counterfactual.
The important issue is that the definition of the counterfactual scenario is made as
consistently as possible between analyses. In some evaluations the ‘with’ scenario included a
higher probability of a benefit being delivered than with the counterfactual scenario.
Cash Flows and Benefit Periods
The annual cash flows for benefits were developed for both ‘with investment’ and for
‘without investment’ scenarios. This entailed subtracting any implementation or adoption
costs from the gross benefits estimated for each scenario. The annual gain due to the research
was obtained by subtracting the net benefit without the research from the net benefit with the
research. A constant degree of conservatism was pursued when finalising assumptions
concerning benefits. Cash flows for benefits were expressed in 2013/14 dollar terms.
The annual costs of the research were used to derive the cash flow for research investment in
the cluster. Costs for each R&D project included the cash contributions of FRDC, as well as
any other resources (cash and in-kind) contributed by third parties (e.g. researchers or
industry). All research costs were converted to 2013/14 dollar terms via the consumer price
index.
A 30 year time frame for benefits was used in all analyses. This was applied in the following
manner. Year zero was assumed to be the year in which the last investment in the cluster
occurred. Benefits were then estimated annually up to and including year 30, unless benefits
were assumed to cease earlier. Hence future expected benefits were also included.
Research investment costs occurred earlier than year zero for all clusters. An example of the
relevant cash flows that were developed is shown in Table 2.2 below. In some cases, it was
possible for benefits to commence prior to year zero, that is, during the period of the
investment.
10 | P a g e
Table 2.2: Example for Annual Cash Flows for Research Costs and for Benefits
Year ended 30 June
2001
2002
2003
2004
2005
2006
2007
2008
Year Number
-6
-5
-4
-3
-2
-1
0
1
Cash Flow Research Costs
x
x
x
x
x
x
x
Cash Flow Benefits
x
x
x signifies a dollar value for that year; a blank signifies no research cost or benefit for that year
…
…
…
…
2038
30
x
x
x
Investment Criteria
The Present Value of Benefits (PVB) and Present Value of Costs (PVC) were used to
estimate investment criteria of Net Present Value (NPV), Benefit-Cost Ratio (BCR) and
Internal Rate of Return (IRR) at a discount rate of 5%. The PVB and PVC are the sums of the
discounted streams of benefits and costs. The NPV is the difference between the Present
Value of Benefits (PVB) and the Present Value of Costs (PVC). The BCR is the ratio of the
PVB to the PVC. The IRR is the discount rate that would equate the PVB to the PVC, thus
making the NPV zero and the BCR 1.
Sensitivity analyses were undertaken in most cases for those variables where there was
greatest uncertainty or for those that were thought to be key drivers of the investment criteria.
Some identified benefits were not valued, mainly due to:
 A weak or indirect relationship existing between the research investment and the actual
R&D outcomes and associated benefit.
 Evidence or examples demonstrating the existence of the benefits were lacking.
 The magnitude of the value of the benefit was thought to be only minor.
Where benefits were found to be both economic and environmental/social in nature, valuation
focussed on the economic benefits to industry and government. Assumptions used in these
valuations were clearly defined. Other environmental and social benefits were described in
qualitative terms.
11 | P a g e
3. Results
3.1 Qualitative Results
Summary of Qualitative Benefits
Table 3.1 summarises the identified principal benefit types from the nine clusters, and
categorises each benefit as economic, environmental or social. Not all of the clusters
demonstrated benefits from each category. More detailed economic, environmental and social
benefits identified for each of the nine clusters are given in Appendices 1 to 9.
Table 3.1: Summary of Benefits in Triple Bottom Line Format
Economic
Increased yields/avoided yield
loss
Environmental
Enhanced sustainability of
ecosystems and the
environment; reduced
biodiversity loss
Reduced environmental
impact of production
Avoidance of restrictions on
aquaculture expansion /
development of emerging
aquaculture industries
Productivity gains via higher
Improved natural resource
growth rates and lowered
management by aquaculture
production costs
industries
Increased technical and
scientific capacity
Increased industry capacity
(e.g. prepare for disease
outbreaks)
Increased sustainability of wild
catch fisheries
Social
Increased technical,
scientific, and research
capacity
Improved research resource
allocation/public policy and
governance
Reduced management costs
to government
Avoided social costs
Increased utility for
recreational fishers and
consumers
Improved community
wellbeing through increased
aquaculture farm income and
associated off-farm
expenditure
Increased access to resources
(or avoided reduced access) by
both wild catch and the
aquaculture industry
Improved research resource
allocation
Higher product quality/
increased demand for seafood
products
Reduced fishery management
costs
Increased net profits to
industry
12 | P a g e
The benefits identified in each of the nine clusters (presented in Appendices 1-9) have been
further classified by type of beneficiary (see Table 3.2). The number of identified benefits
falling into each category is indicated in Table 3.2, and provides a subjective assessment of
the distribution of benefit types. The percentage for each category is presented in brackets
and represents the proportion of the 63 benefits identified.
Table 3.2: Number and Category of Benefits Identified for all Clusters
Fishing
industry
Economic
33
Environmental 0
Social
0
Total
33 (52%)
Other
industries
0
0
0
0
Spillovers
Public
2
10
14
26 (41%)
Total
Overseas
1
0
3
4 (6%)
36 (57%)
10 (16%)
17 (27%)
63 (100%)
Table 3.2 presents a count of the number of benefits. However, Table 3.3 presents a
weighted count of the number of benefits falling into each category. In the individual cluster
reports, each of the identified benefits was weighted using a star system (where *** was a
major contribution; ** was some contribution; and * was a minor contribution). Table 3.3
presents a ‘count’ of the number of stars falling into each category.
Table 3.3: Subjective Magnitude of Contribution of Categories of Benefits Identified for All
Clusters (a)
Fishing
industry
Economic
68
Environmental 0
Social
0
Total
68 (65%)
Other
industries
0
0
0
0 (0%)
Spillovers
Public
3
12
18
33 (31%)
Total
Overseas
1
0
3
4 (4%)
72 (69%)
12 (11%)
21 (20%)
105 (100%)
(a) Figures represent total number of stars in each benefit category
3.2 Quantitative Results
Investment Criteria for Clusters
Table 3.4 presents the investment criteria for total investment in each of the nine clusters
analysed at a 5% discount rate, and over a 30 year time period from the final year of
investment in the cluster. Table 3.5 presents the investment criteria for the FRDC investment
alone. The latter criteria have been estimated by using the FRDC costs only and assigning
part of the total benefits to the FRDC investment by multiplying total benefits by the
proportion of total investment contributed by FRDC. Further details on each of these
investments and the associated results are provided in the individual analysis reports in the
appendices. It is evident from the results there is a wide range in the investment criteria
across the nine clusters of investment. Box 1, following Tables 3.4 and 3.5, helps to explain
how the investment criteria should be interpreted.
13 | P a g e
Table 3.4: Investment Criteria for Nine Clusters of Investment
(5% discount rate; total investment)
Cluster
Theme 1: Biosecurity and aquatic animal health
(YTK, abalone and oysters)
Theme 4: Ecologically sustainable development
(Management)
Theme 5: Governance and regulatory systems
Theme 6: Resource access and allocation
Theme 7: Production, growth and profitability
(Enhancement, nutrition and health)
Theme 7: Production, growth and profitability
(Genetics/selective breeding)
Theme 7: Production, growth and profitability
(Systems and production)
Theme 7: Production, growth and profitability
(Profitability)
Theme 8: Consumers, products and markets
(Consumers/markets, products and supply chains)
All 9 Clusters
PVB
($m)
29.87
PVC
($m)
7.80
NPV
($m)
22.07
173.51
49.39
110.52
15.70
50.63
BCR
3.83
IRR Contribution
(%) by FRDC (%)
14.5
46.3
124.13
3.51
16.4
47.3
28.65
8.03
23.13
81.88
7.67
27.50
3.86
1.96
2.19
17.9
9.5
11.3
35.6
49.8
37.5
85.48
39.00
46.48
2.19
9.1
29.7
174.98
87.61
87.37
2.00
10.1
31.2
12.98
7.05
5.93
1.84
9.5
60.0
30.37
15.88
14.49
1.91
25.5
50.3
684.04 a
266.53a
417.51a
2.57b
-c
37.9d
a
Total for all 9 clusters
Weighted average of 9 clusters; calculated by dividing $684.04m by $266.53m; simple average was 2.59.
c
Weighted average IRR for 9 clusters was not calculated; a simple average for IRR is not presented
d
Calculated by dividing total PVC for FRDC investment only in all 9 clusters, 101.02m (see Table 3.5) by
266.53m
b
Table 3.5: Investment Criteria for Nine Clusters of Investment
(5% discount rate; FRDC investment only)
Investment
Theme 1: Biosecurity and aquatic animal health
(YTK, Abalone and Oysters)
Theme 4: Ecologically sustainable development
(Management)
Theme 5: Governance and regulatory systems
Theme 6: Resource Access and allocation
Theme 7: Production, growth and profitability
(Enhancement, Nutrition and Health)
Theme 7: Production, growth and profitability
(Genetics/Selective Breeding)
Theme 7: Production, growth and profitability
(Systems and Production)
Theme 7: Production, growth and profitability
(Profitability)
Theme 8: Consumers, products and markets
(Consumers/markets, products and supply chains)
All 9 Clusters
PVB
($m)
14.12
PVC
($m)
3.61
NPV
($m)
10.51
BCR
3.91
IRR
(%)
15.0
83.58
23.36
60.22
3.58
17.0
40.91
8.34
19.59
10.19
4.00
8.67
30.72
4.34
10.92
4.01
2.09
2.26
19.9
10.6
12.0
25.35
11.60
13.75
2.19
9.1
56.20
27.37
28.83
2.05
10.5
7.97
4.23
3.75
1.89
9.9
15.18
7.99
7.18
1.90
24.2
271.24a 101.02 a
170.23a
2.68b
-c
a
Total for all 9 clusters
Weighted average of 9 clusters; calculated by dividing $271.24m by $101.02m
c
Average weighted IRR for 9 clusters was not calculated; a simple average IRR was not estimated.
b
14 | P a g e
Box 1: Interpreting Investment Criteria
The concept of discounting - Discounting refers to the application of a ‘discount rate’ that
takes account of the time value of money. That is, individuals value a financial return
tomorrow at a higher rate than a financial return in one year’s time. Future values are
therefore ‘discounted’ to reflect this. A discount rate of 5% has been used in these analyses,
as nominated by the CRRDC.
Present Value of Benefits (PVB) – Refers to the sum of the discounted annual net benefits
from the investment (allowing for any costs to the adopter) over the time period from the
first year of investment, until 30 years after the final year of investment. For the Theme 1
cluster (YTK, abalone and oysters) this value was $29.87 million.
Present Value of Costs (PVC) - Refers to the sum of the discounted annual R&D costs
invested in the research cluster. For the Theme 1 cluster (YTK, abalone and oysters) this
value was $7.80 million.
Net Present Value (NPV) - The discounted value of the benefits of an investment less the
discounted value of the costs, i.e. PVB minus PVC. For the Theme 1 cluster (YTK, abalone
and oysters) this value was $22.07 million. This investment criterion gives an indication of
the size of the benefits, but does not provide a rate of return to the investment.
Benefit-Cost Ratio (BCR) – The ratio of the PVB to the PVC. For the Theme 1 cluster
(YTK, abalone and oysters) this value was 3.83 to 1. This investment criterion gives an
indication of the rate of return and can be interpreted as for every dollar of investment,
there is a return of 3.83 dollars of benefits. Unlike the NPV, it provides no indication of
the magnitude of the benefits.
Internal Rate of Return (IRR) - The discount rate which would have to be applied, for
an investment to have a NPV of zero, i.e. where present value of benefits = present value
of costs. For the Theme 1 cluster (YTK, abalone and oysters) cluster this value was 14.5%.
This criterion gives an indication of return, and can be interpreted as the investment
achieving a return of 14.5% per annum over the assumed time period. The timing of
benefits is a major influence on the IRR, as the earlier the benefits are achieved, the higher
the discount rate needs to be to achieve a net return of zero.
Difference between results for Total Investment and FRDC Investment – Table 3.4
above presented the investment criteria when the total investment in the R&D evaluated is
considered. This includes investment in the R&D by FRDC, the research provider, and any
other investment partners. Table 3.5 presented the investment criteria when FRDC
investment only is considered. It is assumed that the proportion of the total benefits that
accrue to FRDC is equivalent to the proportion of total investment provided by FRDC.
Any differences in BCRs and IRRs between the total investment and FRDC investment are
usually due to the differences in the timing of investment by the different parties.
Confidence Ratings
The results produced for individual analyses are highly dependent on the assumptions made,
many of which are uncertain. There are two factors that warrant recognition. The first factor
is the coverage of benefits. Where there are multiple types of benefits it is often not possible
to quantify all the benefits that may be linked to the investment. The second factor involves
uncertainty regarding the assumptions made, including the linkage between the research and
the assumed outcomes.
15 | P a g e
A confidence rating based on these two factors has been given to the results of each cluster
analysis. The rating categories used are High, Medium and Low, where:
High:
Medium:
Low:
denotes a good coverage of benefits or reasonable confidence in the
assumptions made
denotes only a reasonable coverage of benefits or some uncertainties in
assumptions made
denotes a poor coverage of benefits or many uncertainties in assumptions
made
Table 3.6 summarises the confidence ratings provided for each investment.
Table 3.6: Confidence in Analysis of Each Cluster
Cluster
Theme 1: Biosecurity and aquatic animal health (YTK,
abalone and oysters)
Theme 4: Ecologically sustainable development
(Management)
Theme 5: Governance and regulatory systems
Theme 6: Resource access and allocation
Theme 7: Production, growth and profitability
(Enhancement, nutrition and health)
Theme 7: Production, growth and profitability
(Genetics/selective breeding)
Theme 7: Production, growth and profitability (Systems
and production)
Theme 7: Production, growth and profitability
(Profitability)
Theme 8: Consumers, products and markets
(Consumers/markets, products and supply chains)
Coverage
of Benefits
Medium
Confidence in
Assumptions
Low
High
Medium
Medium
Medium
Medium
Medium
Low
Low
High
Medium
Medium
Medium
Medium
Medium
Medium
Low
16 | P a g e
4. Discussion
Categorisation and Distribution of Benefits
Most benefits identified as emanating from these nine clusters of investment were economic
in nature, however there also were some significant environmental and social benefits
identified.
Public versus Private Benefits
Tables 3.2 and 3.3 provided an indication of the distribution of public and private benefits, by
allocating each of the individual benefits identified for the nine clusters to a particular
beneficiary type. The distinction between the triple bottom line categories and the
beneficiaries is important, as economic benefits can accrue to both the fishing industry and
the public sector.
The analysis showed that when the number of identified individual benefits is considered, the
Australian public captures 41% of the total number of benefits. When these benefits are
subjectively weighted for their relative magnitude, this proportion decreases to 31%. Using
the weighted data, 3% of that 31% of public benefits are economic in nature, while 11% are
environmental and 17% are social. These results demonstrate the significant spillovers of
benefits to the public sector from research targeted at the fishing industry.
Most of the public sector benefits were dominated by social, economic and environmental
benefits that fell to groups outside the commercial fishing industries. These benefits included:
 Enhanced sustainability of ecosystems and/or reduced biodiversity decline
 Increased technical/scientific/research/industry capacity
 Improved natural resource management and reduced environmental impact by
aquaculture industries
 Improved resource allocation and/or public policy/governance
 Reduced fisheries management costs to government and avoided social costs
 Improved regional incomes from aquaculture growth
 Increased utility for recreational fishers and seafood consumers
Distribution of Benefits Along the Fish Supply Chain
Many of the industry benefits identified in these clusters will be passed along the supply
chain (including both the input and market supply chains). These will also impact domestic
consumers as well as foreign consumers in some cases. The way in which the benefits are
distributed along the supply chains will be determined by the supply and demand elasticities
along the chain.
Benefits to Other Industries
The fishing industry is defined broadly as including the commercial wild catch and
aquaculture industries, as well as the recreational and indigenous and cultural sectors. There
were limited benefits to industries outside of the fishing industry.
Benefits Overseas
It is likely that there will be some minor benefits to overseas interests, largely emanating
from foreign delegates who attended conferences and workshops supported by the
investments covered in this report. Such benefits would mostly be in the form of scientific
and industry knowledge exchanges.
17 | P a g e
Additionality
Additionality refers to the degree to which public spillover benefits are being delivered by
FRDC supported research, and how the level of these public spillovers would have changed if
public money to FRDC was reduced. That is, it is the ‘additional’ public spillover benefits
captured by FRDC being able to fund a greater number of projects due to the contribution of
government resources.
The ‘clustering’ approach makes it difficult to address additionality as each of the projects in
a cluster may have been associated with a different industry priority at the time of funding.
To address this issue, a general approach was applied where the overall industry funding
priority of each of the clusters was considered and each cluster provided with a rating of high,
medium or low priority for industry at the time of funding.
None of the nine clusters were considered low priority for industry; there were seven
considered high or medium-high priority and two considered medium priority. For those that
were considered high priority for industry, it was typically assumed that if the government
contribution to FRDC was halved, then the cluster of projects would still have been funded at
a reduced but still substantial level. If there was no contribution at all from the government, it
was usually assumed the research would still have been funded, but at a level closer to 50%
of the actual level. For those clusters of medium priority for industry, it was typically
assumed that if the public contribution to FRDC were halved then the clusters would have
received a reduced investment substantially below the actual, and that if no public funding
were available, funding may have been closer to only 25% of the actual level.
These generalised assumptions lead to the conclusion that if public funding were reduced,
FRDC would still have funded the same research areas, but that the numbers of projects in, or
their levels of funding to, each research area would be significantly reduced, and therefore the
benefits to both the public and industry would also be reduced. Other scenarios that may
affect this result would be if the levy rate was changed or if additional financial contributions
may have been available from research or other partner organisations.
In reality, the funding decisions resulting from reduced public financial support would be
much more complex, and there would be interactions between the differing levels of priority
for FRDC that considers both its industry and public investors, and industry itself. The
response by state agencies and other investors would also have had an influence.
Round 2 Investment Criteria
The total PVB for the nine clusters was $684 m and the total PVC was $267 m yielding a
total NPV of $417 m. The individual cluster BCRs across the nine clusters for the total
investment ranged from 1.8 to 3.9. The simple average BCR was 2.59 and the weighted
average similar at 2.57.
With a total PVC of $267 m, the nine clusters evaluated in Round Two represented a
substantially larger investment than the eight clusters evaluated in Round One, which had a
total PVC of $99.3 m. The aggregate BCR for Round Two was slightly higher at 2.6,
compared to 2.5 for Round One.
Results from Round One and Two present a preliminary indicator of FRDC investment
performance. The third round of evaluations will need to be completed before a final
conclusion for the analysis of the total portfolio is drawn.
18 | P a g e
Confidence in Quantitative Analyses
As presented in Table 3.6, each of the clusters was rated as to the confidence in the coverage
of benefits, and the confidence in assumptions used to value selected benefits. With respect to
coverage of benefits, seven clusters were rated as medium and two were rated as high. With
respect to confidence in assumptions, four clusters were rated as low and five were rated as
medium. Hence, there was greater confidence held in the coverage of benefits than in the
assumptions made.
Reasons for a self-assessed low level of confidence in the assumptions in some clusters were
associated with:
 The lack of specific evidence for impacts of the research, for example, on the extent
of cost reductions or quality/product price changes
 Sensitivity analyses that demonstrated high sensitivity of investment criteria to
changes in key assumptions
 Lack of information or confidence in defining the counterfactual situation
Association with FRDC Key Performance Indicators (KPIs)
There are one or more KPIs specified for each FRDC theme. As there were nine clusters and
six themes involved in Round Two, Table 4.1 shows how many of the projects in each cluster
addressed the KPIs for the corresponding FRDC theme.
Table 4.1: FRDC Key Performance Indicators by Cluster
Cluster
Associated Key Performance
Indicator (s) for Theme
Theme 1: Biosecurity and
aquatic animal health
(YTK, abalone and oysters)
1. Development and dissemination of
protocols, techniques and technologies
to mitigate and minimise the impact of
disease outbreaks
2. Development of knowledge to assist
industry to register vaccines and
veterinary chemicals
1. Development of mechanisms and
technologies to collect economic,
environmental and social data to inform
management processes
2. Improvement of knowledge of the
relationship between environmental
processes and known biological
processes
3. Development of techniques for
incorporation of ecosystem based
fisheries management in fisheries
4. Development of knowledge to help
the industry to meet environmental
standards
1. Development of processes and
technologies to improve the efficiency of
governance and regulatory systems for
Theme 4: Ecologically
sustainable development
(Management)
Theme 5: Governance and
regulatory systems
Proportion of
Projects
Contributing
13/13
0/13
33/50
4/50
8/50
13/50
17/22
19 | P a g e
Theme 6: Resource access
and allocation
Theme 7: Production,
growth and profitability
(Enhancement, nutrition
and health)
Theme 7: Production,
growth and profitability
(Genetics/selective
breeding)
Theme 7: Production,
growth and profitability
(Systems and production)
Theme 7: Production,
growth and profitability
(Profitability)
Theme 8: Consumers,
products and markets
(Consumers/markets,
products and supply chains)
fishing and aquaculture
2. Development of methods to
incorporate economic knowledge into
fisheries management.
1. Development of processes for
efficient, transparent allocation of shares
and associated property rights for all
aquatic resource users.
1. Development of knowledge, processes
and technologies to improve
productivity and profitability of the
commercial sectors.
2. Development of knowledge and
technologies in the areas of
domestication and breeding genetics to
support growth of the aquaculture sector.
1. Development of knowledge, processes
and technologies to improve
productivity and profitability of the
commercial sectors.
2. Development of knowledge and
technologies in the areas of
domestication and breeding genetics to
support growth of the aquaculture sector.
1. Development of knowledge, processes
and technologies to improve
productivity and profitability of the
commercial sectors.
2. Development of knowledge and
technologies in the areas of
domestication and breeding genetics to
support growth of the aquaculture sector.
1. Development of knowledge, processes
and technologies to improve
productivity and profitability of the
commercial sectors.
2. Development of knowledge and
technologies in the areas of
domestication and breeding genetics to
support growth of the aquaculture sector.
1. Development of knowledge and
technologies to support
the industry’s development of new
products
2. Development of knowledge and
technologies to improve seafood value
chains and support trade and market
access
10/22
6/7
7/15
0/15
18/18
18/18
18/28
11/28
9/14
1/14
8/35
31/35
20 | P a g e
Lessons Learnt for Evaluation and Future Investment
A number of lessons learnt for future investment, and for future evaluations, were identified
from these cluster analyses. Some of the issues reported are similar to those identified in the
Round One summary report. General and specific issues are identified below:
General issues
1. The outcomes reported for a few projects (e.g. any usage of the project’s outputs) were
still considered uncertain despite inputs from FRDC personnel, the principal
investigator/organisation and, in some cases, representatives of the associated industry.
When outcomes are uncertain, the actual impact of the project is uncertain and hence any
evaluation of its benefits is highly uncertain. Apart from evaluation exercises such as this
current study, it may be in FRDC’s interest to develop processes for further following up
on past projects to remedy this situation. This raises the issue of resource allocation to
evaluation and the relative importance given to accuracy, coverage and contemporariness.
2. The FRDC project management system was found valuable in being able to extract
funding information by financial year across a range of individual R&D areas. However,
an improvement would be if an annual time series for R&D expenditure within individual
R&D areas could also be made available. This would also be helpful to FRDC for
tracking and reporting R&D investment across areas over time as part of its portfolio
monitoring, management and resource allocation systems.
3. It would be of value to economic assessments such as the current analysis if industry
information on disease costs, including treatment, mortality, and productivity losses could
be monitored over time for various aquaculture industries. As well as aiding evaluation of
past research, such information may be useful in priority setting for future research and in
ex-ante economic analyses.
4. The ratio of FRDC funding to total funding for projects funded by FRDC are worth
recording by cluster as they can be important in assessing the FRDC current and
prospective roles in different R&D areas and where public benefits are manifest but
where industry or other funding is difficult to attract.
5. While the ABARES statistics on production and value by species group and jurisdiction
are extremely useful in economic evaluation, one of the constraints faced by this
evaluation was finding data on the gross value of some fisheries and how these gross
values may have varied year by year. Likewise, there is a lack of readily available
information on standardised yields, costs, and gross margin budgets for the various
aquaculture industries. It would be helpful if improved indicative summary performance
data on the various fisheries and aquaculture industries were available.
6. At the time of the evaluation, there is usually limited evidence available that can be
identified to establish the importance of the knowledge produced by FRDC projects to
policy change and indeed how policy and fisheries management developments are formed
with regard to increments in knowledge. There can be misfits between the information
supplied by scientists/researchers and that required by policy makers. This area may form
the basis for a useful separate social science project focusing on the linkage between
FRDC research investment and policy development and change. One approach could be
for FRDC to review its general and specific impacts on policy changes to which it has
contributed over the past 20 years. At the same time, such a project may elicit the key
21 | P a g e
information needed by those involved in management and policy development to ensure
that FRDC research projects are appropriately targeted.
7. Strategic research can be successful in developing knowledge and understanding but
outcomes of such research can be limited in terms of quickly producing products or
practices that can be utilised by industry in the shorter term. The outputs from many of
the current cluster investments are being used in other projects currently being funded by
FRDC. Any future analysis of such projects should be sure to acknowledge the
contributions of these earlier strategic projects, but at the same time, double counting of
quantified benefits should be avoided.
8. There were sometimes difficulties contacting and obtaining responses from appropriate
personnel, particularly for older projects where some principal investigators had retired or
moved overseas, and other people in the research organisations were not knowledgeable
enough of the specific project and is outcomes to be of assistance. FRDC could attempt to
encourage some form of corporate responsibility for project knowledge to be addressed in
research organisations, but this may be difficult to accommodate. This issue is part of a
larger challenge of improving the tracking of outcomes for all investments.
9. The government’s rural R&D priorities do not accommodate a number of social impacts
such as improved recreational experiences for fishers and improved health and safety.
The implication of this is that these areas of research investment should be given a low
priority by FRDC or that national R&D priorities should be changed to provide some
focus on social benefits. Even the revised national research R&D priorities announced in
July 2015 do not accommodate such important impacts.
Issues relating to specific clusters
1. Many of the projects in the genetics cluster spanned some 10-15 years and many were not
completed before other interacting projects on the same topic commenced. From an
evaluation perspective this superseding of continuing projects made it time consuming to
assess benefits from the earlier project as it was necessary to capture some data from the
newer projects; this extended the workload. For example, two projects in the genetics
cluster commenced in 1994 but did not deliver final reports until 11-13 years later.
One suggestion for consideration would be to change the population definition by moving
it forward in time and include incomplete projects when they had reached a significant
milestone. Another suggestion would be to ensure projects are completed within one year
of their original completion date. If not, and the investment is to be continued, it may be
preferable to commence a new project rather than extend the former project.
2. There were a number of projects in the Consumers/markets, products and supply chains
cluster that were associated with the Seafood CRC. A general impression gained was that
these projects appeared to be associated to some extent with missing reports and a lack of
clear responsibility of reporting to FRDC by the Principal Investigators.
3. A number of projects in the Consumers/markets, products and supply chains cluster were
associated with issues that could be termed industry ‘management and organisational
issues’, rather than R&D issues. In that regard it could be argued that such issues should
be the complete responsibility of, and funded by, the industry. On the other hand,
22 | P a g e
assistance from FRDC in addressing such issues made some contribution to the
facilitation and augmentation of more effective R&D investment in that industry.
4. The projects in the Profitability cluster spanned a wide variety of industries and issues,
and in many respects most did not seem to have much in common except that they
concerned profitability. Projects ranged from scientific trials of new technologies, through
to desktop feasibility studies and information gathering exercises. Such a grouping can be
seen as working against one of the purposes of clustering. The disparate nature of the
projects not only contributes to greater difficulties in impact valuation, but also
potentially decreases the value of the analysis in providing useful input for strategies
regarding future investment priorities. Furthermore, ‘profitability’ is one of the desired
impacts and applies to a whole range of investments in other clusters.
5. Conclusions
The key conclusions from this report are:
1. The majority of the benefits identified from these nine clusters of research investment
have been economic in nature, although significant environmental and social benefits
have also been delivered.
2. The major beneficiary of the impacts of the nine clusters of research has been the
Australian fishing industry, but with still 41% of the number of benefits identified as
being public benefits. There were limited benefits to other industries and to overseas
interests.
3. The NPVs for total investment for the individual clusters ranged from $5.9 million to
$124.1 million and the individual BCRs ranged from 1.8 to 3.9.
4. FRDC investment made up 38% of the total investment in all clusters in present value
terms.
5. When all nine clusters are aggregated, the return to the $267 million total investment
(present value terms) produced a BCR of 2.6 to 1, with benefits (PVB) of $684 million,
and an NPV of $417 million.
6. The confidence in the coverage of benefits in each cluster was rated as medium to high,
while the confidence in the assumptions used was rated as low to medium.
7. It was difficult to draw conclusions regarding the additionality of the investments
analysed. It was concluded that if public funding to FRDC was reduced, spillover benefits
to the public also would be reduced. However, the degree to which this reduction would
occur would depend on the relative priorities for funding of individual research areas and
projects between FRDC, the industry itself and other funders.
8. All five rural research priorities were addressed by between three and nine clusters; with
rural research priority one (Productivity and adding value) being the most frequently
addressed and rural research priority four (Climate variability and climate change) being
the least frequently addressed. Most clusters addressed both supporting priorities.
Priorities addressed by each cluster are described in Appendices 1-9.
9. Most clusters evaluated exhibited a moderate to high proportion of projects addressing the
relevant KPIs for their respective theme.
10. A number of issues were identified with implications for future evaluation of FRDC
investments, some of which may have implications for FRDC priority setting and
management (e.g. industry information on costs and productivity losses, and leverage
factors for different R&D areas).
23 | P a g e
Acknowledgments
The authors of the report would like to acknowledge the contribution by FRDC principal
investigators and industry members who were willing to make comments on drafts and
answer questions regarding industry impacts. Also, a number of FRDC staff assisted most
willingly with access to reports, other information, and reading and commenting on drafts.
Specific acknowledgments to a range of personnel are provided in each of the nine
appendices.
References
Council of Rural Research and Development Corporations (CRRDC) (2007) “Guidelines for
Evaluation”, Canberra
http://www.ruralrdc.com.au/WMS/Upload/Resources/Evaluation/Evaluation%20methodolog
y%20150607.pdf
FRDC (2010) Investing for tomorrow’s fish: the FRDC’s Research, Development and
Extension Plan 2010–2015, Research and Development Corporation, Australian Government,
Canberra <http://www.frdc.com.au/research/current-plan>
24 | P a g e