Assessing Social Benefits:
A Review of Methods
Revised 2007 June 19
COMMUNITY
AND
NEIGHBOURHOOD SERVICES
SOCIAL RESEARCH UNIT
Assessing Social Benefits: A Review of Methods
Prepared by:
Jeffery Halvorsen
Social Research Unit, Social Policy and Planning
Community and Neighbourhood Services
The City of Calgary
Prepared for:
Kay Wong
Corporate Initiatives
Corporate Services
The City of Calgary
© 2007 The City of Calgary, Community and Neighbourhood Services, Social Research Unit
Executive Summary
This report supports The City of Calgary’s User Fees and Subsidies Review, which is a
response to Council Priority 3.11 requesting an update to The City’s user fee policies
and related subsidy strategies. Specifically, this report investigates methodologies
available to assess social benefits in terms of: (1) identifying impacts; (2) monetizing
impacts; and (3) presenting impacts.
Twelve methods for assessing social benefits were reviewed. They are: Cost Benefit
Analysis (CBA); Social Impact Assessment (SIA); Social Return on Investment (SROI);
Social and Ethical Accounting, Auditing and Reporting (SEAAR); Blended Value;
Heritage Framework; Cost Effectiveness Analysis (CEA); Quality Adjusted Life Years
(QALY); Cost of Illness; Willingness to Pay (WTP); Fair Calgary’s Fairness Filter; and
the Canadian Parks / Recreation Association’s Benefits Catalogue.
Understanding these methods provides a conceptual basis and the literacy needed to
understand decisions about the three key questions. Experiments, stakeholder
consultation, surveys, and literature reviews are all suggested methodologies for
identifying impacts. Advice ranges from identifying easily monetized impacts (e.g., Cost
of Illness) to identifying impacts as widely as possible (e.g., Social Impact Assessment).
The preferred method will depend on whether wide data – meaning difficult to identify
and cumulative information – or narrow data – meaning easily identified and monetized
information – are needed for the process of presentation.
i
Abbreviations and Definitions of Key Terms
Abbreviations
CBA
Cost Benefit Analysis
QALY
Quality-Adjusted Life Years
SEAAR
Social and Ethical Accounting, Auditing and Reporting
SIA
Social Impact Assessment
SROI
Social Return on Investment
WTP
Willingness to Pay
Definitions of Key Terms
Baseline
What is the situation without any change, or what would have
happened anyway?
Inputs
Resources invested in an activity
Outputs
The direct and tangible results from an activity
Outcomes
Changes that are either directly or indirectly linked to the activity
Impact
Outcomes less (-) an estimate of what would have happened anyway
Theory of Impact
How an organization’s activities help achieve its objectives
ii
Contents
Executive Summary .................................................................................................... i
Abbreviations and Definitions of Key Terms ................................................................ ii
Contents ...................................................................................................................... iii
1.0
Introduction ................................................................................... 1
2.0
Methodologies Reviewed ............................................................. 1
3.0
2.1
Cost Benefit Analysis (CBA) ..................................................................
2
2.2
Social Impact Assessment (SIA) ............................................................
4
2.3
Social Return on Investment (SROI) ......................................................
5
2.4
Social and Ethical Accounting, Auditing and Reporting (SEAAR) ..........
8
2.5
Blended Value .......................................................................................
9
2.6
Heritage Framework .............................................................................. 10
2.7
Cost Effectiveness Analysis (CEA) ........................................................ 11
2.8
Quality Adjusted Life Years (QALY) ....................................................... 12
2.9
Cost of Illness ........................................................................................ 13
2.10
Willingness to Pay (WTP) ...................................................................... 13
2.11
Fair Calgary’s Fairness Filter ................................................................. 14
2.12
Canadian Parks / Recreation Association – Benefits Catalogue ............ 16
Summary of Key Findings ............................................................ 16
3.1
Identifying Impacts ................................................................................. 16
3.2
Monetizing ............................................................................................. 17
3.3
Presentation ........................................................................................... 18
3.4
Conclusions ........................................................................................... 18
References .............................................................................................. 19
iii
Appendices ............................................................................................. 23
A. Steps for Conducting a Cost Benefit Analysis ........................ 23
B. Green Book Valuation Technique ............................................. 25
C. Analysis of Social Return on Investment ................................. 27
1.0 Introduction ............................................................................................ 28
2.0 Social Return on Financial Investment (SROFI):
A Quantitative Framework ..................................................................... 28
3.0 Social Return on Investment (SROI):
A Qualitative and Quantitative Framework ............................................. 29
4.0 Understanding the Difference between SROFI and SROI ..................... 29
5.0 References for Appendix C .................................................................... 30
D. Heritage Benefits Framework .................................................... 31
iv
Revised 2007 June 19
COMMUNITY
AND
NEIGHBOURHOOD SERVICES
SOCIAL RESEARCH UNIT
Assessing Social Benefits: A Review of Methods
1.0 Introduction
The City of Calgary’s background document on user fees, Underlying Principles Guiding
User Fees and Subsidies Review, provides some guiding principles, indicating that the
full cost of a program should be covered by the individual or group receiving the benefit
from the program (City of Calgary, 2007: 8-9). Benefits to society should be supported
by a tax supported subsidy. Some societal benefits derive from general use and some
from individual use (City of Calgary, 2007: 13-14).
To determine the ratio between subsidy and user fee, the direction offered by the above
principles indicates that the impacts of a program must be identified and attributed to
the beneficiary. Once identified, these benefits may be monetized and a ratio can be
determined. Monetizing societal benefit will offer policy makers a tool for decision
making. Further, the information presented may show the economic benefit of having
proactive instead of reactive policy.
Considering all of these guiding principles, the following research focuses on three key
questions:
1. How can program outcomes be identified?
2. How can program outcomes be monetized?
3. How can program outcomes be presented?
The conclusion reached is that there are numerous methodologies available to answer
each of the above questions. The methods range in the skills required and timeliness.
The rigor will depend on the skills and time available for the analysis.
2.0 Methodologies Reviewed
Twelve methods for assessing social benefits were reviewed. They are: Cost Benefit
Analysis (CBA); Social Impact Assessment (SIA); Social Return on Investment (SROI);
Social and Ethical Accounting, Auditing and Reporting (SEAAR); Blended Value;
Heritage Framework; Cost Effectiveness Analysis (CEA); Quality Adjusted Life Years
(QALY); Cost of Illness; Willingness to Pay (WTP); Fair Calgary’s Fairness Filter; and
the Canadian Parks / Recreation Association’s Benefits Catalogue. Each is described
separately below. Four appendices provide additional detail.
Assessing Social Benefits: A Review of Methods
Page 1 of 31
2.1 Cost Benefit Analysis (CBA)
Description: In Cost Benefit Analysis (CBA), programs are evaluated by identifying the
costs (inputs, fixed, marginal and opportunity costs) and benefits (revenue, impacts and
outcomes). The costs and benefits are then monetized by determining the price per unit
multiplied by the number of units. The presentation of the data can occur in two ways.
The first is dividing the benefits by the costs, which results in a ratio. If the ratio is
above one (1), then the program offers more benefit than cost. The second alternative
is to subtract the cost from the benefit, which reveals a net welfare gain.
Process: The following process is found in Lawrence and Mears (2004), as shown in
Appendix A.
•
Clearly state the question under consideration
•
Determine the perspective: i.e., Society? Business Unit?
•
Identify benefits and costs
•
Assign values to benefit and cost items and compare total benefits and total costs to
determine net value and cost-benefit ratio
•
Address issues of uncertainty using sensitivity analysis: i.e., Being explicit with
assumption and following those assumptions through to the end of the analysis;
offering a range of possible scenarios
•
Incorporate time and discounting, and
•
Articulate the limitations of the methodology and analysis.
Monetizing: CBA monetizing methodology typically takes the format of price per unit
multiplied by unit cost. Swaray, et al. (2005: 149) summarize the methodology used by
Griffith, et al. (1999) in determining the cost of a prison term:
The total cost of a prison term was calculated by multiplying the total [number]
of days by the cost per day. The authors divided the yearly expenditure on
the facility by the average population and further divided the quotient by 365
days.
This example shows a simple method of calculating a cost from publicly available
information. It is also useful when considering the impact of a crime prevention program
because an avoided cost is the same as a benefit (Cohen, 2000: 276).
The Green Book manual published by Her Majesty’s Treasury in the United Kingdom
has developed a valuation technique (see Appendix B). Once benefits and costs have
been identified, a monetized value can be determined from market data. If market data
do not exist, then a Willingness to Pay (WTP) proxy can be employed by either
observing consumer behaviour or conducting a survey asking what people would be
willing to pay to receive the benefit or avoid the cost of good X.
Page 2 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Two methods of conducting WTP surveys are suggested in the Green Book (Her
Majesty’s Treasury, 2003). The first is a Contingent Valuation model in which
respondents are asked what they would pay for good X, or asked to choose from a
predetermined list of options. The second method is a Choice Modeling method in
which respondents are asked a series of questions in which they must choose between
two options (Her Majesty’s Treasury, 2003: 57). This technique is also used in the
United States by the U. S. Office of Management and Budget (1992).
Important Considerations: When determining costs and benefit values, ask the
question: “Will the cost or benefit still be realized if the program is not undertaken?”
(Lawrence and Mears, 2004: 12; see also Her Majesty’s Treasury, 2003: 19).
Value of this Approach: Identifying Impacts. Experimental and quasi-experimental
designs are suggested as possible methods that can be used to identify outcomes in
CBA. Experimental and quasi-experimental designs (Bell, 2006a; 2006b) suggest
defining two random populations, one that receives an intervention (program) and
another that does not. The populations must be large enough to control for chance
differences. These designs will allow the researcher to collect baseline data on the
control group as well as data on the study group. Comparing the two groups will reveal
the outcomes attributable to the program.
Monetizing. Green Book Method in CBA – Determine market value. If none exists then
use Willingness to Pay (see Appendix B).
Presentation. CBA distinguishes between which costs and benefits will accrue to the
various stakeholders, including The City of Calgary and society. Costs and benefits are
often borne by different stakeholders. An investment in subsidizing recreation by The
City of Calgary may result in cost savings to the medical system. “Understandably,
those who pay for an initiative may have a particular interest in costs and benefits that
directly accrue to them” (Lawrence and Mears, 2004: 8). Some impacts can be costs
and benefits, depending on which perspective is viewed. For example, a job that is
created to support a program will be a cost to the business unit funding the program but
provide a benefit to society.
Presentation. In CBA, monetized values can be determined for outcomes, which can be
a useful comparison tool.
Decisions about which outcomes are more beneficial are implicitly made when one
program is funded over another. Monetizing the value a program contributes to society
is one way of making the implicit value, explicit. The important contribution of
monetization is not determining the dollar value a program contributes to society, but
the process of identifying the costs and benefits associated with a problem and a
solution.
Assessing Social Benefits: A Review of Methods
Page 3 of 31
Limitations of this Approach: The analysis will only be as good as the data it is
based on. Many assumptions go into a CBA. Stating these assumptions and using a
sensitivity analysis to provide the range will make the implications of these assumptions
evident to a reviewer (Lawrence and Mears, 2004: 17).
Some impacts such as air quality, pain and suffering can be hard to monetize and the
value determined may be controversial. This may be addressed by presenting CBAs
with and without the controversial elements included. This will ensure that, if the
controversial elements are not considered legitimate, the CBA will still be useful
(Lawrence and Mears, 2004: 18). Finally, a CBA can not account for political pressure.
One program that is popular may not have a good CBA ratio. Much like other ratios and
quantified information, CBA cannot determine priorities; it can only determine efficiency.
However, quantified information is still useful as one tool among many in decision
making (Lawrence and Mears, 2004: 18, see also Appendix C).
Additional Information
Her Majesty’s Treasury, 2003:
www.hm-treasury.gov.uk/economic_data_and_tools/greenbook/data_greenbook_index.cfm
Lawrence and Mears, 2004: www.urban.org/UploadedPDF/411047_Supermax.pdf
2.2 Social Impact Assessment (SIA)
Description: Social Impact Assessment (SIA) identifies and engages stakeholders to
create a base case scenario and a scenario of what will happen if a program is
implemented. Comparing the two scenarios reveals the impacts of a program. This
methodology is useful in answering the key question of how to determine the outcomes
of a program.
Process: The following guidelines are drawn from the Interorganizational Committee
on Principles and Guidelines for Social Impact Assessment (2003: 234-238).
1. Identify and describe interested and affected stakeholders; establish a baseline
(profiles).
2. Identify the key social and cultural issues related to the action or policy from the
community and stakeholder profiles. These indicators may also be determined in
consultation with stakeholders and SIA practitioners.
3. Describe all aspects of social impacts (holistic); include secondary and cumulative
impacts.
4. “It is more important to identify likely social impacts than to precisely quantify more
obvious social impacts” (Interorganizational Committee, 2003: 236).
5. Consider the underrepresented and vulnerable.
Page 4 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
6. Monitor and mitigate.
Key issues may include: “people’s way of life, their culture, community, political
systems, environment, health and wellbeing, personal and property rights, fears and
aspirations,” among other things (Vanclay, 2003: 8; see also www.iaia.org).
Monetizing: This methodology encourages widely identifying impacts, discouraging the
practice of quantifying the easily identifiable vectors when doing so does not allow
breadth of impact to be identified.
Important Considerations: Within the tradition of SIA, special attention is paid to
cumulative impacts. These are situations that result from the combination of direct
impacts and are quite difficult to foresee (see Council on Environmental Quality, 1997).
Vulnerable and low power stakeholders will not usually be involved early on in the
planning process, whereas high power stakeholders will be involved early on because
of access, investment, and interest. This is an important consideration as opinions
formed from early stakeholder engagement can be formative in how a situation is
perceived, giving preference to the perceptions of high power stakeholders
(Interorganizational Committee, 2003).
Value of this Approach: Identifying Impacts. SIA is useful in answering the key
question of how to determine the outcomes of a program.
Limitations of this Approach: SIA is an exercise in thinking through the impacts of a
program. As such, the process can be as limited or extensive as time and resources
allow. However, the resulting data will reflect the investment.
Additional Information
International Association for Impact Assessment: www.iaia.org
Interorganizational Committee, 2003.
Vanclay, 2003.
2.3 Social Return on Investment (SROI)
Description: Social Return on Investment (SROI) is a variation of Cost Benefit
Analysis and uses the same basic concepts of inputs, outputs, outcomes and impacts.
Inputs are the resources invested in an activity. Outputs are the direct and tangible
results from an activity. Outcomes are changes that are either directly or indirectly
linked to the activity. Finally, impacts are defined as outcomes less an estimate of what
would have happened anyway (baseline).
Assessing Social Benefits: A Review of Methods
Page 5 of 31
The following table provides a hypothetical case study using SROI and The City of
Calgary’s (2007) Underlying Principles Guiding User Fees and Subsidies Review.
Description
Program A
Program B
Explanation
Program Cost
$50.00
$50.00
Hypothetical
Social Impact
$50.00 (50%)
$100.00 (67%)
Hypothetical
Individual Impact
$50.00 (50%)
$50.00 (33%)
Hypothetical
Total Impact
$100.00
$150.00
Social + Individual
User Fee
$25.00
$16.50
Cost x Individual % of Impact
Level of Subsidy
$25.00
$33.50
Cost x Social % of Impact
SROI
4
4.5
Total Impact / Subsidy
As we see from the table above, because investment in Program B produces a higher
societal benefit, the general subsidy can be larger and still produce a greater SROI.
The following discussion highlights the limitations and benefits of the SROI tool. It will
also contextualize the SROI ratio in qualitative information found in the literature.
Process: The following is adapted from the Roberts Enterprise Development Fund
(REDF) method (Gair, 2005: 6-7). Several methods for conducting a SROI have been
developed (see Scholten, et al., 2006: 14 for a summary). However, the REDF method
was chosen because it was the first used and provides a good conceptual basis for
understanding SROI. Further, the REDF method builds on the basic concepts identified
in the description section above. Newer methodologies incorporate more stakeholder
consultation and theories of change (how an organization’s activities achieve stated
objectives).
1. Calculate the economic value created (financial indicators): the revenue and costs of
running the program.
2. Calculate the socio-economic value created: monetize those indicators that lend
themselves to monetization (cost of social programs, increased tax base, etc.).
Deduct the specifically social costs associated with the program (counselling, etc.).
These calculations should be done on a per client basis, then multiply by the number
of clients.
3. Calculate the blended value: financial and socio-economic value is then added
together and long-term debt is subtracted.
4. SROI, the Enterprise Value: the blended value is divided by the investment, resulting
in a ratio. If the ratio is over 1 then the return was greater than the investment.
However, there are important considerations that must be understood when
interpreting the ratio (see below).
Page 6 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Monetizing: The REDF’s monetizing methodology is based on the spectrum of
economic, socio-economic and social value creation. Economic indicators are defined
as the financial return from the operations of the enterprise or program (i.e., user fees,
service charges). Socio-economic indicators are those that ‘lend themselves’ to
monetization. These include reduced use of publicly funded services and programs, as
well as an increased tax base. Social indicators, such as psychological impacts are not
monetized but are still included in client profiles (Gair, 2005).
Important Considerations: The utility of the ratio is dependent on the objectives of the
program being evaluated. For example, if a program’s objective is to increase the use
of social services available to the mentally ill, then the social cost would increase with
the success of the program in meeting its objectives. The implication of this is that the
SROI would decrease so if this is not properly expressed or understood, it may result in
decreased funding for a successful program.
Decreasing costs of social services will also result in a smaller SROI ratio. For
example, if the cost of incarceration decreased 20% per person per year, then the
avoided cost of an intervention would decrease by the same amount. The result being
that the SROI of the intervention program would decline even though the program is
successful.
Many of these issues are addressed by making the distinction between SROI and
Social Return on Financial Investment (SROFI). SROFI refers to the quantitative ratio
while SROI analysis refers to the presentation of quantitative and qualitative information
outlining the impact of a program. For further discussion, refer to Appendix C.
Value of this Approach: Monetizing. Defining indicators along the spectrum of
economic, socio-economic and social value creation will increase legitimacy and
usability. The process of monetizing social benefits that are not easily monetized (e.g.,
pain and suffering, happiness, etc.) is controversial (Lawrence and Mears, 2004: 14).
However, including the category of socio-economic indicators, easily monetized
indicators (e.g., decreased use of social programs) can still be included. This expands
the monetary reporting from impacts on the specific organization to impacts on society.
Presentation. SROI Analysis incorporates quantitative and qualitative information that
helps decision makers understand detailed quantitative performance measurement in
context.
Identify Impacts. The SROI literature discusses the OASIS data collection system in
which program participants are surveyed once at the very beginning of their
participation with a program and then every six months for two years regardless of
continued involvement in the program (Twerksy, 2002). The survey may include
questions about homelessness, public assistance, criminal convictions, mental health,
education, and number of dependants. However, the indicators can be tailored to the
specific program being implemented.
Assessing Social Benefits: A Review of Methods
Page 7 of 31
Limitations of this Approach: The SROI ratio is not “meant to be a ‘cross-cutting’
metric of comparison for dissimilar projects. It is not meant to form a ranking system or
similar types of relativistic systems with no context” (Scholten, et al., 2006: 36). The
SROI ratio is a measure of efficiency and using it to determine which sector to invest in
is inappropriate unless the primary objective of the funder is to decrease societal
inefficiency.
Additional Information
Gair, 2005 (review of issues in SROI Analysis): www.redf.org/publications-sroi.htm
New Economics Foundation, 2004: http://www.neweconomics.org/gen/
Olson and Nicholls, 2005:
http://haas.berkeley.edu/responsiblebusiness/conference/documents/SROIFrameworkv.Haa
s.pdfSaraOlsen.pdf
Roberts Enterprise Development Fund: http://www.redf.org/
Social Enterprise Coalition: http://www.socialenterprise.org.uk/
2.4 Social and Ethical Accounting, Auditing and Reporting (SEAAR)
Description: Social and Ethical Accounting, Auditing and Reporting (SEAAR) is a
process that uses consultation to identify relevant indicators to the organization’s
objectives. Once established, these indicators are used for accounting, are audited,
and regular reports are published. This process is very similar to impact assessment
(participatory, includes monitoring and evaluation). The unique contribution of SEAAR
is embedding this process in an organization (i.e., sustainability reports).
Process: The following process of SEAAR is found in Pay (n.d.: 8):
1. Identify the social objectives and the ethical values of the organization against which
its activities are assessed.
2. Define the stakeholder groups.
3. Establish social performance indicators.
4. Begin the accounting process.
5. Audit the accounts.
6. Publish regular reports.
Monetizing: If a monetary value already exists then it can be reported as such, but
there is no specific methodology provided for monetizing non-monetary impacts.
Important Considerations: None Identified.
Value of this Approach: None Identified.
Page 8 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Limitations of this Approach: SEAAR is a framework for integrating SIA and similar
studies into an organization’s operating context.
Additional Information
AccountAbility website: www.accountability.org.uk
Pay (n.d.): www.enterprise-impact.org.uk/pdf/SocialAccounting.pdf
Social Audit Network website: www.socialauditnetwork.org.uk/
2.5 Blended Value
Description: Investors as well as organizations pursing their objectives create wealth.
For-profit firms are thought to create money and non-profit organizations are assumed
to create social benefit. The thesis of blended value is that all organizations create a
mix of social benefit (or cost) and profit. Essentially, all organizations have a social,
economic and environmental impact, and these three impacts can not be separated
from each other (Emerson, 2003b: 1).
Emerson (2003b: 3) identifies six initiatives that are independently trying to develop
answers to the issue of “blended value.” Corporate Social Responsibility (CSR), Social
Enterprise, Socially Responsible Investment (SRI), Double Bottom Line (DBL), Strategic
Philanthropy, and Sustainable Development make up the six fields in which Emerson
groups the current literature. While this work does not provide any original material, it is
a useful reference tool, listing sources for each of the above initiatives. Of particular
interest is the section identifying the work on Non-Profit Measurement and Metrics.
Process: None Identified.
Monetizing: Emerson (2003a: 77) provides an analysis of the current level of research
facing outcome measurement:
There is a natural inclination to focus upon those metrics and approaches that
target easily quantifiable performance indicators as opposed to indicators that
could shed light upon what the actual and desired long-term outcomes are.
Moving from input/output analysis to that of true outcome assessment is
much more challenging than even the most ardent supporters of
measurement may be able to address.
Important Considerations: The City of Calgary has a Triple Bottom Line policy
framework for assessing fiscal, social, and environmental impacts (City of Calgary,
2006b).
Value of this Approach: None Identified.
Limitations of this Approach: No original work.
Assessing Social Benefits: A Review of Methods
Page 9 of 31
Additional Information
Blended Value website: www.blendedvalue.org/
Emerson, 2003a: www.blendedvalue.org/publications/
Emerson, 2003b: www.blendedvalue.org/publications/
2.6 Heritage Framework
Description: The Heritage Framework emerged from work done for the Department of
Canadian Heritage by a consulting firm called the Outspan Group (2003). Their projects
focus on impacts that can be attributed to heritage institutions in Canada, which include:
•
Museums (including art galleries)
•
Archives
•
Historic sites, buildings, and parks
•
Nature parks and conservation areas with interpretation or educational programs
•
Related institutions (exhibition centres, planetariums, observatories, aquariums,
zoos, botanical gardens, arboretums and conservatories).
The framework is a qualitative assessment organizing the impacts into the categories of
personal, commercial and societal benefits. This framework also takes into account use
and non-use value creation, or primary and secondary impacts (Outspan Group, 2003).
Process: The benefits of a heritage institution are divided into three categories
(Outspan Group, 2003; see also Appendix: D):
•
Individual Benefit – benefits accruing to individual stakeholders (users and nonusers).
•
Commercial Benefit – benefits derived from the redistribution of commercial activity
from one area to another.
•
Societal Benefit – benefits that cannot be allocated to either individuals and/or
businesses, yet indivisible and tending to be societal in scope (i.e., public goods).
Monetizing: This is a qualitative impact assessment framework.
Important Considerations: This is an impact assessment specific to heritage
institutions. However, the framework is general enough to be of value to other sectors.
Value of this Approach: Identifying Impacts. This framework offers three categories
around which impacts may be organized. Also the study by Scott (2006) provides a
wide range of identified impacts of museums.
Page 10 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Limitations of this Approach: Utility is limited to identifying impacts.
Additional Information
Scott, 2006.
Outspan Group, 2003: www.pch.gc.ca/progs/ph/pubs/epc-hic/05_e.cfm
2.7 Cost Effectiveness Analysis (CEA)
Description: Cost Effectiveness Analysis (CEA) is used to compare programs with the
same objective (Lawrence and Mears, 2004: 3). It is similar to CBA in the method used
to determine costs – cost or benefit per unit multiplied by the number of units.
Process: This method is very similar to CBA so the process was not investigated.
Monetizing: Swaray, et al. (2005: 149, summarizing Griffith, et al., 1999) provides an
example of how the benefit of an averted crime can be calculated:
•
Total cost of prison term = # of days spent in prison x cost per day
•
Cost per day = (annual expenditure on the prison facility / average prison population)
/ 365
In more general terms this calculation may become:
•
Benefit of program (participant days avoided) = # of days in program x cost per day
•
Cost per day = (annual expenditure on the program / average number of program
participants) / 365
Note: Refer to section 2.3 on the value of SROI, specifically the OASIS system (Twerksy, 2002)
in the section titled “Identifying Impacts.” This may prove to be a useful tool for identifying the
impact on participants. For example, an employment program could ask participants what
services they used before then, after six months, what services they still use.
Important Considerations: None Identified.
Value of this Approach: Monetizing. This method offers a simple calculation that can
be done with data usually published in public reports.
Limitations of this Approach: The type of programs this method focuses on do not
typically charge a user fee (i.e., employment programs, addictions and crime
prevention).
Additional Information
Griffith, et al., 1999.
Swaray, et al., 2005.
Assessing Social Benefits: A Review of Methods
Page 11 of 31
2.8 Quality Adjusted Life Years (QALY)
Description: Quality Adjusted Life Years (QALY) is an eight-step method used to
determine a non-monetary comparative indicator, the monetary value of which is then
determined. In the article describing the method, formulas are provided (French, et al.,
1996: 899).
Process: French, et al. (1996: 897-900) developed a QALY method and applied it to
estimating the dollar value of health outcomes from drug-abuse interventions, using
eight steps to:
1. Determine the set of health problems that are related to drug abuse.
2. Characterize the level of severity, reflecting symptoms, mortality rate, duration of
illness, and functional status.
3. Combine data on average maximum life span and survival rates at different ages to
determine the number of life-years lost and then approximate maximum life span
and survival rates for individuals based on gender, race and current age.
4. Select Health Status Index Scale (such as the Rosser/Kind index provided in
French, et al., 1996: 897).
5. Determine time spent in different health states.
6. Estimate discounted QALY lost for each illness.
7. Estimate dollar value of a QALY.
8. Estimate the dollar value of avoiding each illness.
Monetizing: This process can provide a monetized value for a program that is
population wide.
Important Considerations: This is probably the most rigorous methodology; however,
it is also the most complicated.
Value of this Approach: Monetizing. QALY can provide a monetized value for a
program that is population wide. This methodology can measure and quantify very
difficult metrics such as pain and suffering.
Identifying Impacts. Steps one and two of the QALY process use a literature review to
identify issues.
Limitations of this Approach:
This methodology is health specific.
Three
assumptions are made in this model: “(1) death is equally bad for everyone; (2) a year
of healthy life has equal value for everyone, this ensures QALY is valued equally and
does not vary by gender, race, current productivity, or earnings capabilities; and (3)
quality / quantity-of-life tradeoffs are consistent and uniform” (French, et al., 1996, 898).
Page 12 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
The calculation is highly correlated with the value of life and discount rates used. The
appropriate number for these values is widely debated and no consensus exists. A
possible solution is to offer a range, followed by a recommended number to reflect the
uncertainty introduced by these two unresolved issues (French, et al., 1996: 903).
Additional Information
French, et al., 1996.
2.9 Cost of Illness
Description: Cost of Illness is a health adaptation of the concept of human capital,
which views “an individual as producing a stream of output over the years that is valued
at the individual’s earnings, with the value of household work being imputed” (Hodgson
and Meiners, 1982: 457). To the value of human capital is added the cost of illness
(i.e., healthcare).
Process: Life = program cost + lost wages.
Monetizing: Life = total public spending associated with program (i.e., in the case of
illness, medical expenses) + opportunity cost.
Important Considerations: This is a criticized methodology and is not currently in
popular use in policy (see French, et al., 1996: 894; Miller, et al., 2002: 1).
Value of this Approach: None Identified.
Limitations of this Approach: “Productivity costs are based on observed wages
rather than the full value of work and leisure and the value of avoided pain and
suffering” (French, et al., 1996: 894, 906). Many of the programs that user fees are
considered for are not associated in any way with wages (i.e., heritage institutions).
The result is that this methodology will not be able to value the benefit of these
programs in any way.
Additional Information
Hodgson and Meiners, 1982.
2.10 Willingness to Pay (WTP)
Description: Using the Willingness to Pay (WTP) methodology, goods and services
can be valued by their market price. However, when a market does not exist for those
goods, surveys can determine a population’s willingness to pay for one increment of the
good. The result is a monetized market proxy for good or service X.
Assessing Social Benefits: A Review of Methods
Page 13 of 31
Process: Researchers are able to survey people on the willingness to pay for an
increment of good or service X. The response is correlated with income so the range of
answers is averaged across income groups and the increment of good or service X is
then given a monetized value. Consumers’ willingness to pay can also be determined
through revealed preferences, hedonic pricing is an example. This method involves
“inferring an implicit price revealed indirectly by examining consumers’ behaviour in a
similar or related market” (Her Majesty’s Treasury, 2003: 57).
Monetizing:
preferences.
Based on survey results of people’s willingness to pay and revealed
Important Considerations: This method should not be used without an issue
identification tool. Survey questions should be focused on valuing outcomes, not on
valuing programs or goods themselves because one participant will identify different
outcomes than another participant and this will influence the participant’s willingness to
pay. So this method will offer the best results if participants are asked their willingness
to pay for outcomes as opposed to programs, where outcomes may be ambiguous.
Value of this Approach: Monetizing. Survey results and revealed preferences of
people’s willingness to pay for an increment of good or service X create a hypothetical
market price.
Limitations of this Approach: Hedonic or revealed pricing methods are generally
reliable, however they “cannot estimate the value placed on an asset by people who
make no direct use of it” (Her Majesty’s Treasury, 2003: 58).
Additional Information
Her Majesty’s Treasury, 2003:
www.hm-treasury.gov.uk/economic_data_and_tools/greenbook/data_greenbook_index.cfm
2.11 Fair Calgary’s Fairness Filter
Description: The Fairness Filter is a set of seven principles that are considered when
reviewing policies and programs. In 2005, Fair Calgary staff reviewed the Subsidies
and Fees policy. The recommendations included a discussion paper entitled Ensuring
Affordability (City of Calgary, 2005). This discussion paper reviewed available
frameworks for implementing subsidies.
Process: Fairness Filter Summary (City of Calgary, 2006a):
1. Accessibility – location
2. Availability – scheduling and ease of use
3. Affordability – ability to pay fees and related charges
4. Acceptability – sensitivity to diversity
Page 14 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
5. Accommodation – persons with special needs
6. Adequacy – amount and volume of services
7. Achievements – metrics
The Subsidy Frameworks (City of Calgary, 2005) are twofold:
For Individuals:
•
Targeted Fees: identify and subsidize low-income Calgarians
•
Geographic or Regional Fees: subsidize areas of facilities
•
Zone Fees: i.e., transit zones
•
Facility or Service Specific Fees: identify specific fees to subsidize
•
Variable Fee for a Hierarchy of Services: fees increase with level of service
•
Complimentary Service: fully subsidized fees, and
•
Promotion Fee: subsidize fee for a short period to generate interest in the service.
For Groups:
•
Group Discount: discounted services for groups or members of an association, and
•
Foregone Revenue: Subsidizing organizations so they can pass the lower operating
costs onto clients.
Monetizing: None Identified.
Important Considerations: The Fair Calgary research often refers to the capability
principle: “ensuring that Calgarians are capable of using The City services they value”
(City of Calgary, 2006a: 4; 9). The capability principle is also referred to as the equality
principle. Fair Calgary (City of Calgary, 2006a: 21) contrasts the equality concept with
the “equity approach (benefits principle).”
Discussion: The equality principle’s aim is to ensure that everyone has “equal access”
and that the “greatest advantage goes to the most disadvantaged” (City of Calgary,
2005: 2). This concept is not in contention with the benefits principle (although it makes
the goal more explicit). The idea that synthesizes the two principles is that subsidies
supporting the individuals that are “disadvantaged” will yield a greater return than those
who have the “advantage” because supporting the “disadvantaged” will yield a greater
societal benefit.
The equality principle refers to access to the services people value. While Fair Calgary
may oppose any attempt at quantifying this value, the Willingness to Pay methodology
may be employed to determine the value. Usually individuals’ WTP are aggregated with
income considerations to determine an appropriate value. Perhaps WTP can be used,
but instead of aggregating the data, income groups could first be determined and then
WTP could determine the user fee for each income group.
Assessing Social Benefits: A Review of Methods
Page 15 of 31
Value of this Approach: None Identified.
Limitations of this Approach: None Identified.
Additional Information
City of Calgary, 2006a: http://community.gov.calgary.ab.ca/faircalgary/presentations.html
City of Calgary, 2005: http://community.gov.calgary.ab.ca/faircalgary/policydis.html
Fair Calgary Internal Website: http://community.gov.calgary.ab.ca/faircalgary/index.html
2.12 Canadian Parks / Recreation Association – Benefits Catalogue
Description: The Benefits (Catalogue Canadian Parks / Recreation Association, 1997)
is essentially an annotated bibliography of the literature on the impacts (health, social,
economic and environmental) of recreation. Outcomes are identified on pages xii to xvi.
The body of the document (pages 1-159) reports primary research that supports each of
the outcomes. References can be found on pages 161-203.
Process:
1. Define Outcomes (pp. xii to xvi): Methodology not provided for this step.
2. Identify and quote primary sources supporting outcomes (pp. 1-159).
Important Considerations: This report completes step 1 and 2 of the Quality Adjusted
Life Years (QALY) eight-step method (see French, et al., 1996 and Section 2.8 of this
report).
Additional Information
Canadian Parks/ Recreation Association, 1997.
3.0 Summary of Key Findings
3.1 Identifying Impacts
•
Cost Benefit Analysis:
Experimental and quasi-experimental designs are
suggested as being possible methods that can be used to identify outcomes in CBA.
•
Social Impact Assessment: SIA is useful in answering the key question of how to
determine the outcomes of a program.
•
Social Return on Investment: SROI literature discusses the OASIS data collection
system, in which program participants are surveyed once at the very beginning of
their participation with a program and then every six months for two years regardless
of continued involvement in the program.
Page 16 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
•
Heritage Framework: This framework offers three categories around which
impacts may be organized. The study by Scott (2006) also provides a wide range of
identified impacts of museums.
•
Quality Adjusted Life Years: Steps one and two of the QALY process use a
literature review to identify issues.
3.2 Monetizing
•
Cost Benefit Analysis: The Green Book method of CBA determines market value;
if no market value exists, then use Willingness to Pay.
•
Social Return on Investment: Defining indicators along the spectrum of economic,
socio-economic and social value creation will increase legitimacy and usability.
•
Cost Effectiveness Analysis: This method offers a simple calculation that can be
done with data usually available in public reports.
•
Quality Adjusted Life Years: QALY can provide a monetized value for a program
that is population wide.
•
Willingness to Pay: Survey results and revealed preferences of people’s
willingness to pay for an increment of good or service X create a hypothetical market
price.
Much like impact identification, the process of monetizing impacts has a high correlation
between difficulty, rigor and legitimacy. On one end of the spectrum exists Cost of
Illness and Cost Effectiveness Analysis. These methodologies identify impacts that are
easily monetized (i.e., wages, program costs avoided) but avoid controversial impacts
such as pain and suffering.
Social Return on Investment monetizes impacts that lend themselves to the process
and avoids controversy by including non-monetized impacts as qualitative data.
Willingness to Pay includes more controversial aspects in the process of monetizing
but, through surveys and revealed behaviour, estimate what people would or have paid
for impacts that are not easily monetized. Further, the Green Book method (see
Appendix B) is a process for identifying the best method for determining a monetary
value for an impact. Among the realistically implemented strategies, Willingness to Pay
has the ability to legitimately monetize the most impacts.
The most difficult, legitimate, and rigorous method available is Quality Adjusted Life
Years. This method depends on extensive use of high level medical, statistical, and
economic research. The result is a rigorous method of determining the lost quality of
life which has an associated economic value. This method offers cross-sector
comparable indicators for valuing interventions and programs.
Assessing Social Benefits: A Review of Methods
Page 17 of 31
3.3 Presentation
•
Cost Benefit Analysis: CBA distinguishes between which costs and benefits will
accrue to the various stakeholders (e.g., The City of Calgary and society).
Monetized values can be determined for outcomes, which can be a useful
comparison tool.
•
Social Return on Investment: SROI Analysis incorporates quantitative and
qualitative information that helps decision makers understand detailed quantitative
performance measurement in context.
With regards to presentation, a review of these methodologies reveals that decision
making can be greatly helped by understanding the social, environmental, and
economic implications of the decision at hand, whether or not those implications are
monetized.
3.4 Conclusions
In summary, the research process revealed a three step process that is common to all
frameworks for assessing social benefits and formed the key questions of this report:
(1) identify impacts; (2) monetize impacts; and (3) present impacts. Experiments,
stakeholder consultation, surveys, and literature reviews are all suggested
methodologies for identifying impacts. Advice ranges from identifying easily monetized
impacts (e.g., Cost of Illness) to identifying impacts as widely as possible (e.g., Social
Impact Assessment). The preferred method will depend on whether wide data –
meaning difficult to identify and cumulative information – or narrow data – meaning
easily identified and monetized information – are required for presentation.
Page 18 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
References
Bell, Stephen. 2006a. Experimental Design. Retrieved 2007 June 1 from website:
www.urban.org/toolkit/data-methods/experimental.cfm.
Bell, Stephen. 2006b. Quasi-Experimental Design. Retrieved 2007 June 1 from
website: www.urban.org/toolkit/data-methods/quasi-experimental.cfm.
Canadian Parks/ Recreation Association. 1997.
Canadian Parks / Recreation Association.
Benefits Catalogue.
Ottawa:
City of Calgary. 2007. Underlying Principles Guiding User Fees and Subsidies Review.
(Revised Discussion Paper). Calgary: City of Calgary, Corporate Services.
__________. 2006a. Fair Calgary and User Fees and Subsidies Review. The City of
Calgary, Community and Neighbourhood Services, Fair Calgary. PowerPoint
Presentation dated 2006 May 30. Retrieved 2007 June 12 from intranet website:
http://community.gov.calgary.ab.ca/faircalgary/presentations.html
__________. 2006b. Triple Bottom Line. Calgary: The City of Calgary, Environmental
Management. Retrieved 2007 June 8 from website:
http://content.calgary.ca/CCA/City+Hall/Business+Units/Environmental+Manage
ment/Strategic+Environmental+Initiatives/Triple+Bottom+Line/Triple+Bottom+Lin
e.htm
__________. 2005. Ensuring Affordability: A Discussion Paper. City of Calgary,
Community and Neighbourhood Services, Fair Calgary. Retrieved 2007 June 12
from internal website:
http://community.gov.calgary.ab.ca/faircalgary/policydis.html.
Cohen, Mark. 2000. “Measuring the Costs and Benefits of Crime and Justice.”
Measuring and Analysis of Crime and Justice. NIJ Publication No. 182411: 263316. Washington, DC: National Institute of Justice.
Council on Environmental Quality. 1997. Considering Cumulative Effects Under the
National Environmental Policy Act. Washington: CEQ NEPAnet. Retrieved
2007 May 23 from website: http://ceq.eh.doe.gov/nepa/ccenepa/ccenepa.htm.
Emerson, Jed. 2003a. Blended Value Map. Retrieved 2007 May 28 from website:
www.blendedvalue.org/publications/
Emerson, Jed. 2003b. Blended Value Map: Executive Summary. Retrieved 2007 June
1 from website: www.blendedvalue.org/publications/.
Assessing Social Benefits: A Review of Methods
Page 19 of 31
French M. T., J. A. Mauskopf, J. L. Teague, and J. Roland. 1996. “Estimating the
Dollar Value of Health Outcomes from Drug Abuse Interventions.” Medical Care
Vol. 34, No. 9: 890-910.
Gair, Cynthia. 2005. A Report from the Good Ship SROI. Retrieved 2007 June 7 from
REDF.org website: www.redf.org/publications-sroi.htm.
Griffith, J. D., M. L. Hiller, K. Knight, and D. D. Simpson. 1999. “A Cost-Effectiveness
Analysis of In-Prison Therapeutic Community Treatment.” Prison Journal Vol.
79, No. 3: 352-365. Cited in Swaray, et al., 2005: 149.
Her Majesty’s Treasury. 2003. Green Book: Appraisal and Evaluation in Central
Government. Retrieved 2007 June 5 from website: www.hm-treasury.gov.uk/
economic_data_and_tools/greenbook/data_greenbook_index.cfm.
Hodgson, T., & M. Meiners. 1982. “Cost-of-Illness Methodology: A Guide to Current
Practices and Procedures.” Milbank Memorial Fund Quarterly Vol. 60, No. 3:
429-462.
Interorganizational Committee on Principles and Guidelines for Social Impact
Assessment. 2003. “Principles and Guidelines for Social Impact Assessment in
the USA.” Impact Assessment and Project Appraisal Vol. 21, No. 3 (September):
231-250.
Lawrence, Sarah, and Daniel P. Mears. 2004. Benefit-Cost Analysis of Supermax
Prisons: Critical Steps and Considerations. Retrieved 2007 June 1 from
website: www.urban.org/UploadedPDF/411047_Supermax.pdf.
Miller, Ted; Janusz Mrozek; Charles Calhoun, and W. Brian Arthur. 2002. “Accounting
Frameworks for Societal Cost-Outcome Analyses of Morbidity Reduction.”
Medical Care Vol., 38: 562-582.
New Economics Foundation. 2004. The SROI Primer. Retrieved 2007 June 14 from
website: http://sroi.london.edu/about.html
Olson, Sara, and Jeremy Nicholls. 2005. “A Framework for Approaches to SROI.”
Paper presented at the Haas Social Metrics Conference. Retrieved 2007 June
18 from web site:
http://haas.berkeley.edu/responsiblebusiness/conference/documents/SROIFram
eworkv.Haas.pdfSaraOlsen.pdf.
Outspan Group. (2003). Heritage Institutions in Canada: Characteristics, Impacts and
Benefits. Retrieved 2007 May 30 from website:
www.pch.gc.ca/progs/ph/pubs/epc-hic/05_e.cfm
Page 20 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Pay, Chris. n.d. Social Accounting: A Method for Assessing the Impact of Enterprise
Development Activities? Retrieved 2007 May 23 from website:
www.enterprise-impact.org.uk/pdf/SocialAccounting.pdf.
Scholten, Peter; Jeremy Nicholls, Sara Olson, and Brett Galimidi. 2006. Social Return
on Investment: A Guide to SROI Analysis. Utrecht, Netherlands: Lenthe
Publishers.
Scott, Carol. 2006. “Museums: Impact and Value.” Cultural Trends Vol. 15, No. 1:
45-75.
Swaray, Raymond, Roger Bowles, and Rimawan Pradiptyo. 2005. “The Application of
Economic Analysis to Criminal Justice Interventions: A Review of the Literature.”
Criminal Justice Policy Review Vol., 16, No., 2: 141-163.
Twersky, Fay. 2002. An Information OASIS. Roberts Enterprise Development Fund.
Retrieved 2007 June 8 from website: www.redf.org/download/other/oasis.pdf.
U.S. Office of Management and Budget. 1992. Guidelines and Discount Rates for
Benefit-Cost Analysis of Federal Programs.
Circular No. A-94 Revised
(Transmittal Memo No. 64), 1992 October 29. Washington, DC: U.S. Office of
Management and Budget.
Vanclay, Frank. 2003. “International Principles for Social Impact Assessment.” Impact
Assessment and Project Appraisal Vol. 21, No. 1: 5-11.
Assessing Social Benefits: A Review of Methods
Page 21 of 31
Page 22 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Appendix A: Steps for Conducting a Cost Benefit Analysis
Source: Lawrence and Mears (2004: 8). www.urban.org/UploadedPDF/411047_Supermax.pdf
Assessing Social Benefits: A Review of Methods
Page 23 of 31
Page 24 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Appendix B: Green Book Valuation Technique
Source: Her Majesty’s Treasury (2003: 23).
www.hm-treasury.gov.uk/economic_data_and_tools/greenbook/data_greenbook_index.cfm
Assessing Social Benefits: A Review of Methods
Page 25 of 31
Page 26 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Appendix C: Analysis of Social Return on Investment
Analysis of Social Return on Investment
2007 June 7
Author: Jeffery Halvorsen
Supervisor: Dr. Sharon M. Stroick
The City of Calgary
Community and Neighbourhood Services
Social Policy and Planning
Social Research Unit
Abstract
This report will discuss the utility and limitations of the Social Return on Investment
(SROI) indicator. The distinction is made between the Social Return on Financial
Investment (SROFI) and the more qualitative SROI Analysis. The conclusion reached
is that the SROFI ratio alone can not be used to compare sectors, or in some cases,
organizations for suitable investment. However, SROI Analysis can be used as a useful
tool to present direct and indirect impacts that result from a program, whether the
impact is monetized or not.
© 2007 The City of Calgary, Community and Neighbourhood Services, Social Research Unit
Assessing Social Benefits: A Review of Methods
Page 27 of 31
1.0 Introduction
The Triple Bottom Line (TBL) is a concept that extends performance measurement from
purely financial to the environmental and social impacts an organization makes. The
problem is that very sophisticated quantitative tools exist for measuring financial
performance so management is much easier with these tools.
Social Return on Investment (SROI) is an attempt to develop a tool to measure the
performance of environmental and social investment (Scholten, et al., 2006: 6). SROI
Analysis is a tool that can be used to present the outcomes or impacts that result from a
program or project. Monetized and qualitative information are combined to tell the story
of the impact that may or has resulted from a program.
Decision makers can use this tool to decide between programs to invest in within
sectors (optimizing investment). However, in SROI, the qualitative story is primary and
is supported by the quantitative indicator. This allows investment to be guided, for
example, by principles outlined by The City of Calgary in its Corporate Vision Statement
(City of Calgary, 2005), instead of a misunderstood quantitative indicator.
When research is presented in the form of SROI, a decision maker must understand
what he or she is looking at and must understand the difference between SROI Analysis
and Social Return on Financial Investment (SROFI). If the decision maker does not
understand this distinction, then inappropriate investment decisions may result.
2.0 Social Return on Financial Investment (SROFI):
A Quantitative Framework
Social Return on Financial Investment (SROFI) is a quantitative ratio of the cost and the
benefit of a program. The ratio is calculated using the following concepts (New
Economics Foundation, 2004):
•
Inputs [IN]: resources invested in an activity
•
Outputs [OP]: the direct and tangible results from an activity
•
Outcomes [OC]: Changes that are either directly or indirectly linked to the activity
•
Impact [IM]: Outcomes less (-) an estimate of what would have happened anyway,
less (-) displacement
The ratio is determined by the following calculation: SROI = IN / IM.
For example, if the cost to the medical system of hypertension was $250,000, and a
program that cost $50,000 prevented hypertension – and reduced the $250,000 cost
associated with the health problem – the resulting SROFI would be 5 ($250,000 /
$50,000). If the cost borne by the health care system then declines to $100,000
because the program was effective and fewer services were needed, the resulting
SROFI would be 2 ($100,000 / $50,000).
Page 28 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Seeing the declining SROFI, a decision maker may misinterpret the data and decide to
stop funding the program because of decreasing efficiency. This scenario shows that
the contextual qualitative information is key to evaluating programs and determining
program success using SROFI.
3.0 Social Return on Investment (SROI):
A Qualitative and Quantitative Framework
Understanding the stakeholders (those affected by impacts of a project) and what is
important to the stakeholders is where the SROI process begins. Understanding who
the stakeholders are is a good first step in understanding what the impacts are or will be
(Scholten, et al., 2006: 12). The impact desired is reflected in the objective of the
organization.
Theory of Impact: An organization can usually easily identify the theory of how its
activities reach its objectives. This link between activities and objectives is the
organization’s theory of impact. The SROI Analysis makes this theory explicit.
Calculation and Context: SROI Analysis includes: “a) a specific process by which
number was calculated, b) context information to enable accurate interpretation of
number itself, and c) additional non-monetized social value and information about
number’s substance and context” (Scholten, et al., 2006: 7). Including all of
contextual information will “tell the story” of what is happening over time in
organization and what the organization’s impact is.
the
the
the
the
an
4.0 Understanding the Difference between SROFI and SROI
The foundation of the SROFI indicator is the use of baseline costs. These costs are
vastly different for each societal benefit that is provided by a program or service. The
result is that the SROFI cannot be used to compare performance between sectors. The
SROFI ratio can be easily misinterpreted if used out of its qualitative context. However,
if the ratio is used within the appropriate context it can be a tool to present the story of
an organization’s impact.
Similarly, the ratio SROI is not “meant to be a ‘cross-cutting’ metric of comparison for
dissimilar projects. It is not meant to form a ranking system or similar types of
relativistic systems with no context” (Scholten, et al., 2006: 36). However, the process
of monetizing impact and presenting it using SROI can be used to help decision makers
think about the cost of providing services and the value of subsidizing some services to
help The City of Calgary achieve its objectives. Prevention and intervention programs
are often less expensive over the long term and optimize public money; however, many
programs are not implemented because of cost. SROI can show the long-term cost of
delaying prevention programs and compare it to the relatively small cost of intervention.
Assessing Social Benefits: A Review of Methods
Page 29 of 31
5.0 References for Appendix C
City of Calgary. 2005. Municipal Handbook. Retrieved June 18, 2007 from website:
www.calgary.ca/portal/server.pt/gateway/PTARGS_0_2_104_0_0_35/http;/conte
nt.calgary.ca/CCA/City%20Common/Municipal%20Handbook/Municipal%20Han
dbook.htm
New Economics Foundation. 2004. The SROI Primer. Retrieved 2007 June 14 from
web site: http://sroi.london.edu/about.html
Olson, Sara, and Jeremy Nicholls. 2005. “A Framework for Approaches to SROI.”
Paper presented at the Haas Social Metrics Conference. Retrieved 2007 June
18 from web site:
http://haas.berkeley.edu/responsiblebusiness/conference/documents/SROIFram
eworkv.Haas.pdfSaraOlsen.pdf.
Scholten, Peter; Jeremy Nicholls, Sara Olson, and Brett Galimidi. 2006. Social Return
on Investment: A Guide to SROI Analysis. Utrecht, Netherlands: Lenthe
Publishers.
Additional Resources
New Economics Foundation: www.neweconomics.org/gen/
Roberts Enterprise Development Fund: www.redf.org/
Social Enterprise Coalition: www.socialenterprise.org.uk/
Page 30 of 31
City of Calgary, Community and Neighbourhood Services, Social Research Unit
Appendix D: Heritage Benefits Framework
Source: Outspan Group, (2003). www.pch.gc.ca/progs/ph/pubs/epc-hic/05_e.cfm
Assessing Social Benefits: A Review of Methods
Page 31 of 31