IN OUR OWN WORDS:
A DISCOURSE ANALYSIS OF SUBMISSIONS TO THE
INTERNATIONAL ASSISTANCE REVIEW
ACKNOWLEDGEMENTS
This report is the product of a collective effort, with data collection, processing and analysis conducted by
Carolyn Frank, Alyssa Grant, Bailey Kew, Colin Kornelsen, Nicole Lee, Erika Rossi, and Claire Zhou.
The final report was compiled by CCIC staff with key inputs and contributions from these individuals.
Shantelle Binette generated the scattergraphs that illustrate the variance between submissions.
While many have reviewed and commented on the report, the Canadian Council for International Cooperation (CCIC) assumes full responsibility for any factual errors.
December 2016
* * *
The Canadian Council for International Co-operation (CCIC) is Canada’s national coalition of civil
society organizations (CSOs) working globally to achieve sustainable human development. Our members
represent a broad range of CSOs working in international development and humanitarian assistance –
from faith-based and secular groups to labour unions, cooperatives and professional associations. CCIC
seeks to end global poverty and to promote social justice and human dignity for all.
This publication was produced with financial support from the CSO Partnership for Development
Effectiveness and CCIC’s institutional funders and our members.
39 McArthur Avenue Ottawa, ON K1L 8L7
Tel.: (613) 241-7007 E-mail: [email protected]
Web site: www.ccic.ca
© 2016 Canadian Council for International Co-operation (CCIC). All parts of this report may be reproduced for
educational purposes provided acknowledgement of the source and notification of such use is given to CCIC. The opinions
expressed in this report do not necessarily represent the opinions of CCIC’s members or its funders.
2
TABLE OF CONTENTS
Findings in numbers .................................................................................................................... 4
I. Overview: The International Assistance Review - A Key Moment for Change .................... 5
Assessing the input of the international development and humanitarian community ................................. 5
Summary of process and findings................................................................................................................. 6
II. Methodology for data collection ............................................................................................ 7
Limitations to the methodology .................................................................................................................. 8
III. Top words across all submissions ......................................................................................... 9
Total word count and medians across all submissions—visualized as a word cloud................................10
IV. Divergences and convergences between submissions ........................................................ 12
1) The results as a bar chart – where the submissions diverge ..................................................................12
A. Where do they diverge? GAC vs. CCIC ..........................................................................................13
B. Where do they diverge? All submissions vs. CCIC .........................................................................16
C. Where do they diverge? All submissions vs GAC ...........................................................................19
2) The results as a scattergraph – where the submissions converge ..........................................................21
Findings from analysis of all plots .......................................................................................................21
A note about the methodology ..............................................................................................................22
A. Where do they converge? GAC vs. CCIC .......................................................................................23
B. Where do they converge? All submissions vs. CCIC ......................................................................24
C. Where do they converge? All submissions vs GAC ........................................................................25
V. What does this all mean? The narrative looking forward for Canadian global development
cooperation................................................................................................................................. 27
VI. Conclusion ........................................................................................................................... 31
Annex 1 - List of organizations ................................................................................................. 32
Annex 2 - List of words, groups and subgroups ....................................................................... 33
3
12
NUMBER OF SUBMISSONS
FROM NON-MEMBERS
10
SUBMISSONS FROM
COALITIONS
WORD CATEGORIES
259
WORD INDICATORS
1177
TOTAL PAGES FOR ALL 80
SUBMISSIONS
14.3
AVERAGE PAGES PER
SUBMISSION
432,821
TOTAL WORDS FOR ALL
80 SUBMISSIONS
5410.3
AVERAGE WORDS PER
SUBMISSION
2609
TIMES “WOMEN”, THE
MOST POPULAR WORD,
WAS USED IN ALL 80
SUBMISSIONS
73.4
17
% OF CCIC
MEMBERSHIP MADE A
SUBMISSION TO THE
INTERNATIONAL
ASSISTANCE REVIEW
18.5
4
MEDIAN TIMES “WOMEN”
OCCURS ACROSS ALL 80
SUBISSION
DISCOURSE REPRESENTATION
FINDINGS IN NUMBERS
58
SUBMISSIONS FROM CCIC
MEMBERS
SUBMISSION REPRESENTATION
80
SUBMISSONS FROM
MEMBERS, NON-MEMBERS,
COALITIONS
I.
OVERVIEW: THE INTERNATIONAL ASSISTANCE REVIEW –
A KEY MOMENT FOR CHANGE
In May 2016, the federal government launched a public review of Canada’s international assistance
policy, opening up a consultation period on its proposed six thematic priorities1 and areas of focus – a
process that continued until the end of July 2016. The International Assistance Review was the most
substantive and comprehensive examination of Canadian development policy in at least 20 years. It also
represented an opportunity to craft a new vision for how Canada engages in global development
cooperation in a very new context – following the adoption of the Sendai Framework for Disaster Risk
Reduction, the Sustainable Development Goals, the Paris Agreement on climate change, and
humanitarian disasters that seem to be growing in scope, scale and frequency. Given the scope and scale
of the challenge, it is perhaps not surprising that the IAR process engaged over 15,000 people and that
Global Affairs Canada (GAC) received over 10,000 written submissions.
The Canadian Council for International Co-operation (CCIC) and its members were actively involved in
the IAR process. CCIC’s staff participated in eight high-level and one working-level consultations hosted
by GAC. In addition, CCIC organized eight roundtables, including three roundtables solely for Chief
Executive Officers and Executive Directors (representing 40 organizations) on Delivering Results, and an
expert roundtable on Innovation co-organized with Grand Challenges Canada. CCIC staff also
participated in 12 roundtable consultations organized by CCIC members. During the review process,
CCIC provided over 20 summaries and analysis documents of the consultations for our organizational
members, which many of them used to inform their own submissions.
ASSESSING THE INPUT OF THE INTERNATIONAL DEVELOPMENT
AND HUMANITARIAN COMMUNITY
At the end of the consultation process, CCIC made its own submission and 58 of our 79 members made
their own submissions.
In order to better understand what its members, as well as key coalitions and organizations in the
1
Health and rights of women and children; clean economic growth and climate change; governance, pluralism, diversity and human rights; peace and security; responding to humanitarian crises and the needs of displaced populations; and a programmatic focus on delivering results through effectiveness, transparency, innovation and partnerships.
5
international development and humanitarian community, had focused on in their respective submissions,
in August, 2016, CCIC reached out to key organizations and requested copies of their submissions. A full
list of who we reached out to, and which submissions we received, are available in Annex 1.
Cognizant that it would be difficult to make an objective evaluation and comparison of the content of the
80 submissions received, CCIC decided to analyze each of the submissions against a set of key words that
would at least reflect, at a basic level, the content of the submissions, individually and collectively. For
more details, see the Methodology section.
SUMMARY OF PROCESS AND FINDINGS
This discourse analysis identifies key areas of emphasis, as well as convergence and divergence, in the
CCIC submission, GAC’s discussion paper, and the collective submissions by the broader community.
The convergences and divergences constitute the basis for more in-depth analysis of the submissions –
with the areas of convergence signaling areas where CCIC should continue its advocacy, and areas of
divergence as areas where CCIC could further develop its policy analysis and advocacy.
The assessment also allows CCIC to conduct its own analysis of what some of the most important civil
society actors in the international cooperation community advocated in their submissions. These findings
can be compared with GAC’s “Summary of what we heard” scheduled for release in November 2016.
This report presents the findings of the discourse analysis in words, numbers and pictures and a final
narrative – representing in different ways the collective content of the 80 submissions analyzed. After
articulating the general methodology used to identify and collect the data, the report presents the findings
in several different ways. It starts by presenting the most popular words used across the 80 submissions
(noting the words with the highest total count, as well as those with the highest median count). The report
then looks at areas of major convergence and divergence (i.e. the variance) between CCIC’s submission,
GAC’s discussion paper, and the averages across the 80 submissions in various ways. It concludes by
pulling together the messages that emerge from the different visualizations and analysis to paint a
summary narrative around the overall emphasis that groups underscored in their submissions. Taken
together, CCIC and GAC’s vision (as indicated in their respective submission and discussion paper)
showed the greatest divergence, albeit still with a fair degree of convergence around key issues. In
contrast, while there were less major divergences (and areas of difference in emphasis) between the All
submissions group and GAC, there wasn’t necessarily as much convergence as one might expect.
6
II.
METHODOLOGY FOR DATA COLLECTION
The data collection and analysis process followed a number of steps to determine the frequency with
which certain key words appeared across the different submissions CCIC collected, using CCIC’s own
submission and GAC’s discussion paper as baselines.
CCIC received a total of 80 submissions: 58 submissions from members, 12 submissions from nonmembers and 12 submissions from coalitions.
Where organizations made multiple submissions and provided multiple annexes, the researchers only
assessed the principal submission to the IAR.
To analyze these submissions, CCIC initially came up with a set of 125 key words, drawn from key
messages that CCIC had identified within the consultation process. It then complemented this set of
words by closely reviewing the CCIC submission and GAC’s discussion paper, identifying additional key
words and concepts that appeared in both. Words were then grouped into themes and subthemes, often
helping to signal gaps and identify additional missing words. This draft set of words were then tested
against ten submissions by the researchers, who in their searches looked for additional words or phrases
that lay in proximity to the words they were searching. The list of words was further revised and the
formal review of all submissions began, with the 80 submissions divided evenly between the seven
researchers. The researchers were instructed to identify any additional words that kept re-occurring in
their searches but were not part of the initial list. This led to two further revisions to the key words, and
searches of these new words. Overall, 259 key words were identified.
Finally, the researchers were also instructed to identify the “direction” of 17 key words; for example, in
searching for the word “invest,” researchers were asked to identify “in what?” The direction was included
in comment boxes, and the directions were analyzed across all the submissions to identify any additional
trends.
Researchers used the search function in Adobe Acrobat and Microsoft Word to locate and count the
number of words. There was no manual counting. However, the researchers did have to evaluate each
count since they elected not to include words that were repeated as headers or footers in titles, nor to
include references to words in footnotes and endnotes, or in Annexes.
7
The data was collected in three Microsoft Excel worksheets (organized by Members, Non-Members and
Coalitions). Once all the data had been collected, three additional worksheets were set up to aggregate,
process and analyze the different findings. This included extracting the average, total, and median of
words, both across the three worksheets and the combined set of data. The aggregate findings also helped
isolate the top ten key words (total and median) for all the submissions – members, non-members and
coalitions – as well as in the International Assistance Review discussion paper and the CCIC submission,
and to evaluate different variances across the submissions.
LIMITATIONS TO THE METHODOLOGY
The researchers recognize that there are several limits to this methodology.
This report reflects an analysis of words in text. It is a superficial analysis of content. It does not capture
tone, nor the correlation between different words, nor necessarily the meaning, direction or intent of the
words (except in those cases where the direction was explicitly identified).
Some efforts were made to ensure that the root of a word was captured (searching for “econom” rather
than “economic”, “economy” or “economist”). Still, in some cases, the search words the researchers
identified and used may not have captured all the data (for example, “youth” would not have captured
references to “young people”).
In examining various submissions, many important key words were added to strengthen the data set.
Nonetheless, the researchers may have still missed notable key words that could have important
implications for the overall results. We hope such omissions (which are hard to avoid) are minimal.
The final and perhaps the most obvious limitation to this methodology is human error. The data collection
was done by seven different researchers across 80 submissions. Although the methodology and
parameters for the search were discussed prior, each individual may have approached each document and
key word differently. On occasion, some individuals may have included words in the annexes, citations, or
bibliographic sources as key words, while others did not, leading to differences in the final count for each
submission and the aggregate count. This type of error can affect the consistency and accuracy of the
overall results.
8
III.
TOP WORDS ACROSS ALL SUBMISSIONS
This section looks at the top words across all of the submissions. Since a single submission can easily
bias the total word count for a particular word through high frequency use in a specific submission, the
researchers also looked at the median across all 80 submissions. The median lies at the midpoint of a
frequency distribution of observed values – in this case, of each of the 259 word. Rather than the total
number of times a word appears, the median is an even better proxy of the importance all organizations
place on a word.
In the following table, words that rank in the top ten for both total word count and highest median are
highlighted in bold.
TOTAL WORD COUNT (ACROSS ALL SUBMISSIONS)
HIGHEST MEDIAN (ACROSS ALL SUBMISSIONS)
women (2609)
health (1746)
children (1175)
women (18.5)
local (9)
gender (1085)
local (1039)
girls (1029)
sustainable (8) | economic (8)
need/needs (7) | policy (7) | funding (7)
partnership (6) | human rights (6) | access (6)
economic (1001)
education (982)
youth (954)
civil society (5.5) | gender (5.5)
impact (5) | climate change (5)
health (8.5) |international assistance (8.5)
sustainable (939)
Women is by far and away the most common word, both in terms of total word count and median. Only
five other words make both lists (word count and median): local, health, sustainable, econom(y/ic)
and gender. The focus on gender is not surprising, given the strong emphasis on women and the
anticipated feminist-based approach that the international assistance review aspired to. Strong
references to local, sustainable and econom(y/ic) are perhaps particularly important in the context of
the new Sustainable Development Goals – and its three pillars of environment, society and economy –
and the need for locally-owned development agendas.
9
Many groups also focused on children, girls and youth (high word counts for each).
References to civil society ranks relatively highly in the median count; this is not surprising given the
source of the publications – civil society organizations.
In terms of sectors, health ranks highly – not surprisingly given the government’s focus on Maternal,
Newborn and Child Health over the past five years. The high ranking of education in the total word count
is also notable – a clear signal that education must be part of any new international assistance policy. (The
GAC discussion paper referenced it 20 times, compared with health (28), gender (27) and climate
change (27).)
The prevalence of references to human rights, access (a key element of a human rights based approach),
and climate change signal that these must also be central elements of any new approach, as a well as a
focus on impact (over results).
TOTAL WORD COUNT AND MEDIANS ACROSS ALL SUBMISSIONS —
VISUALIZED AS A WORD CLOUD
While the word count and top median give a sense of the potential top priorities among the submissions
(as above), using a word cloud – where the words with the highest incidence appear larger relative to
other smaller words that are less prominent – helps provide a visual representation of the focus of
organizations within a broader context of words.
To make the word cloud less cluttered, the visualization only includes 50 words. That said, 132 of the 259
words tracked appeared less than 80 times overall in the 80 submissions – or on average less than once
per submission. The following word clouds include: (1) the top 50 words that appeared across all the IAR
submissions, and (2) the top 50 median words that appeared across all IAR submissions
10
A. TOP 50 WORDS THAT APPEAR ACROSS ALL IAR SUBMISSIONS
B. TOP 50 MEDIAN WORDS THAT APPEAR ACROSS ALL IAR SUBMISSIONS
11
IV.
DIVERGENCES AND CONVERGENCES BETWEEN SUBMISSIONS
1) THE RESULTS AS A BAR CHART—WHERE THE SUBMISSIONS DIVERGE
A key element of this research was to identify – by comparing the incidence of words in different
submissions – any convergences and positive or negative divergences between the different submissions.
To calculate these variances, the researchers compared the incidence of the 259 designated key words in
the CCIC submission relative to a) GAC’s discussion paper and b) to the 80 submissions made by
members, non-members and coalitions. The researchers also compared GAC’s discussion paper to the 80
submissions.
The researchers created a multiplier to ensure rough comparability between submissions of different page
length and word count. The researchers noted both the page lengths and the word counts of all the
submissions. To assess the variance between the CCIC submission and GAC’s discussion paper, the
multipliers were CCIC page length (38) divided by GAC page length (28), and CCIC word count (12681)
divided by GAC word count (9559). The respective words across the 259 indicators in the GAC discussion
paper were then increased by these multipliers (1.36 and 1.33 respectively) to make the GAC discussion
paper equivalent in length (pages and word numbers) to the CCIC submission. The variance was then the
difference between the CCIC word count and the revised word count for the GAC discussion paper.
Positive variances (e.g. 10) indicate greater word use by CCIC. Negative variances (e.g. -10) indicate
greater word use by GAC.
A similar process was followed to create multipliers comparing CCIC’s submission with all the
submissions, and GAC’s discussion paper with all the submissions. In this case, the page length and word
counts used to create the multiplier were the average page length and average word count across the 80
submissions. Similarly, the words used to establish the variance were the averages across all 80
submissions for each of the 259 words.
In the aggregate worksheet, words with an incidence higher than +10 or -10 were highlighted. The
researchers also grouped the variances into two groups of higher than ten and higher than 30 – to indicate
medium and high variance respectively.
These graphs illustrate more where the different respective submissions diverge.
12
A. WHERE DO THEY DIVERGE? GAC VS. CCIC
In the comparison between GAC’s submission and CCIC’s, there were 39 words with a variance above 10
or -10, and six with a variance above 30 or -30. This is the highest degree of variance between the three
different comparisons of submissions (GAC vs. CCIC; All submissions vs. CCIC; All submissions vs.
GAC) – with a total of 45 words (of 259) or 17.9%, demonstrating a medium or high variance.
WORDS WITH A VARIANCE ABOVE 10 OR -10
REFERENCED MORE BY GLOBAL AFFAIRS CANADA
WORDS WITH A VARIANCE ABOVE 30 OR -30
REFERENCED MORE BY GLOBAL AFFAIRS CANADA
Growth
Refugees
Conflict
Need/Needs
Children
Inclusive(ness)
Innovation
Health
Education
Peace
Environmental
Girl(s)
Internally displaced persons
Needs (analysis)
Development assistance
Women
Poverty reduction
International assistance
Econom(ic/y)
-28.78
-25.51
-24.45
-23.72
-23.49
-21.17
-19.12
-18.14
-17.53
-15.80
-15.57
-14.47
-13.92
-12.92
-12.25
-11.62
-10.61
REFERENCED MORE BY CCIC
Policy
CSO
Development cooperation
Funding
REFERENCED MORE BY CCIC
Strategy
Enabling environment
Local
Transparen(t/cy)
Partnership
Humanitarian funding
Holistic
Investment in
Dialogue
Public engagement
Inequality
Advocacy
Adaptation
Development effectiveness
Action plan
Sustainable development goals
National
Civil society
Vision
Women's rights
Outcome
-81.48
-38.39
10.00
10.00
10.12
10.39
10.49
10.67
11.00
11.39
11.67
12.00
13.37
14.00
14.35
15.00
16.00
16.06
17.37
19.75
20.00
22.00
25.02
13
31.79
43.73
58.00
60.39
The high variances (above 30) are perhaps most worth noting, as well as some of the variances in the
high teens and 20s.
In terms of rethinking how to talk about the broader agenda of global development cooperation, beyond
traditional conceptions of international development, GAC uses the term international assistance more,
whereas CCIC refers to this more often as development cooperation. This broader focus is welcome;
and the difference between the two terms, one could argue, is just semantics. In this case, however,
semantics is important. From CCIC’s perspective, assistance still conveys an outdated way of referring to
the relationship between donors and recipients. CCIC intentionally focuses on development cooperation
to convey collaboration and partnership.
GAC also emphasises a much stronger role for economic development (and growth) within its vision
than CCIC.
GAC’s discussion paper also focuses much more on issues of conflict (and peace), and the plight of
refugees than CCIC’s. Conversely, CCIC’s submission focuses more on many of the macro elements and
mechanics of the development cooperation relationship – development effectiveness, action plans,
vision and outcomes.
The strong focus in CCIC’s submission on CSOs (relative to GAC’s) is perhaps not surprising given the
Council’s membership, nor, one might argue, is the focus on funding – a clear pre-occupation of CCIC and
its members.
The following illustrates the positive and negative variances between CCIC’s submission and GAC’s
discussion paper, when adjusted for word length.
14
Figure 1 - Variance between GAC’s discussion paper and CCIC’s submission, when adjusted for word length
GAC’s discussion
paper said it more
CCIC’s submission
said it more
15
B. WHERE DO THEY DIVERGE? ALL SUBMISSIONS VS. CCIC
In the comparison between all submissions and CCIC’s, there were 26 words with a variance above 10
or -10, and five with a variance above 30 or -30 – less than between CCIC’s submission and GAC’s
discussion paper. This is the second highest degree of variance between the three different comparisons
of submissions (GAC vs. CCIC; All submissions vs. CCIC; All submissions vs. GAC), although
noticeably less than between GAC’s discussion paper and CCIC’s submission. There is less divergence –
a total of 31 words (of 259) or 12.0%, demonstrating a medium variance.
WORDS WITH A VARIANCE ABOVE 10 OR -10
WORDS WITH A VARIANCE ABOVE 30 OR -30
REFERENCED MORE ACROSS ALL SUBMISSIONS
Children
-23.43
Education
-19.77
Youth
-16.95
Econom(ic/y)
-13.33
Women
-12.44
Access
-11.89
REFERENCED MORE ACROSS ALL SUBMISSIONS
Health
-32.16
REFERENCED MORE BY CCIC
Policy
CSO
Funding
Development cooperation
REFERENCED MORE BY CCIC
Humanitarian assistance
Dialogue
Advocacy
Leaving no one behind
Humanitarian funding
Adaptation
Agenda 2030
Poorest
National
Inequality
Development effectiveness
Action plan
Civil society
Transparen(t/cy)
Women's rights
Vision
Climate change
Partnership
Outcome
Sustainable
10.32
10.33
10.40
10.44
10.83
10.91
11.50
11.58
12.13
13.11
14.03
14.24
15.20
15.35
16.67
18.15
18.93
19.08
19.33
20.49
16
30.92
44.64
45.39
51.85
Interestingly the highest variances were the same for between CCIC’s submission and all submissions, as
between CCIC’s submission and the GAC discussion paper – namely, in the use of development
cooperation, funding, CSOs and policy. In these cases, CCIC referenced these four words substantially
more than both GAC and the average for all submissions.
The other most noticeable variance is the greater emphasis the collection of all submissions placed on
health relative to CCIC’s submission. The collective submissions overall had more of a focus on
children, education and youth, than CCIC’s; conversely, CCIC’s submission included more of a focus on
climate change, partnership, outcome and sustainability.
The following chart illustrates the positive and negative variances between CCIC’s submission and
the collection of all civil society submissions, when adjusted for word length.
17
Figure 2 - Variance between All submissions and CCIC’s submission, when adjusted for word length
ALL submissions
(average) said it more
CCIC’s submission
said it more
18
C. WHERE DO THEY DIVERGE? ALL SUBMISSIONS VS. GAC
In the comparison between all submissions and GAC, there were 16 words with a variance above 10 or 10, and one with a variance above 30 or -30 – less than between either CCIC’s submission and GAC’s
discussion paper, and CCIC’s submission and all submissions. This is the lowest level of variance
between the three different comparisons of submissions (GAC vs. CCIC; All submissions vs. CCIC; All
submissions vs. GAC) - a total of 17 words (of 259) or 6.6%.
WORDS WITH A VARIANCE ABOVE 10 OR -10
WORDS WITH A VARIANCE ABOVE 30 OR -30
REFERENCED MORE ACROSS ALL SUBMISSIONS
Funding
-11.30
Health
-10.56
REFERENCED MORE ACROSS ALL SUBMISSIONS
REFERENCED MORE BY GAC
Private sector
Accountab(le/ility)
Internally displaced persons
Climate change
Sustainable
Agenda 2030
Inclusive(ness)
Peace
Innovation/Innovate
Econom(ic/y)
Refugees
Conflict
Need/Needs
Growth
REFERENCED MORE BY GAC
International assistance
10.04
10.12
10.43
11.12
14.26
14.60
17.25
17.50
18.04
18.89
20.17
20.34
21.84
24.65
58.37
As with the previous comparisons, the biggest variance is in the substantially higher use of international
assistance in the GAC discussion paper relative to all submissions. However, given the focus of the
Review (International Assistance) this is perhaps not surprising.
The following chart illustrates the positive and negative variances between all submissions and GAC’s
submission, when adjusted for word length.
19
Figure 3 - Variance between All submissions and GAC’s discussion paper, when adjusted for word length
ALL submissions on
average said it more
GAC’s discussion paper
said it more
20
2) THE RESULTS AS A SCATTERGRAPH—WHERE THE SUBMISSIONS CONVERGE
Another way of looking at the divergence and convergence between submissions is through a
scattergraph.
The following graphs map out the frequency of each word used by each group (CCIC, GAC, All), and
help to visualize the convergence and divergence in their respective discourse. Each word is given
coordinates based on the number of times it was mentioned in the text of each group, and plotted on the
graph accordingly. For example, if the word “growth” was mentioned 16 times by one party and 20 times
by another, “growth” would be given the coordinates (16, 20) and plotted at that point.
Words closest to the diagonal axis, with green points, are the words with the greatest convergence – that
is, the words that they “agreed” upon the most or that were said an equal number of times by each party.
The closer to the green diagonal line, the more the parties “agreed”. The higher the words appear up the
axis the more times they were mentioned by both parties - meaning the words were used more frequently
in their discourse, and were presumably more important.
Words along the X and the Y axis, with red points, are the words of highest divergence – words
mentioned several or many times by one party but not at all mentioned by the other party, in other words,
where the two parties “disagreed”. The higher the point on either axis, the higher the divergence, and the
more the parties “disagreed”.
To allow as many words as possible to be plotted, the axes were restricted to 45 words. Therefore, words
with an exceptionally high frequency (mentioned over 45 times by either party) could not be plotted in
their actual location. To ensure these words were captured on the graph, they were plotted along the outer
extremities of the plot – either along the top axis or the far right axis. Their actual coordinates are
included beside the word.
FINDINGS FROM ANALYSIS OF ALL PLOTS
Analysis all of the scattergraphs showed the following findings:

In all the graphs, the most frequently mentioned word by both parties was Women. This was also
the word with the least divergence – Women was mentioned nearly equally by all parties.
21

The words in which there were convergence and divergence varied slightly when comparing each
group to one another. Of the three groups compared, CCIC and All submissions converged around
the most number of words, and CCIC and GAC diverged the most. (Similar findings to the previous
section where CCIC and GAC diverged the most; but an interesting contrast with the previous
section, where GAC and All submissions diverged the least, and by assumption seemed to converge
the most).
Compared with the bar charts, these scattergraphs are helpful for identifying key areas of convergence (as
well as highlighting key areas of divergence).
A NOTE ABOUT THE METHODOLOGY
For selecting the words to be plotted on the scattergraph, the researchers used the following method.
Firstly, data was viewed in aggregate and narrowed down to the top 150 words used by all parties. To do
so, all words not or only rarely mentioned by all parties were removed. This included removing words
used less than once by all parties (CCIC, GAC and All submissions), unless a word was used more than
twice by another party.
Secondly, following this initial selection, data was narrowed down further to 115 words by removing
words with a low frequency and variance. All words were removed that were mentioned less than five
times by all of the submissions combined, unless there was a variance of more than plus or minus four
between the three parties (CCIC, GAC and All submissions) - which implied the word was important to
one group.
Finally, the total number of words for each comparison was then narrowed down even more to best
highlight words that were most important for both parties in a particular comparison.2
2
For each comparison, words that were 1) not mentioned by either party 2) not mentioned by one party and mentioned under 5 times by the other
party and 3) words mentioned over once but under 10 times by one party with a variance of between 2 and 5 were removed from the graph.
22
A. WHERE DO THEY CONVERGE? GAC VS. CCIC
Figure 4 - Convergence and divergence of GAC discussion paper with CCIC submission to the IAR
This graph shows quite a bit of divergence between GAC and CCIC, evidenced by a large number of
words on both the X and the Y axis. In other words, a large number of words were said frequently by one
party and not at all, or much less, by the other party.
As noted in the previous section, words mentioned most by GAC and least by CCIC include Economic,
Growth, Conflict, Children, International Assistance, and Inclusive. These words were all mentioned
at least 20 times more by GAC than they were by CCIC. Words mentioned most by CCIC and least by
GAC include Development cooperation, Funding, CSO, Outcome, and Women’s rights. These words
were all mentioned at least 20 times more by CCIC than by GAC.
GAC and CCIC aligned most closely on the following words: Women, Sustainable, Impact, Trade,
International Development, Gender, Regional, Vulnerable, Food Security, Gender Equality,
Climate Change and Accountability.
23
B. WHERE DO THEY CONVERGE? ALL SUBMISSIONS VS. CCIC
Figure 5 - Convergence and divergence of CCIC submission with All submissions to the IAR
This graph demonstrates that the CCIC submission and All submissions were fairly aligned. There are no
words along either the X or the Y axis – meaning there were no words that were mentioned frequently in
the CCIC IAR submission and completely absent from All Submissions, and vice versa.
The words of greatest convergence are Women, International Cooperation, Impact, Social, Most
vulnerable, Trade, International assistance, Inclusive, Water, Systems, and Agriculture.
The words said more frequently by CCIC and missing from the All submissions group are Development
cooperation, CSO, Funding, Outcome, Partnership, Climate Change and Policy, all said
approximately 20 times more by CCIC than by all other IAR submissions.
The words more frequent in all submissions and mentioned less in the CCIC submission are Children,
Education, and Youth, all mentioned approximately at least 15 times more in All submissions than in
the CCIC submission.
24
C. WHERE DO THEY CONVERGE? ALL SUBMISSIONS VS. GAC
Figure 6 - Convergence and divergence of All submissions with GAC’s discussion paper
This plot reveals a fairly aligned discourse between GAC and All submissions, as there is a lack of words
along the both the x and the y axis- GAC and All submissions agreed on more words than did GAC and
CCIC. There were no words mentioned by All submissions that did not appear in the GAC discussion
paper. There is also a notable lack of words of high divergence in the bottom right half of the graph,
meaning that All submissions did not use a lot of words that were not used by GAC in the IAR discussion
paper.
There are many more words of convergence in this graph than in the two previous graphs. Words of
highest convergence are (in order of importance) – Trade, CSO, Investment, Vulnerable, Social,
Access, Civil Society, Youth, Local, Health, and Women.
Many words were mentioned by the GAC discussion paper that were not frequently mentioned by All
submissions – words along the top of the plot like Growth, Conflict, International Assistance, Needs,
Economy and Sustainable were among those of highest divergence.
25
Words used most by GAC and least by All submissions are – Conflict, Innovation, Refugees, Peace,
Inclusive, and International Assistance. These words were all used at least 20 times more by GAC than
they were by All submissions.
Words used most by All submissions and least by GAC include – Funding, Systems, Leadership,
Feminis(t/m), and SDGs. These words were on average mentioned five times more by All submissions
than by GAC. This divergence is much lower in degree than most other divergences seen in the dataset.
26
V.
WHAT DOES THIS ALL MEAN? THE NARRATIVE LOOKING FORWARD FOR
CANADIAN GLOBAL DEVELOPMENT COOPERATION
It is possible from all of this analysis to piece together an image of where the international development
and humanitarian community placed the strongest (and least) emphasis in their submissions, and key
areas where they agree or disagree with GAC (and CCIC). The words that make up this narrative are
drawn from the various sections in this report.



It includes the words with the overall highest incidence (both median and word count).,
It also includes the words that have the highest incidence per sub-section (prior to collecting the
data, the 259 words were organized into 17 thematic sections and 16 sub-sections). One word is
included for every five words in a sub-section. So for example, a sub-section with four words has
one word in the narrative; a sub-section with six words has two; and one with 13 words has three. A
second criterion for inclusion in the narrative was that the word also had to appear more than 80
times across all the submissions (so on average at least once per submission).
Finally, it includes some direction in terms of the words where there are major divergences or
convergences with GAC.
So what do the submissions say?
The vision that Canadian civil society brings to the International Assistance Review is firmly grounded in
the framework of the Sustainable Development Goals (808); promoting human rights (862), peace (566)
and gender equality (416); and addressing conflict (528) and inequality (235). It is a vision for a broad
agenda that extends beyond international development (309) to consider broader issues of international
assistance (889) and international cooperation (223). The analysis makes clear civil society’s position that
this approach must be guided by the Official Development Assistance Accountability Act (148) and key
tenets of a human rights-based approach (101), including access (747), accountability (402),
inclusiveness (351) and participation (289), as well as child protection (129). Central to any feminist
(309) approach that the government pursues must be the advancement of gender equality (416) and
empowerment (379). CSOs want to see impact (554) more than just results (470), and they recognize that
achieving impact requires strengthening systems (369).
27
In terms of target populations, there is a strong focus and convergence (among CCIC, GAC and All)
around the need to work with the most vulnerable (509) and marginalized (214) populations, in particular
women (2609) and girls (1029). These two are often grouped together (409). Key target populations also
include children (1175) and youth (954). To ensure no one is left behind, addressing needs (913) through
needs analysis (305) will be important.
In terms of sectoral areas of support, there was a focus on health (1746) and gender (1085), a high degree
of support for education (982), with strong support in the areas of climate change (719), governance
(442) and peace and security (253) and agriculture (257).
In terms of mechanics, CSOs, in particular CCIC, expect that policy (890) and advocacy (123), research
(248), learning (213), monitoring and evaluation (89), and public engagement (131) must accompany
programming. Funding (874) mechanisms must be structured in such a manner that they are long-term
(283) and responsive (118).
Realizing all of these objectives will require partnerships (748); working with local groups (1039), in
particular civil society (744), and connecting this work regionally (convergence CCIC and GAC); and
innovating (587) where possible. Groups placed differing degrees of emphasis on economics (1001) and
growth (378) in development (greater divergence between GAC and CCIC, and GAC and All), relative
to the social (greater convergence between GAC and All) dimensions; but still recognized that trade
(288) has an important role to play, alongside the private sector (270). Furthermore, more efforts (by
GAC and civil society) must take into account environmental (260) considerations, ensuring solutions
are sustainable (939) and that populations, communities and ecosystems are resilient (367).
Realizing this ambitious vision and substantive agenda will require that the government make strong
investments (627) in global development cooperation and demonstrate leadership (383).
28
CREATE YOUR OWN NARRATIVE FROM THESE WORDS!
TOP WORD COUNTS PER SECTION AND SUBSECTION
VISION
TARGETED
POPULATIONS
LOCAL (1039); SUSTAINABLE (939); NEED/NEEDS (913); SUSTAINABLE
DEVELOPMENT GOALS/SDGS (808), CIVIL SOCIETY (744), VULNERABLE (509),
LEADERSHIP (383), INCLUSIVE(NESS) (351), TRADE (288), INEQUALITY (235),
MARGINALIZED (214)
WOMEN (2609), CHILDREN (1175), GIRL(S) (1029), WOMEN AND GIRLS (467),
YOUTH (954)
DESIRED
OUTCOMES
POLICY (890), PARTNERSHIP (748), INNOVATION/INNOVATE (587), IMPACT
(554), ACCOUNTAB(LE/ILITY) (402), SYSTEMS (369)
PRINCIPLES
NEEDS (ANALYSIS) (305), ODA ACCOUNTABILITY ACT (148), CHILD PROTECTION
(129)
CHANGING
LANDSCAPES
INTERNATIONAL ASSISTANCE (889), INTERNATIONAL DEVELOPMENT (309),
INTERNATIONAL COOPERATION (223)
FEMINIST
APPROACH
GENDER EQUALITY (416), EMPOWERMENT/ING (379), FEMINIS(T/M) (309)
HUMAN
RIGHTS
HUMAN RIGHTS (862), ACCESS (747), GOVERNANCE (439), PARTICIPATION
(289), HUMAN RIGHTS BASED APPROACH (101)
ENVIRONMENT RESILIENCE/RESILIENT (TO?), (367)ENVIRONMENTAL (260)
29
GROWTH
ECONOM(IC/Y) (1001)
SECTOR
HEALTH (1746), GENDER (1085), EDUCATION (982), CLIMATE CHANGE
(719), GOVERNANCE (442), AGRICULTURE (257)
INVEST
INVESTMENT IN (627)
MONITORING
& EVALUATION LEARNING (213), MONITORING AND EVALUATION (89)
ADVOCACY
ADVOCACY (123)
RESEARCH
RESEARCH (248)
PUBLIC
ENGAGEMENT
PUBLIC ENGAGEMENT (131)
PEACE
PEACE (566), CONFLICT (528)
FUNDING
FUNDING (874), LONG-TERM (283), PRIVATE SECTOR (270), RESPONSIVE (118)
30
VI.
CONCLUSION
Canadian civil society organizations operating globally recognize the importance and the opportunity of
the federal government’s comprehensive review of Canada’s international assistance.
The feedback that different civil society organizations submitted through the review’s consultation
process reflects both their diversity of views and their consensus on some key areas of focus. Most
prominently, Canadian international development and humanitarian CSOs are clearly looking for a
substantive government focus on women, girls, and gender; health; education; climate change; and
economic issues. CSOs expect the government’s approach to these issues be locally-focused and
sustainable, with attention paid at all times to ensuring the protection and promotion of human rights and
to ensuring gender-transformative change. In terms of both content and process, these priorities align well
with the Sustainable Development Goals – the ultimate goal of which is to leave no one behind in
inclusive and sustainable development. These priorities also reflect the broad nature of Canada’s
international assistance efforts, which encompass issues under the purview of many different departments,
ranging from GAC to Finance Canada to Environment and Climate Change Canada.
In order to successfully implement these diverse and complex substantive objectives, the government will
need to break down traditional programming silos in favour of a more integrated and comprehensive
approach. The analysis conducted here also suggests that CSOs believe government should make a
particular effort to align policy and programming, develop learning and research capacities within
government, and encourage diverse forms of partnership, innovation, and multi-sectoral engagement.
31
ANNEX 1 - LIST OF ORGANIZATIONS
ActionAid International
Action Canada for Sexual Health and
Rights*
Africa Study Group
Aga Khan Foundation Canada*
Aidwatch Canada*
Alberta Council for Global
Cooperation*
Americas Policy Group*
Association québecoise des organismes
pour la coopération
internationale*
Atlantic Council for International
Cooperation*
Avocats sans frontières
BC Council for International
Cooperation*
Canada World youth*
Canadian Association for the Study of
International Development*
Canadian Bar Association
Canadian Christian Relief and
Development Association*
Canadian Coalition for Climate
Change and Development
Canadian Cooperative Association*
Canadian Feed the Children*
Canadian Foodgrains Bank*
Canadian International Development
Platform
Canadian International Education
Policy Working Group
Canadian Labour Congress*
Canadian Lutheran World Relief*
Canadian Network on Corporate
Accountability
Canadian Red Cross*
Canadian Society for International
Health*
CARE Canada*
CECI*
Centre de coopération internationale en
santé et développement*
CIVICUS
Coady International Institute*
CODE*
CoDev*
Council of Canadians with
Disabilities*
Crossroads International*
CUSO*
Development & Peace*
Dignitas*
Engineers Without Borders
Equitas*
Farm Radio International*
Federation of Canadian Municipalities
Food Security Policy Group
Handicap International
InterAgency Coalition on AIDS and
Development*
International Child Protection Network
of Canada
Inter-Council Network of Provincial
and Regional Councils
International Institute for Sustainable
Development
Inter Pares*
KAIROS Canada*
Manitoba Council for International
Cooperation*
Mennonite Central Committee*
Micronutrient Initiative*
MNCH Network of Canada
32
Northern Council for Global
Cooperation*
Ontario Council for International
Cooperation*
L’Oeuvre Leger*
Oxfam Canada*
Oxfam Quebec*
Plan Canada*
Presbyterian World Service and
Development*
Primate’s World Relief and
Development Fund*
Results Canada*
Save the Children*
SOCODEVI*
SOS Children's Village*
Steelworkers' Humanity Fund*
SUCO*
UNICEF Canada
United Church of Canada*
USC Canada*
Veterinarians without Borders/World
Animal Protection*
VIDEA*
Volunteer sending agencies
Water, Sanitation and Hygiene
Coalition
Women's Rights Policy Group
World Renew*
World Vision*
WUSC*
YMCA*
ANNEX 2 - LIST OF WORDS, GROUPS AND SUBGROUPS
VISION
1.
2.
3.
4.
5.
6.
7.
8.
9.
10.
11.
12.
13.
14.
15.
16.
17.
18.
19.
20.
21.
22.
23.
24.
25.
26.
27.
28.
29.
30.
31.
32.
33.
34.
35.
36.
Vision
Policy framework
Policy coherence (for what)*
Diplomac(y/tic)*
Defence
Trade
Leadership (in what)
Sustainable development
goals/SDGs
Agenda 2030
Paris (agreement on climate)
Sendai
Universality
Inclusive(ness)
Exclusion
Integrated
Intersection(ality) (of)
Holistic
Stand alone (stand-alone,
standalone)
Cross-cutting (cross cutting)
Sustainable
Civil society
CSO
NGO
Small and medium sized
organizations (SMOs)
Local
national
Regional
Multilateral
Coalition
network/s
Whole-of-government
Whole-of-Canada
Whole-of-society
Gap (Gaps) (what?)*
Leaving no one behind
Diverse (to describe what?)*
37.
38.
39.
40.
41.
42.
43.
44.
45.
46.
47.
48.
49.
Marginalized
Poverty alleviation (alleviate
poverty)
Poverty reduction (reduce
poverty)
Extreme poverty
Root causes
Inequality
Agents of change
Poorest
Most vulnerable
Vulnerable
In need
need/needs
Asset
TARGETED POPULATION
51.
52.
53.
54.
55.
56.
57.
58.
59.
60.
61.
62.
63.
64.
65.
66.
67.
68.
69.
70.
Adolescent girls
Boys
Children
Cooperatives
Diaspora
Entrepreneurs(hip)
Famil(y/ies)
Girl(s)
Indigenous people
Internally displaced
populations/persons (IDPs)
LGBT
Minorit(y/ies)
Mother(s)
People with disabilities
Refugees
(Micro) Small and medium sized enterprises
Smallholder (small-scale)
farmers
Social enterprises
Women
Women and girls
33
71.
Youth
DESIRED OUTCOMES
71.
72.
73.
74.
75.
76.
77.
78.
79.
80.
81.
82.
83.
84.
85.
86.
87.
88.
89.
90.
91.
92.
93.
94.
Strategy (which/or what?)*
Policy (which/for what)*
Action plan (which/for what)*
Impact
Outcome(s)
Results
Data
Basket/bucket approach
Systems (moving into)
Silos (moving out of)
Collaboration
Consortium
Dialogue
Multi-stakeholder
(Multistakeholder)
Partnership
Public private partnership
(Private public partnership)
Accountab(le/ility)
Global citizens
Social justice
Solidarity
Transparen(t/cy)
Innovation/Innovate (how?)*
Social innovation
Technical innovation
PRINCIPLES
95.
96.
97.
International human rights
Child protection
Early, child and forced
marriage (ECFM)
98. Humanitarian principles
99. neutral(ity)
100. Impartial(ity)
101. Istanbul Principles
102. Aid effectiveness
103. Development effectiveness
(how do they define it)*
104. Paris Declaration/Principles
105. Conventions (list which)*
106. CSO partnership policy
107. ODA Accountability Act
(ODA)
108. Multi-stakeholder/
Multistakeholder
109. Enabling environment
110. Disabling environment
111. needs (analysis)
112. Vulnerability (analysis)
CHANGING LANDSCAPE
113.
114.
115.
116.
117.
118.
119.
120.
121.
122.
123.
124.
125.
126.
Development cooperation
International development
Foreign aid
Development assistance
International assistance
International cooperation
Emerging (economy or
economies)
Conflict state
Failed state
Fragile state
Least developed (country or
LDC)
Low-income (Country or LIC)
Middle-income (country or
MIC)
Public good
136. Gender violence/Genderbased violence
137. Masculinity
138. Power
139. Women, peace and security
HUMAN RIGHTS
140.
141.
142.
143.
144.
145.
146.
147.
148.
149.
150.
151.
152.
153.
154.
Feminis(t/m)
Feminist approach
Gender equality
Women's rights
Empowerment/ing
Gender discrimination
Gender inequality
Gender relations
Gender transformative
Human rights
Human rights based approach
Rights-based development
Labour (rights)
Indigenous (rights)
Social protection
Access
Dignity
Informed
Participation
Quality
Governance
Pluralism
Democracy
Diversity
175.
176.
177.
178.
179.
180.
181.
182.
183.
184.
185.
186.
187.
188.
189.
190.
ENVIRONMENT
155.
156.
157.
158.
159.
160.
161.
162.
163.
FEMINIST APPROACH
127.
128.
129.
130.
131.
132.
133.
134.
135.
169.
170.
171.
172.
173.
174.
Environmental
Adaptation
Mitigation
Biodiversity
Ecosystems
natural resources
Resilience/resilient (to?)*
Sustainability
Water (not in the context of
WASH)
GROWTH
164.
165.
166.
167.
Growth (what kind?)*
Econom(ic/y)
Social
Livelihoods (descriptors?)*
SECTOR
168. Agriculture
34
191.
192.
193.
194.
195.
196.
197.
198.
199.
200.
201.
202.
203.
204.
205.
206.
Capacity building
Child protection
Climate change
Clean energy
Conflict prevention
Disaster risk reduction or
DRR
Economic Development
Education
Emergency preparedness
Energy
Environmental protection
Extractives
Fishing
Food aid
Food security
Food sovereignty
Forestry
Gender
Governance
Health
Humanitarian assistance
Information and
Communication Technology
(ICT)
Infrastructure
Mining
Maternal, newborn and child
health (MNCH)
Malnutrition
nutrition
Peace & security
Public engagement
Reconstruction
Relief
Recovery
Renewable energy
Sexual and reproductive
health and rights/ choices
Shelter and housing
Technical assistance
Trade
Transporation
MONITORING AND EVALUATION
209. Monitoring and evaluation
(Monitoring & evaluation;
M&E, M & E)
210. Indigenous OR traditional
knowledge
211. Knowledge sharing
(knowledge-sharing)
212. Learning
213. Data collection
214. Statistical capacity (Statistics
capacity)
215. Best practices
216. Community of practice
217. Peer learning
ADVOCACY
218. Advocacy
219. Policy and (Advocacy) and
policy
220. Policy development
RESEARCH
221. Research
222. Evidence (evidence-based)
223. Scale-up
224. Existing practices
PUBLIC ENGAGEMENT
225. Public engagement
226. new narrative
227. Global citizen (Global
citizens, citizenship)
PEACE
228. Peace (not just peace &
security)
229. Peacebuilding (peacebuilding)
230. Peacekeeping (peace-keeping)
231. Conflict
232. Stabilization
233. Fragility
234. War
FUNDING
235.
236.
237.
238.
239.
Funding framework
0.7
Timetable (time period?)*
Funding
Humanitarian funding
(funding for humanitarian)
35
240. Climate funding (climate
finance)
241. Decentralized
242. Long-term
243. Multi-year
244. Predictable
245. Responsive
246. Funding mechanisms
247. new and additional
248. Countries of focus
249. Alternative sources of
financing/Alternative sources
of funding*
250. Innovative financ(ing, e
mechanisms)*
251. Development Finance
Initiative (DFI)
252. Remittances
253. Tax collection
254. Tax avoidance
255. Capital flight
256. Private sector
257. Multinational/Transnational
258. Corporate accountability
259. Corporate (social)
responsibility
36