Evaluation of RNTC programme
2005-2010
Final report
Final report – December 2011
Evaluation of RNTC
programme 2005-2010
Corina Dhaene (ACE Europe)
Dirk Barrez (Journalist, Global Society vzw)
Patrick Vander Weyden (FocusUp)
Acknowledgement
ACE Europe would like to thank everybody that contributed to this
evaluation. Former and current management and staff of RNTC have
been most collaborative in sharing information and views with the
evaluators. Alumni have responded very well to the e-questionnaire and
partners in the South in Zambia, Uganda and Indonesia have dedicated
time and attention to the evaluation teams. This has allowed to apply a
participatory approach and contributed to the quality of the evaluation
report and its conclusions and recommendations.
Executive summary
1
This evaluation report is the product of the Evaluation of the Programme of the Radio
Netherlands Training Centre (RNTC). The evaluation was coordinated by ACE Europe and
implemented by a team of three international consultants and three national consultants. The
evaluation was commissioned by the Ministry for Foreign Affairs in the Netherlands (DSO, Social
Development Department, Education and Research Division) and executed in the period MayDecember 2011.
2
RNTC wants to contribute to the development of professional media skills in developing countries
and to the use of media for development and education. Its’ mission is thus twofold: development
of media and media for development. RNTC activities over the last 10 years have been targeted
at individual media professionals and media organisations and consisted mainly of (i) trainings of
individuals (in various settings) and (ii) support to capacity building of both governmental and
non-governmental/private actors in the media sector.
3
RNTC is currently executing its second multi-year plan 2009-2012 with a budget of 17.587.590
euro. The multi-annual plan is co-financed by the Ministry of Foreign Affairs through the budget
for International Higher Education. The funding is a maximum of 6.932.737 euro for 4 years (or
39% of the total budget presented in the multi-annual plan. Other main sources of funding in this
period are Nuffic (through the NFP scholarship programme) and MFS (for projects on community
radio in African countries).
4
The evaluation concerned the programme of RNTC between 2005 and 2010 which covers two
multi-annual plans. The focus was on the activities executed by RNTC in this period. The
evaluation looked in particular at: (i) the international courses in Hilversum provided to midcareer professionals, (ii) the provision of tailor made courses to media or media related
organisations in the South and (iii) the development of projects with media or media related
organisations in the South (community radio projects and support to media training institutions).
5
The main objective of the evaluation as formulated in the ToR was to assess to which extent
RNTC is realising and achieving its strategy, objectives and results, thereby looking at efficiency,
effectiveness, sustainability and issues of accountability and capacity building.
6
The evaluation was executed in three phases: (i) inception phase, (ii) data collection through equestionnaires, semi-structured interviews with stakeholders in the Netherlands, case study on
community radio based on evaluations of several projects and interviews and work sessions with
RNTC alumni and partner organisations in Zambia, Indonesia and Uganda, (iii) analysis,
feedback sessions with the Ministry and RNTC and reporting.
7
The evaluation answered 4 evaluation questions.
pag. 5/78 ACE Europe / Evaluation of RNTC / Final report December 2011
Key evaluation questions (EQ)
EQ1: Are the activities mentioned in the programme executed efficiently?
EQ2: Was the organisation able to formulate and provide relevant and tailor made interventions for
media professionals, media organisations and training institutes taking into account the context and
the challenges?
EQ3: Was the organisation able to realise changes in capacity at the level of the media
professionals, media organisations and training organisations?
EQ4: what are the longer term effects of interventions and services provided to media
professionals, media organisations and training organisations?
8
The main conclusions are the following:
9
RNTC is a an organisation working in a professional manner and offering quality products to mid
career professionals and their organisations. The main interventions are trainings and (to a
lesser extent) support to the capacity strengthening of media organisations. Clearly, the capital
of RNTC rests for a great deal with its pool of trainers and the way they are managed.
10
Theory of change - The first part of the mission, ‘development of media’ rests upon the firm
conviction that change in media can be realised by training individual people and more in
particular mid-career professionals. The strong focus on the indivual as actor of change was
successfully translated in training courses and modules of high quality and effectiveness can be
rated as high.
11
The second part of the mission of RNTC, ‘media for development’ was not well translated in
operational terms. Clearly, international education has remained the core business of RNTC and
a full integration of development projects and approaches aimed at organisational capacity
building was not realised, unless partly within specific projects. ACE Europe concludes that
investing more in capacity building of media organisations to influence on the media environment
was not possible for RNTC as a relatively small organisation.
12
ACE Europe concluded that the negotiations with the Ministry for the current multi-annual plan in
the course of 2008 had a clear influence on the formulation of the mission and objectives by
RNTC. Although the DSO funding was primarily aimed at supporting international and
professional education to mid-career professionals, the requirements of the ‘subsidiebeschikking’
overstrechted the scope of the DSO funding and the RNTC had to take development action as
well without DSO providing the funding.
13
This created a field of tension for this evaluation: based on the mission statement of RNTC the
training institute has also been assessed as a development organisation (using the DAC criteria).
This approach nevertheless allowed ACE Europe to better identify the added value and potential
of RNTC. ACE Europe thus concluded that (i) through the training of mid-career professionals
RNTC can indirectly influence on the capability of media organisations to deliver results, (ii) that
RNTC has been able to articulate better its sensitivity for development and (ii) that RNTC has
developed a niche product/approach on community radios which has great development
potential and potential for learning about integrated capacity building.
14
Conclusions on efficiency - RNTC has been able to attract external funds for projects in the
South but its dependency on DSO funding for its functioning and the provision of the international
courses is clear.
15
With its international courses and the projects on community radio, RNTC offers two strong
products. All findings support the conclusion that the international courses, funded through DSO
and Nuffic are high quality products with a good price/quality ratio. Price of the courses did not
significantly rise of in the period 2005-2010. With its community radio projects RNTC
demonstrated capacity to explore the boundaries of a training institute: local capacity needs for
community radio in the South were well combined with the strength of RNTC which is Training of
Trainers (ToT) organised in situ, of good quality and learner centred
16
RNTC holds a strong view on how to strengthen capacities of individuals (vision on adult
learning) which was recently documented in a trainers’ manual. Interventions for capacity
strengthening at organisational level are not that well supported by a consolidated and shared
policy. Yet, RNTC community radio projects seem to follow a more integrated approach: training
of individuals is combined with support to radio stations and their organisational development
(more in particular their capability to be, to do and to relate).
17
RNTC has a system to ensure follow-up on a number of performance indicators at the level of
activities. RNTC has also developed several instruments to collect information about the
appreciation of trainees from the international courses in Hilversum. These data are used to
monitor and improve the quality of the courses. There are some weaker points in the capacity of
RNTC with regards to effective monitoring and evaluation: RNTC does not conduct specific
tracer studies of alumni, data collection at the level of trainees from tailor made courses is far
less developed and generally the instruments, tools and systems of RNTC to capture and
document change processes in the South are limited.
18
Conclusions on relevance - RNTC is able to provide tailor made services to mid-career media
professionals and to media organisations. Interventions are relevant in several respects: with
regards to the broader context of media development, the spearheads and policy of Dutch
development cooperation, the requirements of the media sector and the required skills for media
professionals within the context of democratisation. Alumni (and organisations) have strongly
confirmed the relevance of the RNTC trainings.
19
ACE Europe noted that tailor made interventions with organisations in the South depart from
perceived needs as expressed by the beneficiary organisation and are most often connected to
the capability of the organisation to deliver results (or capability to do). Within community radio
projects, a broader analysis of capabilities of the beneficiary radio stations is executed.
pag. 7/78 ACE Europe / Evaluation of RNTC / Final report December 2011
20
When assessing relevance, the evaluators also noted the efforts of RNTC to connect to civil
society through establishing cooperation with NGOs such as HIVOS, War Child and Press Now
which are seen as closer to civil society.
21
Conclusions related to effectiveness - RNTC engaged mainly in international courses and in
community radio projects (together 3/4 of the total budget): for both interventions effectiveness
can be rated as good. The international courses are effective instruments to strengthen the
professional capacity of mid-career media professionals. They induce changes in the domain of
professional skills, work ethics and mind-sets. More in particular the course topics, RNTC
approach (practical assignments with feedback), the quality and mentorship of trainers and the
international environment have contributed to this. For many alumni, this course has changed
their professional career for the better. The link between strengthened individuals and
strengthened organisations is not evident though. Although one alumnus can be an actor of
change, many other conditions in the environment determine whether or not the alumnus with
her/his acquired skills can change something in the organisation.
22
Tailor made trainings and projects primarily strengthened the capability of beneficiary
organisations to deliver results: increased professional skills amongst staff were noted next to
more efficient organisation of the work and the newsroom, development and introduction of new
tools and products, etc. Generally, the effectiveness of tailor made interventions at
organisational level and their sustainability in Zambia and in Uganda was limited. More succesful
experiences with tailor made interventions in Indonesia point at the importance of the set-up of
the intervention (sufficient budget, longer intervention or engagement) and the readiness of the
beneficiary organisation (ownership, view on how to integrate aquired skills) as factors of
success.
23
Conclusions regarding longer term effects - Through its courses and the alumni, RNTC has
been able to contribute to increased concern and professional skills for providing information
about relevant social issues (relevant to the people) looking at issues from different angles
(balanced reporting) and this in different parts of the world. There is some evidence that
beneficiary media organisations generate more information to be used by the public to hold
government accountable. This is more clear at the level of projects and more comprehensive
tailor made interventions: a better connection to national news, translating news to local context,
reporting on relevant development issues could be noted in a number of cases. Further, there is
evidence from the field (field missions and case on community radio) that changes in the
capability of media organisations to deliver results contributed to increased opportunities for the
public to express their opinion about relevant social issues through social media, phone ins, and
feedback sessions with the public. This can be considered as an important building block for
supporting public debate and dialogue.
24
Finally, it is important to note that the RNTC community radio projects have sustained existing
media in places where freedom of expression is under threat, as such identifying and nurturing
nascent media capacity which can form a basis for further development once the environment
becomes more conducive. Further, the RNTC way of working supports the diversity of the media
landscape: encouraging media professionals to depart from local reality and to identify the most
appropriate format and packaging allows more diversity.
For the recommendations, ACE Europe has taken into account, the conclusions, the
opportunities in Dutch development cooperation, the strengths of RNTC and the weaker aspects,
the mission of RNTC and the fact that RNTC (in the course of 2011) has transferred its project
related activities to another organisation, Free Press Unlimited.
ACE Europe formulated 7 recommendations which are the following:
1. Translate the mission ‘media for development’ in more operational terms (from the
perspective of RNTC as an educational institute). This in order to consolidate and further explore the
potential of the RNTC sensitivity to development.
2. Develop an organisation wide (but light) M&E system that focuses on the changes and the
competences at the level of media actors and professionals. This to highlight and document better
the effects of trainings and other interventions.
3. Ensure that RNTC is more explicit about its vision on capacity building (both at individual and
at organisational level). This to consolidate already existing experience.
4. Sustain and market the international courses as a strong RNTC product and niche.
5. Ensure that the niche of supporting community radios, is sustained, even now when the
projects have been transferred to Free Press Unlimited.
6. Develop a third niche, that of supporting the capacity development of training institutes or
training initiatives in the South. This to allow the RNTC approach to be more wide spread in the
South.
7. Continue to operate as a provider of training ‘at request’ or based on service provider
contracts with other NGOs/donors (taking into account the lessons learnt from the success of
international courses).
pag. 9/78 ACE Europe / Evaluation of RNTC / Final report December 2011
List of abbreviations
DSO
Social Development Department, Education and Research Division
(Dutch Ministry of Foreign Affairs)
GIEBTA
Georgian Initiative for the Enhancement of Broadcasting Training
Activities
HRD
Human Resources Development
ID
Institutional development
INFORMOTRAC
Initiative For Mobile Training of Community Radio
MATRA Programme
Maatschappelijke Transformatie Programme
M&E
Monitoring and Evaluation
MFS
Mede Financieringsstelsel
NESO
Netherlands Education Support Office
NFP
Netherlands Fellowship Programme
NIMD
Netherlands Institute for Multi-Party Democracy
OD
Organisational development
PANOS
Greek word, referring to a ‘beacon’, lighting the way
RNTC
Radio Nederland Training Centre
RNW
Radio Nederland Wereldomroep
SBM
School for Broadcast Media
SEERTI
South East European Radio Training Initiative
TMF
Thematische Medefinanciering
ToT
Training of Trainers
Table of Content
1
Introduction
13
1.1
Context and objective of the evaluation
13
1.2
Object of the evaluation
14
1.3
Evaluation questions
18
1.4
Evaluation approach, techniques and limitations
19
2
Theory of change
25
3
Evaluation question 1: findings on efficiency
27
4
5
6
3.1
Input-output
27
3.2
Organisation of the work
31
Evaluation question 2: findings on relevance
35
4.1
Ability to provide tailor made services to mid-career professionals
36
4.2
Ability to provide Tailor made services to organisations
37
4.3
Addressing the challenges involved in the development of responsive and
responsible media
39
Evaluation question 3: findings on effectiveness
45
5.1
Changes in the capacity of media professionals
45
5.2
Changes in the capacity of media organisations and training institutes
48
Evaluation question 4: findings on sustainability and longer
term results
51
6.1
pag. 11/78
Role of alumni in realising (organisational) change
ACE Europe / Evaluation of RNTC / Final report December 2011
51
7
8
6.2
Role of media organisations in development debate
54
6.3
Contribution to improved access to quality training for media professionals
55
6.4
Changes in media environment
55
Conclusions and recommendations
57
7.1
Analysis: reflection on theory of change, interventions and effects
57
7.2
Conclusions
58
7.3
Recommendations
61
Annexes
65
8.1
List of documents consulted
65
8.2
Overview of interviews in the Netherlands
66
8.3
Overview of tailor made courses
68
8.4
Overview of projects
71
8.5
Results from e-questionnaire alumni
73
8.6
Field mission report
73
8.7
Community Radio Case description
73
8.8
Financial information from RNTC on costs per trainee
74
25
1
Introduction
1.1
CONTEXT AND OBJECTIVE OF THE EVALUATION
The Radio Netherlands Training Centre or RNTC was set up in 1968 by the Ministry of Foreign
Affairs and the Wereldomroep RNW) as an institute for international education in the field of
media and development. RNTC still falls under the responsibility of the general director of the
RNW - though with a strict financial division between the two - and is advised by an ‘Adviesraad
RNTC’.
26
As a training centre, RNTC is aimed at improving professional skills of mid career professionals
in the media sector in developing countries. Its activities over the last 10 years have been
targeted at individuals and media organisations and consist of (i) different types and forms of
trainings, (ii) capacity building of both governmental and non-governmental/private actors in the
media sector (radio, TV, internetand written press, media training institutes, NGOs with a media
component) and (iii) (support to the) production of educational materials. RNTC wants to
contribute to the development of professional media skills in developing countries and to the use
of media for development and education.1 To support the development aim, RNTC wants to pay
specific attention to marginalised groups in society and strengthen responsible and responsive
media in developing countries and countries in transition , focussing on their public service riole
and their role as agents of change.
27
RNTC is currently executing its second multi-year plan 2009-2012 with a budget over 17.587.590
euro.2 The plan is executed with 14 FTE of staff.3 In June 2011, there was a partial merger of
RNTC (project related activities) with two other Dutch NGOs (Press Now and Free Voice) which
has lead to the creation of a new organisation: Free Press Unlimited. The execution of the RNTC
programme is ensured through a contract with Free Press Unlimited which determines that
former RNTC workers dedicate max 40% of their time to support RNTC activities.
28
The multi-annual plan is partly financed by the Ministry of Foreign Affairs in the Netherlands
(department DSO, budget for International Higher Education) through the Subsidiebeschikking of
February, 19th 2009 (DCO/OO-010/09) for an amount of maximum 6.932.737 euro for 4 years
(or 39% of the total budget presented in the multi-annual plan). The Memorandum of December
18th 2008 on the execution of the contract states that 4.292.224 euro of this funding (or 61% of
the DSO funding and 24,4 % of the total budget for the RNTC programme) is earmarked for
activities (to be further specified by RNTC in its multi-annual plans) and 2.640.513 euro as
contribution to the financing of the core activities of RNTC (which is 38% of DSO funding and
1
Mission of RNTC: multi-year plan 2005-2008, 3 and multi-year plan 2009-2011, 4.
Ministry of Foreign Affairs, Memorandum of December 2008, page 10.
3
Since the beginning of June 2011, RNTC is working with a smaller team of 6 FTE, while 8 FTE (9 people) have moved to
Free Press Unlimited.
2
pag. 13/78
ACE Europe / Evaluation of RNTC / Final report December 2011
15% of the total budget). It was understood4, both by the Ministry and the RNTC that the DCO
contribution for activities was – primarily – for funding the organisation of the regular international
courses in The Netherlands for individual mid career professionals and for refresher courses for
alumni of these international courses. To al lesser extent the funding could be used for capacity
building of media organisations in partner countries (seed money to explore opportunities for
further cooperation). It is RNW that signed the contract with the Ministry.
29
In the Subsidiebeschikking, it was determined that the programme of RNTC would be evaluated.5
The ToR for the evaluation were drafted by the Ministry in consultation with RNTC. The Ministry
awarded the assignment to ACE Europe, a Belgian consultancy firm, on the basis of a technical
offer. This report is the final output of this evaluation.
30
The main objective of the evaluation as formulated in the ToR is to assess to which extent RNTC
is realising and achieving its strategy, objectives and results, thereby looking at efficiency,
effectiveness, sustainability and issues of accountability6 and capacity building. The ToR did not
require the evaluators to execute an impact assessment or an institutional evaluation of RNTC.
31
The ToR for this evaluation requested that the evaluation would align with another evaluation,
commissioned by the Ministry looking at the NFP scholarship programme as managed by
NUFFIC. Due to a late start of the NFP evaluation, this was not fully realised7.
1.2
32
OBJECT OF THE EVALUATION
The focus is on the activities executed by RNTC in the period 2005-2010 . This period covers two
multi-annual plans: the full plan of 2005-2008 and first half of the plan 2009-2012. The evaluation
looked at the activities of RNTC disregarding their source of funding8 and more in particular: (i)
the regular international courses in Hilversum provided to mid-career professionals (and the
refresher courses organised in the South to a lesser extent), (ii) the provision of tailor made
courses to media or media related organisations in the South and (iii) the development of
projects with media or media related organisations in the South.
33
When looking at the objectives/results in the muli-annual plans (see table below), the evaluation
focused on objectives 1, 2 and 4 (for the multi-year programme 2005-2008, with particular
4
This is not mentioned in any document but confirmed in interviews and discussions with the current DCO representative
and the former and current management of RNTC.
This was also determined in the previous Subsidiebeschikking: an evaluation was planned in 2007, but not executed by
the Ministry.
6
in 2008, the Minister of Development cooperation requested the Dutch organisations for development cooperation to
initiate projects around local accountability, in cooperation with each other and with Dutch Embassies. RNTC did not
develop specific separate initiatives labelled as ‘accountability projects’. However, the study in the inception phase
revealed that accountability runs through many RNTC activities.
7
A concept report for this evaluation is expected to be ready in February 2012. ACE Europe took into account the ToR for
this evaluation when preparing the interview guidelines and questionnaires. This to allow some comparison.
8
The idea is that the contribution of DSO to the core functioning of RNTC has made it possible for RNTC to undertake its
activities. As such, DSO funding has contributed to the overall programme of RNTC. Therefore, activities which are not cofunded by DSO have been taken into account in this evaluation.
5
attention to community radio but not on e-learning or the production of educational materials in
Latin-America) and on results A, B, C and D. As such, ACE Europe’s evaluation covers more
than 3/4 of the budget of the multi-annual plans.
Objectives 2005-2008
Activities
Objective 1: strengthening professional
capacity of individuals
-International / basic course
-refresher course
-tailor made course
-e-learning
Objective 2: strengthening local training
capacity
-Partnerships
-Development of project proposals
-implementation of projects
Objective 3: support to networking
-Partnerships
network/websites
-courses
-new networks
-RNTC alumni network
Objective 4: support to media in relation to
development
-Production of materials
-Support to community radio
-projects aimed at diversity
Objective 5: professional capacity of
professionals at RNW
-Courses in consultation with RNW staff
-Loopbaan-ontwikkelingstraject of 42 training modules
Results 2009-2012
Activities
Result A:
People working in, with and for media have
increased their professional activity
-International / basic course
-refresher course
-tailor made course
Result B:
Media organisations have increased their
capacities to fulfil their role
-needs assessment/strategic advice
-tailor made courses
-projects
Result C: Organisations and institutions
working in or for the media sector have
increased their capacities to develop and
sustain an enabling environment for the media
-needs assessment/strategic advice
-tailor made course
-projects communication faculties
-projects training institutes
Result D:
Non-media organisations and networks have
increased their capacities to work with media
for development
-needs assessment/strategic advice
-projects participation and diversity
-projects media in education
Result E:
Professional networks and public involvement
-(active) participation in networks
Result F: M&E
34
In the following, the three main types of activities under evaluation will be described. An overview
is provided in annex.
pag. 15/78
ACE Europe / Evaluation of RNTC / Final report December 2011
1.2.1 REGULAR INTERNATIONAL COURSES AND ALUMNI
35
RNTC offers a number of international courses in the Netherlands each year. This is the most
important activity of RNTC in terms of budget which is in conformity with the history of the RNTC,
the DSO funding and its core business. The duration of the course varies between 2 weeks and
12 weeks. The topics and the learning methods of each course are defined and can be adjusted
(a full overview of content and learning methods has been prepared by RNTC to inform the
evaluators and can be consulted on request). RNTC also organises refresher courses in the
South where alumni of the region are invited to. All of these courses are co-funded through NFP
scholarships administered by Nuffic (except for the 2 weeks course on broadcast management in
2008-2009)9.
9
Year
Courses
2005
12 weeks: International course broadcast journalism (topic: conflict and cohesion)
12 weeks: International course soaps and society
6 weeks: International course internet for journalists
Refresher course in Nepal: “social conflict and cohesion in South (East) Asia: the role
of broadcast media”
Refresher course in Sri Lanka: “New programme formats for the public broadcast
media in Asia”
2006
12 weeks: International course broadcast journalism (topic: globalisation)
12 weeks: Educational programme production (topic: children and development)
6 weeks: International course training the trainers
Refresher course in Kenya: “New Programme Formats for the Public Broadcast Media
in Africa"
2007
12 weeks: International course broadcast journalism (topic: migration)
12 weeks: International course facts and formats
6 weeks: International course internet for journalists
Refresher course in Zambia: “Commercial pressures and ethical dilemmas in
broadcast journalism”
2008
12 weeks: International course broadcast journalism (topic : sports and society)
12 weeks: International course soaps and society
6 weeks: International course training the trainers
2 weeks: International course broadcast management
Refresher course in Vietnam: “Cross Media Approaches in Development
programming”
Refresher course in India: “training the trainers: Fresh approaches to broadcast
training”
2009
12 weeks: International course broadcast journalism (topic: social conflict and
cohesion)
12 weeks: International course educational programmes (topic: youth and
development)
6 weeks: International course internet for journalists
2 weeks: International course broadcast management
Refresher course in Ethiopia: “getting the untold stories told – the power of journalistic
storytelling”
Refresher course in Zambia: “using educational media to engage children, young
people and women in development”.
NFP requires that new courses are first tested before they can be accepted in the NFP scheme.
2010
36
12 weeks: International course broadcast journalism (topic: good governance and the
broadcast journalist)
12 weeks: International course facts and formats
6 weeks: International course internet for journalists
2 weeks: International course broadcast management
According to the RNTC database the total number of alumni is 457 (2005-2011). The profile of
the alumni can be described as follows:
−
When analyzing the frequencies of participants in the different types of international
courses10, we find that more than 37 per cent of the alumni did participate in the International
course Broadcast Journalism. About 17 per cent of the total number of alumni attended the
international course Internet for Journalists. 15 per cent of the alumni participated in the
international course Facts and Formats. The international courses ‘Soaps and Society’ and
‘Training the Trainers’ are less often organized and thus represent a lower percentage of
alumni. Least important in number of participants are the courses ‘Educational Programme
Production’ and ‘Broadcast Management’: the target group for these courses is much
smaller.
−
Most of the course participants have a professional position in their home country in Radio
and/or TV sector. Positions in internet and written press are underrepresented.
−
Most of all alumni (56 %) are living in Africa and about 38 % of the alumni are from Asia.
Both groups make up about 94 per cent of the course participants of RNTC. Another
important fact is that both men and women are almost equally represented for the total group
of participants as well as for the group of participants of the two most important continents
Africa and Asia. RNTC closely follows the requirements of the NFP scheme focusing on
scholarships for women and people from sub-Saharan Africa.
−
Analyzing the country of origin of the RNTC alumni provides us with the following picture: the
457 alumni are coming from 56 countries. Most alumni are coming from Kenya (N=52 –
11,4%), Nigeria (N=42 – 9,2%), Nepal (N=34 – 7,4%), Uganda (N=30 – 6,6%), Zambia
(N=26 – 5,7%) Ghana (N=26 – 5,7%), Indonesia (N=25 – 5,5%). All other countries have
percentages below five per cent.
1.2.2 TAILOR MADE COURSES
37
RNTC provided the evaluators with an overview of tailor made courses executed in the South:
funding can be DSO, other donors or the beneficiary organisations11. In total 46 tailor made
10
Raw data are available at ACE Europe. All raw data will be put at the disposal of RNTC.
Beneficiary organisations that partly finance the tailor made course are not easy to identify through the budget.
Beneficiary organisations that can be identified in the years 2008-2010) for tailor made courses having funded their training
11
pag. 17/78
ACE Europe / Evaluation of RNTC / Final report December 2011
courses have been organised of which the majority in Asia and in Africa (sometimes with study
visits and short trainings in the Netherlands). In terms of budget expenditure the activities labeled
as tailor made courses or on the spot courses are less important.12 Each year generally around
4% of the total budget of RNTC was spent on tailor made projects13. Generally, RNTC does not
establish or continue formal partnerships over the tailor made courses, with Bhutan Broadcasting
Services as a clear exception (3 tailor made courses in this period). This means (i) that tailor
made course in general do not lead to formal partnerships14 and (ii) that longer term
engagements are based on concrete activities. RNTC distinguishes different types of
organisations its database.15 Tailor made courses are primarily targeted at media houses (24 in
total) and to a lesser extent to training/educational institutes (10) and development organisations
(mainly Northern and international). An overview of the courses is provided in annex.
1.2.3 PROJECTS
38
RNTC provided the evaluators with an overview of projects executed. In total, 4016 projects were
executed by RNTC in the period concerned, of which 23 in African countries and 12 in LatinAmerican countries, with the remaining in Asia. Project execution is the second most important
activity of RNTC (in budget terms) and is financed from external sources (other than the DCO
funding for activities). It should be noted that some of the projects have the character of a tailor
made course (or combine several tailor made courses). RNTC distinguishes different types of
organisations: in the period 2005-2010, RNTC mainly worked with media organisations (9
partners), 11 development partners (mainly Dutch NGOs or international organisations) and 17
training/educational institutes. Seven of these project have been studied more closely in this
evaluation through field missions (2 projects) and the case on community radios (see further).
1.3
39
EVALUATION QUESTIONS
ACE Europe formulated 4 key evaluation questions based on the ToR for this evaluation, the
technical offer and desk study. These questions were further detailed through judgement criteria
and indicators such serving as evaluation framework and guidelines (for more details, see
are (through the overview of tailor made courses): the Dutch Bernard van Leer (in 2010) ICTA (Sri Lanka in 2010) and El
Faro (El Salvador 2008).
12
It should be noted that outside of the courses, labelled as tailor made courses or ‘on the sport courses’, RNTC has
engaged with other partners in development of courses, such as e-learning courses in Latin America and in Asia (with
partners AMIC, FELAFACS) which have not been counted here.
13
When only looking at the DSO funding the picture is comparable: in 2005, 2% f the DSO funding was spent on tailor
made courses, in 2010 this percentage was around 3,7.
14
There is evidence (also from the field) that RNTC keeps the door open by engaging in non formal contacts with
organisations.
15
The types are: media, training/educational institutes, development organisations, government, consultancies, other and
RNTC/TNW, It should be noted though that media houses and training institutes can be government owned/part of
government. The type ‘government’ refers to ministries.
16
The country of intervention is taken as starting point. However, interventions in more than one country are sometimes
connected to one particular project/donor.
inception report). The framework was also designed to allow to identify and describe the quality
and added value of the RNTC activities.
40
The evaluation questions are related to the DAC criteria of relevance, efficiency, effectiveness
and sustainability. The relation to the DAC criteria is highlighted in the table below. The issue of
local accountability is addressed in evaluation question 2 and 3. The issue of capacity building is
addressed in evaluation question 3.
Key evaluation questions (EQ)
DAC criteria
EQ1: Are the activities mentioned in the programme executed
Efficiency
efficiently?
EQ2: Was the organisation able to formulate and provide relevant and
Relevance
tailor made interventions for media professionals, media organisations
and training institutes taking into account the context and the
challenges?
EQ3: Was the organisation able to realise changes in capacity at the
Effectiveness
level of the media professionals, media organisations and training
organisations?
EQ4: what are the longer term effects of interventions and services
Sustainability
provided to media professionals, media organisations and training
organisations?
1.4
41
EVALUATION APPROACH, TECHNIQUES AND LIMITATIONS
Execution of the evaluation - This evaluation was executed in three phases with a clear output
for each phase (included in annex): (i) inception phase (output: inception report), (ii) data
collection phase through e-questionnaires, case study of community radio projects and field
missions in Uganda, Zambia and Indonesia17 (one report with the findings of the three field
missions, results of e-questionnaires, community radio case description) and (iii) phase of
analysis and conclusions (final report).
42
The evaluation was executed by a team of three people: Corina Dhaene (ACE Europe) took the
lead of the evaluation and executed the field mission to Uganda, Patrick Vander Weyden (Focus
Up) was responsible for the e-questionnaires and executed the mission to Zambia, Dirk Barrez
(Global Focus) contributed to the development of the evaluation framework as a media
17
The ToR proposed that Indonesia and two African countries would be included in the sample. It was important to choose
countries with a significant proportion of alumni and with evidence of RNTC activities aimed at strengthening media related
organizations in the South. ACE Europe validated this proposal in the inception report and used a set of criteria to select
the African countries. As such Zambia and Uganda were chosen.
pag. 19/78
ACE Europe / Evaluation of RNTC / Final report December 2011
professional and executed the mission to Indonesia. In each of the countries visited, local
consultants have been selected to compose teams: Ezki Suyanto (Indonesia), Agnes Phiri
(Zambia) and Denis Jjuuko (Uganda).
43
Study of projects and tailor made trainings - Through the field missions and the case on
community radio, different projects and tailor made trainings have been studied and
organisations have been visited. The tables below give an overview of organisations visited
through field missions and projects studied in this evaluation. A total of 16 projects has been
studied, of which 10 have been implemented in Sub Sahara countries in Africa. In addition, ACE
Europe has taken into account existing external evaluations of other projects; e.g. evaluations on
the projects GIEBTA (in Georgia) and SEERTI (in Bulgaria).
18
Field mission: (type of) organisations visited
National
Other media
Media educational institute
broadcasting
organisations
Uganda
(representing
6,6% of
alumni)
-Uganda
Broadcasting
Corporation
(UBC)
(-Mega FM
Radio Gulu)
Indonesia
(representing
5,5% of
alumni)
19
Zambia
(representing
5,7 % of
alumni)
Zambia
National
Broadcasting
cooperation
-Uganda
Radio
Network
-WBS
Television
Makerere University
-Jakarta Post
-Asosiasi TV
Lokal Indonesia
-Trans TV
(-Metro TV)
School for Broadcast Media
Zambia
Educational
Broadcast
Services (part
of Ministry)
Zambia Institute for Mass
Communication/Educational
Trust
NGO
specialised in
media
Uganda
Women’s
Media
Association
(UWMA)
NGO
active in
media
TPO
Alliance of
independent
Journalists
(AJI)
Field mission: projects/tailor made courses studied and their funding (source: RNTC)
DSO
contribution
Other contribution
part of 30.000
3.028,00
0,00
Makerere University, Department
of Mass Communication
7.500,00
0,00
University Makerere
EUR 6.417
TPO
part of 21.440
13.320,00
0,00
Year
Country
Beneficiary/partner
Budget (euro)
2005
Uganda
Uganda Media Women's
Association, Mama 101.7 FM
2010
UGANDA
2010
Uganda
18
20
Organisations mentioned in Italic have not been involved in tailor made trainings or projects but only employed a
significant amount of alumni (3 or more in the period 2005-2010). Those between brackets have not been visited during the
field mission.
19
Two organizations in Zambia each organized an RNTC refresher course in the period 2005-2010: Zambia National
Broadcasting Cooperation in 2007 (on "Commercial pressures and ethical dilemmas in broadcast journalism") and Zambia
Institute for Mass Comunication in 2009 on "Using educational media to engage children, young people and women in
development".
20
This means that the budget is taken from a budget line which is not allocated as such to this particular initiative but is
dedicated to tailor made projects or preparation of new projects under one of the anticipated results.
2006
Indonesia
School for Broadcast Media
122.292,00
0,00
BBC: 121.775
2007
Indonesia
Asosiasi TV Lokal Indonesia
94.297,00
0,00
NESO: 89.787
2009
Indonesia
The Jakarta Post
55.649,00
0,00
NESO: 47.876
0,00
NESO: 66.068
0,00
RNW: 125.545 EVD:
472.287
10.193,00
0,00
7.862,00
0,00
2010
Indonesia
20102013
Indonesia
2007
Zambia
2010
Zambia
Alliance of Independent
76.435,00
Journalists - Jakarta Chapter (AJI)
School for Broadcast Media (in
cooperation with 4 regional
597.832,00
training facilities
Zambia Institute of Mass
part of 52.314
Communication, Educational Trust
Educational broadcast services BS
part of 20.400
Case community radio: projects studied and their funding
Year
Country
Title
Beneficiary/part
ners
2003-2006
GuinéeINFORMOTRA
Centro de
Bissau
C I, initiative for Treinamento
Sierra Leone
mobile training
Informorac
Senegal
of community
(Guinea
radio
Bissau)
2007-2010
DRC
Liberia
Guinée
Conakry
Informotrac II
(part of
programme
'Strengthening
Media:
Empowering
people' with
Press Now)
Informorac
(Senegal)
31 stations, nr
of trainings not
mentioned, 850
participants
(gender division
not clear)
Training Centre
Informotrac
(Sierra Leone)
Remacob
Liberia Media
Centre (LMC)
4RGUI
1.255.887 (for
the project on
community
radio)
2010-2012
Liberia
Informotrac
Liberia Media
Centre
2009-2011
RDC
Renforcement
des capacités
en gestion des
radios de
proximité du
Congo' (FCI I)
Fédération des
Radios de
Proximité de la
RD Congo
(FRPC)
Renforcement
des capacités
des radios
locales au
Congo“ (EU)
Réseau
National des
Journalistes
des Radios
Communautair
es du Congo
2009-2011
pag. 21/78
Brazzaville
Budget total
(euro)
3.527.792
413 trainings for
65-75 radio
stations, with
financial
support to 3240 radio
stations
Training for
partner staff (13
in total)
630.016 euro
For 17 radio
stations
199.164 euro
25 radio
stations, 67
trainings (no
information
about nr of
participants)
341.926 euro
(30 months)
8 radio stations,
1 federation,
340 people
trained through
47 trainings (1/3
ACE Europe / Evaluation of RNTC / Final report December 2011
DSO
funding
0,00
Other
contribution
DGIS, TMF
(budgetlijn
Thematische
medefinancie
ring)
0,00
MFS
0,00
EC
0,00
DFIDCooperation
Francaise –
SIDA
0,00
EU
2009-2011
Surinam
Lokale Radio Een stem in
ontwikkeling
3 radio station
with OLON
(Organisatie
Lokale
Omroepen
Nederland) and
RNW
RNTC
submitted on
behalf of RNW
44
women)
320.449 euro
0,00
Twinning
between 6 radio
stations, nr of
trainings, nr of
participants and
gender division
not clear
UTSN Dutch
Twinning
facility
Evaluation techniques - To answer the evaluation questions and collect information on the
international courses, tailor made courses and projects, ACE Europe used the techniques of
desk-study, semi-structured interviews, e-questionnaires, self-evaluation workshops with
partners and alumni in the South:
−
Information on the international courses was mainly collected through e-questionnaires to
alumni and their employers;
−
Information on tailor made courses and projects was mainly collected through field missions
(Indonesia, Uganda and Zambia), desk study of related documents, and e-questionnaire to
beneficiary organizations.
45
The table below gives an overview of the stakeholders involved. An overview of people
interviewed and documents consulted is attached in annex.
Stakeholders
Techniques of data collection
RNTC management
−
Semi-structured interviews
RNTC staff
−
Semi-structured interviews
Trainers from the RNTC pool
−
Semi-structured interviews
−
Semi-structured interviews
Staff from the Ministry of Foreign
Affairs
−
semi-structured interviews
Staff from Dutch embassies (in
particular: Nicaragua, Rwanda,
Zambia, Uganda and Indonesia)
−
−
Questions per email
Semi-structured interviews
Staff/management from other Dutch
NGOs (in particular: Press Now, War
Child, NIMD, Nuffic, HIVOS)
RNTC alumni of international basic
courses and refresher courses
Employers of alumni
−
e-questionnaire
For alumni in the countries of the field mission:
−
self-evaluation workshop based on the technique
of the Most Significant Change
−
semi-structured interviews
−
e-questionnaire
For organisations in the countries of the field mission:
−
semi-structured interviews
46
Organisations in the South that
benefited from tailor made courses
and projects
−
−
Donors (active in media sector)
−
e-questionnaire
case study (on community radio projects in
Africa)
For organisations in the countries of the field mission:
−
semi-structured interviews with management and
staff
−
self-evaluation sessions based on reconstruction
of time line and capacities
Semi-structured interviews
Limitations – Limitations are connected to the availability of data at the level of RNTC and the
application (see also inception report) of the evaluation techniques:
47
The M&E system of RNTC has been primarily collecting data regarding the appreciation of
alumni of basic international courses in Hilversum thus also answering to the information
requirements from Nuffic regarding the NFP scholarships. RNTC does not dispose of a system
to aggregate or analyse these data. There has been no systematic data collection about effects
and results at organizational level. Such data are available though when looking at specific
projects (depending on the requirements of the donors for that project). As such, there is little
information to highlight changes at the level of organisations. The collection of additional data by
ACE Europe within the framework of this evaluation is not enough to fully cover for the gap of
data at the level of RNTC. As such, the evaluation is not able to pronounce itself on the
effectiveness and impact of the full RNTC programme (and all its activities).
48
The e-questionnaire to alumni was a success: in total mails were send to 437 addresses of
alumni from the period 2005-2010, of which 49 appeared to be wrong. 388 alumni thus received
the questionnaire. In total 202 alumni have fully replied. This represents a response of 52%,
which can be considered as very high. From the results, it appears that the group of respondents
was representative for the total population with regards to gender and to region. Therefore reweighing of results was not necessary.
49
The e-questionnaires to organisations: this evaluation activity was not successful. This is
explained mainly by the weaker link between RNTC and beneficiary organizations: the core
business of RNTC remains the training of individual professionals. The response rate on the
survey of organisations was 17 %, which is too low to make any valid and reliable conclusions.21
In total, 49 valid email adresses of tailor made organizations and project organizations could be
delivered to ACE Europe through RNTC: 8 organisations replied, 34 did not and 7 e-mails
bounced. Alumni were asked to give the email address of their superiors in their current
21
The 17% is on the e-mails send out, thus not the total population of organisations that have interacted with RNTC. The
guiding principle on response rates is that we need at least a response rate of 40 percent. We take this “40 percent rule” as
a minimum percentage in the case we do have some information about the total population. In this case (survey
organization), we did not have information of the total population (same variables for all the organizations). By no means it
was possible to compare the profile of the respondents with the total population of organizations.
pag. 23/78
ACE Europe / Evaluation of RNTC / Final report December 2011
organization22: 98 (on 202 alumni) filled in a email address of their superiors; 21 of these
addresses were the address of the alumni (maybe being self-employed?), 12 addressed were
not correct. As such, 65 employers of alumni have received a questionnaire. Of these employers
only 9 answered the questionnaire. This was to be expected as the international courses are
aimed at individual professionals and not their organisations.23 In total two reminders were sent.
50
The field missions in general: although there were some difficulties (see overall mission report),
the evaluators succeeded in getting a fairly good overview of interventions in the three countries.
The mission in Zambia was the most challenging as national elections were organised in the
week prior to the mission and a lot of knowledgeable contacts of RNTC were (no longer) present
in their organisations. This made it difficult to reconstruct with the respondents the history of their
organisation and the evolution in capacity development.
51
The case study on community radio: the case study was based on internal and external
(evaluation) reports regarding 6 projects (5 in Africa and 1 in Surinam). These reports allow to
identify the way of working of RNTC but are less usefull to provide evidence on effects of the
interventions and the changes realised at the level of radio stations and the general public. When
effects are described these are usually not substantiated by figures or examples.
52
As such, this evaluation will (i) describe and highlight the way of working of RNTC (under
efficiency, including quality), (ii) assess the relevance of the RNTC interventions based on the
field mission findings and the study case on community radio, (iii) highlight the results of basic
international courses, mainly based on the perception of the alumn and (iv) assess the effects of
the RNTC tailor made courses and projects based on the findings of the field missions and in
relation to the projects of community radio.
22
When preparing for the questionnaire, it became clear that the RNTC database does not contain the details of the
employers of the alumni. Therefore it was decided to ask the alumni to give ACE Europe the contact of their employer
23
Organisations are involved in the application procedure: the applicant has to present a letter proving his position with a
certain employer and the support of this employer. This requirement is part of the Nuffic application procedure and aims at
increasing the effects of the individual training on the organisations in which the alumni are active.
2
Theory of change
The change theory was reconstructed in the inception report based on desk study and interviews
with RNTC staff. The application of this theory of change will be discussed further in the report.
To create a dynamic and democratic society, RNTC identifies the need for free exchange of
information, room for expression and active citizenship. Media play a vital role in this process.
The role for media lies in information, education, participation (giving voice and facilitating
dialogue) and providing a platform for accountability (Multi-year Plan 2005-2008, 2). The media
however are under pressure in fragile states, lack basic quality criteria in general and offer little
room for individual citizens and marginalized groups to participate in the public debate (Annual
Report 2009, 6).
Therefore, RNTC wants to contribute to the development of professional media skills in
developing countries and to the use of media for development and education.24
The solution is primarily sought in the provision of (professional) education to mid career media
professionals (Multi annual plan 2005-2008, 2), through individual capacity building, through
developing partnerships with training institutions in the South (strengthening local capacity for
training) and through support to networks (supporting media professionals in dealing with rapidly
changing environments, including the network of alumni). Secondly the solution is sought in the
development of educational materials related to development topics and projects (community
media and diversity).
The services provided by RNTC aim to apply the concept of integrated capacity building at both
the individual and the organizational level. The Multi-year Plan 2009-2012 states that this
capacity building should strengthen media organizations and media professionals to realize their
potential as actors in development processes. Therefore RNTC wishes to focus on media
organizations that aspire to fulfill a public service role and are rooted in society, reflecting society
and serving society.
The focus on the public service role of media means supporting responsible and responsive
media that are committed to contribute to development, paying particular attention to
participation and diversity, development and social cohesion and education (Multi-year Plan, 1517). This explains the focus of RNTC on programmes in which media and marginalised groups
are supported to cooperate in a participatory, inclusive manner, working in post-conflict states
and using media in educational programmes.
24
Mission of RNTC: multi-year plan 2005-2008, 3 and multi-year plan 2009-2011, 4.
pag. 25/78
ACE Europe / Evaluation of RNTC / Final report December 2011
53
The multi-year plan 2005-2008 marked important shifts in the way of working of the RNTC (which
was already visible in the years preceding). The Plan (page 6) highlighted the evolution of a
situation of :
54
−
courses in the Netherlands to a combination with activities in the South;
−
focus on knowledge and skills to the positioning and role of media and responsibilities of
media professionals;
−
individual training to capacity building at organizational and institutional/sector level;
−
one-dimensional training to blended learning (with a combination of techniques);
−
an orientation on media organizations to a broader orientation towards other actors in the
field.
The Annual Report of 2006 pointed also to an increased linking with development e.g. link with
development related themes such as peace building, globalisation, participation, through
community radios and production of educational materials (Report 2006,3).
55
These evolutions have continued over the years and have influenced the focus of the work of
RNTC. Despite the changes mentioned in the above, the Multi-annual Plan 2005-2008 is very
much driven by the provision of 'products': trainings, development of websites, developing of
curricula, development of networks of media-professionals and media products and designing elearning products. In the years 2009-2012, the production side of the RNTC work was more seen
as a means to an end and integrated as such. The Plan 2009-2012 was more driven by a 'multiactor approach', formulating results and activities for different types of organisations (media
houses, training/educational institutes and NGOs/organisations of civil society). Focus on design
of e-learning and educational production (mainly in latin-America) became less prominent.25
Throughout these changes, the interaction with the alumni and the management of the alumni
network remained very important in the plans.
56
The year 2010 seemed to mark important changes again: the office in Costa Rica closed down, a
new website was put online (seeking to improve information about the services of RNTC, talks
about a merger with Free Voice and Press Now started (finalised in March 2011 with the creation
of a new organisation Free Press Unlimited). With the Ministry of Foreign Affiars it was agreed
that the existing contract between RNW/RNTC and the Ministry would be maintained for the
remaining contract period. In practice this meant that all non-DSO funded activities were
transferred to FPU in 2011; all DSO funded activities remain under the responsibility of
RNW/RNTc for the contract period. Pressure on the financing of RNTC is increasing:
government policy is leading to a restriction of NFP scholarships urging RNTC to search for
paying students and organisations. Prudence in hiring new staff and reduced level of investment
following the uncertainties about future funding are high on the agenda and the start of a masters
programme with Leiden University is postponed.
25
Obviously; educational production in Latin-America could be easily left to local organisations (often lead by RNTC
alumni) providing them with a source of income. Further, the support of networks was not evident, one of the reasons being
the lack of finances to support networking.
3
Evaluation question 1: findings on efficiency
ACE Europe has been looking at the: (i) the input-output ratio (mainly cost of courses in relation
to quality) and (ii) the organisation of the work (quality and M&E, management of trainers).
Information was collected through study of documents, finances and semi-structured interviews
with RNTC staff and staff from peer organisations. Findings are mainly based on study of
international basic courses, tailor made courses and projects. Below, we formulate the main
findings.
The conclusion is that RNTC is a professional organisation demonstrating genuine concern for
costs and efficient planning, which provides quality trainings and developed a solid approach for
supporting development of community radio in the South. Beneficiaries are able to compare the
quality with what is offered by other international donors and educational institutes and rate the
quality of RNTC as particularly high(er). They refer to: flexibility, high quality of trainers, approach
towards adult learning going beyond demonstration and exposure through practical assignments
combined with mentorship and specific feedback techniques. There is room for improvement in
aggregating and analysing data on effects of training. RNTC has been able to attract external
sources for projects and has demonstrated increasing capacity to work with other NGOs in an
effort to link individual capacity building to organisational and media sector development. The
external funds have supported the development of a particular approach which is combining
trainings with organisational development of community radios. DSO funding for the core
business of RNTC (international education) is the most important funding source and RNTC
remains dependent on the DSO funding for running its organisation. Below, the main findings are
described in detail.
3.1
57
INPUT-OUTPUT
The inception report gave already an overview of the RNTC interventions and the budget. A
number of issues are repeated here with additional information on sources of funding,
cost/trainee for the international basic courses and quality.26
26
A very detailed analysis of input/output and effects cannot be provided: the performance indicators mentioned in the
RNTC reports are not directly linked with the budget lines.
pag. 27/78
ACE Europe / Evaluation of RNTC / Final report December 2011
58
The overall turn-over of the RNTC programme was around 4 million euro annually (except in
2007, following investments in the Informotrac programme). In the following paragraphs, the
main findings related to the income side and the expenditure of the full RNTC programme will be
described. Findings are mainly based on the financial sections in the RNTC year reports and on
calculations made by RNTC regarding percentage of external funding and cost/trainee of the
regular international courses (see annexes).
59
Looking at the income side - There are three important sources of funding27: (i) the Ministry –
DSO covering the RNTC activities in the field of international education and the costs for running
the organisation, (ii) Nuffic-NFP scholarship scheme for short courses (basic international
courses and refresher courses) and (iii) MFS funding for the Informotrac programme. In 2010,
these three sources covered 73% of the total budget (4.040.000 euro): 42% of the total budget in
that year was covered by DSO (activities and core funding), 16,6% by Nuffic and 14% by MFS.
60
The Ministry – DSO is thus the most important funder. This funding is allocated mainly to the
regular international courses (covering costs not included by Nuffic/NFP scholarships) and costs
for the running of the organisation. RNTC is highly dependent upon the funds to cover the non
collectable costs for the courses and to cover its core functioning: the functioning of RNTC is
mainly covered by DSO funding (at 98% in 2010)28. DSO funding became also more important to
finance the tailor made courses with partners in the South: from 10% in 2005, over 24% (2006),
21% (2007), 75% (2008) and 36% in 2010. IN these cases DSO fundig was used as seed money
(or ‘vliegwielgeld’). External funds (see further) mainly cover costs for specific projects as most
donors are not likely to fund the regular training activities. As such, RNTC has not been able to
diminish its dependency on the ministry for the financing of its core functioning. A strategy was
developed to find external funding for activities (such as increased cooperation with Press Now
and Free Voice) but this did not yet solve the dependency on the DSO funding for the functioning
of the organisation. .29
61
The DSO funding for regular international courses that are part of the NFP scheme covers: (i) the
non collectable participant related costs such a costs for lodging of participants (which are
generally higher than what NFP accepts) -the tuition fee and a handling fee (are financed by
NFP, (ii) course costs that are not directly related to participants (and as such not eligible for NFP
such as staff and freelance teachers). The 2010 tuition fee was set at 4.542 euro for the 12weeks courses, 1.720 euro for the 6 weeks courses and 1.015 for the 2 weeks course.30 31
27
A detailed view on the different sources was made for the years 2008-2010. The figures cannot be presented for the
previous years: the contract with the Ministry for the programme 2009-2011 specified that detailed figures on income and
expenditure had to be presented. This requirement was not in the previous contract (see Subsidiebeschikking DCO/OO325/04, article 3, page 3.: “Het jaarplan behelst een overzicht van de activiteiten met de daarvoor benodigde personel en
materiële middelen en de doelstellingen. De begroting behelst een overzicht van de geraamde inkomsten (inclusief
subsidie, eigen inbreng van de aanvrager, middelen verschaft door derden (…) en uitgaven van de aanvrager, voorzover
deze betrekking hebben op de activiteiten waarvoor subsidie is aangevraagd.”.
28
Small part of this funding was transferred to the office in Latin America (Costa Rica) until it was closed this year: this
office generated its own funds.
29
This changed in 2011 with the development of a marketing plan (which was not shared with the evaluator).
30
Source: NFP annual report 2010, 56.
31
The tuition fee of RNTC was determined 5 years ago by RNTC based on the following considerations: (i) the tuition fee
was supposed to remain below 150% of the average tuition fee from Dutch educational institutes providing media training,
(ii) costs that were not able to allocate to one participant (such as costs for lecturers), (iii) costs to organise joint events with
62
When looking at the different sources of income32: RNTC provided a detailed overview for the
years 2008-2010, clearly identifying the BUZA sources (including DSO, Nuffic but also Nuffic
NESO, MATRA), MFS funding and non-MFS/non-BUZA. The overviews can be obtained on
request. From this, the following can be concluded: RNTC succeeded in attracting additional
external funds and in increasing the % with regard to the total budget33: between 2008 and 2010,
the percentage raised from 9% to 18 %. The overview also demonstrates that the additional
external funds are generally small funds (many different donors), except for EU funding. Amongst
the donors are also Dutch NGOs such as NIMD and Press Now.
63
Looking at the expenditure - The highest volume of expenditure is related to courses/training
activities in general and to support of community radio projects.
64
The costs per trainee for the basic international courses and the refresher courses can be
calculated. Distinction had to be made between the type of course (variable duration) and the
number of trainees. The detailed calculation per type of international course (2 weeks, 6 weeks
and 12 weeks and per year is attached in annex. When using the figures of 2005 and 2010, the
following can be concluded: (i) the total cost of the trainings did not significantly increase over the
years. E.g. one 12- weeks course in 2005 cost 552.000 euro for 28 participants or 19.700 euro
per participant. The same course in 2010 cost 560.000 euro for 30 participants or 18.700 euro
per participant. (ii) the contribution of DSO per participant varied between 45% (for the same 12weeks course in 2005), 50% (in 2007 and 2009) and 39% (in 2010).
65
In general, the evaluators did not find evidence of great discrepancies between what was
planned/budgeted and what was executed/spent, except for some projects where extension
periods (budget neutral) were necessary to carry out the planned activities.34 Since 2010, the
organization has become more prudent in its expenditure which explains a lower expenditure
than budgeted. This is influenced by the current Dutch policy to cut a number of costs in
development cooperation.
66
When looking at concern about costs – The evaluators have noticed some concern about
costs at the level of RNTC staff. Staff states that the culture within the organisation does not
allow ‘to make costs, just because there is a budget’. Examples of cost concern are: working
with local experts, negotiating deals for housing and meals of trainees for the regular
international courses in Hilversum, negotiation with trainers on their price/day, organization of
joint trainings within projects, using focal points of RNW instead of opening own/new local offices
and working through local partners, etc. Examples from the community radio projects are:
trainings for the staff of local radio’s are in situ or as close to the radio station as possible:
the group of trainees. This tuition fee can be indexed every year. Nuffic decided to raise the fee in 2007 (one time
operation) to compensate for the fact that max costs for housing, meals and pocket money were determined.
32
RNTC uses small budgets (called ‘vliegwielgeld’) to start up specific tailor made interventions with organisations in the
South. IN the reports, this is referred to as or ‘partnerships’ or ‘preparing project proposals’ (period 2005-2008) or as.
‘needs assessment’ and tailor made training (2009-2010)
33
A number of factors contributed to this: the MFS rules required that RNTC was able to cover minimum of 25% of income
(not taking into account funds from Buza).
34
This was mainly due to overambitious planning with regards to the context of the development countries, in combination
with staff turnover at the level of RNTC.
pag. 29/78
ACE Europe / Evaluation of RNTC / Final report December 2011
partners go to the field and the local radio station with some basic equipment. In some cases,
radio stations are grouped for reasons of efficiency (projects in Africa). Local/regional expertise is
valorized: trainers/experts of the region are used for needs assessment and analysis or training.
These are known to RNTC through the Alumni network or through former projects/projects in
other countries (e.g. Informotrac I, Brazzaville). Equipment is mainly imported (which is said to
be cheaper and RNTC can buy without VAT).
67
On/off interventions in the South are based on contracts with the beneficiary organisations and
contain a budget in which the different costs are specified. From the field mission in Zambia and
in Uganda, the evaluators did not find evidence that costs have been thoroughly discussed with
the partners. In most of the cases, no financial contribution from the partners was required which
explains why partners did not consider costs too much. Partners generally offered a contribution
in kind. The case is different for the projects in Indonesia. Projects in Indonesia have been
submitted by the partners under the NESO financing facility which requires that local
organisations design the project and the budget.
68
When looking at quality – Input from alumni, peers, beneficiary organizations, evaluation
reports give a ‘uni-sono’ positive appreciation of the quality of expertise provided by RNTC with
regards to training and more in particular35: the trainers, the content/topics provided, the learning
methods (learner centred) and the approach of combining theory with practical assignments and
ensuring feedback and coaching. From the analysis of the most significant change with the
alumni during the field missions and the comparison made by alumni and their employers with
training initiatives offered by other international training institutes the evaluators can conclude
that the RNTC courses offer added value and differ from other courses: RNTC courses go
beyond demonstration and exposure and beyond technical aspects of the profession, they are
about integrating knowledge and insights in the daily work and aim at changing the mindset of
the trainee. Although the courses do not give access to a master degree, they were seen by the
alumni as ‘masters squeezed into 12 weeks’. Alumni responding to the e-questionnaire also
compared RNTC courses to other courses they have followed (which was the case for ¾ of the
respondents): 80% of them values the RNTC course higher and refers to the method and
approach (92%), the content (83%) and the organization of the course (78,5%). Beneficiary
organizations met during the field missions expressed clear appreciation for the training expertise
of RNTC, its flexibility and its approach towards (adult) learning. The evaluation reports on the
projects for community radio and interviews with peers highlight the openness, the pragmatic
approach and flexibility, the commitment to tailor made approaches and the educational
expertise of RNTC (for basic in situ training). This in situ training focusing on basic skills is
considered by PANOS Afrique (see MFS evaluation of Informatrac II) to be a clear niche for
RNTC (not duplicating other interventions). Also Press Now highlights the specific educational
and training expertise of RNTC and its quality in ToT and curriculum development allowing
organizations to formalize and improve their training capability (e.g. in comparison to Press Now
35
Employers of alumni (met during the field missions) and Dutch embassies (field mission visit) have less view on the
quality of the RNTC courses.
which says to be more activist and aimed at supporting people and organizations active in media
to take actions).36
69
ACE Europe concludes that RNTC offers high quality for trainings at individual level. ACE Europe
is not able to compare however the price/quality ratio of the courses, more in particular for the
basic international courses because a benchmark does not exist.
70
ACE Europe also concludes from the analysis of the work of RNTC with organisations that a
certain expertise of RNTC for capacity building at organisational level has been developed more
in particular within BUZA funded projects (TMF, MFS, MATRA). The project set-up stimulated to
develop a more integrated approach at partner/organisation level. This could be witnessed more
in particular in the projects aimed at supporting community radio in Africa. The experiences have
led to the development of a kind of ‘product’ in which organisational development and
professional development of individuals is combined. Outside of these funds/projects, RNTC did
not develop a solid or even implicit approach towards organisational capacity development since
it did not consider this to be the core business of RNTC.
3.2
71
ORGANISATION OF THE WORK
The work processes within RNTC are underpinned by the ‘Handboek
Kwaliteitsmanagementsysteem RNTC’. RNTC has an ISO certificate since 2005 and is
committed to improving its business processes. The ISO audits in the annual reports confirm and
highlight continued progress of RNTC processes. Increasing attention was paid also to M&E and
internal learning. ACE Europe did not find (policy) documents further operationalising the theory
of change, view on capacity building or partnerships.37
72
From the findings of the field mission and the case study on community radio projects, ACE
Europe can conclude that efficiency of execution was good with clear agreements on
implementation of activities. Weaker aspects have been: turnover of staff at the level of RNTC (in
some projects, such as SBM, GIEBTA), timely identification of particpants (and their needs) for
training in the south, follow-up on activities, overestimation of capacities in the South for project
implementation, not always fully understanding what offered trainings were about (e.g. Indonesia
and SBM project: ToT is not about topical issues)38.
73
M&E - The procedures of NFP require that each educational institute has a basic system in place
which explains the fact that this type of data has been well kept. The RNTC administration
developed over the years and is supported by a database Selligent (since 2008). Besides the
database, several instruments (mainly questionnaires) have been developed to record
36
Press Now can refer to a very concrete example where the comparative advantage of RNTC can boost a Press Now
project in Moldavia (school for journalism in Chisinau).
37
Information on how to deal with partnerships in the handbook are very operational by nature.
38
Trainees in the SBM project thought with the first ToT that the training would also be on enhancing their skills in radio
production. This was not the case. The issue was solved with the second training where the ToT was combined with a
training on radio production offered by an Indonesian lecturer.
pag. 31/78
ACE Europe / Evaluation of RNTC / Final report December 2011
appreciation of the target group: intake assessment, in between assessments, end evaluation
and evaluation after 6 months per participant).39 Generated data are available in digital format
since 2010. Data are not aggregated per course or at the level of RNTC and mainly serve as
operational instruments: to monitor and improve quality of courses (logistics, content, trainers); to
prepare for the next course, brief the trainers or highlight problems. The RNTC does not conduct
specific tracer studies: follow-up on the career path of alumni is not done in a systematic way.
74
Data collection and follow-up on results and appreciation by target groups is far less developed
when looking at the other activities of RNTC, e.g. refresher courses or tailor made courses.
75
Data on performance - The year reports highlight the number of trainees and organisations
reached/involved. These performance indicators thus give information about the output of the
RNTC and can be used to calculate the input-output relation. In general, the year reports are
mainly activity oriented. Data on performance and results are not aggregated or analysed.
76
Data on organisations - RNTC is coming from a purely people oriented institute targeting media
professionals as such to an institute which pays more and more attention to individuals in relation
to their organisation and to organisations as such. This evolution explains why information
related to the (partner) organisations of RNTC was not collected in a systematic way over the
years. E.g. contact details about different organisations are not readily available. RNTC however
confirmed that contact details can be collected if needed for the purpose of this evaluation.
77
Data on projects - same remark. Data collection follows the requirements of the donor, e.g. EC,
MATRA, MFS II. As such, RNTC disposes of a number of evaluation reports related to several of
its projects and data per project. The data are not aggregated or analysed at the level of RNTC.
The MFS evaluation on community radio (Informotrac II, page 10) states that MFS indicators
formulated were not very helpful for tracking results, but refers to the use of a basic checklist
(effective in tracking progress, highlighting issues of concern and good basis for dialogue with
partners) which is commended as an effective system for monitoring progress.
78
To conclude: different types of data are available to inform RNTC about performance and about
results (as perceived by individual trainees of basic/international courses). The ISO audit of 2009
(Annual Report 2009, 9) assesses that the planning and monitoring of projects and programmes
is well organised and transparent. ACE Europe however concludes that RNTC does not yet
disposes of a system to aggregate or analyse the data and to highlight changes at the level of
organisations. This is also confirmed by an ISO audit (Annual Report 2007, 10) which states that
improvement can be realised related to the analysis of collected data about the performance of
RNTC and measuring satisfaction of different client groups. RNTC is aware of the M&E
challenges to serve learning and accountability and has taken several initiatives to improve its
M&E system, up till know not yet with convincing results. The main stumble block seems to be
the formulation of good indicators.
39
ACE Europe learned that data about appreciation of participants are also collected by Nuffic through monkey survey.
These data are not analysed; ACE Europe is still looking into the possibilities of using these data as an additional source of
information.
79
Internal learning - It should be noted that RNTC invested in internal learning (see doc house of
knowledge). Several initiatives have been taken together with Press Now: e.g. trainings provided
by MDF on context analysis, needs assessment and planning (2007), courses on management
of change (2009) with focus on intercultural communication, performance and appraisal systems,
training on communication and leadership skills. Over the years, RNTC increasingly involved its
trainers (a pool of over 200 trainers, mainly from The Netherlands which is in fact the main
‘capital’ of RNTC) in the learning through trainer of trainer courses (one in 2008 with
development of a training manual, one in 2009, 2 in 2010). In 2009 a learning trajectory was
developed with a policy advisor from the University of Maastricht. An interesting project
developed within RNTC (2009) is the project 'mutual learning - the case of educational media'
which aims at developing a methodology to share and secure knowledge and experience evident
in individual people with the institution.
80
From the field missions, the following can be noted on efficiency: overall, the evaluators can
conclude that the efficiency of the interventions was good. The set-up was generally quickly and
easily concluded for tailor made interventions and even for the project with SBM in Indonesia:
requests were clear (in Indonesia thanks to interaction on the application for funds between
beneficiary organizations and NESO), general needs for training easily defined, budget available,
clear cut training activities, RNTC proposed the expertise. RNTC was able to apply its RNTC
principles and way of working (interactive and active, learner centred approach, high quality of
trainers) although some stumble blocks appeared: organizations selected the participants late in
the process (therefore detailed needs analysis in some cases still had to be conducted at the
start of the course, knowledge of English was not fully ensured (Indonesia mainly), attendance of
management could not be enforced, interventions up to 10 days did not allow to practicing
sufficiently the theory. Reporting was mainly done by RNTC trainers based on participants’
evaluation sheets, using several formats but with particular attention to what worked well/not so
well and recommendations; there was no evidence of particular follow-up on these (to verify to
what extent effects were translated to the organisation or integrated).
81
From the case on community radio, the following can be noted:
−
RNTC is not present in the South and works through local partners who in turn interact with
beneficiary radios. The partners can be federations of radios (are preferred partners and
always considered first), training institutes or other type.
−
Partnerships between RNTC and organizations in the South are project oriented and
contract based and aimed at realizing the project funded by a particular donor. In one report
(on Informotrac II) it is said that partners are very much seen as a means to an end: the
project aims at ensuring capacity to deliver support to the community radio stations. Clearly,
efforts are primarily focused on the capability of the partners to deliver results: develop a
programme, reach beneficiaries in the whole country, having infrastructure, financial means
and people in place to deliver. Attention is paid to the internal organization and more in
particular mechanisms for planning, follow-up and evaluation and models for financial
sustainability.
pag. 33/78
ACE Europe / Evaluation of RNTC / Final report December 2011
−
Reporting is very much a responsibility of the partner organization: it is seen as part of the
capacity building. Reports are assessed by RNTC on the basis of a checklist.
−
RNTC seems to have developed a kind of clear cut and understandable ‘product’ or
approach in supporting development of community radio. This approach seems to work quite
well with view to assisting partners/beneficiary organisation to roll out training programmes in
the field and can be applied within different donor formats and in some variations (see the
various RNTC projects. The approach was developed within the first Informotrac project and
is used as a basis for new projects (e.g. the one in Surinam). RNTC works with partner
organizations (newly set up in the first project but later on existing federations/networks of
community radio stations were selected) who in turn have contracts with (some of) their
members. RNTC mainly interacts with the partners through provision of budget and ToT.
This TOT follows the general RNTC ‘values’ (learner centred, focus on learning cycles and
adult learning, learning by doing). The partner interacts with the radio stations (on the basis
of contracts) and the support is integrated (training, follow-up and coaching, provision of
equipment) and looks at the whole radio station: quality of programmes, interaction with
community and development actors, capacity for income generation and efficient
management based on clear legal status, governance structures and action plan.40 This
integrated approach at the level of community radios is less obvious in Surinam (and in
Indonesia).
−
The clear approach eases interaction with partners (clear, good and regular communication
with RNTC in general, timely and relevant feedback to reports, pragmatic management of
partner relations, appreciation of flexible and genuine partnership relation) and efficient M&E
systems (focused on output of training activities and performance of local radio stations with
some challenges though to collect reliable data at the lowest level).
−
The level of capacity (of individuals) was generally overestimated: project set-up was
sufficiently flexible to allow adaptation in approach and content of activities, but generally this
caused some delays41 or deleting certain activities (e.g. e-learning platforms and coaching
through the internet which is too sophisticated, e.g. Informotrac I). Donors require projects to
be finished within a certain period and this did not allow taking sufficient time for processes
of capacity building to be completed.
40
41
RNTC is considering three main strands for it support to capacity: ‘to be’, ‘to do’ and ‘to relate’.
Delays were sometimes also related to staff turnover at RNTC, e.g. Informotrac I.
4
82
Evaluation question 2: findings on relevance
ACE Europe has been looking at the following judgement criteria: (i) did RNTC provide tailor
made services to mid-career professionals, (ii) to media organisations and media training
institutes,(iii) did these interventions address the challenges involved in the development of
responsive and responsible media?
83
Information was collected through e-questionnaires to alumni, interviews and work sessions with
organisations in the countries of the field mission and information from the case study on
community radio.
84
The main conclusion on relevance is that through the regular international courses, RNTC has
been able to deliver tailor made services to mid-career professionals in the media sector meeting
their needs. Course development rested upon analysis of previous courses (on a case by case
basis) involving trainers and alumni. When looking at tailor made interventions and projects,
RNTC worked with partners that are well positioned in the sector and in their country: they have
a certain reputation and track record and could make a difference. ACE Europe concludes that
RNTC demonstrated great commitment to meeting the needs of the beneficiary organisations.
Interventions financed through external sources (other than DSO) and with bigger budgets were
based on a certain analysis of capacity of the beneficiary organisation and had a broader scope
combining training of individual staff members with organisational development (with focus on the
capability of the organisation to deliver results). There is some room for improvement in
analysing requests for interventions: RNTC guidelines and quality handbook did not provide a
systematic approach or tools to analyse requests against the background over the overall
capacity of the beneficiary organisations. Small and punctual tailor made interventions funded
with DSO seed money could generally not address the different challenges faced by a
beneficiary organisation and did not realise the objective of starting off longer term partnerships.
In its programme and interventions aimed at supporting responsive and responsible media,
RNTC has primarily responded to the challenge of low professional capacity. However, RNTC
also demonstrated to be development sensitive in its trainings (through the training assignments,
the choice of topics), through cooperation with other development actors and through its support
to community radio.
85
Below, the main findings are described in detail.
pag. 35/78
ACE Europe / Evaluation of RNTC / Final report December 2011
4.1
86
ABILITY TO PROVIDE TAILOR MADE SERVICES TO MID-CAREER
PROFESSIONALS
Under this heading, ACE Europe is looking at the regular international courses in Hilversum (and
the refresher courses) which is the main support of RNTC to media professionals at mid-career
level.42
87
RNTC has been able to deliver tailor made services to media mid-career professionals: the
needs of the trainees were clear, alumni confirm that the trainings addressed their needs and
found added value in the RNTC trainings when compared to other trainings (see in the above
under quality).
88
Courses, topics, modules and assignments are defined before hand, based on experience with
previous trainings and through interaction between the RNTC course coordinator, the trainers
and selected alumni (the latter more recently, according to interviews with RNTC staff). The
experience with previous trainings is well documented: written evaluations of alumni and of
trainers are available. This experience, combined with an analysis of general media trends also
inspires and informs the course coordinator to define topics for refresher courses.
89
Before the course starts, RNTC collects information about the trainee (background and
expectations) through the NFP application procedure. Each applicant is also contacted to check
his/her knowledge of English and expectations. At the start of each training (course opening),
time is allocated to getting to know the trainees better.
90
During each course several meetings with alumni and trainers are organized to check on the
quality and relevance of the course. Trainees are encouraged to develop and update a personal
workbook (recording their work and the reflection on, translation of and application of what was
learned during the day).
91
The alumni met during the field missions confirmed that the trainings met their needs. They also
valued the RNTC course as particularly high when compared to other courses. This was
confirmed by the results of the questionnaire to the alumni. All respondents said that most likely
they would recommend RNTC courses to other colleagues. Those that followed a refresher
course (around 20%) all found these very relevant to their work.
42
Other than this, there was also the alumni network but RNTC did not invest much in the management of its alumni
network (until recently). Interaction with alumni is not organized in a systematic way and this was also highlighted during
the field missions. There is a lot of evidence though of interaction between trainers and alumni of a particular course.
RNTC has been seeking opportunities for meeting alumni in the South (in combination with working visits and/or the
organisation of refresher courses), e.g. Kenia in 2006, Zambia in 2007, The Philippines in 2008. In 2008 an alumni
community on facebook was developed which is used to get input from alumni on new ideas for courses. Another way of
interacting with mid-career professionals was through networks, but this was not very successful and became less
important in a context of new social media (see Annual Report 2008).
4.2
92
ABILITY TO PROVIDE TAILOR MADE SERVICES TO ORGANISATIONS
RNTC demonstrated a great commitment towards meeting the needs of beneficiary
organisations and to respond to their perceived needs. RNTC tries to get an understanding of
needs of individual people to be trained (mainly through the beneficiary organization) and
possible gaps in identification and understanding are usually dealt with quickly by RNTC trainers.
An indepth analysis of the overall capacity of beneficiary organizations and partners is usually
not done and RNTC works with perceived needs..
93
When looking at organizations a distinction should be made between tailor made courses
(financed by RNTC or other donors) and projects. In both types of intervention, there is a strong
focus on the design and provision of tailor made courses (ToT, topical trainings) for individuals
aimed at influencing the quality of the media profession. The role of RNTC is however different:
−
In tailor made courses, RNTC identifies and provides the trainer for the full course. The
interaction between RNTC and the organization is focused on training for selected staff
members. Needs of trainees are discussed with the management and sometimes through a
study visit. Trainings financed by RNTC (or the beneficiary organization in a limited number
of cases) are organized on small budgets (for 1 week) and within the country of the
organization. Trainings financed by other donors can be longer (more budget) and can be
organized elsewhere: e.g. tailor made trainings funded by Nuffic NESO Indonesia were
organized for 2 weeks and could take place in the Netherlands. Often more than one
organization was involved (following a requirement of Nuffic Neso). Trainings financed by
other donors also required a more elaborated funding proposal: e.g. for Nuffic Neso this
proposal had to be drafted by the beneficiary organization and forced the organization to be
explicit about its perceived needs. From the field missions and additional interviews with
trainers, it appeared that notwithstanding a kind of needs analysis43, the view on the actual
participants was often not clear at the start of the course: the organizations identified the
participants late in the process (or the participants changed), the organization did not have a
good view on the needs of participants, or the course topic proposed by RNTC and what it
entailed exactly was not well understood. This required a flexible approach from the trainers.
ACE Europe concludes from the findings of the field mission and reports of trainers and
beneficiary organizations that the RNTC trainers were quite flexible and capable of adapting
their input and approach to meet the needs of the participants.
−
In projects (that go beyond the provision of tailor made courses), RNTC engages with a
partner organization that can cover a certain country or area within the country (and who in
turn identifies, selects and supports beneficiary organizations at a lower level). The RNTC
input is a combination of funding (through external donors) and interaction with the partner
on project implementation, curriculum development and ToT. Trainers are trained to analyse
needs of beneficiary media organizations, to design a training programme and to roll out this
training programme in the field. There is no evidence of an in depth and systematic capacity
43
It appears that trainers sometimes engage with trainees before hand trying to get a better understanding of the
background and experience of the trainee. This is not a general RNTC approach.
pag. 37/78
ACE Europe / Evaluation of RNTC / Final report December 2011
needs analysis at the level of the partner organizations: what is looked at is the capability of
the partner to implement the project and to develop training programmes and support to
beneficiary organisations. The identification of needs of individual trainees encounter similar
challenges as with tailor made courses. It should be noted that for the community radio
projects in Africa, RNTC encourages the partners to conduct a broader needs analysis taking
into account the full organization with regards to its being, doing and relating.
94
Overall, the conclusion from the field missions is that relevance of the interventions was good.44
Tailor made interventions were relevant when taking into account history and capacity of the
organizations involved, but are when looking at Zambia and Uganda- limited in scope, mainly
targeting the individual staff members and their technical skills, thus influencing on the capability
of organizations to deliver results (read: improving the output /media products/trainings). The
evaluators have doubts about the priority of this need (in Uganda and in Zambia). The
interventions in Indonesia are the exception and were more looking beyond the needs of
individual staff. The main explanation is in the funding of the interventions and the requirements
and quality of preparation of the interventions that go with it: the activities in Indonesia have all
been financed by NESO, those in Zambia and Uganda with ‘vliegwielgeld’45 from RNTC/DSO.
The idea was that punctual interventions could serve the identification of future cooperation on a
more long term basis, but this idea did not work: the available DSO funding was too small to
cover costs for longer term projects and RNTC nor the partners were able to identify funding
sources to support longer term projects in the countries concerned.
95
When looking at the community radio projects, the following can be concluded from the study of
documents:
−
External organizations and evaluators stressed the knowledge of the RNTC staff of the local
context.
−
Generally the context description highlighted similar trends regarding the community radios
(mostly pointing at gaps): community radio is important in the rural area, professional or
minimum technical equipment is lacking, professional capacity of staff/journalist is weak,
management capacity is weak and capacity to mobilize funds is not developed.
−
RNTC pays attention to joint analysis with partners and stakeholders: e.g. future search
analysis (Liberia, Informotrac I) with different stakeholders in the projects in Africa ensuring
as such building ownership and cross-sectoral networks (but no clear evidence to
substantiate the functioning of these networks was found in the documents).
44
During the field missions, ACE Europe assessed the relevance of the interventions taking into consideration: (i) the
positioning of the organization (what does it mean in the sector and what can it mean for media development, thus how
relevant is it to work with this type of organization ?), (ii) the context in which the organization is operating and (iii) the
history and capacity of the organization: was the intervention relevant when looking at the history and capacity of the
organization at the time of the intervention?
45
This ‘vliegwielgeld’ refers to the money used by RNTC to explore opportunities for partnerships, also seed money. They
were funded by the budget line for ‘on the spot trainings’ or the budget lines to support media organisations and training
institutes. Budgets were usually too limited.
−
There is a dedication to meeting the needs of beneficiaries. RNTC ensures that the partners
assess the level of capacity of radio stations, using a checklist which takes into account
where the radio wants to go (vision). This assessment determines how the input will be
provided (e.g. type of equipment needed, type of training needed). This does not always
guarantee a full and clear picture of needs: some beneficiary organizations have expressed
the fact that they were too little involved in the exercise (e.g. Suriname and Informotrac II).
However, generally, the combination of hardware (equipment) and software (training) is
appreciated and welcomed as relevant by the beneficiaries.
−
The MFS evaluation of Informotrac II rates the relevance of the interventions as high with
regards to the objectives of the programme and the needs of the beneficiaries. Other reports
also highlight the importance of media interventions in these post-conflict areas and fragile
states in Africa: many rural communities rely on rumors and storytelling for information about
what is going on in the country: community radio can provide accurate information translated
to the local context.
4.3
ADDRESSING THE CHALLENGES INVOLVED IN THE DEVELOPMENT OF
RESPONSIVE AND RESPONSIBLE MEDIA
4.3.1 WHAT ARE THE CHALLENGES AND HOW HAVE THEY BEEN ADDRESSED BY
RNTC?
96
The challenges - The design of the RNTC multi-annual programme is based on a general
analysis of the media environment in developing countries and the firm belief that media have a
role to play in a structural approach to fight poverty by stimulating active participation in social
processes which should be realised through information about relevant social issues.
97
For RNTC, media as such are not just channels through which messages are sent, but parts of
civil society with a public role. Therefore it is important for RNTC that media are rooted in society,
reflect society and serve society (are ‘responsive and responsible media’). The main challenges
identified by RNTC for realizing that type of media are: the fact that the media are under pressure
in fragile states, they lack basic quality criteria in general and offer little room for individual
citizens and marginalized groups to participate in the public debate (Annual Report 2009, 6).
98
Addressing the challenges - The RNTC multi-annual plans do highlight these identified
challenges although a clear match between the challenges and the proposed objectives/results is
not fully clear: the issue of media under pressure, fragile states, participation of marginalized
groups and the issue of having public debate is not clearly or particularly addressed by the
objectives and results.
99
The proposed interventions as such do not directly address the participation of individual citizens
and marginalised groups in the public debate nor do they organise the public debate (unless
pag. 39/78
ACE Europe / Evaluation of RNTC / Final report December 2011
through phone-in programmes at local radio stations in some examples): they offer to create
some conditions by influencing on the providers of news and media programmes (, educational
programmes) through training. Generally, the RNTC interventions proposed do not address the
media environment. Some Informotrac projects in Africa demonstrate a concern for the ‘media
environment’ and the development of a legal and administrative framework (which is upcoming in
the countries): signing MoUs with ministries (Informotrac I, Brazzaville, Liberia).
100
Further, the challenges are not addressed by the multi-annual/yearly plans in a fully coherent
way. There are many different types of interventions with their own dynamics in fact all requiring
their proper intervention strategy: e.g. supporting networks encounters other challenges than
producing educational products. There are also a lot of smaller stand alone interventions (app.
1/3 of the programme). This is explained by the fact that RNTC is dependent on donor money:
donor opportunities determine whether RNTC will develop a certain intervention. Another
explanation is that RNTC is not pro-actively promoting (or even communicating) its expertise or
‘selling’ its programme: being primarily an educational institute RNTC does not have a particular
development agenda.RNTC reacts upon requests (demand driven) and to opportunities for
funding to some extent.
101
Two types of intervention (two of which are the most important in budget terms) are however
more coherent and have a recognisable and clear intervention strategy: (i) the regular
international courses address the quality of the media profession(als) and the public service role
of media to report about relevant social issues that appeal to the public. (ii) The projects (more in
particular on community radio) address low quality of the media professionals, lack of equipment
and management of media including business models and involvement of civil society in
management and operations. There might be to other types of intervention that are clear-cut, but
these have not been studied by the evaluation(i) the production of educational materials
addressing the need to have more attention for marginalised groups and relevant developmental
issues; (ii) the development of e-learning courses aimed at easing access to training for
professionals in remote areas and reaching more trainees.46
102
Training of and support to local radio stations demonstrate a concern to support the emergence
of ‘media for development’. This is envisaged through the development of specific products that
pay attention to local issues (all projects, through not succeeded though in Surinam),
participation of listener in listener clubs (Brazzaville) and in development committees
(Informotrac projects) but there is little information in the reports to substantiate the concrete
output and quality (e.g. how many development committees, how do they function, what is their
influence, …?).For these project interventions in the South, RNTC executes additional analysis of
country context taking into account main elements of legal/administrative framework, position of
partner organisations and the challenges they are facing.
46
Information from evaluations of projects in Africa, Bulgaria and Georgia however suggests that RNTC has not been very
strong in implementing this type of e-learning. This might have been better developed in Latin-America having less
financial stumble blocks
4.3.2 WHAT DOES ‘MEDIA FOR DEVELOPMENT’ MEAN AND HOW DID RNTC
CONTRIBUTE TO THAT?
103
There is no clear definition at the level of RNTC about what ‘media for development’ means.
However, different interventions however can be understood as ‘contributing to development’ in a
broader sense and they demonstrate the development sensitivity of RNTC. To identify relevant
interventions, ACE Europe has also taken into account the way the Ministry sees a role for media
in development.
104
For the Ministry ‘media for development’ as such are not a topic in development policy. Media
are understood to be important:
−
in the promotion of good governance which is a cross cutting issue underpinning the four
spearheads in development cooperation being security and legal order, water, food security,
sexual and reproductive health47 and
−
In supporting the human rights policy, more in particular the promotion of freedom and
democracy (through freedom of speech, access to internet and social media, strengthening
the media landscape and promoting accountability and critical journalism.48
The issue of accountability was particularly important for the Ministry as it was mentioned in the
Subsidiebeschikking with RNTC (2009).
105
Below, we will describe different RNTC interventions, their contribution to development and the
effects where possible ( e.g. based on field missions and case study on community radio
projects):
−
Media for development was in some cases translated by RNTC as ‘using media for
development and education’. This was particularly clear in the production of educational
materials (for NGOs and international organizations). ACE Europe did not come across
interventions directly helping civil society organizations and movements in particular
countries/regions to create or develop media that are responsive to their needs or to use
them in supporting their (development) agenda.
−
The regular international courses of RNTC address development issues through particular
topics or assignments. E.g. the topics of the courses for broadcast journalism around which
the full training of 12 weeks is build were in the respective years: conflict and cohesion,
globalization, sports and society, social conflict and cohesion, good governance and the
broadcast journalist. Practical assignments in the courses on Facts and Formats dealt with
47
Ministry of Foreign Affairs (2011) Letter to the house of Representatives presenting the spearheads of development
cooperation policy.
48
Ministerie voor Buitenlandse Zaken (April 2011). Verantwoordelijk voor vrijheid. Mensenrechten in het buitenlands
beleid, 6-7.
pag. 41/78
ACE Europe / Evaluation of RNTC / Final report December 2011
health issues (in 2007) or water and sanitation (2010). The international courses on
Educational programme production focused on children and development (in 2006) and on
youth and development (in 2009). Next to topical issues, the way of training addressed the
capacity of the media professional to relate to the public, the audience and specific target
groups. The following was noted during field missions:
(i)
The alumni highlighted their increased attention for the public/the audience and could
give clear examples of being sensitive to diversity: ensuring that different groups of
people are interviewed when covering a topic, understanding the reality of certain
groups to ensure relevance of the product, paying more attention to content
development and packaging of the information.
(ii) In the three countries visited there were examples of new programme formats and topics
being introduced by the alumni that are trying to be more relevant to the public and
to address development issues. Only a few examples: carving out new niche with
childrens’ programmes (TRANS TV Indonesia, diversify content of health
programmes with information on gender, human rights, law and politics (NGO
working with radio Zambia), programme for tobacco farmers allowing interaction
between the farmers and the tobacco factory (Radio Kitara, Uganda).
(iii) Alumni stated that RNTC courses generally contributed to this by (i) highlighting the
importance of the responsibility to pass on information to the public (because
‘people might use this information for their development’), (ii) by practicing the
technique of looking for information on the ground (course assignments to go to
schools and associations in villages and cities) and (iii) by stressing the importance
to appeal to the public and to understand what kind of information is relevant to
them.
−
The RNTC refresher courses (7 in the concerned period) also addressed development
related topics such as social conflict and cohesion (in 2005) in the Asia region, or
commercial pressures and ethical dilemmas in broadcast journalism (in 2007, East Africa).
−
RNTC has developed a number of initiatives with Dutch NGOs and in cooperation with Dutch
Embassies49 that have addressed issues of freedom of press, balanced reporting. Examples
are mentioned in the table below: The reports on these activities do not give clear
information about effects or sustainability of these interventions.
Partner
NIMD
Description of activity
2006-2007: Latin-America50
International seminar
Role of RNTC
Through the office in Costa Rica
(in cooperation with RNW)
49
From interaction with different embassies (Uganda, Rwanda, Indonesia, Zambia) ACE Europe understands that the
embassies did not take particular initiatives on accountability in cooperation with Dutch NGOs. Further: there are no
specific programmes aimed at media and media development.
50
ACE Europe contacted the Dutch Embassy in Nicaragua but did not yet receive a reply.
NIMD
NIMD
Debate on the links between media-politics and
democracy
Involving both media professionals and
politicians
Funded by external donors
Training facilitators for the
debate
Preparing materials and
dissemination and sharing of
conclusions
2007-2008: Nicaragua
Establishing platform for civil society to
exchange ideas and best practices on
democratisation, social cohesion and regional
integration
Funded by external donors and the Dutch
Embassy in Nicaragua (direct interaction with
NIMD, not RNTC)
Through the office in Costa Rica
2010: Latin-America
Three public debates
Through the office in Costa
Rica/Columbia
Input of technical expertise (live
stream, recording), network in
the media sector/media
professionals and knowledge
about media organisations
Training facilitators for the
debate
Preparing materials and
dissemination and sharing of
conclusions (alternative and
innovative way of reporting)
Funded by the Dutch Embassies in Nicaragua,
Costa Rica (direct interaction with NIMD, not
RNTC)
Organisation of the platform and
the learning exchange.
Preparation and dissemination
of materials
NIMD
2010: Burundi51
Training to develop the capacity of
(representatives of) 39 political parties in
Burundi to work with the media to communicate
their programme and policy views to potentials
voters.
Topics treated: the principles of
communication; the media landscape in
Burundi; understanding the media; press
release and giving an interview Implemented
with NIMD’s partner in Burundi, the Burundi
Leadership Training Programme.
Funded by NIMD
Design of training and executing
the training
Dutch
Embassy in
Rwanda52
2007-2008: Rwanda
Training of lecturers with the Great Lakes
Media Centre (GLMC) in Kigali: 12 lecturers
from Rwanda, Burundi and RDC.
Beneficiaries: GMLC, school of journalism in
Butare, 7 radio stations in Rwanda, Burundi
and RDC
Topics: training skills, professional journalism,
course development and sustainability and
organisational strengthening
Part of the training was organised in the
Netherlands
Funded by the Embassy and the RNTC
RNTC organised and provided
the training
51
ACE Europe has not yet been able to organise an interview with Pepijn Gerrits from NIMD.
It was not possible to speak to the contact person at the Embassy: she has left, her replacement send through some
documents from the archives. He confirms that the Centre in Kigali still exists but there are no contacts with the Embassy.
52
pag. 43/78
ACE Europe / Evaluation of RNTC / Final report December 2011
−
The RNTC projects on community radio and support to training institutes (in Indonesia,
Bulgaria and in Georgia) all have a component of improving professionalism of media people
through TOT. Trainers are encouraged to develop topic trainings on particular issues, related
to development. There are several examples of this: election reporting in Bulgaria, reporting
on minorities and children’s issues in Georgia, social conflict and cohesion and local election
issues in Indonesia, etc.). The documents on community radio projects did not provide
concrete examples.
−
Reports used for the case on community radio generally do not give a lot of information
about the longer term (development) effects, yet quite clear examples of some longer term
effects of the RNTC community radio projects were presented in relation to voice of the
general public in rural areas and the education of the public (see further under evaluation
question 4)
4.3.3 DOES RNTC WORK WITH THE APPROPRIATE PARTNERS TO ADDRESS THESE
CHALLENGES?
106
With partners, ACE Europe refers to beneficiary organisations in the South (and not technical
and financial partners or peers – since these relations, mainly aimed at implementation of
trainings or other initiatives have not been further analysed).
107
Based on the field missions and the case study on community radio, ACE Europe concludes that
RNTC generally works with partners that are well positioned in the sector and in their country:
they have a certain reputation, a certain track record) and could make a difference. However,
most of the partners remain weak when looking at their overall capacity and are not always able
to make the difference. From the field missions, it appeared that several organizations had many
other challenges (besides training needs) posing risks to the effectiveness and sustainability of
the tailor made course. These challenges were not taken into account in the design of the tailor
made course. The set-up of a tailor made course did not allow this.
108
The interventions for tailor made courses were based on a clear request from the South for
training. Generally, a request should clarify the need for training because RNTC does not react
upon all requests with a training proposal and some requests are not taken on by RNTC. The
quality handbook of RNTC offers some guidelines in the process of analysing a request. ACE
Europe finds that an analysis of the request against the background of the overall capacity of the
organisation is not included in these guidelines.53 From the field missions, there was no evidence
that training requests as such were challenged on the basis of an overall capacity analysis. .
RNTC is the one to develop a first proposal for training which is always discussed with the
organization.
53
See RNTC (2010) Handboek Kwaliteitsmanagementsysteem RNTC, page 63.
5
109
Evaluation question 3: findings on effectiveness
ACE Europe has been looking at the following judgement criteria: (i) changes in the capacity of
media professionals, (ii) media organisations training organisations that can be attributed to
RNTC interventions.
110
Information was collected through e-questionnaires of alumni, semi-structured interviews with
alumni and employers and work sessions with organisations and other stakeholders in the
countries of the field mission and information from the case study on community radio. Below, we
formulate the main findings.
111
The main conclusion on effectiveness is that the effects of individual capacity building through
the regular international courses are high. Alumni (and their employers) have noted changes in
professional skills, work ethics (including team work and mentoring other colleages) and mindset. Almost half of the alumni have seen positive changes in their career, mainly thanks to the
RNTC course. When looking at interventions with organisations in the South, the conclusion is
that the most obvious effects were related to strengthened skills of individual professionals. The
effectiveness regarding organisational change was clearer within interventions that had a
broader focus. The most important effects at organisational level were related to the capability of
organisations to deliver results: more professional staff, better equipment and better organisation
of news rooms have lead to increased output (better programmes/trainings reaching more
people/trainees). Evaluation reports on community radio suggest that interventions have
strengthened local radio stations and their interaction with the public. Overall, these changes
have not been well captured in the reports thus weakening the capacity of RNTC to substantiate
better statements on effects and outcome.
112
Below, the main findings are described in detail.
5.1
113
CHANGES IN THE CAPACITY OF MEDIA PROFESSIONALS
Under this heading, changes at the level of RNTC alumni are described. RNTC is clearly able to
influence on professional skills, work ethics and mind-set which can be considered a strong
result beyond pure technical issues.
114
From the e-questionnaire it is clear that following an RNTC regular international course in
Hilversum has important effects on the career perspectives of alumni: 47% of the respondents
highlighted that they changed position after the course; this position was appreciated as being
more in line with their competences and entailed more responsibility. 81% of the respondents
pag. 45/78
ACE Europe / Evaluation of RNTC / Final report December 2011
with changed position attributed this change to the RNTC course. Of those that changed
positions, half remained in the same organisation. In general, 86% of all RNTC alumni remained
active in the media sector. There was generally a high appreciation of change in competences
thanks to the course; appreciation was less outspoken for participants in the courses Facts and
Formats, Soaps and Society and Internet for Journalist when looking at increased insight in
business aspects of the media sector. Overall, participants confirmed that the courses have
helped them more in particular to develop their professional network, develop new types of
content and formats and technical skills.
115
During the field mission, several alumni were involved in a workshop on the most significant
change: 33 alumni have been involved of which 19 women.54 Changes were mentioned by
participants in several domains. The main domains that come out as trends over the countries
are: (i) professional skills and knowledge, (ii) work ethics and (iii) personal growth and changed
mind-set. When looking at what participants identified as their personal Most Significant Change,
the accent is generally on changes in the technical domain (basic and multimedia skills and
packaging information) and in personal growth (mainly career move). As a group, the participants
in Zambia highlighted their stronger professional skills and work ethics (including attention for
balanced reporting; looking to a topic from different angles), the participants in Indonesia
stressed the (revolutionary) changed mind-set (“re-born journalist”) and the participants in
Uganda named increased professionalism as MSC, with alumni being more professional and
innovative thanks to the technical skills gained.
116
The changes and factors within RNTC causing these changes (trends based on the replies by
the beneficiaries) are the following:
−
Professional (technical) skills – The changes mentioned are: (i) improvement of basic
skills as journalist/programme maker (better use of equipment and application of tools,
acquiring/improving techniques of writing, interviewing, hosting, presenting, doing research)
or trainer (improved needs analysis and more practical approach to training (less knowledge
based training, no longer ‘pumping’ the trainees, (ii) creating content
(selecting and
choosing stories) to interest the audience, (iii) packaging information in the most appropriate
way to deliver to the audience in an attractive way. The above mentioned changes have
increased the control of the participants over their job, the quality and the output and
creativity. (iv) Using media to change people’s mind-sets was only mentioned a few times
(once in every country).
“The most significant change for me is that I now understand media can be used to change
people’s mind-sets. We can take quality education to remote areas and promote positive
changes” (Alumnus Zambia)
54
The alumni interviewed during the field mission were coming from radio/TV media houses (in Indonesia mainly from TV).
It is not possible to say which houses were community owned or had community representation. The group of employers
portrayed a mix of privately owned organisations and government owned (the latter in particular in Zambia). NGOs and civil
society organisations were much less represented in this group. The alumni mainly followed the international course on
Broadcast journalism, followed by Internet for Journalists and Facts and Formats courses.
“The most significant change for me is the improvement of packaging information into attractive
programmes for farmers as such increasing the audience and the advertisement for our radio
station. It is the most significant because the financial situation of the station improved. There is
a good balance between what is relevant for the audience and what might interest advertisers. I
also believe that we have empowered the farming community by listening to them.” (Uganda
alumnus)
From the discussion during the workshop, it appeared that participants mostly related these
changes to the RNTC quality of trainers, the availability of equipment and the content of the
courses (attention for specific topics).
−
Work ethics (as a journalist) – The changes mentioned are: (i) accepting the practice of
feedback (being more critical towards proper performance and at the same time more
tolerant towards colleagues), (ii) better able to work independently (organize your proper
work) and to deliver (on time), (iii) accepting hard work (pushing, taking on challenges and
leaving the comfort zones), (iv) more grounded in team work (which was highlighted as
crucial in broadcasting). Generally this seems to have led to improved team work, ability to
work with others (and sometimes even coach and train colleagues and newcomers), ability to
work with limited resources and increased output of the participant. (v) Sensitivity to
participation, inclusiveness and voice of vulnerable/marginalized groups in society was
mentioned less frequent and particularly in Zambia: the awareness that media can have an
impact on what people do and the awareness that every story needs to be looked at from
different angles (balanced) reporting was quite outspoken.
“The most significant change for me was that I am now a competent and credible journalist who
can work with everyone. It is the most significant because I am really doing a better job, being
more tolerant, accepting feedback and understanding the importance of psychological aspects”
(Alumnus Indonesia)
The participants mostly related these changes to the RNTC approach of delivering the courses
and more in particular the attention for (giving and receiving) feedback (‘psychology’), the
practical individual and group assignments within an international and foreign environment
(forced to work) and the mentorship by the trainers.
−
Personal growth and mind-set - The changes mentioned are: (i) growing self-confidence in
the current job. Sometimes that has lead to pursuing further professional training (more in
particular in Zambia and in Uganda), (ii) career growth (more responsibility) or career moves
(promotion) for significant number of people (see table on overview of career path of alumni
after participating in an RNTC course), (iii) change in the way of thinking (more critical
analysis of news, deeper analysis, new mindset.
“My mind is no longer set before hand, I allow input from different people and never accept a
‘truth’ as such, always trying to take things one step further and trying to find out what is behind
things’. (Uganda alumnus)
pag. 47/78
ACE Europe / Evaluation of RNTC / Final report December 2011
The most significant change for me is a character change and personal growth. Because I am
now more creative, even able to ‘sell’ innovations in programme formats, writing proposals for
funding, becoming a better presenter that is always looking at what will work best for my
audience” (Uganda alumnus)
These changes were mostly related to the RNTC motivational and inspiring international setting
of the course and the RNTC approach (see the above). The career moves in Uganda are
sometimes caused by the RNTC certificate only (which serves as a kind of quality label and adds
to the competitiveness on the labour market).
117
Explanatory factors - The causes of the changes gives us an idea of the strong aspects and
added value of the international courses. It should be noted that the careful selection process of
applicants and the criteria (being mid-career, having a certain position within the organisation,
support from the organisation) also partly contributed to the results. Another explanatory factor
lies in the process of the regular international courses: the set-up of the courses give
responsibility for the effects of the course to the trainee: trainees are encouraged to think about
personal goals, they should keep a personal workbook, there are the feedback sessions. As one
alumnus in Uganda stated: 'You do not have to be self-motivated for this course; if you are not
motivated RNTC makes sure that you find your motivation'.
118
Confirmation by employers – The evaluation was not able to get a firm confirmation from a
large group of employers. To the extent possible, the evaluators have tried to talk to superiors
and managers in the media organizations during field missions. The main changes mentioned by
the alumni are generally confirmed by (this limited number of) interviews with employers:
capacity to produce different types of programmes, better grounded in team work, engagement
and taking on responsibilities stand out as most noticeable changes.
5.2
119
CHANGES IN THE CAPACITY OF MEDIA ORGANISATIONS AND TRAINING
INSTITUTES
The evaluation has looked at the organisations involved in tailor made courses and projects of
RNTC. Findings are based on a selection of those (see in the above).
120
From the field missions, the evaluators noted that tailor made courses and projects mostly
improved on the technical skills of the participants/individual staff members. Effects at
organizational level are generally to be awaited or remain limited in Uganda and Zambia (maybe
with the exception of ZEBS where the organizational output seems to have changed
considerably: more educational programmes of better quality). The interventions were basically
on/of and very short (max of 10 days) which did not allow sufficient practicing (or the application
of the RNTC approach). Generally: effectiveness and sustainability of these tailor made
interventions were very limited.
121
In Indonesia, more clear effects have been noted at the organisational level: these changes are
mainly related to the capability to deliver results: e.g. better organisation of news room, capacity
to design and provide trainings, increased capacity for online reporting. This has lead to
increased output of better quality (which was significantly different when compared to the past):
trainings on new topcis (such as conflict), creation of a website, carving out a niche for childrens'
programmes, more online reporting, etc. Changes in Indonesia are to a lesser extent also related
to the capability to learn and adapt: reflection can be noticed within organizations about how they
are operating/managed and how they should position themselves in the rapidly changing media
environment. The evaluators noted that a certain organizational ‘readiness’ (ownership of change
processes amongst management) in Indonesia made the difference. Next to the funding: funding
from Nuffic Neso made it possible to have a more comprehensive training, funding from EVD
allowed to develop a project allowing longer term engagement and trajectory of trainings
combined with organisational support. Many challenges for sustainability though remain, also in
Indonesia
122
Tailor made courses had some clear advantages when compared to the international courses:
they allowed to train a critical mass within an organization, participants do not need to have a
degree and can be younger professionals, they are less costly (although input of the
organization, liberating its staff increases), management and staff can be part of the same
course, they force the trainer and the participants to make the best of the equipment available55.
But there are also clear disadvantages/challenges identified through the field missions: needs
analysis of participants was not always properly done, there were little guarantees that trainings
will effect on the capacity of the organization in a broader sense (and little control on how this
can happen) especially in Zambia and in Uganda, trainings were generally too short to ensure
sufficient practicing. The evaluators also noted that generally, the participation of women seems
to remain at maximum 1/3.
123
Many of the changes noticed by the respondents were attributed to the RNTC interventions,
though not solely.56
124
When looking at the projects on community radio, the picture related to effects shows mixed
results:
−
At the level of the partners: there is some evidence in the reports to substantiate statements
about partners becoming more robust organizations. Evidence is related to the capability of
the organization to deliver results. Reports on projects in Africa point at the fact that partners
are now able to define training and annual activity plans and are more able to deliver
trainings but also that they are not financially viable. Improvement in reporting (also financial)
and external relations is noticed as well. Effects seem to be particularly strong for the Media
Centre in Libera and the turnover of the organization quadrupled in a few years time. Effects
55
It should be noted however, that several respondents stressed the fact that RNTC always told them that even with little
and bad equipment one can make a good story.
It should be noted however, that time during field missions and availability of respondents was too limited to dig deeper
on other external factors or to get more conclusive confirmation from external stakeholders. E.g. in Indonesia: management
of media organisations referred to RNTC as one of the factors contributing to change.
56
pag. 49/78
ACE Europe / Evaluation of RNTC / Final report December 2011
at organizational level in Surinam are unclear: it is expected that organizations will integrate
trainings in their work plan, but activities to ensure that they will have the capacity to do so
have not been undertaken. It was mentioned by RNTC that ownership of the project by the
Surinam partners (especially for the ‘software’ and development issues) was clearly weaker.
One partner in DRC seemed to have serious governance issues (REMACOB, see report on
Informotrac I) but is currently implementing 5 projects. Finally, it seems hard for the partners
to retain their staff which was trained; this also makes it difficult to support innovation and
learning at organizational level. It is not clear which partners supported under the older
projects are still active.
−
All reports stress the importance of managerial training and developing business plans at the
level of partners to ensure the sustainability of the capability to deliver training/services. It
appears from the reports that management and financial training sessions have not always
been well internalized or remembered by the beneficiaries.
−
The reports give little information on output, effects and outcome at the level of the
beneficiary radio stations (unless the MFS evaluation on Informotrac II, see further). The
report on the Liberia project states that the project is providing institutional capacity building
but that it is impossible to measure effects (such as: participation of target group in
management and programme making because date are not collected, higher market share
because there is no audience survey). The 2011 activity report on RDC highlights the
difficulties to measure impact and proposes the technique of storytelling as a way of
identifying examples of improvement.
−
Reports on the community radio projects in Africa suggest that (although still very fragile and
financially not sustainable), the radio stations are better organised (basic equipment and
structures in place, voluntary staff in place) and embedment/ancrage in local society is
emerging (through advisory boards, but their actual functioning is not assessed in the
reports). The MFS evaluation on Informotrac II states that local radio stations have staff with
increased knowledge, confidence and professionalism and gives examples of changed
output at the level of the stations as mentioned by the members of the community: improved
quality of sound, better reporting and programming, more relevant content and programmes,
easier to tune in etc. The report on Liberia mentions: increased coverage (thanks to the
equipment), increased percentage of spoken word, increased awareness concerning
reporting on development issues. RNTC stresses the fact that the majority of the radio
stations are still in the air after receiving support: this is explained by the fact that better
coverage and programming leads to higher income from public announcements57, allowing
the stations to cover their running costs.
57
Sources of income mentioned are: obituaries and other family related announcements (biggest source), church
(broadcasting service), telecom companies, commercials (limited), gifts, sponsored programmes (e.g. Unicef), government.
6
125
Evaluation question 4: findings on sustainability and
longer term results
ACE Europe has been looking at the following judgement criteria: (i) the role alumni have played
in their organisation and in the media environment of their country, (ii) the role of media
organisations in the development debate, (iii) the contribution to improved access to quality
training for media professionals, (iv) the contribution to changes in the media environment in the
countries.
126
Information was collected through e-questionnaires of alumni, semi-structured interviews with
alumni and employers and work sessions with organisations and other stakeholders in the
countries of the field mission and information from the case study on community radio. Below, we
formulate the main findings.
127
The main conclusion on longer term effects is the following: the fact that the international courses
for individual mid-career professionals realize changes in mind-sets and work ethics, with the
majority of alumni remaining active in the media sector strongly points at longer term effects.
There is evidence to conclude that individual alumni have been able to introduce changes in their
organisations: integration of internet and social media in the organisation, setting-up training
iniatives and introducing new ways of packaging information strengthened the capability of their
organisation to deliver results. The effects are less evident when looking at other capabilities at
organizational level: it is quite a leap from the strengthened individual capacity to organizational
change. Further, the changes at organizational level attributed to the RNTC international and
tailor made courses are small and do not fundamentally change the role and policy of the media
organization as an actor in organizing or feeding the development debate. This kind of change
can probably not be expected from such trainings because it requires longer term organizational
change processes. Nevertheless, ACE Europe can conclude that the way of working of RNTC in
general has indirectly contributed to the diversity of the media landscape. Finally, it should be
noted that RNTC projects have been able to nurture nascent media capacity even in post-conflict
areas.
128
Below, the main findings are described in detail.
6.1
129
ROLE OF ALUMNI IN REALISING (ORGANISATIONAL) CHANGE
From the questionnaire, it is clear that 90% of the alumni have prepared a back home action
plan: they were encouraged to do so at the end of the course in Hilversum and RNTC organised
specific sessions to assist them in thinking over how to put into practice what they had learnt. In
this action plan, 80% of the alumni that prepared such a plan also looked at organisational issues
or changing the way of working within their organisation. The answers to the questionnaire
pag. 51/78
ACE Europe / Evaluation of RNTC / Final report December 2011
provide examples from the alumni on changes, particularly in the following domains: integration
of internet and social media in the organisation, setting-up training initiatives and introducing new
ways of packaging information.
−
Better integration of internet and new (social) media in the organisation: e.g. introduction of
internet radio in the radio station, establishing a website for the radio station, linking the NGO
to community radio for advocacy through a facebook page, introducing or redesigning a
website with tailor made content, opening facebook and twitter accounts to improve online
presence leading to 31.800 followers on twitter and 460 friends on facebook.
−
Setting up training initiatives inhouse and taking new initiatives: preparation of a new
curriculum to train community radio journalist and providing training through ToT of the
editors in the stations, establishment of a new training centre in the country for journalism
and computer training, organization of regular small training seminars, training colleagues to
improve the quality of video production, engagement in (re-)training of staff even when not
assigned to do so, organizing training on sports for women journalists and building a network
with them.
−
Introducing new ways of packaging information: using the documentary format in the radio
production, enriching news bulletins with reportages, introducing wrapper formats with cases,
quizzes and interactive elements combined with educational materials to support discussion
programmes, introducing a news bulletin for children, introducing a news magazine that
focuses better on the target group, improving the presentation ways of the children’s show
(changing graphics, introduction of prizes and introducing social media), creative way of
presenting news and development projects on the radio station.
130
From the field missions, ACE Europe can conclude that the fact that the international courses for
individual mid-career professionals realize/are able to realize changes in different domains and
more in particular in mind-sets and changed work ethics strongly points at longer term effects.
Further:
−
The fact that the alumni largely remain active in the media sector and for a great deal in
more senior positions supports the conclusion that RNTC alumni can be actors of change in
the media sector;
−
Alumni usually contribute to team work and take on informal roles of mentorship and
coaching of colleagues who have been not able to receive training from RNTC.
131
alumni can influence on media organizations and more in particular on the capability of their
organisation to deliver better products/trainings in a more efficient way. Influence on the
capability to learn from experiences is to a lesser extent supported by the practice of feedback
and improved team work. The practice of feedback was generally not institutionalized within the
organizations (exception of Trans in Indonesia). These changes in capability resulted in more
output that is of better quality and more innovative. In several cases this has lead to increased
audience and even more income for the organisations (some examples in Indonesia and in
Uganda, Zambia were provided by the alumni. The effect is less evident when looking at other
capacities at organizational level: it is quite a leap from the strengthened individual capacity to
organizational change. Influence can happen through one person only, but there are many
factors that need to be in place to support this: the position of the alumnus in the organization,
openness of management/superiors to change, the view of the organisation on training and
capacity building. In some cases, the type of media are determinant: e.g. smaller privately owned
radio stations, media organizations dealing with educational products/trainings and not with news
offer easier opportunities for change. Last but not least, contextual factors play a role: political
frictions, intimidation, pressure to find financial resources. Clearly the limits in influencing change,
in Zambia and in Uganda, have motivated some (frustrated) alumni to start their own initiative
next to their employment: e.g. own media consultancy or radio (Uganda), e.g. production/media
house in Zambia.
132
As such, the international courses have their limitations: it should be accepted that direct
influence of international courses on the employers can be facilitated by RNTC but not managed
nor guaranteed.
133
From the field missions, the evaluators learn that the main stumble blocks to achieve more
effects from the international courses (and tailor made courses) were related to:
−
Context: (i) financial pressures (in the three countries): most of the media houses are under
high pressure to attract income from advertisement and at the end of the day, this
determines what is relevant news or what is interesting and what not, (iii) the media
environment: the environment in Uganda is closing up (leading to self-censorship) or highly
politicized (Zambia) which leaves little room for wider changes.
−
The view held by organizations about their capacity development: the organizations did not
have an elaborated training/capacity building plans. Many young people come in (high
turnover) and are thrown in the bath, which makes it difficult to create a critical mass of ‘new’
journalists. Much is left to the initiative of the alumnus. Even with organizations that are more
open to consider the need of training: there is no organizational mechanism to prioritize and
plan for training, training of individual staff is always based on request, short trainings that
are organized for the staff are often ad hoc (urgent needs or opportunities), the link between
training and organizational plan and priorities is weak and capacity for consolidation is weak
(sharing of information and knowledge of the training is mostly not done, unless informally
with the superior). There are some exceptions however: UWMA in Uganda, Trans TV in
Indonesia. Also Jakarta Post and ATVLI in Indonesia thought over their training needs and
discussed these more thoroughly with NESO.
−
Management in organizations: managers in media often come from another sector (not
journalists or programme makers, producers), they do not understand the difficulties or
challenges of the profession (‘easy job’) and they do not really understand the real quality of
pag. 53/78
ACE Europe / Evaluation of RNTC / Final report December 2011
the RNTC course or are not open to the opportunities alumni can offer (Uganda58, Zambia,
not in Indonesia). The RNTC module on how to return back to home (which is integrated in
all courses) was helpful but not sufficient according to the participants (also in Indonesia).
6.2
134
ROLE OF MEDIA ORGANISATIONS IN DEVELOPMENT DEBATE
Typically, the changes at organizational level attributed to the RNTC international courses and
tailor made courses are small and do not fundamentally change the role and policy of the media
organization as an actor in organizing or feeding the development debate. This kind of change
can probably not be expected from this type of trainings because it requires longer term
organizational change processes which an individual alumnus cannot easily initiate or influence.
Having an effect on other capabilities of the organisation than the capability to deliver results
might require a different set of skills and approaches that have not been strengthened by the
RNTC course.
135
When looking for the longer term effects caused by the community radio projects, the following
can be stated:
−
The MFS evaluation on Informotrac II quite rightly states that the RNTC programme on
community radios is relatively small, involving only a small proportion of the numerous actors
who affect a countries’ media arena and are unlikely to see the impact of their work at the
broader country level. The same could be concluded for the other RNTC projects which have
a limited scale in number of partners and stakeholders involved and people reached (when
taking into account the full population of the country..
−
Only a limited number of reports give some clear examples of the real longer term effects of
the project in relation to voice of the general public and education of the public.
(i)
Voice of the general public in the rural areas: e.g. programmes are more responding to
local issues, local organisations are able to reach their/a public, radios are becoming
the primary source for local and national news. The reports on Informotrac I and II
demonstrate increasing effects in relation to listener’s participation through phoneins, sms interaction, interviews and questionnaires. The Informotrac I review gives
several examples of listeners sharing their opinions about corrupt government
officials, peace development, deteriorating road conditions, scolarity issues, etc. The
MFS evaluation on Informotrac II mentions the examples of women joining women’s
organisations or setting up new organisations after having heard about opportunities
for development and income generation. Structural listener’s involvement in
58
See for e.g. the experience of one alumnus in Uganda who had new ideas (change website and information items that
were often Kampala based and elitist and allowing localizing issues through the regional focal points) but could not
implement them in his organisation: editor was interested, but management not really, although the idea was and still s still
relevant. Alumni now has taken his ideas to his own initiatives: through his consultancy firm he is supporting e.g the Forum
for Women and Democracy (an NGO that implements community radio programming). He is also starting his own radio
station.
programming and management seems to be less strongly developed: generally, the
reports do not give information about the ownership of the stations (who is the
owner? or the involved in management). One report (on Liberia) states that it still
remains to be seen whether the stations will (continue) to spread reliable
information).
(ii) The education of the public: clearly, partners and radio stations have been recognised
by national and international organisations as good vehicles for development
campaigns or messages (see Informotrac I and II). The reports give however too
little information on the educational output and effect of the projects.
6.3
CONTRIBUTION TO IMPROVED ACCESS TO QUALITY TRAINING FOR MEDIA
PROFESSIONALS
ACE Europe concludes that RNTC has been able to contribute to increased opportunities for
quality training for media professionals in different ways, albeit limited (several smaller
interventions that are not building up or still have to demonstrate effectiveness):
−
Working with training institutions: from the inventory of projects and tailor made courses, it is
clear that training/educational institutes are an important partner for RNTC (1/3 of beneficiary
organisations);
−
By focusing on TOT in projects : see community radio projects in Africa, projects in
Indonesia, Bulgaria and Georgia;
−
By the international courses and the contribution of alumni to training and learning in their
organisation;
−
By the model of basic, in situ training (developed within the community radio projects)
bringing a different approach to learning and improving professional quality in remote areas,
offering training to professionals without degrees;
−
By introducing e-learning (although interventions have not been assessed within this
evaluation) to allow better access to training materials also for people in remote areas and
professionals without degrees, mainly in Latin-America and in Asia (to a lesser extent).
6.4
136
CHANGES IN MEDIA ENVIRONMENT
RNTC generally did not directly address the media environment and as such effects on media
environment are limited. In community radio projects efforts were done to link up with authorities
and ministries but results are not convincing: there is no evidence to state that the projects have
an impact on the legal and administrative environment. It is however important to note that the
pag. 55/78
ACE Europe / Evaluation of RNTC / Final report December 2011
RNTC projects involve sustaining existing media in places where this is under threat, as such
identifying and nurturing nascent media capacity which can form a basis for further development
once the environment becomes more conducive) (MFS evaluation on Informotrac II). Further, the
way of working of RNTC is an important contribution to the diversity of the media landscape:
encouraging media professionals to depart from local reality and to identify the most appropriate
formats and packaging allows more diversity. Community radio projects have an enormous role
to play to promote diversity in rural areas.
7
Conclusions and recommendations
7.1
137
ANALYSIS: REFLECTION ON THEORY OF CHANGE, INTERVENTIONS AND
EFFECTS
The first part of the mission, ‘development of media’ rests upon the firm conviction of RNTC that
change in media can be realised by training individual people and more in particular mid-career
professionals. The strong focus on the indivual as actor of change was successfully translated in
international training courses and modules of high quality and effectiveness can be rated as high.
138
ACE Europe was less convinced by the second part of the mission of RNTC, ‘media for
development’: the theory of change related to this (as reconstructed on the basis of the multiannual plans of RNTC) is quite encompassing and contains sweeping statements and objectives,
for e.g. about integrated capacity building and involving marginalised groups. ACE Europe finds
that these objectives have not been well translated in operational terms: operational guidelines
on how to set the focus, how to identify and select appropriate interventions, developing
indicators to measure change, etc. have not been defined. Interventions in the multi-annual plans
under the objectives of realising media for development did not always clearly address the
challenges. Financial resources to develop this part of the mission had to be identified outside of
the DSO funding and as such interventions has to be tailored partly to the requirements of
several donors.
139
Overall, ACE Europe finds that the general and actual scope of RNTC interventions was more
limited than might have been expected from the mission statement and the multi-annual plans.
Clearly, international education has remained the core business of RNTC and a full integration of
development projects and approaches aimed at organisational capacity building was not
realised, unless within specific projects. ACE Europe concludes that investing more in capacity
building of media organisations to influence on the media environment was not possible for
RNTC as a relatively small organisation: working with organisations requires additional/other
expertise, probably more staff to analyse context, choose partners in a more strategic way and
monitor and evaluate changes.
140
Clearly, the ambitions were set too high. This created a field of tension for this evaluation: based
on the mission statement of RNTC, the training institute has also been assessed as a
development organisation focusing on organisational and media sector development using the
DAC criteria and applying the model of the 5 core capabilities. As such, ACE Europe has
concluded that the effects of RNTC interventions at organisational level with view to creating a
conducive environment for democracy through media were limited.
pag. 57/78
ACE Europe / Evaluation of RNTC / Final report December 2011
141
On the other hand, thanks to this approach ACE Europe has also been able to better identify the
added value and potential of RNTC. ACE Europe thus concludes that (i) through the training of
mid-career professionals RNTC can indirectly influence on the capability of media organisations
to deliver results, (ii) that RNTC has been able to articulate better its sensitivity for development
and (ii) that RNTC has developed a niche product/approach on community radios which has
great development potential and potential for learning about integrated capacity building.
142
ACE Europe concludes that the negotiations with the Ministry for the current multi-annual plan in
the course of 2008 had a clear influence on the formulation of the mission and objectives of
RNTC. Although the DSO funding was primarily aimed at supporting international and
professional education to mid-career professionals, the requirements of the ‘subsidiebeschikking’
overstrechted the scope of the DSO funding and RNTC had to take development action as well
without DSO providing the funding. As such, the multi-annual plan has been formulated to
address these requirements but practice demonstrated that this was not evident.
7.2
143
CONCLUSIONS
RNTC is a an organisation working in a professional manner and offering quality products to mid
career professionals and their organisations. The main interventions are trainings of different
types and within different settings and -to a lesser extent- support to the capacity strengthening
of media organisations.
144
Clearly, the capital of RNTC rests for a great deal with its pool of trainers: it is good that RNTC
invests more in the management of this training pool. ACE Europe feels that this approach/work
is currently very much connected to one function/person within the organisation. It might be good
to start documenting the practice and the lessons learnt
145
Conclusions on efficiency - The organisation has been able to attract external funds but its
dependency on the DSO funding for its functioning remains. This core funding is clearly
important to be able to provide the international courses and became increasingly important for
the tailor made interventions.
146
Although there was no benchmark available to use in this evaluation, all findings point at the
international courses, funded through DSO and Nuffic as a high quality product with a good
price/quality ratio. Price of the courses did not significantly rise of in the period 2005-2010.
147
With its international courses and the projects on community radio RNTC can present two strong
products. With its community radio projects, RNTC demonstrated its capacity to explore the
boundaries of a training institute: it has shown flexibility and developed a clear niche product
where local capacity needs for community radio were quite well combined with the strength of
RNTC which is TOT and basic in situ training of good quality and learner centred. Both products
seem to be replicable to a certain extent: the international courses offer inspiration for developing
tailor made and in situ trainings, the product of community radio works rather well in different
countries. Even here though, the link with the mission 'media for development' and the attention
for capacity development at organisational level could be more explicit and could consider more
than mainly the capability to deliver results.
148
RNTC holds a strong view on how to strengthen capacities of individuals (vision on adult
learning) which is documented recently in the trainers’ manual. RNTC is weaker in considering
what is necessary to use individual capacity to strengthen organisations. Attention for capacity
building at organisational level is present in projects. There is however not a consolidated and
shared view at the level of RNTC about capacity building: what is it ? What are the challenges?
How can we support it? Experiences within community radio projects can be considered as a
good starting point to reflect upon this: within these projects a more integrated approach is
followed and radio stations are assessed along their being, doing and relating.
149
Conclusions on relevance - RNTC is able to provide tailor made services to mid-career media
professionals and to media organisations.
150
Interventions are relevant in several respects: with regards to the broader context of media
development, the spearheads and policy of Dutch development cooperation, the requirements of
the media sector (such as attention for innovative ways of working, online reporting, social
media) and the requirements for being a media professional (being able to perform the job from a
technical point of view) within the context of democratisation (attention for the public, looking at
issues from different angles, choice for relevant topics to cover).
151
Interventions with organisations in the South depart from the perceived need as expressed by
the beneficiary organisation which is often connected to the capability of the organisation to
deliver results. There is no evidence that a broader analysis of different capabilities and capacity
is undertaken. As such interventions are relevant but do not necessarily address the most
crucial needs of an organisation at a certain time. This poses risks to effectiveness and
sustainability. Especially the on/off tailor made trainings of 1 week are not effective nor
sustainable: RNTC argued that these trainings are generally considered as first steps in longer
term partnerships, but there is no evidence that it is working like this (e.g. partners do usually not
reappear in the list of projects).
152
When assessing relevance, the evaluators noted the efforts of RNTC to connect to civil society
through establishing cooperation with NGOs such as HIVOS, War Child and Press Now which
are seen as closer to civil society. It has not been studied to what extent these cooperations
contain targeted actions or a strategy to connect civil society movements in the South to (their)
media: how to ensure that social movements get ‘their media’ that are interested in their issues,
are willing to give them voice and as such supporting information and mobilisation for change.
153
Conclusions related to effectiveness - RNTC engaged mainly in basic international courses
and in community radio projects (together 3/4 of the total budget): for both interventions
effectiveness can be rated as good.
pag. 59/78
ACE Europe / Evaluation of RNTC / Final report December 2011
154
The international courses are effective instruments to strengthen the professional capacity of
mid-career media professionals. They lead to a strong combination of changes in the domain of
professional skills, work ethics and mind-sets. Changes are resulting from the RNTC course
topics, the approach (practical assignments with feedback), the quality and mentorship of
trainers and the international environment. For many alumni, this course has changed their
professional career to the better. The link between strengthened individuals and strengthened
organisations is not evident though: RNTC courses can only facilitate. Although one alumnus can
be an actor of change, many other conditions in the environment determine whether or not the
individual with newly acquired skills can change something in the organisation.
155
Tailor made trainings and projects have realised changes at the level of organisations, primarily
strengthening the capability of these organisations to deliver results: mainly increased
professional skills were noted amongst the staff, more efficient organisation of the work and the
newsroom, new tools and products, etc. Generally, the effectiveness at organisational level and
sustainability of tailor made interventions in Zambia and in Uganda was limited. Depending on
the set-up of the intervention (sufficient budget, longer intervention or engagement) and the
readiness of the beneficiary organisation (ownership, view on how to integrate aquired skills),
results are clearer and output at organsiational level is stronger. This was clear from the
experience in Indonesia. As such, tailor made interventions can be effective under certain
conditions.
156
Conclusions regarding longer term effects - Through its courses and the alumni, RNTC has
been able to contribute to increased concern and professional skills for providing information
about relevant social issues (relevant to the people) looking at issues from different angles
(balanced reporting) and this in different parts of the world. Notwithstanding the weaker capacity
of RNTC to capture and document change processes, there is some evidence that within media
houses that have interacted with RNTC more information is generated which could enable the
public (better) to hold government accountable for its policy and development interventions. This
is more clear at the level of projects (in comparison to tailor made interventions), for e.g. in
community radio projects: a better connection to national news, translating news to local context,
reporting on relevant development issues can be noted. Further, there is evidence from the field
(field missions and case on community radio) that changes in the capability to deliver results
contributed to increased opportunities for the public to express their opinion about relevant social
issues through social media, phone ins, feedback sessions for the public. But it would go too far
to speak about public debate and dialogue.
157
It is important to note that the RNTC community radio projects involved sustaining existing media
in places where this is under threat, as such identifying and nurturing nascent media capacity
which can form a basis for further development once the environment becomes more conducive.
Further, the way of working of RNTC is an important contribution to the diversity of the media
landscape: encouraging media professionals to depart from local reality and to identify the most
appropriate format and packaging allows for more diversity to develop. Community radio
projects have an enormous role to play in promoting diversity in rural areas.
7.3
158
RECOMMENDATIONS
For the recommendations, we have taken into account, the conclusions, the opportunities in
Dutch development cooperation, the strengths of RNTC and the weaker aspects, the mission of
RNTC and the fact that RNTC has transferred its project related activities to another
organisation, Free Press Unlimited.
159
The recommendations are the following:
160
1. Translate the mission ‘media for development’ (from the perspective of an educational
institute) in more operational terms: what does it mean: what changes do we think are needed
to respond to current problems? How can we get there and realise these changes? What
competences are needed with media actors and professionals to support the process and realise
the changes? Where can RNTC support the development of these competences taking into
account: the strengths of RNTC (experience, expertise, relations) and activities of other players?
How is RNTC going to do this: what type of interventions are best fit for what kind of objective
envisaged, how to balance pro-activeness and being responsive to requests (what criteria will be
used to discuss and select new opportunities?), what competences should be strengthened at
the level of RNTC and within the trainers’ pool? RNTC could be guided by its proper idea that
media should reflect society, represent society and be enrooted in society. ACE Europe would
like to stress here that the recommendation is not that RNTC tries to be a development
organisation, but that the organisation better defines its scope and contribution to development.
161
2. Develop an organisation wide (but light) M&E system that focuses on the changes and the
competences at the level of media actors and professionals (in connection with model/approach
that RNTC is choosing to support capacity development – see next recommendation). It is
suggested in some RNTC documents that it is too difficult to measure change. However, ACE
Europe believes that a follow-up on change processes is possible, for e.g. through process
indicators. The translation of the mission should help to identify useful mechanisms and
indicators for measuring change. It might be interesting to check out the opportunities offered by
the mechanism of outcome mapping. Using the technique of storytelling is interesting but still
needs to be connected to a certain evaluation framework in order to analyse changes. The
current system of evaluation of international courses at individual level is strong and can be kept:
it should be verified however (i) to what extent RNTC can invest more in aggregation and
analysis of data on a number of indicators, (ii) to what extent new/other indicators can be
integrated and (iii) to what extent data can be more systematically collected at the level of
employers (or a sample of them) to assess influence on organisations.
162
3. Ensure that RNTC is more explicit about its vision on capacity building and integrates
this vision. RNTC is focusing on individual capacity building and on organisational capacity
building. The individual capacity building is underpinned by a strong understanding about how
adult learning works. This understanding is shared with the trainers. It is good to make it more
explicit. This will help to sustain the RNTC approach when new trainers coming in. It will also
provide a good instrument to assist training/educational institutes in the South to build their
capacity in this respect: ACE Europe noted that RNTC develops a ToT approach in different
pag. 61/78
ACE Europe / Evaluation of RNTC / Final report December 2011
projects (SBM, SEERTI, GIEBTA, community radio projects) in order to create a pool of trainers,
but did not find evidence on a strategy to ensure that organisations can manage this pool.
163
When looking at organisational capacity building: this approach is developed with the community
radio projects in Africa and envisages three aspects of capacity: to be (vision and mission and
business models), to do (activities and programmes) and to relate (with community and other
actors). This approach could be better documented and analysed to turn it into a more explicit
and organisation wide approach. ACE Europe argues that even with a small team and limited
capacity, RNTC can invest in capacity building of organisations if the intervention is focused (see
point 6 of the recommendations). ACE Europe also suggests taking a look at the Dutch study on
the framework of 5 core capabilities to understand better the conditions and approaches for
support to sustainable and endogenous capacity development processes within organisations in
the South.59
164
4. Sustain and market the international courses as a strong RNTC product and niche:
based on the findings, ACE Europe concludes that the RNTC courses add value (beyond
technical training), are effective and can even produce longer term effects. It is important to
market this product better and to communicate better about the added value, the RNTC
approach and the effects of the courses. Given the management challenges in most media
institutions, ACE Europe feels that RNTC should and could build further upon the management
course it is already offering: e.g. invite employers of alumni to apply for the courses, develop
further the content of the course to demonstrate better to managers the opportunities of bringing
more exciting and relevant products, ‘sell’ the management course to donors and provide the
course in a number of regions. It might require some investment from the side of RNTC to study
better the managerial and financial aspects of media as this is often the achillesheel of many
organisations and projects and to attract experienced lecturers to this end.
165
5. Ensure that the other identified niche, that of supporting community radios, is
sustained, even now when the projects have been transferred to Free Press Unlimited. Many
development NGOs/donors see media and community radio as an instrument within projects:
they have little idea on how to actually start up or support genuine media initiatives that are
sustainable. They also tend to see media as communication tools and not as instruments having
the potential to empower social movements/civil society. ACE Europe suggests that the ‘product’
of community radio should be sustained. It was not very clear to ACE Europe, based on the
study of documents, to what extent the approach has been successful in ensuring that the
community radio stations are strengthening the capacity of social movements to pursue their
(development) agenda. Efforts should be done to document this better. Most of the projects also
depart from the medium and encourage the medium to involve members of civil society. But what
is the perspective of the social movements and how is this taken into account?
166
6. Develop a third niche, that of supporting the capacity development of training institutes
or training initiatives in the South. This niche balances between RNTC as a training institute
59
Ministry of Foreign Affairs (2011). Facilitating Resourcefulness. Synthesis report of the evaluation of Dutch support to
capacity development. IOB Report nr. 336.
and RNTC as development oriented organisation: ACE Europe believes that RNTC has proven
to be able to move beyond strict educational approaches and has demonstrated to be
development sensitive in designing training approaches and training modules. This
recommendation rests upon the identified strength of RNTC (its educational expertise and
development sensitivity) and on the experience which is already being build to a certain extent
through different projects funded by EVD and MATRA. The recommendation is also motivated by
a clear demand in the South (the alumni): there are many training opportunities in the South but,
according to the alumni and their employers, the quality offered is not comparable. The
recommendation is further underpinned by a shift in Dutch government policy suggesting that in
the future Dutch development cooperation (including the embassies) will keep a focus on TVET
and professional education (but less on basic education). Finally, it is important to support
improved quality of and access to training for media professionals in the South and to secure
funding for this through a multi-annual plan: other than DSO, most funders are not very keen on
funding capacity strengthening of training institutes as such.
167
Developing this niche means investing in the analysis of own practice: more in particular analysis
of what is needed to support genuine capacity development at the level of training institutes. E.g.
establishing a pool of trainers through ToT is not sufficient: a trainer’s pool needs to be managed
(coached, nurtured, renewed, etc). E.g. providing ToT by an RNTC trainer (see SBM) is effective
but entails sustainability risks: it could be envisaged to provide trainings through tandems (one
RNTC trainer and one local trainer), sharing responsibility for coaching and preparing future
training strategies. Developing this niche will further require that RNTC becomes more pro-active
in presenting its capacity and ‘product’, in analysing the environment in order to identify
promising training initiatives in the South and in selecting partners (based on criteria developed
through the operationalisation of the mission ‘media for development’). Partners should not
necessarily be educational institutes that already have a reputation (such as universities).
Developing this niche should also entail interaction with embassies and analysing the
opportunities offered by the policies and programmes that they are envisaging.
168
7. Finally, ACE Europe finds that RNTC can continue to operate as a provider of training
‘at request’ or based on service provider contracts with other NGOs/donors. Given the fact
that core funding in the future is no longer possible (which means that a considerably higher
volume of activities is needed to cover the running costs of the organisation), RNTC should
market its expertise and experience. ACE Europe recommends that this activity would go beyond
pure service provision, meaning that RNTC needs to analyse requests based on a number of
criteria (see also recommendation in the above on operationalising the mission). Further, RNTC
can pro-actively propose packages for effective and sustainable trainings: e.g. refusing trainings
for more than 20 people for one week because the lesson learnt from the international courses is
that effectiveness is supported by the RNTC approach (hands-on and feedback through group
and individual assignments) which requires sufficient space and time to practice and organise
feedback. This will inevitably require a longer term or a build up as a trajectory where two weeks
of training seems to be an absolute minimum. RNTC can also point at the fact that an explicit
vision/practice of the organization on training and capacity building might be required/should be
developed, because people need to feel safe (to make mistakes), need to be liberated and new
pag. 63/78
ACE Europe / Evaluation of RNTC / Final report December 2011
skills need to be integrated afterwards. This means that this kind of interventions need to be
prepared by pro-active research and marketing from the side of RNTC.
8
Annexes
8.1
LIST OF DOCUMENTS CONSULTED
ACE Europe (2011). Technical offer for the evaluation of RNTC’s Three Year Plan 2009-2012.
De Vaan Communicatie (2009) Strategisch waarderingsonderzoek onder financiers,
opdrachtgevers en partners van RNTC
Gallagher, M. (2010) Capacity building Indicators for Media, Education and Development.
Hancock, A. and Galin Borodinov (s.d.) An evaluation of the South East European Training
Initiative
Ministerie van Buitenlandse Zaken (2004). Beschikking en beoordelingsmemorandum.
Ministerie van Buitenlandse Zaken (2009), Beschikking en beoordelingsmemorandum.
Ministerie van Buitenlandse Zaken (2011). Terms of Reference. Evaluation of RNTC’s Three
Year Plan 2009-2012.
Ministerie van Buitenlandse Zaken (2011). Terms of Reference. Evaluation of the Netherlands
Fellowship Program (NFP).
Ministerie van Buitenlandse Zaken (2011) Verantwoordelijk voor vrijheid. Mensenrechten in het
Buitenlands Beleid.
Ministry of Foreign Affairs (2011) Letter to the House of Representatives presenting the
spearheads of development cooperation policy.
Ministry of Foreign Affairs (2011). Facilitating Resourcefulness. Synthesis report of the
evaluation of Dutch support to capacity development. IOB Report nr. 336.
NFP (2010) NFP Annual Report 2010.
Nuffic (2008) NFP. Guidelines for Dutch Institutions.
Pasveer, B. (2010) A house for/of knowledge. Imaginaries to guide knowledge sharing and
learning in a (small) development organisation.
pag. 65/78
ACE Europe / Evaluation of RNTC / Final report December 2011
RNTC/INTERNEWS (2010) Georgian Initiative for the enhancement of broadcast training
activities (GIEBTA). Final progress report July 2009-June 2010.
RNTC, Meerjarenplan 2005-2008.
RNTC, Multi-year Plan 2009-2012.
RNTC, annual plans and reports for the years 2005, 2006, 2007, 2008, 2009 and 2010.
RNTC (2007) Forty years of satisfied customers. Baseline study looking into the long-term
customer satisfaction and the direct or indirect impact of RNTC’s products and services (in
cooperation with bureau Sensz).
RNTC (2010) Handboek Kwaliteitsmanagementsysteem RNTC
Van de Pol, P. and van Geffen, E. (2008) RNTC Trainers Handbook.
8.2
OVERVIEW OF INTERVIEWS IN THE NETHERLANDS
Institution
Name, position
date
RNTC
Peter Veenendaal, general manager
RNTC
28/06/2011, 29/06/2011
14/07/2011
21/11/2011
(discussion final report)
Gerhard Timmerman, financieel
administrateur RNTC
28/06/2011
06/10/2011
Tony Wilkinson, projectcoördinator
Internationale Cursussen RNTC
28/06/2011 en 29/06/2011
06/10/2011
28/10/2011
(debriefing field mission
21/11/2011
(discussion final report)
Bernadette van Dijck, voormalig deputy
manager RNTC (nu Deputy head of
strategy and business development
RNW)
29/06/2011
28/10/2011
(debriefing field mission
21/11/2011
(discussion final report)
Tineke van Kooten (voormalig project
coördinator RNTC, nu bij RNW)
29/06/2011
Jan Van galen (projectmanager bij
RNW)
28/06/2011
Lem Van Eupen, adjunct directeur
(Strategie, Business Development en
14/07/2011
RNW
Innovatie), voormalig manager RNTC
Free Press
Unlimited
Leon van den Boogerd,
projectcoördinator RNTC, nu bij Free
Press Unlimited
29/06/2011
Eva Oosterwegel, verantwoordelijke
alumni en communicatie, nu bij
Free Press Unlimited
28/06/2011
Bram Truijen
14/07/2011
27/09/2011
Yvonne van Hees, senior
beleidsmedewerker (dossierbeheerder
RNTC)
14/07/2011
28/10/2011
(debriefing field mission
21/11/2011
(discussion final report)
Jannie de Graaf, beleidsmedewerker
(dossierbeheerder Nuffic)
14/07/2011
Ministerie van
Buitenlandse
Zaken, Directie
Duurzame
Economische
Ontwikkeling
Karin Wigman, beleidsmedewerker
(voormalig dossierbeheerder RNTC)
16/06/2011
Nuffic
Angela van Gennip, team coördinator
NFP
13/07/2011
Miriam, beheerder short courses
13/07/2011
Elsien Rauws, beheerder refresher
courses
13/07/2011
NIMD
Heleen Schrooyen, staff member
10/10/2011
HIVOS
Lou Scholten, programmamedewerker,
‘Expression and engagement’
17/10/2011
Dutch Embassy
Nicaragua
Silvia Porras
Contacted by mail, no
Ministerie van
Buitenlandse
Zaken, Directie
Sociale
Ontwikkeling/ond
erwijs en
onderzoek
response
Netherlands
Ministry of
Foreign Affairs
pag. 67/78
Wouter Biesterbos, Senior Policy
Officer
Good Governance Division (DMH/GB)
Human Rights, Gender, Good
Governance and Humanitarian Aid
Department
ACE Europe / Evaluation of RNTC / Final report December 2011
05/10/2011
War Child
Nicole Engering, Manager Institutional
Funding and Foundations Unit
07/10/2011
Dutch Embassy
Rwanda
Eric Van Oudheusden
Communication by mail
(October 2011)
8.3
OVERVIEW OF TAILOR MADE COURSES
Overview of tailor made courses 2005-2010
Courses in Africa 2005-2010
Year
Country
Organisation/partner
Type of Organisation
2005
TANZANIA
Television Zanzibar
Media
Uganda Media Women's
2005
UGANDA
Association, Mama 101.7 FM
Media
2006
BURUNDI
Search for Common Ground
Development
2006
NIGERIA
Farm Radio Network
Development
2006
NIGERIA
Food and Agriculture Organization
Development
SOUTH
Institute for the Advancement of
AFRICA
Journalism
2006
Training/Educational
Zambia Institute of Mass
2007
ZAMBIA
2007
ZAMBIA
2008
BOTSWANA
2008
KENYA
Communication, Educational Trust
Training/Educational
Zambia Institute of Mass
Communication, Educational Trust
Training/Educational
Ministry of Education, Educational
Broadcasting Division
Government
Kenya Institute of Mass
Communication
Training/Educational
SOUTH
2008
AFRICA
Cape Town Community TV
Media
2008
TANZANIA
Orkonerei Radio Service
Media
2009
IVORY COAST
ONUCI
Media
2009
South Africa
Cape Town Community TV
Media
Radio Darfur Network
Media
Sudan/the
2009
Netherlands
FORCOM (Forum Nacional das
2010
MOZAMBIQUE
Rádios Comunitárias)
Media
2010
UGANDA
TPO
Development
2010
Zambia
Educational broadcast services
(ZEBS)
Media
Courses in Latin-America (2005-2010)
2006
Columbia
Faculty Javeriana Bogota
Training/educational
2008
El Salvador
El Faro, digital newsservice
Media
2008
Brasil
Radio Poneira, community radio
Media
2009
Suriname
Surinaamse Televisie Stichting
Media
RNW network affiliates (in
cooperation with RNW Spanish
2009
Central America
Department
Media
2009
Dutch Antiles
BTP, Bureau Telecom and Post
other
2009
Colombia
Ministry of Culture Colombia
Government
Nicaragua,
Guatemala, El
Salvador,
Honduras
2010
Panama
Trust of the Americas
Training/Educational
2010
Nicaragua
Trust of the Americas
Training/Educational
Courses other continents (2005-2010)
2006
Indonesia
School for Broadcast Media
Training/Educational
2007
Indonesia
Asosiasi TV Lokal Indonesia
Media
2009
Indonesia
The Jakarta Post
Media
pag. 69/78
ACE Europe / Evaluation of RNTC / Final report December 2011
Alliance of Independent Journalists
2010
Indonesia
- Jakarta Chapter (AJI)
Media
Centre for independent Journalism
2005
Hungary
Budapest
Media
The
2005
Netherlands
Jeugdjournaal
Media
2005
China
Bejing People's Radio
Media
State Administration for Radio,
2006
China
Film and Television (SARFT)
Training/Educational
2006
Bhutan
Bhutan Broadcasting Services
Media
2006
Irak
Institute for International
assistance and Solidarity (IFIAS)
?
State Administration for Radio,
2007
China
Film and Television (SARFT)
Training/Educational
2007
Nepal
Radio Nepal
Media
2007
Bangladesh
Ekushay television
Media
2007
Bhutan
Bhutan Broadcasting Services
Media
2008
Bhutan
Bhutan Broadcasting Services
Media
2008
India
Radio Station Voice of Tibet
Media
2009
Tibet
Voice of Tibet
Media
Council of Europe (via Directorate
General of Human Rights and
2009
Ukraine
Legal Affairs)
Development
The
2010
Netherlands
Bernard Van Leer Foundation
Development
2010
Sri Lanka
ICTA
Government/media
8.4
OVERVIEW OF PROJECTS
Overview of projects 2005-2010
Projects in Africa 2005-2010
Year
Country
Organisation/partner
2003-2006
GUINEA-BISSAU
Centro de Treinamento Informorac
Training/
Educational
2003-2006
SENEGAL
Informorac Senegal
Training/educational
2003-2006
SIERRA LEONE
Training Centre Informotrac
Training/educational
2006
MALI
Plan West Africa
Development
2006-2007
ETHIOPIA
Unicef Ethiopia
Development
2007
NIGER
UNICEF Niger
Development
2007
SOUTH AFRICA
Southern Africa Communications for
Development (SACOD)
Media
2007
SOUTH AFRICA
KRAZYBOYZ Digital
Consultancies
2007-2008
RWANDA
Great Lakes Media Centre
Government
2007-2008
RWANDA
National University of Rwanda, School of
Journalism and Communication
Training/
Educational
2007-2010
GUINEA
4RGUI
Training/educational
2007-2010
LIBERIA
Liberia Media Centre
Training/educational
2007-2010
CONGO,
DEMOCRATIC
REPUBLIC OF
REMACOB
Training/educational
2008
TANZANIA
Young People's Media Network of
Tanzania (YOMnet)
Media
2008-??
Sudan
Press Now
Media
2009
SOUTH AFRICA
Cape Town TV
Media
2009-2011
CONGO,
DEMOCRATIC
REPUBLIC OF, 5
provinces
Fédération des Radios de Proximité de la
RD Congo (FRPC)
Training/educational
2009-2011
CONGO,
REPUBLIC OF
THE (Brazzaville)
Réseau National des Journalistes des
Radios Communautaires du Congo
Media
2010
BURUNDI
NIMD, Netherlands Institute for Multiparty
Development
pag. 71/78
ACE Europe / Evaluation of RNTC / Final report December 2011
Type of
Organisation
Democracy
2010
MALI
Reseau de Communication AADAR
Media
2010
SUDAN
Stichting Warchild nederland
Development
2010
UGANDA
Makerere University, Department of Mass
Communication
Training/Educational
2010
ZAMBIA
Educational Broadcasting Services
Media
2010-2012
LIBERIA
Liberia Media Centre
Training/educational
Projects in Latin-America (2005-2010)
20062008/09
Colombia
Universidad de los andes Bogota
training/educational
2006-2007
Latin-America
NIMD
Development
2007-2008
Nicaragua
NIMD
Development
2007-2008
Colombia
Ministry of Culture Bogota
government
2009 - 11
Surinam
Surinaamse Televisiestichting
media
2009 - 11
Surinam/The
Netherlands
3 radio station with OLON (Organisatie
Lokale Omroepen Nederland) and RNW
media
2006-??
Costa Rica
INA
training/educational
2009
Peru
IPYS Peru (Instituto Prensa y Sociedad)
training/educational
2010
Costa Rica
World Vision
development
2010
Costa Rica
Red Cross Regional Reference Centre
on Community Education for Disaster
prevention
Development
2010
Latin-America
NIMD
Development
Brasil
Foundation Itaipu Technological Park,
International Hydroinformatics Centre at
Itaipu Binacional
development
training/educational
2010
Projects other continents (2005-2010)
2005
worldwide
consortium of broadcasters (GER, UK,
NOR, NL) under lead of European
Federation of journalists
2002-2006
Bulgaria
Media Development Centre
Training/Educational
2006-2007
2008
Iran
Sharzad News and Internationaal
Informatiecentrum en archief voor de
vrouwenbeweging
Media
2008-2010
Georgia
Internews Georgia
Training/Educational
2010-2013
8.5
Indonesia
School for Broadcast Media (in
cooperation with 4 regional training
facilities (Java, Sulawesi, Sumatra and
Kalimantan)
RESULTS FROM E-QUESTIONNAIRE ALUMNI
Is available as separate document.
8.6
FIELD MISSION REPORT
Is available as separate document.
8.7
COMMUNITY RADIO CASE DESCRIPTION
Is available as separate document
pag. 73/78
ACE Europe / Evaluation of RNTC / Final report December 2011
Training/Educational
8.8
FINANCIAL INFORMATION FROM RNTC ON COSTS PER TRAINEE
NFP short course
12 weekse cursus
2010
aantal deelnemers
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
2009
aantal deelnemers
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
2008
aantal deelnemers
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
2007
aantal deelnemers
30
219
341
560
7.300
11.400
18.700
17
100
5.900
108
6.400
208
12.300
10.500
10.500
21.000
22
52
133
185
13.500
9.900
23.400
19
233
12.300
151
7.900
384
20.200
29
304
305
609
21
284
208
492
18
6 weekse cursus
25
2.400
6.000
8.400
2 weekse cursus
10
37
40
77
NFP refresher course
12 weekse cursus
2 weekse cursus
17
190
11.200
183
10.800
373
22.000
0
0
0
wel gehouden
niet NFP
25
254
10.200
257
10.300
511
20.500
6
123
129
wel gehouden
niet NFP
15
233
15.500
151
10.100
384
25.600
-
3.700
4.000
7.700
25
0
0
0
0
40
200
3.100
3.300
40
5
127
132
100
3.200
3.300
20
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
2006
aantal deelnemers
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
2005
aantal deelnemers
bijdrage DSO totaal*/per deelnemer
bijdrage NFP totaal*/per deelnemer
206
202
408
11.400
11.200
22.600
87
145
232
-
8.500
11.900
20.400
12
97
8.100
83
6.900
180
15.000
-
9.000
10.700
19.700
18
96
5.300
106
5.900
202
11.200
20
169
238
407
28
252
300
552
3.500
5.800
9.300
194
247
441
* in 1.000 euro
bron
aantal deelnemers
bijdrage NFP totaal
bijdrage DSO totaal
afrekening NFP
jaarrekening, werkelijke lasten
jaarrekening, werkelijke lasten
De tuition fee die Nuffic aan RNTC vergoed is in 2007 verhoogd. Dit ter compensatie van de gemaximeerde
declarabele kosten voor huisvesting, eten & drinken en zakgeld.
De genoemde aantallen deelnemers zijn exclusief deelnemers welke niet door Nuffic gefinancierd worden.
pag. 75/78
ACE Europe / Evaluation of RNTC / Final report December 2011
7.800
9.900
17.700
21
202
9.600
231
11.000
433
20.600
22
196
8.900
214
9.700
410
18.600
3
60
63
200
3.000
3.200
20
1
62
63
100
3.100
3.200
40
0
110
110
0
2.800
2.800