Go to content

Evaluation of Capacity Development in Danish Development Assistance

Evaluation 2015.06

Summary

Support for capacity development is a cornerstone in Danish development cooperation.

This evaluation assesses Denmark’s support to capacity development as an integral part of sector support to priority countries. The focus of the evaluation is on support to enhancing the capacity of public sector organisations’ performance as regards relevance, coverage and quality of services and regulations.

The evaluation concludes that Denmark has facilitated capacity development processes that have contributed to increased organisational performance in most of the supported interventions.

The evaluation finds that good and comprehensive guidelines for support to capacity development exists and suggest that if they had been applied more effectively during planning and implementation better results would have been forthcoming. In particular, the evaluation points to the lack of a systematic and comprehensive approach to planning and reporting on capacity development.

The evaluation has been carried out by Nordic Consulting Group, Denmark.

Table of Contents

Acknowledgments

Abbreviations and terminology

Executive Summary

1 Introduction

2 Approaches to capacity development

3 The evaluation sample

4 Answering the evaluation questions

5 Hypotheses

6 Lessons learned

7 Conclusions and recommendations

The following separate annexes to the evaluation report can be downloaded from http://evaluation.um.dk:

Annex A Terms of Reference

Annex B Evaluation Approach and Methodology Annex C Portfolio Screening Note

Annex D Desk-based Review Report

Annex E Nepal Country Study Report

Annex F Uganda Country Study Report

Annex G Tanzania Country Study Report

Annex H Danida’s Approach to Capacity Development Annex I Capacity and Capability Concepts

Annex J Validation of Hypotheses

Annex K List of Persons Met

Annex L List of Documents Consulted

Ministry of Foreign Affairs of Denmark
Asiatisk Plads 2,
DK-1448 Copenhagen K
Phone: +45 33 92 00 00
Fax: +45 35 44 62 01
um@um.dk
www.um.dk


Acknowledgments

The evaluation team would like to thank all stakeholders who have participated in the Evaluation by giving their time and attention. The team would especially like to thank representatives from government organisations, state institutions, civil society organisations and the private sector who have taken time out of their busy schedules to meet with the team and participate in focus group discussions. Their input has been invaluable to understand how capacity development has been integrated and managed in Danida’s development assistance and how improvements for future capacity development interventions could be made.

A great thanks to staff at Danida’s Evaluation Department in Copenhagen and staff at the Danish embassies who have spent a lot of time digging out documentation on the many programmes that have been reviewed and assessed as part of this Evaluation. A special thanks to the Danish embassies in Nepal, Uganda and Tanzania in which country studies were conducted; we especially appreciated the assistance they provided in organising the country visits and their engagements to dialogue with us.

Special thanks also to the members of the Danida Evaluation Reference Group who have guided the evaluation team along the way and provided critical comments and feedback on the Evaluation’s deliverables.

Abbreviations and terminology

ADB Asian Development Bank
AIDS Acquired Immuno Deficiency Syndrome
CD Capacity Development
CSO Civil Society Organisation
CPI Corruption Perception Index
Danida The term used for Denmark’s development cooperation, which is an area of activity under the Ministry of Foreign Affairs of Denmark
DBR Desk-based Review
DCA Dynamic Capability and Capacity Approaches
DCDSP Danida Capacity Development Programme
DFC Danida Fellowship Centre
DFID Department for International Development
EC European Commission
ECDPM European Centre for Development Policy Management
EU European Union
EQ Evaluation Question
EVAL Danida’s Evaluation Department
GBS General Budget Support
GCP Guidelines for Country Programmes
GIZ German Agency for International Cooperation
GPM Guidelines for Programme Management
HDI Human Development Index
HIV Humane Immunodeficiency Virus
LDC Least Developed Countries
LGCDP Local Government and Community Development Programme, Nepal
M&E Monitoring and Evaluation
MfDR Management for Development Results
NGO Non-Government Organisation
Norad Norwegian Agency for Development Cooperation
NSA Non-State Actor
OECD/DAC    Organisation for Economic Cooperation and Development/Development Assistance Committee
ODA Overseas Development Assistance
PDIA Problem-Driven Iterative Adaptation
PEFA Public Expenditure and Financial Accountability
PFM Public Financial Management
PIU Project Implementation Unit
PMG Programme Management Guidelines
PPP Public-Private Partnership
RMB Results-Based Management
ROACH Results-Oriented Approach to Capacity Change
Sida Swedish International Development Cooperation Agency
SPS Sector Programme Support
SWAp Sector-wide Approach
TA Technical Assistance
ToC Theory of Change
ToR Terms of Reference
UN United Nations
UNDP United Nations Development Programme
USAID United States Agency for International Development
WB World Bank
WBI World Bank Institute

Danida Programmes/Components

aBi The Agribusiness Initiative, Uganda
ACP The Anti Corruption Programme, Uganda
APHFTA Association of Private Health Facilities in Tanzania
ASPS Agricultural Sector Programme Support, Bangladesh
BSPS Business Sector Programme Support, Tanzania
CCAM Climate Change Adaptation and Mitigation, Vietnam
CODEC Community Development Centre, Bangladesh
ESAP Energy Sector Assistance Programme, Nepal
ESP Environmental Support Programme, Indonesia
HRGG Human Rights and Good Governance, Nepal
HSPS Health Sector Programme Support, Tanzania
HUGOU Human Rights and Good Governance Advisory Unit
JWESSP Joint Water and Environment Sector Support Programme, Uganda
MDTF Multi Donor Trust Fund, Nepal
NPTF Nepal Peace Trust Fund, Nepal
PFMRP Public Financial Management Reform Programme, Tanzania
POSCIS Strengthening the Comprehensive Capacity of the Inspectorate System
RAS Revenue Administration Support, Nepal
RTI Rural Transport Infrastructure, Uganda
SSRP School Sector Reform Programme, Nepal
TACAIDS Tanzania Aids Commission
UGOGO Uganda Good Governance Programme
UGP Uganda-Growth Programme

Terminology

Capacity is a frequently used term that is often applied without either being specifically defined or grounded in theory. This inevitably creates confusion, especially when applied to varied contexts. The Evaluation applies capacity and capability as elaborated below.

Capacity:[1] For a given organisational entity, its capacity will be the resulting effect actually generated from the assemblage, orchestration, mobilisation and manifestation of its capabilities (abilities, assets, resources and authorities). Capacity indicates the level of effectiveness of an organisation at whatever level is being examined: the organisation, a division or a sub-unit.

Capability: To achieve any pre-defined level of performance, organisations involved must decide how they will organise (mix, acquire, etc.) assets, resources; individual, managerial, strategic and operational abilities, as well as any entitlements, authorities and delegations they may have in order to be able to best meet expected performance levels. When organised (i.e. designed, mixed, orchestrated, grouped, deployed, etc.) to enable a functional or socio-technical system to perform, the integrated groupings are called the capabilities of the organisation, and they are characterised in large measure by: a) what they can produce, b) how much they can produce in a given time period or space boundary, and c) whether they are sufficient to enable the organisations to achieve expected performance levels.

A simple illustration of the difference between capability and capacity

A Tanzanian household of four in a middle-class neighbourhood has everything it needs to prepare healthy Tanzanian traditional cuisine: adequate stoves, proper utensils, fuel, the right spices and a person (persons?) who have the know-how and skills to prepare and organise family meals. It thus has the capability to generate Tanzanian cuisine, and a maximum capacity to prepare them for (say) 8-10 people. It does not have the capacity to prepare meals for 50 people. That household does not have what it takes (no capability) to prepare Japanese cuisine (i.e. not the right ingredients or the right utensils, etc.), and therefore has no capacity for the preparation of Japanese cuisine at any level. Importantly, just because the household buys Japanese soy sauce does not mean it has the capability required to prepare (the more) relatively complicated Japanese cuisine.


[1] OECD/DAC’s definition on capacity development is more general and is presented in Section 2.2, Box 3.

Executive Summary

Background

This report is concerned with the “Evaluation of Capacity Development in Danish Development Assistance”. The Evaluation forms part of the “Joint Scandinavian evaluation of support to capacity development” involving three Scandinavian development agencies: Sida, Norad and Danida. Capacity development has been an important theme in most donor supported development interventions over the last couple of decades – with the aim that supported organisations had acquired a sufficient level of capacity to execute their mandated functions at the time of the donors’ exit.

This Evaluation serves both learning and accountability purposes: 1) “With regard to learning, the Evaluation will aim to produce knowledge that enables policy, strategy and decision makers to design good strategies for support to capacity development and to review, adjust or discard planned and ongoing interventions based on previous experience with support to capacity development; and 2) With regard to accountability, the Evaluation will aim at assessing results of support to capacity development and to what degree it represents value for money in terms of relevance, effectiveness and efficiency”. While also contributing to the Joint Evaluation, the Danida part deals specifically with Danida supported interventions.

Scope of the Danida capacity development Evaluation

A total of 21 Danida-supported interventions – in which capacity development (CD) was an explicit intention – are included in the evaluation sample. The focus of the Evaluation has been on public sector organisations’ performance as regards relevance, coverage and quality of services and regulations. The Danida interventions included in the evaluation sample were launched and implemented between 2006 and 2015 (one programme will be completed in 2018) and cover seven countries. Field studies were conducted in Nepal, Uganda and Tanzania. The sectors included in the sample were: government and civil society, education, health, agriculture, business, energy, transport, water, environment and climate. The Danida-supported interventions have been implemented in joint programmes with other donors (bilateral and multilateral) in highly complex environments. The thrust of Danida’s engagements has been at the sector level – generally with interventions at all administrative levels (national, regional, local) involving interactions with line and sector ministries, local governments, civil society and the private sector. National poverty reduction strategies and sector legislation, policies and plans provided the directions for the support.

Evaluation approach and methodology

The Danida supported interventions have been assessed in relation to the OECD/DAC CD guidelines (2006) and the Danida CD guidelines (2006 and 2011). The OECD/DAC Guidelines summed up what had been learned about CD over the past decades and what was seen as the state-of-art at the time, and represented a “Capacity Development Results Framework”. The international CD praxis involved three interdependent levels of intervention: the enabling environment (national/sector framework), the organisational level, and the individual level. Appreciating the interactions between the three levels of CD means recognising the important role of systemic factors in enabling or blocking change. Alternative CD approaches have since then emerged, some of which are based on an experimental and learning oriented approach and implemented gradually over time.

Findings

What worked well in the achievement of CD results: Danida’s adherence to its programme management guidelines – as regards sector-wide approaches, ownership and partner-led implementation – has contributed to creating conducive environments for endogenous CD processes that in most cases increased organisational performance. The majority of the sector programmes in the evaluation sample have performed well and have experienced a significant increase in sector-outcomes in terms of coverage and quality, which is well documented in progress and other reports, see the two cases in the box below. Danida-supported interventions were to a large extent designed to reflect external and internal contextual factors in the overall programme design, which implicitly also included CD interventions. Danida has consistently demonstrated a commitment and a willingness to respond to the stated priorities and strategies of the priority countries and partner organisations – and is recognised as a trusted partner, largely because of its relative flexibility to adapt to changing contexts and its long-term commitment.

Attention to CD can make a difference for achieving sector-outcomes

The Nepalese Alternative Energy Promotion Centre (AEPC) succeeded in providing alternative energy solutions (solar, biomass and hydro energy, and improved cooking stoves) to more than 1 million households. The organisational set-up for disseminating energy services comprised district and village development committees mandated to improve energy services; NGOs involved with promotional and management activities; private companies engaged in supplying, installing and maintaining rural energy solutions; and financial institutions managed financial affairs and insurance. Subsidies for energy installations were provided through the Rural Energy Fund. The Danida support was provided through the Energy Sector Assistance Programme, Phase II 2007-2012. The capacity development activities included among others: review of policies and preparation of a national rural energy plan; improvement of the recipient organisations’ management practices; capacity development of partner institutions, energy cooperatives, and the financial sector; training of trainers for local partner institutions; facilitation of credit line availability; and awareness creation for media, key local players and health personnel.

The Tanzania Commission for AIDS (TACAIDS) succeeded in establishing a non-medical prevention and care system for prevention of HIV and AIDS that contributed to reducing the national HIV prevalence from 7% in 2003/2004 to 5.3% in 2011/2012 among the population between 15 and 49 years. A national funding mechanism was established to fund district and civil society stakeholders’ engagement in the prevention activities. The support to TACAIDS was part of Danida’s Health Sector Programme, Phase IV 2009-2014. The capacity development activities included among others: operationalisation of the National Multi-sectoral Strategic Framework (NMSF) for combat of HIV and AIDS; institutional and capacity development of TACAIDS for effective coordination of the NMSF implementation; a new organisational structure for TACAIDS; capacity development of TACAIDS at central, regional and local levels; non-government sector supported to implement the NMSF at community level; and multimedia production and dissemination of prevention material directed at the youth.

Lessons learned: By addressing all three levels of the CD results framework (enabling environment, the organisational and the individual level) as appropriate for the development intervention, the chances for achieving sector-outcomes will be higher – more so if the CD tasks are well interlinked at the three levels and between the levels.

The Danida-supported programmes generally had few technical advisers and donor funding for CD was mainly used for training, equipment, and systems development.

A major part of the work that led to increased organisational performance – in terms of generating sector-outcomes – has been undertaken by the organisations’ own staff.

In some cases, the domestic revenue base for financing the sector’s recurrent expenditures was expanded by introducing user charges. An important feature in some programmes was the engagement of the private sector and civil society organisations (CSOs) acting in complementary roles to public service providers and thus contributing to the expansion of public service delivery capacity. In conclusion, it is reasonable to assume that the organisations’ internal capacity has grown by some measure through endogenous processes whether planned or unplanned.

What worked less well in the achievement of CD results: Changes in CD outcomes have not been tied to predefined results frameworks. While results-based management (RBM) has generally been applied and emphasised in Danish assistance for sector-outcomes, results-based CD in Danida programmes – as introduced in 2005 and building on the “Logical Framework Approach” (1996) – has to a large extent not been applied. Assuming that capacity grew simultaneously with the generation of sector-outcomes, it is essentially not possible to judge the actual extent to which the supported organisations’ capacity increases have been directly or indirectly influenced by Danida’s support. It is thus not possible to establish a direct link between sector-outcomes and CD outcomes. In the absence of direct evidence on CD results, the achievement of sector-outcomes is regarded as circumstantial evidence for CD results.

Although CD interventions have been included in programme documents and have been subject to progress monitoring and reviews, it is evident that a consistent and comprehensive approach to CD has generally not been applied. The Danida-supported programmes were not able to overcome many CD related organisational and enabling environment constraints – as opposed to what was the stated ambition in programme documents. An appropriate risk analysis of the CD context could have helped to highlight challenges so that these could be counteracted. Danida has always aligned itself to the strategies of the recipients, even if the expected results were not optimal and has in a few cases continued to support organisations that were not committed to change. The main thrust of CD interventions has been on the individual level and to some extent on policies and strategies related to the sectors’ enabling environment. The organisational level has only received limited attention, while arguable this should have been at the centre for enhancement of the partners’ performance – and for creating sustainable organisations.

It appears that Danida staff, technical advisers and consultants alike have not focussed on the strategic role that CD can play in bringing about developmental effects. Technical advisers posted in the partner organisations were often underutilised and were not adequately involved in facilitating organisational changes. This is in contrast to Danida’s TA guidelines as well as the preference as expressed by some partner organisations that Danida’s TA/CD support should support the organisations’ staff in playing a key role in programme implementation. The reduction of human resources in the Danish embassies and the increasing complexity of programme planning and implementation, also raise the question regarding the division of work between embassy staff and TA advisers and how this could be organised differently.

Danida’s capabilities to manage CD processes are in need of improvement. CD practices have not been as effective as they could have been, largely because of: a) under-specificity of results; b) inadequate CD management practices; c) ineffective oversight, project management and supervision; and d) inadequate systems to support CD. Danida needs to develop not only the capacity of target organisations, but also the capability of recipient country managers and Danida personnel (including TA advisers). Despite the renewed emphasis on CD, as expressed in the Paris Declaration (2005) and the Accra Agenda for Action (2008), Danida’s management has not been able to lift the CD ambitions into practical implementation. The Evaluation’s findings are in line with many published reports from leading development organisations such as the WB, ADB, EU and DFID.

A changing global development agenda: The global development agenda will undergo significant changes in the coming years. From a CD point of view, there will be a need to keep track of significant development trends and how these generally and for CD specifically will influence Danida’s future development assistance. The outcome document of the Addis Ababa Action Agenda (July 2015) for “Financing of Development” presents an ambitious financing framework that includes concrete policy commitments. The adoption of the post 2015 development agenda in New York in September 2015 and the new universal climate change agreement in Paris in December 2015 are two other essential events that will shape the future development agenda. The Danish Ministry of Foreign Affairs published “More Denmark in the World” in December 2014 presenting Denmark’s visions and foreign policy interests. The new Government that was sworn in in June 2015 may further elaborate the visions for Denmark’s development assistance.

A recent OECD survey (2015) provides insights into what partner countries anticipate will be their main development challenges within five to ten years, and how they expect their relations with donors to evolve to meet these challenges. The survey finds that the demand for development cooperation will remain strong given the economic and environmental challenges that lie ahead. The 40 partner countries surveyed expect donors to shift to a more enabling role in coming years: providing vital finance in support of government-led sector programmes; delivering more and better technical assistance and policy support; and leveraging more private finance. Resources for development cooperation can be expected to be under heavy pressure from other priorities. There is therefore a need to focus on development effectiveness and on the sustainability of development interventions. Appropriate attention to CD could provide feasible options for addressing development challenges.

Conclusions and recommendations

Overall conclusions

The findings above lead to the following conclusions:

  • Danida has, through its adherence to the programme management guidelines, facilitated the creation of conducive environments for endogenous CD processes that have contributed to increased organisational performance in most of the Danida-supported interventions;
     
  • Support for CD was most successful when:
     
    • Internal and external contextual factors were taken into consideration;
       
    • There was strong ownership and commitment on the part of the organisations involved;
       
    • All three levels – the enabling environment, the organisational level and the individual level – were addressed simultaneously with specific attention to the organisational level in order to enhance organisational performance;
       
    • Wherever relevant, the private sector and the CSO were drawn into the implementation in a complementary way to enhance service delivery and coverage;
       
    • The demand side was addressed, e.g. through support for civil society calling for transparency and accountability in the provision of services;
  • While results-based management was emphasised as a Danida policy, most CD interventions were not results-based or were not managed as such;
     
  • Systematic attention to CD and organisational development could have promoted recipient organisations’ performance and productivity further and thus contributed to more effective development;
     
  • While CD and TA guidelines of good quality were available – and the intent of CD was well expressed in Danida documents generally – they have rarely been applied effectively during planning and implementation of development inter ventions;
     
  • Management at Danida headquarters and embassies could have engaged more effectively in CD during programme design and implementation.

Addressing capacity development in future

It is evident that CD requires increased attention in order to bring about development effects more effectively.

The Evaluation recommends that:

  1. Danida should recognise the significance of CD and its potential contribution to development effects – consistently with its policy statements and guidelines. In that regard Danida should emphasise CD more prominently in its programmes and their management. Danida should – in priority countries – continue its practice of long-term engagement in line with national priorities and partner driven interventions, with a view to achieving sustainable public sector organisations and supportive enabling frameworks.
     
  2. Danida should consider if the Danida Fellowship Centre (DFC) could become the focal point for CD – with due reference to the 2014 CD Policy – keeping track on CD trends and strategies. For this to be effective, DFC should interact more closely with the Department for Technical Advisory Services.

Programming and integration of CD

A new paradigm for programme management “Guidelines for Country Programmes” (GCP) was introduced in 2013 and updated in 2015 for Denmark’s priority countries. While the GCP makes frequent references to CD, they could provide more overall direction and guidance on the design, planning and implementation of CD initiatives with due reference to the specific CD and TA guidelines. In short, whereas the GCP provides a sense of policy direction, it does not really provide practical support to those faced with the management of the CD “programme cycle”. The CD and TA guidelines are from 2011 and 2009 respectively and could benefit from being updated to correspond with the new paradigm for programme management.

The Evaluation recommends that:

  1. The Guidelines for Country Programmes should – in connection with the next revision – emphasise CD more strongly as the cornerstone for development, and make due references to planning and operational issues as stated in the CD and TA guidelines. Danida should apply its change model (Theory of Change) in a way that integrates CD outcomes, which should be dynamic to reflect how the enabling environment must be managed as the intervention proceeds, and how organisational outcomes are achieved in a planned and systematic way. Danida should ensure that its interventions are clearly designed to reflect: the specific contexts and other realities of the planned intervention; as well as the conceived risks and how these could be mitigated. An experimental and learning approach should be applied for high-risk situations or where local contexts do not enable a full results-based approach. Danida should continue to support the demand side with a view to strengthen public organisations accountability.
     
  2. The CD and TA guidelines should be updated with an emphasis on their practical use during planning and implementation of programmes with an emphasis on CD outcomes and impact in public organisations and their potential cooperation with the private sector and CSOs. The TA guidelines should be updated to elaborate the role of technical advisers in capacity and capability development in the partner organisations and how TA advisers should interact with embassy staff. CD and TA guidelines could be combined in order to emphasise the interrelation between CD and TA.

Management of CD interventions

Management of CD interventions will require increased awareness and attention by Danida programme officers and national partners’ programme managers.

The Evaluation recommends that:

  1. Danida should assess what competences are required among its staff (including Danida advisers) at headquarters and embassies to address CD comprehensively in programme planning and implementation, and how CD is integrated in the overall programme design to ensure that CD receives adequate attention. Furthermore, Danida should consider what means could be applied to raise knowledge and competence among its staff, ex. through short courses, E-learning, etc. More generally, Danida needs to improve its knowledge management systems to the point where they are actually used as references by personnel.
     
  2. Danida should assess – consistently with its 2011 CD guidelines – what basic requirements and competences should exist in partner organisations to enable supervisors and programme officers to lead the CD process and manage the maintenance and further development of capacity and capabilities over the long term – and what would be the means of developing such competences.
     
  3. Danida should develop a rapid assessment framework for CD that could be applied for appraisals, reviews and progress monitoring to ensure that CD is properly integrated in the programme design and adjusted to changing circumstances. The use of such framework could also be a significant source of learning as well as demonstrating Danida’s corporate accountability and assurance frameworks.

1 Introduction

1.1 Evaluation background

This Synthesis Report is concerned with the “Evaluation of Capacity Development in Danish Development Assistance” that was launched in September 2014.

The Evaluation forms part of the “Joint Scandinavian evaluation of support to capacity development” involving Danida, and the Norwegian and Swedish development agencies Norad and Sida. Capacity development has been an essential issue in most donor supported development interventions over the last couple of decades – the perception being that supported organisations would have acquired a sufficient level of capacity to execute their mandated functions at a satisfactory level at the time of the donor’s exit. While there are positive examples of well performing organisations, experience has shown that many supported organisations continue to encounter problems with having adequate capacity to perform – despite substantial donor support and their own efforts to develop and maintain capacity. While it is relatively easy to explain why CD efforts did not succeed as anticipated, it has become more difficult to explain why others succeed.

This Evaluation attempts to unpack the causes for and learn from successful CD interventions. Based on a long list prepared by Danida, a sample of selected Danida development interventions has formed the basis for the Danish part of the Evaluation, in which CD was an explicit intention, whether alone or as part of broader objectives (see evaluation approach below). The thrust of the sample has been on public sector institutions, some of which have interacted with civil society organisations (CSOs) and the private sector as a means of increasing the capacity and effectiveness of public service delivery.

As part of the joint effort, Norad and Sida have, in parallel, conducted corresponding CD evaluations for their development assistance. The characteristics of the three Scandinavian donors’ development assistance vary considerably but the results of the three evaluations can be considered to be complementary, given the number of approaches and strategies that can be used in CD and the wide variety of contexts in which each donor has provided support. Each donor has conducted case studies in three countries (see Table 1), which provides amble scope for analysis and extraction of lessons learned.

Table 1: Countries selected for in-depth assessment of supported interventions
Sida Norad Danida
Kenya Malawi Tanzania
Cambodia Mozambique Uganda
Bosnia/Herzegovina Vietnam Nepal

1.2 Evaluation purpose

As stated in the Terms of Reference (ToR) for the Evaluation, the purpose of the Evaluation is to improve decision-making and strategy development in Danida regarding support to capacity development in developing countries. The purpose as stated in the ToR has both learning and accountability elements:

  1. “With regard to learning, the evaluation will aim to produce knowledge that enables policy, strategy and decision makers to design good strategies for support to capacity development and to review, adjust or discard planned and ongoing interventions based on previous experience with support to capacity development.
     
  2. With regard to accountability, the evaluation will aim at assessing results of support to capacity development and to what degree it represents value for money in terms of relevance, effectiveness and efficiency”.

The primary intended users and audience for the Evaluation are managers and staff within Danida as well as Sida and Norad. The intended secondary audiences are government and institutions in partner countries, other aid agencies, ministries of foreign affairs, and various intermediaries involved in development cooperation including multilateral institutions. The evaluation results will also be communicated to the general public and political systems for accountability purposes.

1.3 Evaluation approach and methodology

The evaluation approach and methodology builds on the ToR that provide the scope for the Evaluation, attached as Annex A, which includes two annexes: 1) Approach Paper; and 2) Specification of Methodology (Danida). In addition, two supporting documents were provided: 1) Literature Review for the Joint Evaluation on Capacity Development (Sida); and 2) Methodological approaches to evaluate support to capacity development (Norad).

Danida is the client for the Danish evaluation team, which is accountable to Danida’s Evaluation Department (EVAL). Danida, Sida and Norad have established a Steering Group and an Advisory Group with members from each agency that oversee and guide the conduct of the Joint Scandinavian Evaluation. Three joint sessions between the Steering Group and the three evaluation teams were held: i) in Oslo on 29-30 September 2014 to launch the Evaluation; ii) in Copenhagen on 8-9 December 2014 to discuss the Inception Reports; and iii) in Stockholm on 13-14 April 2015 to discuss the Country Reports and the template for the respective Synthesis Reports. A Danida Reference Group has, furthermore, been established to provide feedback and advice to the Danish evaluation team. An overview of the evaluation approach and methodology is provided in the next two subsections. A detailed elaboration of the approach and methodology is presented in Annex B.

Evaluation approach

Broadly speaking, the approach of the Evaluation contains five steps, where each step builds on the findings and development of the previous:

  • Step 1: Portfolio Screening and Literature Review
     
  • Step 2: Desk-based Review
     
  • Step 3: Country Case Studies
     
  • Step 4: Final Evaluation Report prepared by each of the three evaluation teams
     
  • Step 5: Synthesis Report integrating the results from the Sida, Norad and Danida teams, which will be prepared with contributions from the three teams.

Portfolio Screening: An initial list of eligible development interventions – that were considered relatively successful – was prepared by Danida’s Evaluation Department (EVAL), from which a long-list of 47 development interventions was selected by EVAL. The majority the 30 interventions selected for preliminary screening was taken from this list of 47 interventions, but complemented with development interventions from the initial list in order to have a representative coverage of sectors, countries, and size (budget-wise). The guiding selection criteria are presented in the ToR (Annex 2, Section 2). A note on the results of the Portfolio Screening is attached as Annex C.

Desk-based Reviews: For the more detailed Desk-based Review (DBR), the selection 17 of the 30 interventions was based on the evaluation team’s assessment on the CD content of each intervention for the more detailed DBR. It was found that adequate information was not available for three of the selected interventions. Each intervention was subject to a document review; the main outcome of which being an Assessment Note structured according to the evaluation questions as regards relevance, efficiency, effectiveness, impact and sustainability. Information was fed into a common Scorecard that was used by the three parallel CD evaluations.[2] The assessment notes and extracts from the Scorecard are presented in the “DBR Report”, attached as Annex D.

Country studies: Based on the Portfolio Screening and in consultation with EVAL, Nepal, Uganda and Tanzania were selected for more in-depth country studies – since only three countries should be included for field studies.[3] The country selection was made with due consideration given to the countries selected by the Sida and Norad evaluation teams. The Danida supported interventions chosen for further assessment during the country visits were those from the DBR plus a few additional interventions, which could offer interesting insights.[4]

The country visits included consultations with government (central and local), CSO and private sector stakeholders and embassy staff. Focus group discussions and questionnaire survey were conducted at the end of the country visit with key stakeholders. A survey was conducted in each country of other donors’ interventions having a CD content. Briefing and debriefing sessions as well as intervention-specific discussions were held with embassy staff. A country report was prepared for each of the three countries recording of observations and findings; response to the generic hypotheses in ToR; and presentation of conclusions on past, current and future CD support. The country reports are attached as follows: Nepal in Annex E, Uganda in Annex F, and Tanzania in Annex G.

Evaluation methodology

Theory of Change (ToC): For years, donors have sought to model their CD process and their support to CD into an explicit change logic model. The ToC is elaborated on a version of a hypothetical results-chain, which is assumed to happen at the organisational level as a result of the support provided by the donor(s) in conjunction with the efforts and resources provided by the partners. It takes into account, but does not specify either the various contextual parameters or determinants assumed to influence the results of the CD effort, or the motivators and enablers that are internal to the partner and other agencies that form part of the institutional framework and organisational ecosystem.

Evaluation Questions: The ToR listed 15 evaluation questions (EQs). The EQs have been slightly adapted to enable a structured evaluation approach in accordance with the OECD/DAC evaluation criteria. A distinction has also been made whether the EQ is a ‘research question’ meaning that the answer to the question emerges through document reviews, interviews, etc.; or whether it is an ‘analytical question’ meaning that the answer to the question is based on the evaluation team’s judgement based on the answers to the “research” questions. Accordingly, the analytical questions follow the research questions in their sequencing.

Focus Areas and Hypotheses: The ToR presented four focus areas and corresponding hypotheses, which are seen as critical dimensions of capacity, capacity development and support to CD, see Table 2. The four hypotheses were tested as part of the country assessment of the selected interventions.

Table 2: Focus areas and hypotheses

Focus Area Hypothesis
i The relevance and opportunity of a “best fit approach” for CD support are well adapted to specific intra- and inter-institutional dynamics and the wider context. Donor support to capacity development is (more) effective when it fits the drivers for and constraints to change.
ii Within the “best fit dimension”, the appropriateness and legitimacy of external (donor) involvement in different dimensions of capacity development, and whether some processes may be so complex and demanding that the ability of donors to add value is limited. Donor support to capacity development is (more) effective when donors engage in dimensions of capacity development, where external agencies are likely to be able to contribute (not too complex for outsider facilitation) and when donor involvement is found appropriate and legitimate.
iii The merits of looking beyond the supply side of public sector institutions to foster broader accountability relations or other types of collaboration with e.g. civil society, private sector, media and oversight institutions. Donor support to capacity development is (more) effective when one looks beyond the “supply-side” or “push” approaches that only work from the inside in public organisations, aiming also to foster broader accountability relations.
iv How a results-focussed approach to aid for capacity development can serve to improve learning and accountability among aid agencies in future. Donor support to capacity development is (more) effective when it uses results sensibly to measure progress, correct course and learn.

Source: a) Focus areas from Terms of Reference; and hypotheses from Norad’s draft “Developing ‘scorecards’ for CD interventions, 16 September 2014”, which is more elaborate than those presented in the Approach Paper (ToR Annex1). The joint donors approved the hypotheses.

Capacity development

The Evaluation relates its findings and conclusions on capacity development to Danida’s approaches as elaborated in its guidelines as well as to emerging approaches. Danida’s approaches to capacity development (CD) and technical assistance (TA) have evolved significantly over the past decade; an account of this evolution and the interrelationship with Danida’s programme management guidelines are presented in Annex H. The Evaluation’s use of the capacity and capability terms is further discussed in Annex I. A detailed analysis of the CD related hypotheses (ref. Table 2) is presented in Annex J.

1.4 Limitations

Overall the Evaluation did not encounter any problems that it could not resolve through intellectual or logistical accommodation. It was encouraging to share findings with the personnel of the embassies and to find that their assessments were largely in line with those of the Evaluation. The embassies assisted by providing detailed comments to the country reports, thus improving the level of reliability and validity of the Evaluation’s findings. We have also had very interesting methodological discussions with EVAL and have been able to quickly move ahead with the assignment.

This Evaluation of capacity development has its own challenges however, including:

  • The joint methodological approach for the Evaluation limited the flexibility for adjusting the evaluation questions and the scorecard to become more rational.
     
  • Much of the detailed documentation and data regarding how and why an intervention was designed in a particular way was not readily available.
     
  • Progress reports and other intervention-based documentation rarely spoke of capacity progress and capacity development outcomes even less. This made the Evaluation to rely more than necessary on individual perception, rather than empirical evidence.
     
  • Simply noting that a capacity development initiative, action or activity contributed to “organisational strengthening”, for example, was not particularly helpful for the objectives of the Evaluation; the Evaluation therefore sought to identify in what ways those investments and efforts contribute to the attainment of organisational performance (at outcome levels). The documented information to do that was scarce and the Evaluation had to rely on interpretations of stakeholders and proxy measures.
     
  • There were many occasions where misunderstanding and confusion occurred because of a lack of common understanding of what were, in fact, the various aspects and concepts of CD. Baselines and results frameworks were almost non-existent, making analysis of progress towards expected CD goals and objectives difficult.
     
  • Many managers, engaged in programme formulation and implementation from Danida and partner organisations, had moved on and were not available.

Despite the above limitations, it is the opinion of the Evaluation that its findings, lessons learned, conclusions and recommendations are valid and can provide inputs to improve decision-making and strategy development in Danida regarding support to CD in developing countries.

1.5 Structure of the evaluation report

Following the Introduction, Chapter 2 provides an account of: 1) Danida’s approach to CD by examining Danida policies, strategies and guidelines on programme management, technical assistance and CD; and 2) emerging approaches to CD. Chapter 3 presents the evaluation sample and its characteristics. Chapter 4 presents the rationale for answering the 15 evaluation questions (as stated in the ToR with some modifications) and provides the answer to these, while also providing the evidence on which lessons learned, and conclusions and recommendations are based. Chapter 5 discusses the four hypotheses (as stated in the ToR) and comments on their validity, the result of which feed into conclusions and recommendations. Chapter 6 draws the lessons learned from the Evaluation. Finally, Chapter 7 presents the Evaluation’s conclusions and recommendations.

The lists of persons met in the field and documents consulted concerning the country studies are included as appendices to the Country Study Reports. Other persons met are listed in Annex K, and additional relevant documents consulted are attached in Annex L.


[2] The purpose of the DBR is to collect consistent information across interventions and donors to be able to compare interventions. Therefore, it has been emphasized not to deviate from the scorecard instructions as developed by the Sida evaluation team.

[3] ToR, Section 6.

[4] ToR Annex 2, Country study design: “All interventions supported by the agency (Danida) in that country that fall into the scope defined are eligible for further study”.

2 Approaches to capacity development

2.1 Danida’s approach to capacity development

Approaches applied during the evaluation period

Danida’s approach to capacity development has been closely linked to its policy framework, which has evolved under influence of the High Level Fora on aid effectiveness: the Rome Declaration (2003); the Paris Declaration on Aid Effectiveness (2005)[5]; the Accra Agenda for Action (2008); the Busan Partnership for Effective Development Co-operation (2011); and the Mexico Partnership for Effective Development Cooperation – Building Towards an Inclusive Post-2015 Development Agenda (2014). The Accra Agenda for Action has policy statements on capacity development (like the Paris Declaration), see Box 1.

Box 1: The Accra Agenda for Action and Capacity Development, 2008

Together, developing countries and donors will take the following actions to strengthen capacity development:

  1. Developing countries will systematically identify areas where there is a need to strengthen the capacity to perform and deliver services at all levels – national, sub-national, sectoral and thematic – and design strategies to address them. Donors will strengthen their own capacity and skills to be more responsive to developing countries’ needs.
     
  2. Donors’ support for capacity development will be demand-driven and designed to support country ownership. To this end, developing countries and donors will: i) jointly select and manage technical cooperation; and ii) promote the provision of technical cooperation by local and regional resources, including through South-South cooperation.
     
  3. Developing countries and donors will work together at all levels to promote operational changes that make capacity development support more effective.

Programme Management: Danida’s programme management is intricately interwoven with the technical assistance and CD modalities. The Danida interventions included in the evaluation sample were launched between 2006 and 2010 and completed between 2011 and 2015. This implies that it was mainly the programme management guidelines from 2003 to 2009 that were used for the formulation of the Danida supported interventions, whereas those from 2009 to 2011 were the basis for a major part of the annual or biannual programme reviews – providing the prospect of changing directions and emphases of programmes under implementation.

Danida’s “Guidelines for Programme Management,” (GPM) were issued in 2003 and replaced the 1998 Guidelines. A sequence of editions was issued up to and including the 2009 GPM. The aim of the GPM was to move from separated Danida activities towards the application of the ‘Sector Wide Approach’, ‘Sector Budget Support’, and alignment with the national partner’s procedures and formats. The national policy framework should be analysed including the poverty reduction strategy with due attention to the ‘Millennium Development Goals’ (MDGs). In line with the commitments of the Paris Declaration (2005) and the Accra Agenda for Action (2008), the underlying principle of the GPM was to strengthen and use country systems to the maximum extent possible in the implementation of development assistance to public partners. The update of the GPM in 2011 was prompted by several events, most notably the adaption of the new overall policy for Danish development assistance entitled “Freedom from Poverty – Freedom to Change (2010)”. The various editions of the PMG made references to CD and TA guidelines that were valid at the time. The guidelines for CD and TA from 2004 to 2011 are shown in Box 2. Reference is made to Annex H for further details.

Box 2: Danida TA and CD guidelines 2004-2011

Technical assistance guidelines

  • Technical Assistance in Danish Bilateral Aid – Policy Paper, February 2004
     
  • Guidelines for Technical Assistance, June 2009

Capacity development guidelines

  • A Results-Oriented Approach to Capacity Change, February 2005
     
  • Guidance Note on Danish Support for Capacity Development, August 2006
     
  • Addressing Capacity Development in Danish Development Cooperation, Guiding Principles and Operational Steps, January 2011
     
  • Applying Political Stakeholder Analysis – How can it work? 2011

The TA Policy Paper 2004 was available to inform the TA modality of all selected Danida interventions. The 2009 Guidelines for Technical Assistance were only able to inform a few of the sample’s interventions at the formulation stage, but had subsequently good prospects for doing so during implementation in connection with annual or biannual reviews. The two CD guidelines from 2005 and 2006 were available from the onset. The two CD guidelines from 2011 may have come into some use in connection with annual or biannually reviews or special studies.

Technical Assistance (2009): CD is regarded as the general objective of TA. TA is defined as the transfer, adaptation and mobilisation and utilisation of service, skills, knowledge and technology. Technical assistance includes long- and short-term advisers and consultants, training activities, study tours, seminars and institutional cooperation (twinning arrangements). The 2009 Guidelines emphasise national ownership, alignment to national procedures and harmonisation of TA.

Capacity Development (2011): Managing own affairs successfully, is an integral part of CD. Results-focused CD is much in line with the 2010 Danish development strategy. The 2011 Guidelines posit that CD depends on: the broader contextual drivers and constraints to change; and ownership and leadership of those whose capacity is to be developed – as external partners can only play a secondary role. CD is as much a political as a technical process, taking time and requiring flexible adaptation to the changing context. The focus is on public sector organisations where the CD is particular complex, but the principles are equally valid for non-state stakeholders. The core issue of ownership of CD processes is discussed in terms of change readiness and the emphasis is on getting processes and results right. This implies detailing results in the often ‘missing middle’ between a narrow focus on what donors do and the long-term organisational and sector outcomes and impacts that have to be the focus of sector policies and plans. Individuals and organisations can be incentivised and assisted, but eventually, CD is an endogenous process.

Current Danida approaches

Guidelines for Country Programmes, 2013 and 2015: The Strategy for Denmark’s Development Cooperation “The Right to a Better Life, 2011/2012” was adopted in 2011 and has the dual purpose of reducing poverty and promoting human rights – expressed through four strategic priority areas: 1) Human Rights and Democracy; 2) Green Growth; 3) Social Progress; and 4) Stability and Protection. Consequently in 2013, Denmark embarked on a ‘Country Programme Concept’ as reflected in the “Guidelines for Country Programmes”, which replaced the 2011 PMG. Henceforth, the country programmes should relate to the Busan Agenda, i.e.: ownership of development by partner countries; results focus; inclusive development partnerships; and transparency and accountability. Denmark supports the broadening of focus from ‘effective aid’ to a focus on cooperation for ‘effective development’ with a strong emphasis on results and efficiency in all forms of cooperation. The 2009 TA Guidelines and the 2011 CD Guidelines continue to remain valid.

Policy for Danida Capacity Development Support Programme, 2014: The purposes of the Danida Capacity Development Support Programme (DCDSP) are to: a) contribute to enhanced organisational performance and long-term organisational development for development partners in the South; and b) offer capacity development support for the promotion of the Danish strategic priorities for development cooperation, and to promote the principles of participation, accountability, non-discrimination and transparency. The Danida Fellowship Centre (DFC) is mandated to implement the DCDSP. DFC elaborated an Action Plan for its operations for 2014-2017 setting the framework for how DFC may assist the Danish embassies and their national partners in Danida priority countries with formulation and facilitation of CD interventions in country programmes.

2.2 Emerging approaches to capacity development

This section outlines CD approaches that have emerged among the donor community in recent years. These approaches are not so much ‘new thinking’ as an attempt to provide pragmatic and operational guidance to achieve effective CD outcomes. In many ways these approaches are complementary to those that have been formulated by Danida during the same period of time (and vice versa).

OECD/DAC working towards good practice

The OECD/DAC Guidelines “The Challenge of Capacity Development: Working Towards Good Practice (2006)” sum up what has been learned about CD over the past decades.[6] The international CD praxis has for some time involved three levels of intervention: the enabling environment (national/sector framework), the organisational level, and the individual level – the levels are interdependent. Appreciating the interactions between the three levels of CD means recognising the important role of systemic factors in enabling or blocking change. This approach entails a concern with the wider political economy of change, but continues to treat CD as a distinct challenge at entry point. Table 3 illustrates capacity development levels, elements and approaches.

Table 3: Capacity Development Results Framework

Capacity development level Some elements that define capacity Main CD approaches

Enabling environment

Socio-political interests and linkages

Social and economic factors

Policy frameworks

Strategies and plans

Laws and regulatory mechanisms

Institutional landscapes

Resource allocation among publicsector functions

Reform processes

Organisational level

Mandate, legitimacy, credibility

Values, political interference

Organisational structure and processes

Planning and approval procedures

Systems and tools including M&E

Knowledge management

Staff levels, qualificationsand delegated authority

Organisational development

Individual level

Skills and knowledge

Experience

Attitude

Competences

Human resources development

It is important to begin by asking the question “capacity for what?” and focus on the specific capacities needed to accomplish clearly defined goals. The Guidelines outlines a systematic approach to a flexible and “best fit” approach to CD, which calls for a systematic effort to think through what might work in the particular circumstances. This can be done by ensuring that adequate attention is given to both individual and organisational issues and to the enabling environment for CD at each stage in a process of four main steps or recurrent tasks: i) understanding the international and country contexts; ii) identifying and supporting sources of country-owned change; iii) delivering support; and iv) learning from experience and sharing lessons. These steps are likely to be a continuous and iterative process.

Capacity development – unleashing, strengthening and maintaining of capacity – goes well beyond the technical cooperation and training approaches that have been associated with CD in the past. The stock of human capital and the supply of general and technical are important. However, a country’s ability to use skilled personnel to good effect depends on incentives generated by organisations and the overall environment. CD is necessarily an endogenous process as it involves much more than awareness of technical subjects and general organisational principles. The OECD/DAC definition on CD is presented in Box 3.

Box 3: The OECD/DAC definitions on capacity development, 2006

Capacity is the ability of people, organisations and society as a whole to manage their affairs successfully.

Capacity development is the process whereby people, organisations and society as a whole unleash, strengthen, create, adapt and maintain capacity over time.

Promotion of capacity development refers to what outside partners – domestic or foreign – can do support, facilitate or catalyse capacity development and related change processes.

Problem-Driven Iterative Adaptation[7]

The authors suggest that reform dynamics are often characterised by ‘isomorphic mimicry’ – the tendency to introduce reforms that enhance an entity’s external legitimacy vis-à-vis donors and their support, even when they do not demonstrably improve performance. Such strategies of isomorphic mimicry in individual projects, policies and programmes add up to ‘capability traps’ – a dynamic in which governments constantly adopt “reforms” to ensure ongoing flows of external financing and yet legitimacy never improve.

Capability traps can be avoided and overcome by fostering different types of interventions. The authors propose that efforts to build state capability should: i) aim to solve particular problems in particular local contexts; ii) create an ‘authorizing environment’ for decision-making that encourage experimentation and ‘positive deviance’; iii) involving active ongoing and experiential (and experimental) learning and the iterative feedback of lessons into new solutions, doing so by; iv) engaging broad sets of agents to ensure that reforms are viable, legitimate and relevant – that are politically supportable and practically implementable. This kind of intervention is proposed as an alternative approach to enhancing state capability, and is called Problem-Driven Iterative Adaptation (PDIA). Capability is defined as what it takes to function effectively.

Development interventions can be analysed at three levels: agents at front line and in leadership positions; organisations inhabited by agents; and the environment or ecosystems of organisations. Essentially the powerful agents or elites are commonly considered the most embedded in their contexts, and thus are often the less likely to perceive the needs for change, to have access to ideas for change, or to risk their interests in pressing for change. In contrast, agents at the periphery – or front line – are less embedded in extant rules, which is partly why they also benefit less from them. Their low embeddedness make them more open to criticizing incumbents and to entertain change; but they lack the power to make it happen. Change is only possible if something bridges the agents with power to those with ideas. This could be a direct link or third party link between a central leader and front line agent. Such bridge could open the elite to an alternate awareness of their reality and spur a process of entrepreneurship, through which multiple agents combine to define and introduce change in their contexts. These could be organisations or individuals.

PDIA advocates for changes to happen through incremental steps, especially when operating in uncertain and complex contexts where reformers are unsure what the problems and solutions actually are. Such process is termed ‘muddling through’, which implies taking a gradual approach to addressing particular problems. A case survey of 30 studies has demonstrated that PDIA is viable, possible, and could be a key to success.[8] Much of the literature on public sector reforms reflects on their failures. The research investigates which of two competing theories best explain why some reforms exhibit positive deviance – one being the ‘Solution and Leader Driven Change’ (SLDC) and the other PDIA. The bulk of the evidence from the study suggests that the PDIA is relatively more successful than SLDC. It seems that PDIA and SLDC are two viable paths through which positive deviance can emerge; although PDIA seems to provide the wider path for more positive deviance. The research further noted that many reforms are actually a combination of PDIA and SLDC approaches.

Dynamic Capability and Capacity Approach

Dynamic Capability and Capacity Approach (DCA) is a strategic way of ensuring that an organisation constantly has the means to execute its mandate and reach its objectives (expected results). It enables organisations to ensure that they have put into place “what it takes” to perform as required and to adapt to changes that can have an impact on its ability to achieve its results (strategic, operational, tactical, etc.). Each organisation has many capabilities to enable it to perform; the interplay of these capabilities, and their efficiency at producing results, is a key concern of managers at all levels. The DCA emphasises the ability to react adequately and timely to internal and external changes that requires a combination of multiple capabilities.[9] DCA refers to “the capacity of an organisation to purposely create, extend, or modify its resource base” (Helfat et al., 2007). The DCA concept was originally conceived to address private companies’ need for adjusting its capabilities to rapidly changing market situations, but has subsequently been adapted to be applicable to public sector organisations in developing as well.

The DCA is a strategic and operational model linking an organisation’s specific mandate and its expected results to the possible mixes of means (abilities, resources, authorities) that must be put into place to achieve those results. DCA is based on a logical development framework supported by a set of fundamental assumptions (possible scenarios concerning the level of available resources, the role of technology, in-house capabilities, etc.) related to the proactive management of the organisation’s performance through the identification of the capabilities required. DCA is also founded on a series of management and organisational concepts that provide a more refined understanding of the agendas surrounding capacity and capabilities development than merely improving transformational efficiency (inputs to outputs) or increasing the number of outputs.

The DCA has among others been tested in the road and telecom sectors in Ghana.[10] A methodology for DCA for Sector-Wide Development has been developed.[11]

A continuum of CD approaches

The OECD (2006) elaborated approach and PDIA (2012) may represent a continuum with the two approaches being the two extremes. The DCA and PDIA are having similarities but also differences. DCA requires reactions so it is in this way similar to PDIA, but DCA is also much more pre-planned and pre-determined than PDIA. DCA identifies objectives and is a model outlining expected results using log frames. It does not have the same emphasis on experimentation and learning as PDIA. The Danida 2011 CD Guidelines are similar in approach to the 2006 OECD approach. Other approaches have emerged, ex. “Capacity Development Practice Note” (UNDP 2008); “The Capacity Development Results Framework” (WB 2009); The “5Cs” (ECDPM 2011)[12]; etc.

The organisation’s capacity and capability

The main thrust of CD that is part of development assistance is focussed on an organisational entity such as an institution or an association that has a collective goal and is linked to an external environment; this could be termed as the ‘organisational ecosystem’[13] – the context of which is influenced by political, economic and social factors and the way they change over time. An organisation exists to generate outcomes and impacts. Assuming that a public sector organisation has an approved mandate and strategic performance targets, a selection among strategic and operational options will define the framework within which the outcomes would be generated. Thus, how much of those outcomes that should be generated within a given time or space, given assumed levels and means of production, defines the results (what and how much) that the organisation is expected to generate.

To design and manage a CD intervention, the desired end-state of the organisation (or group of organisations) should be defined, i.e. the warranted level of impact and the associated outcomes. That definition can be obtained through a variety of means including through participative and/or iterative processes, but it must eventually be clearly stated so that all organisational processes can be focussed on the achievement of that outcome. CD interventions should typically be defined at the outcome level because that is the highest level over which some degree of influence can be applied by the management responsible for the change process. In the same logic, capacity interventions should not be focussed on “part” of a process unless the rest of it is known to be as effective and efficient as it needs to be.

For a given organisational entity, its capacity is the resulting effect generated from the assemblage, orchestration, mobilisation and manifestation of its capabilities, (abilities, assets, and mandates). An organisation’s capacity is determined by its level of effectiveness; i.e. coverage, quantity and quality of services, products and regulations. CD interventions could be considered for the entire organisation, a division or a sub-unit. To achieve a higher level of performance, the organisation’s management must decide how it will organise (mix, acquire, etc.) organisational abilities (individual, managerial, strategic and operational), assets and other resources it may require.

The organisation’s mandate and entitlement may also need to be adjusted to be compatible with the warranted level of performance. When organised (i.e. designed, mixed, orchestrated, grouped, deployed, etc.) to enable a functional or socio-technical system to perform, the integrated groupings are called the capabilities of the organisation.


[5] The Paris Declaration on Aid Effectiveness, March 2005 has five principles related to: ownership, harmonisation, alignment, results, and mutual accountability.

[6] OECD. 2006. The Challenge of Capacity Development: Working Towards Good Practices. The three-level analysis was also presented in Guidelines for Sector Programme Support, Section 4.2 “Organisational and Institutional Analysis” (Danida, 1998).

[7] The following excerpts are derived from “Matt Andrews et al. 2012. Escaping Capability Traps through Problem-Driven Iterative Adaptation (PDIA)”.

[8] Matt Andrews. 2013. Explaining Positive Deviances in Public Sector Reforms in Development.

[9] The concept is defined by Teece et al., (1997), as the firm’s ability to integrate, build, and reconfigure internal and external competences to address rapidly changing environments.

[10] A version of the DCA model is used in the Danida-financed telecom development programme in Ghana.

[11] LeBlanc and Bealieu. 2013. “Dynamic Capacity Approach for Sector-Wide Development: Methodology Design for Management. of Capabilities’ Development. Case: Ghana Roads Transport Sector – A Pilot Application”, published on behalf of the European Commission.

[12] European Centre for Development Policy Management (ECDPM). 2011. Bringing the invisible into perspective: Reference document for using the five capabilities (5Cs) framework to plan, monitor and evaluate capacity and results of capacity development processes. The document builds on the ECDPM Discussion Paper. 2007. A balanced approach to monitoring and evaluating capacity and performance – A proposal for a framework.

[13] Business ecosystem is a corresponding term, originally used for the business sector but which also has a general application. The concept was conceived by J. F. Moore. 1993. Predators and Prey: A New Ecology of Competition. Harvard Business Review.

3 The evaluation sample

A brief presentation of the evaluation sample is made in this chapter. The selection of Danida interventions for the desk-based review implied that seven countries are represented in the Evaluation. Basic information on the seven countries is presented in Tables 4 and 5. For five of the countries, the Human Resources Development Index (HDI) scores remain relatively stable from 2005 to 2013, but Uganda and Vietnam experienced a drop in the score – Vietnam quite significantly. Whereas six countries saw an increase in their Corruption Perception Index (CPI) score from 2007 to 2014, only Uganda received a lower score and the ranking dropped from 111 to 140. The macro-level indicators provide information on the national level context, which are deciding factors for a country’s prospect for institutional and CD, and for having access to skilled manpower. As mentioned earlier, Nepal, Uganda and Tanzania were selected for more in-depth country studies.

Table 4: Human Development Index (HDI) and Corruption Perception Index (CPI)
Country HDI in 2005 HDI 2013 CPI 2007 CPI 2014
Score Rank Score Rank Score Rank Score Rank
Bangladesh 0.547 140 0.558 142 2.0 162 2.7 136
Indonesia 0.728 107 0.684 108 2.3 143 3.2 114
Mozambique 0.384 172 0.393 178 2.8 111 3.0 109
Nepal 0.534 142 0.540 145 2.5 131 3.1 116
Tanzania 0.467 159 0.488 159 3.2 94 3.3 111
Uganda 0.505 154 0.484 164 2.8 111 2.6 140
Vietnam 0.733 105 0.638 121 2.6 123 3.1 116

Source: HDI, UNDP Human Development Reports. CPI, Transparency International. Note: The 2005 HDI information was published in 2007 and HDI information in 2014.

Table 5: Poverty headcount ratio and GNI per capita
Country Income Category Poverty headcount ratio
at national poverty lines
(% of population)
GNI per capita,
Atlas method 2013
(Current USD)
Bangladesh Lower middle income/LDC 31.5 (2010) 1,010
Indonesia Lower middle income 11.3 (2014) 3,580
Mozambique Low Income/ LDC 54.7 (2009) 610
Nepal Low Income/ LDC 25.2 (2010) 730
Tanzania Low Income/ LDC 28.2 (2010) 860
Uganda Low Income/ LDC 19.5 (2012) 600
Vietnam Lower middle income 17.2 (2012) 1,740

Source: The World Bank.

Besides the 14 Danida interventions selected for the desk-based review, seven additional interventions have been included to expand the scope of for the country studies in Nepal, Uganda and Tanzania. Of these 21 Danida supported intervention, 16 are sector programmes that have three of more components – of which 10 cases have different executing agencies for the components, turning these into sub-programmes. Table 6 overleaf presents the entire sample by country indicating the sector (DAC sector classification) and implementation period. Table 7 outlines the cooperation framework as regards: cooperation with other donors; government level of implementation – central and/or local government; cooperation with one or more national executing agencies; engagement with civil society organisations; and engagement with the private sector. Further information on Danida supported interventions can be found in the Desk-based Review Report (Annex D) and the Country Study Reports (Annexes E, F and G).

Table 6: Evaluation sample of interventions supported by Danida
Country Programme/ project title Sector Year
Bangladesh Community Development Centre (CODEC) Government & civil society 2007-2012
Bangladesh Agriculture Sector Programme Support, Phase II (ASPS II) Agriculture & fishing 2006-2013
Indonesia Environmental Support Programme Phase 2 (ESP2) Environment 2008-2012
Mozambique General Budget Support (GBS) Programme General budget support 2010-2014
Mozambique Support to Public Sector Reform and Statistics, Phase II Government & civil society 2008-2012
Nepal Human Rights and Good Governance Program, Phase III (HRGG III) Government & civil society 2009-2013
Nepal School Sector Reform Plan (SSRP) Education 2009-2012
Nepal Energy Sector Assistance Programme, Phase 2 (ESAP II) Energy generation & supply 2007-2012
Nepal* Nepal Peace Trust Fund (NPTF) (UNDP/EU) Government & civil society 2010-2013
Nepal* Developing Capacities for Effective Aid Mgt. and Coordination (UNDP) Government & civil society 2010-2015
Nepal* PFM Support, Multi-Donor Trust Fund (MDTF) (WB) Government & civil society 2011-2016
Nepal* Revenue Administration Support Project (RAS) (GIZ) Government & civil society 2013-2015
Tanzania Health Sector Programme Support, Tanzania, Phase IV (HSPS IV) Health 2009-2014
Tanzania Business Sector Programme Support – Phase III (BSPS III) Business & other services 2008-2014
Tanzania* PFM Reform Programme, Phase II (PFMRP IV) Government & civil society 2012-2017
Uganda Anti-Corruption Programme, Phase II (ACP II) Government & civil society 2008-2011
Uganda U-Growth Programme (UGP) Agriculture and transport 2010-2013
Uganda* The Uganda Good Governance Programme (UGOGO) Government & civil society 2011-2016
Uganda* Joint Water & Environment Sector Support Programme (JWESSP) Water & sanitation 2013-2018
Vietnam Strengthening the Comprehensive Capacity of the Inspectorate System Government & civil society 2006-2014
Vietnam Climate Change Adaptation and Mitigation (CCAM) Climate 2009-2015

* Additional interventions included in the country studies, but not in the desk-based review

Table 7: Cooperation framework
Country Programme/ project title Donor
coop.
Government
level
National
partner
Civil
society
Private
sector
Bangladesh Community Development Centre (CODEC) MD L SA  
Bangladesh Agriculture Sector Programme Support, Phase II (ASPS II) MD N&L MA
Indonesia Environmental Support Programme Phase 2 (ESP2) MD N&L MA
Mozambique General Budget Support (GBS) Programme MD N SA    
Mozambique Support to Public Sector Reform and Statistics, Phase II MD N MA    
Nepal Human Rights and Good Governance Program, Phase III (HRGG III) MD N&L MA
Nepal School Sector Reform Plan (SSRP) MD N&L SA  
Nepal Energy Sector Assistance Programme, Phase 2 (ESAP II) MD N&L SA  
Nepal* Nepal Peace Trust Fund (NPTF) (UNDP/EU) MD N&L SA  
Nepal* Developing Capacities for Effective Aid Management and Coordination (UNDP) MD N&L SA    
Nepal* PFM Support, Multi-Donor Trust Fund (MDTF) (WB) MD N&L MA  
Nepal* Revenue Administration Support Project (RAS) (GIZ) MD N SA    
Tanzania Health Sector Programme Support, Tanzania, Phase IV (HSPS IV) MD N&L MA
Tanzania Business Sector Programme Support – Phase III (BSPS III) MD N&L MA  
Tanzania* PFM Reform Programme, Phase II (PFMRP IV) MD N&L SA    
Uganda Anti-Corruption Programme, Phase II (ACP II) MD N&L MA  
Uganda U-Growth Programme (UGP) MD N&L MA  
Uganda* The Uganda Good Governance Programme (UGOGO) MD N&L MA  
Uganda* Joint Water & Environment Sector Support Programme (JWESSP) MD N&L SA  
Vietnam Strengthening the Comprehensive Capacity of the Inspectorate System MD N&L MA    
Vietnam Climate Change Adaptation and Mitigation (CCAM) SD N&L MA

MD: Multi-donor. SD: Single donor. N: National level. L: Local level. MA: Multi-agency. SA: Single agency.

The sample of development interventions comprises: sector programmes with three or more components – most of which have had previous phases; and minor programmes (in terms of budget) with a previous phase. The sector programme components constitute two types of frameworks: a) one where the components are part of a coherent entity; and b) another where the components are separate entities related to each their set of recipient organisations. In the latter case, a sector programme will constitute two or more ‘sub-programmes’. Examples of the type of CD activities as perceived in the programme documents are provided in the boxes below for a subset of the evaluation sample.

Box 4: Nepal, Human Rights and Good Governance, Phase III (HRGG III) 2009-2013

Denmark has supported human rights and good governance in Nepal since the early 1990s. The HRGG III was designed to support initiatives by government and state institutions and civil society organisations in order to deepen democracy and contribute to realisation of human rights and effective, inclusive and accountable local governance. The development objective was: A functional and inclusive democracy based on respect for human rights established. HRGG had three inter-related components with each their executing agency: 1) Inclusive Democracy; 2) Human Rights and Justice; and 3) Local Governance.

Inclusive Democracy; capacity development activities include:

  • Preparation of policy and background papers.
  • Capacity development: media, young leaders.
  • Training for youth and leaders of disadvantaged groups.
  • Advocacy on youth involvement; and inclusion of concerns for women and marginalised groups.

Human Rights and Justice; capacity development activities include:

  • Training of staff on institutional development; and land rights activists, and community leaders.
  • Human rights education and awareness raising of communities.
  • Advocacy on human rights issues, legislation and policies; and media to inform the public debate.

Local Governance; capacity development activities include:

  • Policy framework for decentralisation.
  • Capacity development of: communities and marginalised groups to assert their rights; local governments to deliver basic services; and central government and national NGOs.

The role of technical advisers includes institutional development. The total budget is EUR 53.2 million. There is no specific CD budget for the two first components, but the Local Governance component has a CD budget of USD 6 million.

Box 5: Nepal, Energy Sector Assistance Programme, Phase II (ESAP II) 2007-2012

The first phase of ESAP commenced in 1999 with the objective of creating a sustainable rural renewable energy sector in Nepal within a timeframe of 20 years, which included support to the Alternative Energy Promotion Centre (AEPC), rural energy investments, and technological support. The second phase commenced in 2007 with the aim of providing access to clean, cheap and reliable energy in remote rural areas. The ESAP II had three components: 1) Institutional strengthening of the rural energy sector; 2) Rural energy investment/Rural Energy Fund; and 3) Technical support. The components are highly interrelated.

The capacity development activities include:

  • Review of policies and preparation of National Rural Energy Strategic Plan.
  • Strengthening of the coordination of rural energy stakeholders.
  • Improvement of the recipient organisations’ management practices through management training.
  • Conduct of capacity building of other partner institutions, energy cooperatives, and the financial sector.
  • Establishment of a network for knowledge sharing and coordination.
  • Facilitation of credit line availability.
  • Training of trainers for local partner institutions.
  • Awareness creation for media, key local players; and health personnel.
  • Development of IEC materials.

The role of technical advisers and consultants includes institutional development and training. There is no specific CD budget. The total budget is EUR 53.2 million, of which the Institutional Component is budgeted at EUR 4.4 million.

Box 6: Tanzania, Health Sector Programme Support, Phase IV (HSPS IV) 2009-2014

Denmark has supported the health sector in Tanzania for decades. HSPS IV (2009-14) is in line with the Third Health Sector Strategic Plan (Mainland) 2009-2014, the Second Zanzibar Health Sector Reform Strategic Plan 2006-2010, and the National Multi-Sectoral Strategic Framework (NMSF) for HIV/AIDS 2008-2012. The HSPS IV had three components (sub-programmes) with each their executive agency: 1) Support to the health sector in Tanzania Mainland; 2) Support to the health sector in Zanzibar; and 3) Support to the multi-sectoral response to HIV/AIDS.

Support to the health sector in Tanzania Mainland, capacity development activities include:

  • Funding support to health basket fund (HBS).
  • Management capacity, governance and systems at regional and district hospitals.
  • Systems and capacity for procurement, storage, delivery and use of drugs – and quality control.
  • Policies and regulations for and capacity of private health service providers.
  • Institutional arrangements and capacity building for public-private partnerships.

Support to the health sector in Zanzibar; capacity development activities include:

  • Funding support to district health services through the Health Service Fund (HSF).
  • Organisational development: mandates for ministry departments, organogram and job descriptions.
  • Development of health service standards and health information system.
  • Enabling district officers to integrate health in district planning.
  • Conduct of capacity building for financial officers, district health management teams, and medical cadres.
  • Advocacy towards national health priorities; and support to demand driven system of preventive maintenance.
  • Outsourcing of technical services and promotion of private-public partnerships.

Support to the multi-sectoral response to HIV/AIDS (Mainland), capacity development activities include:

  • Funding support to operationalisation of the NMSF for combat of HIV and AIDS.
  • Institutional and capacity development of TACAIDS for effective coordination of the NMSF implementation.
  • Implementation of a new organisational structure for TACAIDS.
  • Capacity development of TACAIDS at central and regional levels; and government institutions at district level.
  • Non-government sector supported to implement the NMSF at community level.
  • Multimedia production and dissemination of prevention material directed at the youth.

The role of technical advisers includes institutional development and training. All three components have specific CD budgets excluding technical assistance. The total budget is DKK 910 million of which DKK 155 million is allocated to CD. Other budget items also have CD elements, which are not specified.

Box 7: Tanzania, Business Sector Programme Support, Phase III (BSPS III) 2008-2014

The development objective was: Socially balanced economic growth through more coherent, transparent and sustainable institutions, including legal and regulatory environment that support development of the business sector in Tanzania. The BSPS III had three components with each their executive agencies: 1) Improved Business Environment comprising strengthening of business environment, advocacy, and enhanced capacity of the private labour market; 2) Better Access to Markets comprising international trade negotiations and trade business education; and 3) Development of micro, small and medium enterprises (MSMEs).

Improved Business Environment, capacity development activities include:

  • Reforms and regulations for business development.
  • Capacity development of private sector organisation to facilitate improvement of the business environment.
  • Capacity development of employer and labour organisations.

Better access to Markets, capacity development activities include:

  • Capacity building for international trade negotiations.
  • Education for promotion of trade and business.

Development of MSMEs, capacity development activities include:

  • Funding support for acceleration of investments and financing of commercial agriculture and agribusiness.
  • Capacity development of MSMEs for food processing and marketing.
  • Funding contribution to Financial Sector Deepening Trust.

The role of technical advisers includes institutional development and training. The total budget is DKK 500 million. The two first components have a CD budgets totalling DKK 102 million.

Box 8: Uganda, Anti-Corruption Programme, Phase II (ACP II) 2008-2011

The programme aimed at supporting the government efforts of strengthening the fight against corruption. The implementation was framed within Uganda’s Poverty Eradication Action Plan. The overall development objective was: to enhance accountability and reduce corruption in the administration of public duty and service delivery in Uganda. The immediate objective was: to build the capacity of anti-corruption agencies and non-state actors to effectively engage in fighting corruption. The ACP II had three components with each their executing agency/partner: 1) Support to the Inspectorate of Government; 2) Support the Directorate of Ethics and Integrity; and 3) Support to civil society and the private sector.

Capacity development activities include:

  • Capacity development for complaints processing.
  • Preparation and implementation of organisational development plan for Public Procurement and Disposal of Public Assets Authority (PPDA).
  • Capacity enhancement of CSOs.
  • Improved procedures and accountability of Poverty Action Funds.
  • Enhancement of media capacity.

The role of technical advisers includes institutional development and training. The total budget is DKK 31 million, of which DKK 6.6 million is allocated to CD activities.

Box 9: Uganda, U-Growth Programme (UGP), 2010-2013

The U-Growth Programme was aligned to the national policy framework for rural economic growth. This policy framework consisted of the National Development Plan, the Government's Manifesto entitled: “Prosperity for All”, and the Competitiveness and Investment Climate Strategy for Sustainable Maintenance of District, Urban and Community Access Roads. The development objective was: To continue building a self-sustaining export-led economy, in which all Ugandans share the benefits of growth. The UGP had three components with each their executing agency/partner: Public Sector Agricultural Support; 2) Rural Infrastructure; and 3) The Agribusiness Initiatives.

Public Sector Agricultural Support, capacity development activities include:

  • Institutional reform of the agricultural sector.
  • Capacity to implement the Development and Strategic Investment Plan (DSIP).

Rural Transport Infrastructure, capacity development activities include:

  • Enhancement of the Ministry’s capacity (Ministry of Works and Transport) to manage local government roads.
  • Development of and training in low-cost seals technology.
  • Enhancement of communities’ capacity to implement community access.
  • District staff trained in labour-based technologies for district roads rehabilitation and maintenance.
  • Refresher courses for labour-based contractors.
  • Revision of Mount Elgon Labour-based Training Centre’s (MELTC) curriculum.

The Agribusiness Initiatives, capacity development activities include:

  • Expansion of delivery mechanisms for financial services.
  • Expansion of financial institutions’ capacity to provide targeted financial products.
  • Sanitary and Phyto-Sanitary standards and management systems developed.
  • Awareness of complying with food safety standards.
  • National commodity organisations capacitated to host “centres of excellence” providing access to knowledge, data and expertise.
  • Support to the “Gender for Growth” (G4G) Fund.

The role of technical advisers includes institutional development and training. The total budget is DKK 530 million, of which DKK 61.7 million is allocated to CD activities.

4 Answering the evaluation questions

The 15 evaluation questions as elaborated in the ToR and adapted by the Evaluation are answered in this Chapter drawing on the evidence gathered from the desk-based review, consultations with stakeholders in Nepal, Uganda and Tanzania, the ‘focus group surveys’, and the various documents consulted. The evaluation questions and answers are arranged in accordance with the OECD/DAC evaluation criteria. The answers are stated right after the questions followed by the evidence gathered by main themes relating to the answers.

4.1 Relevance

Alignment to needs, contexts and external influence (EQ1, EQ2)

EQ1: To what extent have Danida’s CD-related interventions been designed to reflect external and internal contextual factors? (Including: institutional and socio-political dynamics; external influences such as demand and accountability; and complementarity support from other organisations)

EQ2: To what extent were Danida’s CD-related interventions designed to help a sector and/or an organisation to attain a targeted (higher) level of capacity?

Answer to EQ1: The Danida supported interventions were to a large extent designed to reflect external and internal contextual factors generally in the overall programme design, which implicitly included the CD interventions. Since 20 of the 21 interventions were multi-donor funded, Danida has demonstrated its interest and willingness to harmonise with other donors in pursuance of a ‘sector-wide approach’. Correspondingly, Danida has demonstrated its ability to promote the demand side and accountability through a high level of engagements with CSOs/NSAs (ref. country reports).

Answer to EQ2: The Danida supported interventions were to a large extent designed with the intent of helping a sector and its organisational ecosystem to achieve a higher level of capacity. Public organisations have in a number of interventions entered into partnership with the private sector and CSOs as a means of expanding the capacity for service delivery. The specific design for achieving CD results was not well articulated in the programme design, except for one case.

Alignment and ownership: The Danida supported interventions were generally adapted to the countries’ poverty eradication strategies and plans where such existed. Generally, great efforts were made to align with country systems and to promote government ownership at central as well as at local level. Most interventions included engagement with local government authorities (18 of 21), which in many cases were the direct service providers. Cooperation with CSOs was promoted to strengthen the demand side to facilitate that public services meet the needs of the direct beneficiaries.

Contextual factors: Most of the Danida interventions included in this Evaluation have had one or more previous phases (mostly five-year phases), implying that the context and how it had evolved should have been well known and that there were lessons learned that could have been fed into the formulation of the ‘new phase’. Strategies could accordingly be well adapted to the political, social and economic factors that prevailed at the time of formulating the interventions – including considerations on how the factors may evolve and the risks that may be encountered. The Danida supported interventions were well adapted to the external and internal contextual factors at the design stage, as presented in the DBR of 14 of the Danida supported interventions, see Table 8. The scores on the three external factors are satisfactory, whereas the adaptation to the recipient institutions’ capacity needs is comparatively slightly less.

Table 8: Adaptation to context – DBR sample of 14 interventions
Political and
Policy Context
Social and
Cultural Factors
Legal
Framework
Needs and capacity
of recipient institution
1. Very strong 1 1. Very strong 1 1. Very strong 1 1. Very strong 1
2. Strong 5 2. Strong 7 2. Strong 5 2. Strong 3
3. Medium 8 3. Medium 5 3. Medium 6 3. Medium 6
4. Weak 0 4. Weak 0 4. Weak 0 4. Weak 2
5. Very weak 0 5. Very weak 0 5. Very weak 0 5. Very weak 1
6. Not adapted 0 6. Not adapted 0 6. Not adapted 0 6. Not adapted 0
7. Insufficient data 0 7. Insufficient data 1 7. Insufficient data 2 7. Insufficient data 1

Note: Desk-based review – scorecard results (Variable 8). The scores are derived from the DBR Scorecard based on the Evaluation’s assessment.

The ‘sector-wide approach’ and harmonisation: Danida promoted the ‘sector-wide approach’ where relevant and cooperated with other donors engaged in the same sector with a view to addressing the sector’s needs comprehensively. A majority of the programmes (20 of 21) were planned and implemented in cooperation with bilateral/ multilateral donors. Danida was the lead donor in a number of programmes (11 of 21) and thus had comparatively more leverage in defining the scope of the support – including the TA and CD approaches. In some cases compromises were to be made as regards implementation strategies and the relative emphasis on development aspects including CD approaches. The enabling environment (policies, strategies, legislation, national plans, etc.) was in most cases external to the donor-supported programmes, and thus influence could not be directly exerted. In some cases though, Danida supported CSOs to work on the enabling environment (ex. government policies concerning accountability and justice in Uganda). Improvements of the enabling environment were seen by the donors as important contributions that needed to be brought about in order to achieve development outcomes. CSOs/NSAs were in many programmes (12 of 21) a contributing factor for downward and upward accountability and as a means of working more closely with targeted communities.

Organisational development: Although considerable emphasis was placed on CD/institutional strengthening in programme design, the relative emphasis of CD were on improvement of officials’ competence emphasising functional aspects and on the enabling environment (through internal as well as external national partners), and less so on organisational development of the supported agencies. In a few cases ‘strategic organisational plans’ were developed by the national partners with TA support, but were not implemented due to lack of resources (ex. ESAP and SSRP in Nepal). Although the intent in most programmes was to improve the supported organisations’ performance, hardly any of the programmes under evaluation conducted capacity baseline studies and capacity needs assessments (one exception is the water sector programme in Uganda, see Box 10). The capacity gaps and underlying organisational governance culture and context were accordingly not adequately analysed, which could have led to alternative considerations on the CD approach. The CD support mainly included systems development support, training courses, equipment, TA and short-term consultancies. In a few cases the governance culture of the targeted organisation was not conducive and the difficulty in bringing about change was grossly underestimated – resulting in limited or no CD (ex. Bangladesh ASPS II, see Box 12).

Cooperation between the public and private sectors was in a number of programmes (8 of 21) essential for delivery of results (ex. energy services, roads construction and maintenance, water supply, environmental management and health services). Correspondingly, cooperation with CSOs as regards to delivering of public services at community level was also applied to expand the capacity.

Box 10: Two extremes of national ownership

Uganda, Joint Water and Environment Sector Support Programme 2013-2018

The Ministry of Water and Environment is the first ministry in Uganda that redefined its understanding of capacity development from human resources management to a more holistic approach that addresses capacity development not only within the ministry but for ‘water and environment sector’ as a whole, as elaborated in the ministry’s Capacity Development Strategy. The Strategy emphasises a results-oriented, integrated capacity development approach starting with the analysis of performance gaps and priorities; and integrating activities on three levels: the individual level, the organisational level, and the level of the enabling environment. The programme is still under implementation, so lessons can only be partially learned. Danida provides funding to the Joint Partnership Fund (JPF) in the amount of EUR 58.5 million being one third of the JPF. Danida provides three technical advisers, one being an institutional specialist.

Bangladesh, Agriculture Sector Programme Support, Phase II 2006-2013

An ASPS II results framework was established, which was mainly concerned with quantitative aspects at the local level, but which did not include actions that could address the apparent capability gaps at the national level. The interventions were not designed in accordance with the executing agencies’ low level of capacity, as documented in review and progress reports. The programme did, however, reach its intended target groups through Farmer Field Schools and Farmer Clubs in which training and extension services were provided in cooperation with local governments by trainers and facilitators that were mobilised through CBOs. The continued involvement of CBOs would to some extent depend on financial and TA support provided through ASPS II. The case is an example on how the local government level continues to carry out the programme tasks, despite the lack of ownership at and support from the national level (positive deviance).

Danida’s perceived role in CD (EQ3)

EQ3: Based on their past experience with donors generally, to what extent do partner organisations and key intervention stakeholders want Danida to adjust its role (if at all) in capacity development?

Answer to EQ3: Partner organisations and key intervention stakeholders generally appreciated the role Danida played in CD, but expressed preference for Danida’s TA/CD support to complement their organisations’ capabilities in a way that supported the organisations’ staff in playing a key role in programme implementation through active participation in planning and executing of development tasks.

Partners’ overall perception of Danida: Generally, Danida is seen as a credible development partner by national partners. Danida’s comparative advantage can be ascribed to its: long-term commitment, high degree of flexibility, and high level of engagement. The general perception among partner organisations and key intervention stakeholders is that Danida/embassies and the TA advisers have played a constructive role in facilitating improvements of the enabling environment and in improving some aspects of organisational performance. The sentiment expressed by some staff and stakeholders was that TA should be tailored to fit the organisation’s capabilities and that advisers should act in an advisory capacity in accordance with the mentoring approach. Being confronted with the partial lack of CD achievements intended in the programme design, some interviewees acknowledged that a more rigorous results-based approach would have been the preferred modality. The outcome of the focus group discussions (in Nepal, Uganda and Tanzania) confirmed that the role Danida played corresponded satisfactorily to the country needs and that the thrust of the intended CD results as stated in programme documents was appropriate, but that implementation was lacking.

Role of embassies in facilitation of CD: The national partners see the Danish embassies as an important dialogue partner, especially as regards challenges related to the enabling environment. The embassy staff is capable of having a high-level dialogue with government counterparts; and is seen as providing valuable insights and suggestions to the national partners and their downstream stakeholders. The embassies are also an important link between the national partners and the Danida advisers in addressing complex institutional issues and in being able to suggest solutions to practical implementation problems. The embassies are seen to have competent senior programme managers; many have been with the embassy for many years – and thus constitute the institutional memory – and who know the intricacies of the political and social contexts well and are able to oversee programme implementation.

Role of TA in undertaking CD tasks: Formerly, Project Implementation Units (PIUs) were extensively used for programme management – sometimes with the result that the executing agencies’ roles and responsibilities were somewhat absorbed by the PIU. With the advent of the ‘ownership principle’, Danida abandoned the PIU modality after Denmark became signatory to the Paris Declaration in 2005 – either completely or replaced them with TA having a ‘partnership’ and ‘facilitation’ role. In cases where TA is overly effective there is a risk of restricting partners’ participation, which in turn may result in partner organisations’ internal resources are not adequately mobilised and endogenous CD processes not supported. Unfortunately, the Evaluation also identified that there is a perception among some partner agencies that one of the roles of the TA is to provide oversight on programmes and to generate accountability mechanism to Danida (i.e. reporting and management of activities).

Where TA was integrated into executing agencies, Danida advisers worked closely with senior management. The TA advisers often had multiple functions, including oversight/ project monitoring, facilitation and nudging, provision of advice and gap filling. Although all TA advisers have a capacity building role as specified in their ToR – in accordance with the TA Guidelines – the areas in which they worked were likely to result in some gap filling activities (ex. assist in preparation of annual work plans, and reporting and coordination tasks).

There are good examples of how TA transferred knowledge and skills, designed and implemented systems; but overall the TA did not produce the magnitude or the type of CD changes that were described in programming documents. Only a small part of that weakness can be directly attributed to TA advisers personally. Overall, the management of TA for CD purposes has not been satisfactorily undertaken for a number of reasons, (ex. either CD results were not adequately defined, or achievement of CD results were not adequately pursued, the willingness to change was not mobilised, etc.). There are cases where TA has been much appreciated, especially so where needed technological capabilities were developed that contributed to achievement of the planned results (ex. BSPS and TACAIDS in Tanzania, JWESSP Uganda, and ESP2 Indonesia).

Adequacy of and alignment with Danida’s guidance framework (EQ4)

EQ4: To what degree is Danida following its guidelines to CD support?

Answer to EQ4: Danida has adhered well to its programme management guidelines, among others in terms of sector-wide approaches and creation of partnerships based on the ownership principle. Danida has been less stringent in following its TA and CD guidelines, especially as regards ex-ante analyses of the recipient organisations’ capabilities and capacity and the specific interventions needed to address capability/capacity weaknesses.

Overall, Danida has adhered well to its guidelines for programme management in all of its interventions – including the ownership and partnership principles. Although reference was made to the CD guidelines in the programme management guidelines, these were not adequately applied during programme implementation – even when the intent of CD was expressed in programme documents. In some cases CD targets were set, which would have made it possible to pursue results-oriented CD, but limited ex-ante analysis were undertaken of the organisational, technical and human resource contexts prior to the launch of the programmes. The TA guidelines provide limited guidance on how to transfer knowledge and skills to counterparts. The Evaluation finds that there is a need for improving preparation of technical assistance inputs with a view to contributing to the partner institutions CD processes.

4.2 Efficiency/outputs

Danida’s capability to manage CD processes (EQ5)

EQ5: To what degree were the capabilities typically required to manage capacity development processes in place among the donor agencies and partner institutions? (Ex.: policies; procedures; guidance; skills and abilities; delegated authorities; change management plans and management systems; monitoring and other systems; incentives; accountability, and adequate time for key management resources).

Answer to EQ5: Danida’s capabilities to manage CD processes are in need of improvements. Reasons for this being a combination of CD not been given adequate attention in Danida at the corporate and decentralised levels and access to CD expertise is not made readily available to Danida programme managers. The recipient organisations’ managers – who are responsible for executing the programmes – are similarly not well equipped to manage CD processes and are constrained by not being delegated adequate authority to change directions by their superiors.

Level of complexity of CD tasks: The efforts required to manage CD processes depend on the complexity of the services, products and regulations to be provided in a given donor supported programme. If CD encompasses all three levels of the CD Results Framework (ref. Table 3) the complexity will be relatively higher than if CD relates to the individual level only – even though the individual level may be complex by itself depending on the specificity of themes, and number and categories of people to be trained. The Evaluation’s ToR lists six parameters as shown in Table 9.[14] The scorecard scores demonstrate that the programmes have been executed in highly complex contexts. In multi-donor programmes, the complexity is also affected if several donors have to agree on a joint approach to CD.

Table 9: Complexity of the capacity to be developed – DBR sample of 14 interventions
Specificity of products
and services
Scope/distribution
of capacities
Incentives to perform
(CD recipients)
1. High 8 1. High 5 1. High 5
2. Medium 6 2. Medium 8 2. Medium 6
3. Low 0 3. Low 0 3. Low 0
4. Insufficient data 0 4. Insufficient data 1 4. Insufficient data 3
Interest of elites Affinity with dominant
norms and values
 
1. High 3 1. High 4
2. Medium 4 2. Medium 2
3. Low 4 3. Low 2
4. Insufficient data 3 4. Insufficient data 6

Note: Desk-based review – scorecard results (Variable 12). The scores are derived from the DBR Scorecard based on the Evaluation’s assessment.

As regards Danida’s management of CD, it is important to distinguish between what is in place at the corporate level, i.e. what exists to enable capacity management to take place under a policy and management framework that applies to all of Danida, and what is in place so that the planning and execution of interventions can take place at the embassies – the decentralised level.

CD at corporate level: Danida has a comprehensive policy framework and communication protocols in place to ensure that its strategic directions are known. Compared to other donors[15], the Danida policy framework is more open and thus leaves room for flexibility. The corporate management framework allows for innovative approaches. Although conceptually clear, management topics in the guidelines are not always as clear as practitioners would like them to be (a finding based on interviews with Danish embassy staff, consultants and focus group participants), particularly on CD and its related issue of organisational development. Many topics would benefit not only from a more explicit implementation guidance but also access to good cases and to “experts” that could provide advice (on topics such as results management, result indicators, evaluation of CD, ex-ante evaluation of capacity related interventions, risk analysis, the use of logic models such as Theory of Change to name a few).

CD at the decentralised level: While many of the issues raised for the corporate level also apply to the decentralised level, there are a number of issues that need to be addressed. The embassies consulted agreed that there need to be a way of ensuring that programme managers (embassy programme officers and national civil servants) are qualified to oversee not only sector issues but also associated organisational development issues. Managers need to know more about how to manage CD in their sectors in accordance with the programme cycle and be aware of the complexities with which they are confronted – including how to manage CD to achieve outputs that would lead to the anticipated outcomes. Discussions with embassy officials also brought forth the difficulty they have in selecting the most appropriate capacity/capability/organisational development option due to a lack of intelligence gathering and analysis in the country or sector.

A number of embassy staff interviewed noted that they were not well acquainted with Danida’s guidelines on CD and that they had not been well equipped (through training or systems support) to manage CD. The processes in place relegated CD design to the sector specialists who may not have had adequate insight into CD and most often based human resource development on pedagogical strategies rather than on andragogical strategies.[16] Reviews and evaluations, they noted, focussed on inputs and outputs and less on the outcomes to which these would lead. Furthermore, embassy staff noted that they are not directly responsible for supervising the execution of CD tasks, which is the responsibility of the recipient organisation. A mechanism that encourages the introduction of capacity/capability development expertise into the embassies’ country operations could facilitate such oversight function. Such support should be directed to the embassies to facilitate their dialogue with the partner organisation.

A CD support facility: The mandate given to DFC[17] in the ‘Policy for DCDSP’ is compatible with the above observations on the need for CD expertise. Such support should not be considered as an administrative overhead, but as a legitimate content expense for ODA because its sole purpose is to support the achievement of programme results. DFC could – on a demand basis – facilitate the integration of CD interventions in the formulation of country programmes in Danida priority countries in cooperation with the Danish embassies and national partners. DFC is currently in dialogue on CD with a number of embassies (Kenya, Myanmar, Bangladesh and Pakistan) in accordance with DFC’s broadened mandate. DFC is also offering a number of thematic courses of relevance for Danida’s country programmes.[18] DFC has a network of management & public administration/training institutions in Kenya, Tanzania, Uganda, Mozambique, Ghana and Nepal that it can draw upon and broker partnerships in relation to training and CD consultancy services.

Organisational performance and accountability: In terms of accountability, the Evaluation found that even if reports and evaluations stated that efficiency and effectiveness of CD were wanting, the supported organisations were seldom held accountable. Except for the PSAS Component of the U-Growth Programme in Uganda (see Box 11) no intervention was stopped or significantly changed even if it was not likely to achieve its ultimate capacity objectives – one explanation could be that objectives were not explicitly defined. The ASPS II in Bangladesh (ref. Box 10) is another example where support to the national agencies continued despite a low level of performance and ownership. The two cases jointly with the support to the Inspectorate System in Vietnam represent the lower end of organisational performance in the evaluation sample.

Box 11: Public Sector Agricultural Support (PSAS) Component – low level of organisational performance

The PSAS Component of the U-Growth Programme 2010-2013 in Uganda was designed to help the Ministry of Agriculture, Animal Industry and Fisheries (MAAIF) to develop the strategic and operational capabilities it required to oversee the development and implementation of the government plans under its responsibility, including its policy frameworks, regulatory frameworks and on-the-ground service delivery. This was to be done through the PSAS objective: “to support MAAIF in its preparations to receive sector budget support and subsequently to support implementation of the Agricultural Sector Development Strategy Investment Plan and its associated bankable programmes”. The MAAIF has long had significant difficulties in generating the required capabilities to carry out its mandate: public services supporting agricultural development (research, extension/advisory, regulatory services and other domains) have been constrained with very low levels of capacity due to poor in-house abilities and skills, modest budget allocations and ineffective use of allocated resources due to serious governance and accountability problems, and political interference. Danida decided not to continue the support to PSAS for a number of reasons, one being the low level of performance.

Deviations from planned CD processes: The Evaluation discussed with embassy officials, the supported organisations’ managers, and focus group participants if the programme or component had ever defined an alternate plan if the CD process did not proceed as planned – typically called a “plan B”. Respondents noted that deviations from the agreed CD process were generally only brought to the attention of managers once the deviation was a significant problem and corrective action per se was no longer possible. Part of the problem was that the management and oversight functions were made “official and formal” requiring fixed and infrequent meeting times and the participation of people with heavy agendas. The level of authority delegated by the organisation’s executive officers to the management level in the supported organisations was relatively low, especially considering that the managers were almost always very senior officials. If capability and capacity development is to be dynamic – being able to respond to changing contexts – then means need to be found to unshackle these management issues.

Value for money (EQ6)

EQ6: To what extent do the results of CD interventions represent value for money, (i.e. the same results could have been obtained with a lower level of resources (investments) if a different CD strategy had been applied)?

Answer to EQ6: The fundamental challenge in answering this evaluation question has been how to deal with a concept (value for money) when the interventions did not link investments made to CD results sought. The Evaluation observed that most programmes/components did not have a separately budgeted CD element that was clearly linked to defined results.

The appropriateness of the selected CD strategies and options for achieving stated objectives: In every case studied, the problématique was well defined and the need for systemic outcomes was made clear. Some CD strategies were effective in achieving stated objectives (ex. institutional development at TACAIDS in Tanzania; technology development and testing in rural road surfacing in Uganda (see Box 12); higher-level education as a strategy for knowledge transfer for trade negotiations in Tanzania; external consultancies to develop results-based approaches in the water sector in Uganda; study tours to Denmark for a critical mass of judges in Uganda (see Box 13); and long-term training for critical mass of cadres of education administrators in Nepal). Other CD strategies were either not effective (ex. Public Sector Agricultural Support in Uganda) or there was limited specific finality against which to judge CD success (ex. local governance in Nepal, support to the Human Rights Commission in Nepal, health in Tanzania, etc.).

Box 12: Rural Transport Infrastructure (RTI) – improved rural accessibility through cost-effective CD

The RTI component (U-Growth) builds on the experience from Danida’s support to the road sector in Uganda since the mid 1990s. The aim was to support local governments in improving and maintaining district and community access roads through labour-based technologies and use of small-scale contractors. The Mount Elgon Labour-based Training Centre (MELTC) was established to train local government officers for road management and contractors for construction, rehabilitation and maintenance. Rural roads networks in good condition provide access to market and reduce travel time and costs. The main thrust has over the years been on gravel roads (including bridges and storm-water drains), but in recent years low-cost sealed roads has been introduced.

The RTI CD component was quite effective in reaching all (111 in 2014) districts through the MELTC training services and is seen as a cost-effective way of disseminating knowledge and skills development. The implementation benefitted from good leadership at the MELTC and a high level of motivation from its staff and partners. All of the CD outputs of the RTI component have been generated. Observation during field visits indicates that the capacity development outcomes are well on their way to being sustainable, with the proviso that continued financial and operations support will continue to be forthcoming from the Ministry of Works and Transport. The Uganda Road Fund (as in other African countries) was operationalised in 2010 to secure maintenance funding for the national and local road networks.

Box 13: ACP – critical mass of judges Uganda but uncertainty of institutional outcomes

Denmark has had a long-term strategic partnership with the Judiciary in Uganda dating back to 1989 with earmarked funding in specific areas. The support has among others been earmarked to introduce a performance enhancement mechanism and small claims procedure, which in turn has been supposed to lead to outcomes such as access to justice, improve observance of human rights and efficient use of resources. There has been a number of CD interventions such as training courses (including a training course in Denmark) and a TA adviser attached to the Judiciary. The Danish support has addressed with a relatively high degree of success a range of specific capabilities issues in the Judiciary. The ‘Small Claims Procedure’ has been credited with improvement of the case disposal rate because of its simplicity. Other Danida supported initiatives have contributed to enhancing the corporate values in the Judiciary in the direction of respect for human rights and improved accountability. However, the support has been extended without a thorough initial assessment of the institutional constraints and without an overall plan for how the institutional capabilities, and the interplay of these capabilities, would enhance the capacity of the institution to deliver. The programme reached many of its outputs because management was committed to change. The support has been sustained over long time, it has concurrently worked on many initiatives pointing in the same direction, it has seemingly trained a critical mass of judges, and it has contributed to changing rules and procedures. The support to the Judiciary, however, also illustrates the need for more consistent use of results-based management and results-based CD. The Danish support has been aligned to the Sector Investment Plan. However, the Judiciary Strategic Investment Plan does not include a results framework outlining targets and indicators, and CD outcomes, so it is not possible to determine whether the institution overall reached its intended outcomes and objectives and whether these are sustainable.

The Evaluation observed that Danida documentation was not complete on the reasons why selected CD strategies were chosen over others, or even which options had been selected. The CD solution presented in programme documents were rarely logically linked to the organisational outcomes except in a general way with major assumptions not analysed and risks not being spelled out. The management of CD initiatives was clearly underemphasised in most initiatives and supervision was clearly not taking place. In this way, Danida has taken risks with its CD initiatives – among others in terms of investments not leading to appropriate results.

Conversion of inputs to results: Most interventions budgeted the type of inputs that would be used (ex. short-term consultants, training) but not the results that the inputs would have on the organisational performance sought. Field and desk research has shown that the certain types of resources have generated the outputs planned, but which did not directly contribute to outcomes. Examples include some of the training on health in Zanzibar, anti-corruption in Uganda (ref. Box 13), and agriculture in Uganda. There are a number of reasons why managers involved did not take that aspect into consideration, including the sourcing of training not being specific enough to perceived requirements; short-term technical advisers not being tasked with CD assignments, or long-term technical advisers working on tactical solutions to process improvement rather than on strategic level systems improvement. Based on the above, and from a purely efficiency perspective, Danida has invested in activities that did not directly produce the higher level of performance.

The relative costs of inputs versus results achieved: Individual training is expensive, especially if travel is involved compared to collective in-country training. Long-term advisers are a big investment on Danida’s part and also on the part of the recipient organisations. Discussions with TA advisers (health sector in Tanzania, education in Nepal, water sector in Uganda) on the supply and demand for technical advisers demonstrated that experts were frequently underutilised and poorly positioned to be able to influence policy decisions generally and organisational strategies specifically. Substantial investment in individual training was made in connection with the PFM Reform Programme in Tanzania, the size and outcome of which were debated (see Box 14).

Box 14: PFM Support in Tanzania – trying to pursue a result-based approach to capacity development in an allowance driven context

Denmark has supported the Public Financial Management Reform Programme (PFMRP) since 1998. The PFMRP IV aims at enhancing revenue mobilisation, planning and budgeting, transparency, accountability, efficiency and effective use of resources and implementation through five core result areas of the PFM system; 1) revenue management and tax administration, 2) budget and planning, 3) budget execution, accountability and transparency, 4) budget control and oversight, 5) change management, monitoring and communication. The current CD activities include a significant number of short-term training events and workshops, among others 514 local government revenue officers, 300 national and local government officers on cash management, 300 internal auditors on auditing of development projects, etc. The work plan and budgets for PFMRP end up being composed of budgets with significant allocations for allowances. DPs have continuously expressed unease with such spending patterns – not least due to the general recognition of problems associated with allowance spending. During the previous phase of the PFMRP, disagreement on allowance spending made the programme almost to come to a halt and caused the World Bank to withdraw.

During the previous phase of the PFMRP, disagreement on allowance spending made the programme almost to come to a halt and caused the World Bank to withdraw. However, the Government and (other) development partners have since come together in a new phase of reform – not least because of positive developments in overall PFM capacities as measured through the regular PEFA scoring framework that indicated clear improvements at the level of PFM capacity outcomes up to 2013 compared to 2010. An advantage of the PEFA assessment is that it is considered an objective tool by both Government and development partners and therefore could be used to build trust between partners. Nevertheless, whereas it is clear that overall institutional PFM performance have improved over the years, then the specific contributions of the short-term training interventions and workshops to the degree of institutional performance have remained debatable. In recognition of the challenges related to specific CD intervention – in particular the short-term training, it has been decided to undertake a study of the PFMRP training.

The Evaluation did not identify any case where supervisors were trained first and then proceeded to upgrade the knowledge, skills and attitudes of their employees, a mechanism that is universal in its application and much less costly than individual development. Overall, the field research confirmed that Danida’s inputs into CD tended to be time and learner constrained in the sense that once the input had been provided (ex. training or short-term advice provided by external consultants), the learner stopped having access to the source of the knowledge and was not in a position to share the input gained with others through networking, supervision or peer-to-peer knowledge transfers. Implementing leveraging mechanisms would have added costs but those would in all likelihood have been marginal compared to the initial input. Further, the mechanisms could have been designed to leverage individual level efforts so that they were replicated at the organisational level. The cost-effectiveness of CD investments would have been greater if results were disseminated more widely and better absorbed in the organisation.

4.3 Effectiveness/outcomes

Outcomes of Danida’s interventions (EQ7, EQ8)

EQ7: To what degree have interventions achieved the planned results at outcomes level, and to what degree is there a correlation between the interventions, and observed capacity improvements of the partner institutions?

EQ8: What are the possible unintended effects (positive and negative) of support to CD?

Answer to EQ7: The Danida supported interventions have achieved planned sector-outcomes to a satisfactory degree as verified through progress monitoring. The achievement of sector-outcomes implies that there have been corresponding capacity improvements in partner organisations, whether these have been achieved through targeted CD interventions or not. The applied CD approaches were not geared to achieving higher levels CD outcomes.

Answer to EQ8: The Danida supported interventions have encountered both positive and negative unintended effects. Positive effects include: value sharing due to exposure to Danish administrative practices; and realisation of the private sector’s potential to contribute to social and economic development. Negative effects include: distortion of public service delivery due to donors’ focus on specific issues and creation of parallel structures.

Achievement of planned outcomes: While there was evidence to support the contention that Danida supported sector programmes have contributed to the successful achievement of sector-outcomes, there was very little similar documented evidence to determine whether CD objectives had been met. In fact, most programme documents did not identify what organisational outcomes the CD interventions (Danida’s, other donors’ and recipient organisations’ combined efforts) were supposed to achieve. That is not to imply that Danida did not contribute to improvements in the overall capacity of organisations; in fact, there are many examples where reviews and completion reports agree that increases in service levels have taken place (ex. ESAP II in Nepal; aBi component of the U-Growth in Uganda, and TACAIDS component of HSPS IV in Tanzania), but the extent to which the organisational outcomes – expected to be generated by CD – actually occurred is unknown. The extent to which Danida supported interventions have achieved planned sector-outcome is presented in Table 10. Overall, the achievement of sector-outcomes is satisfactory with 11 Danida supported interventions out of 14 having a score of either high or medium. The scores are the overall rating of the programmes and it is emphasised that these relate to the sector-outcomes and not to CD.

Table 10: Achievement of planned sector-outcomes – DBR sample of 14 interventions
Country Programme Year DBR Score
Bangladesh Community Development Centre, CODEC 2007-2012 High
Bangladesh Agriculture Sector Programme Support, Phase II 2006-2013 Medium
Indonesia Environmental Support Programme Phase 2 (ESP2) 2008-2012 Medium
Mozambique General Budget Support Programme 2010-2014 Medium
Mozambique Support to Public Sector Reform and Statistics 2008-2012 Medium
Nepal Human Rights and Good Governance Programme, Phase 3 2009-2013 Medium
Nepal School Sector Reform Plan (SSRP) 2009-2012 Medium
Nepal Energy Sector Assistance Programme (ESAP II) 2007-2012 High
Tanzania Health Sector Programme Support, Tanzania, Phase IV 2009-2014 Very low
Tanzania Business Sector Programme Support – Phase III 2008-2014 Low
Uganda Anti-Corruption Programme in Uganda 2007-2011 Medium
Uganda U-Growth Programme 2010-2013 Medium
Vietnam Strengthening the Comprehensive Capacity of the Inspectorate System 2006-2014 Medium
Vietnam Climate Change Adaptation and Mitigation 2009-2015 No/ Insufficient data

Note: DBR – scorecard results (Variable 17: Achievement of planned outcomes). The scores are derived from the DBR Scorecard based on the Evaluation’s assessment. Medium is the average score in the options provided, which has been translated into being “satisfactory”. The seven Danida supported interventions, which were not included in the DBR, achieved the planned outcomes to a satisfactory degree.

In programmes with ‘delinked’ components (ex. HSPS IV with three different executive agencies with each their mandate and target groups) there could be considerable difference in level of achievements between the components. Whereas the support to the Mainland and Zanzibar health ministries were less successful, the support to TACAIDS was successful in contributing to sector-outcomes (see Box 15). In Nepal the School Sector Reform succeeded in improving access to education with coordinated and interrelated interventions (see Box 16).

Box 15: TACAIDS, decrease of HIV prevalence in Tanzania

The main thrust of Danida’s support to the combat of HIV and AIDS in Tanzania (Component 3 of HSPS 4) has been on non-medical prevention and care with a view to addressing the imbalance of resource allocation in relation to medical care and treatment. The National Multi-Sectoral Grant was the main mechanism for channelling funds to local governments and non-government stakeholders for non-medical prevention and care interventions – enlisting their support and services. The main CD outputs were: a fund transfer system to local government, a communication and information system, and development of local capabilities. A comprehensive local government structure, systems and knowledge for managing the HIV and AIDS response has been developed for local government to enable them to target the specific contextual factors in their locality – as there is a great variation in the HIV prevalence across the country and in the causes for transmission of HIV. The overall HIV prevalence decreased from 7.0% in 2003/04 to 5.3% in 2011/2012 among the population between 15 and 49 years.

Source: Danida-TACAIDS. 2014. Programme Completion Report: HSPS IV Tanzania, Component 3.

Box 16: Improved access to education in Nepal

The SSRP adopted a sector wide approach including the entire organisational network from the Ministry of Education to the individual school with the aim of “Ensuring that by 2015 all children, particularly girls, children in difficult circumstances and those belonging to ethnic minorities, have access to an complete free and compulsory primary education of good quality”. During the past decade Nepal has made impressive gains in the schooling sector in terms of access, equity and completion. Primary education Net Enrolment Rate (NER) had reached 95%, Primary cycle completion rate stands at 77%; the proportion of out-of-school children has declined to 11%; gender parity in primary NER has already been achieved; and disparities in education access across income groups and ethnic/caste groups have decreased significantly (WB 2014). SSRP continued strengthening its decentralised management system – particularly at school and district levels. Significant gains have been made in terms of systems coverage, regularity and accessibility of information. The SSRP CD framework included all entities in the school system’s organisational set-up. The CD interventions targeted the stakeholders at entity level. During the early years of support to the education sector, long-term MA and PhD education was provided in Denmark and short-term teacher training at the National Centre for Education Development, which represented an upgrading of the capacity that proved to be useful in the later phases of support to the education sector.

Organisational performance and capacity: An organisation’s performance is linked to its organisational capacity and motivation, and the external environment. Organisational change occurs: a) in response to factors in the organisations’ external environment, b) because of changes in their internal resources (ex. financial, technological and human), and c) as a result of fundamental shifts in values within the organisation.[19] In those cases where sector programmes have generated sector-outcomes (ref. Table 8) with the support of Danida and other donors, the organisations’ capacity to deliver services, products and regulations have increased. The issue, however, is how much of that capacity increase has happened internally in the organisations and how much will be retained when the donor support ceases. The majority of the sector programmes in the evaluation sample that have performed well have generally had few technical advisers and donor funding was used for systems development, training and equipment. Most of the work that has led to increase of the organisational performance has been undertaken by the organisations’ own staff – and as such it is reasonable to assume that the organisations’ internal capacity has grown by some measure through endogenous processes whether planned or unplanned.

Applied CD approaches in partner organisations: The Evaluation examined the extent to which the CD actions that were undertaken actually responded to a perceived need for organisational change on the part of the recipients. In reality, the CD efforts during implementation were likely to be opportunistic (i.e. decided on the basis on what was available and where, rather than on what was needed). Examples of the foregoing include sending already-qualified Nepalese education officials to Denmark to examine the application of systems there, even though their supervisors already had examined the systems; or sending short-term health consultants to Tanzania for mandates they did without counterparts, direction or follow-up. These efforts were not change-oriented and most of the CD activities examined were not designed to fill a specific capacity gap.

The Evaluation noted that most of the CD activities and approaches that were applied were traditional in nature (ex. training courses, technical advisers, study tours), and were selected without a direct link to expectations of organisational performance improvement. Approaches such as mentoring, directed twinning, technology-assisted skills and knowledge acquisition, distance learning, directed research, partnering and other approaches were seldom used. The target organisations examined did not adopt a change strategy based on supervision and direct management. When discussed in the field, the target organisations noted that while senior managers invariably managed the time, input and resource allocation (including budget) aspects of CD initiatives, they rarely managed outputs or outcomes.

Correlation of CD parameters and sector-outcomes: The desk-based review indicates that there is a positive correlation between the degree to which interventions have been: aligned to findings and recommendations in preparatory studies, aligned to the recipient organisations’ institutional development strategy, and adapted to the recipient organisations’ needs and capacity – and the achievements of the planned sector-outcomes (reference is made to Table 12 in the DBR Report). Correspondingly, it is indicated that there is correlation between the degree to which sector-outputs have been delivered (related to systems, structures and individual knowledge and skills) and the level of achieving sector-outcomes – suggesting that the likelihood of achieving outcomes is higher when appropriate outputs have been produced (reference is made to Table 13 in the DBR Report). There is evidence, however, to show that just because a public sector organisation has generated outputs it does not necessarily mean that it has achieved the expected outcomes fully.

Unintended effects: With respect to unintended effects of support to CD, the Evaluation did not bring forward any review and progress reports or document that specifically included unintended effects. Respondents and observation identified a small number of non-planned positive effects including: a) a heightened sense of community practice and value sharing in education administration and management that has resulted from exposure to Danish education administration by senior Nepalese officials; b) a change in paradigm concerning justice administration in Uganda as a result of a critical mass of judges having had the opportunity to examine Denmark’s system in detail; c) a realisation that social and economic objectives could be leveraged through private sector engagement as in ESAP II in Nepal, AHFTA in Tanzania, and aBi in Uganda (see Box 17). Another example is RTI in Uganda (ref. Box 12).

Box 17: Examples of private sector contributions to social and economic objectives

Energy Sector Assistance Programme, Phase II (ESAP II), Nepal 2007-2012

ESAP worked directly with rural communities to provide a variety of energy services and financial support options. Financial support was provided through the Rural Energy Fund – through which subsidies were provided and on which the expansion of rural energy depended. The organisational set-up comprised district and village development committees mandated to improve energy services for their constituencies; NGOs involved with promotional and management activities; private companies engaged in supplying, installing and maintaining rural energy solutions; and financial institutions providing managing financial affairs and insurance. ESAP support reached more than one million households in remote areas – about one fifth of Nepal’s households.

Association of Private Health Facilities in Tanzania (AHFTA) – HSPS 4, Tanzania 2009-2014

AHFTA received support under Component 1 “Support to health sector Mainland” as part of Subcomponent 1.3 “Support to strengthening the non-government health sector and public private partnership (PPP)” that focused on institutional strengthening to improve the quality of the policy dialogue and on facilitating participation of the private sector in provision of public health goods of good quality. APHFTA were able to put into place a network of private facilities that work in tandem to provide basic services to the public.

The Agribusiness Initiative (aBi) – Component of the U-Growth Programme, Uganda 2010-2013

The aBi has shown itself to be highly efficient at generating jobs and at increasing the household and enterprise incomes. It builds capacity from the perspective of the private sector and uses economic and organisational argumentation in tandem with access to financing at acceptable rates to incite beneficiaries to invest in increased value-added production bases. There are two CD elements: the capacity of aBi to develop capacity in *others, and the capacity of the partners to provoke capacity changes all along the value-chain. Thousands may benefit directly from the interface of aBi and its Gender for Growth subcomponent, including women and men who are convinced that “working as a family” is not only going to help the status of women but also the well-being of their partners.

Negative unintended effects were also noted, including: a) a loss of focus on system-wide health service delivery caused by programmes that are mega-funded by large donors such as the Millennium Development Account, USAID and the Global Fund (AIDS and malaria) by setting-up parallel structures and non-use of country systems; and b) the perverse incentive of the per-diem practice in Tanzania. Managers in Tanzania have noted that they have discovered examples where individuals have steered donors into particular approaches based on training for themselves and their subordinates, or where there has been significant travel for participatory processes, so that they could benefit from allowances and per diem.[20] While per-diem and allowance practices may not have a direct effect on CD outcomes, they distract the focus from what is actually required and result in misuse of resources when the sole purpose of training is to have access to per-diems and allowances.

Experience with results-based approaches (EQ9)

EQ9: To what extent have results-based approaches contributed to learning on how to increase the effectiveness of CD support within and across interventions?

Answer to EQ9: Results-based CD approaches have only been applied to a limited extent in the Danida supported interventions and learning on how to increase CD effectiveness through results-orientation has thus not been possible to obtain. Some CD interventions that are not results-based are recognised as providing successful results.

The evaluation question specifically asked to conclude on the effect that results-based approaches may have had on capacity and capability development, specifically with an emphasis on the associated learning and its link to increased effectiveness. In essence, the field surveys, the focus group discussions, the desk-based reviews all point to the same finding: specific results related to CD were generally not clearly stated in CD inter ventions and programmes were not managed with a view to CD results.

Baseline studies and capacity need assessments (geared to address capability or capacity gaps) were not conducted with few exceptions, notably the PFM reforms in Nepal and Tanzania where the Public Expenditure and Financial Accountability (PEFA) standards were used as external norms and results used as a proxy indicator of capability. The results-based management (RBM) experience of the water sector in Uganda is interesting: after years of Danida support that had mixed levels of success, a few senior managers of the responsible ministry decided to develop a “Made-in-Uganda” approach to results-based management (see Box 18).

Box 18: JWESSP 2013-2018 – a planned CD approach for water and environment in Uganda

The Ministry of Water and Environment has developed a broad and deep approach to CD – the “Water and Environment Sector Capacity Development Strategy 2013-2018” – that encompasses and integrates the enabling environment, including the context, the political motivations and constraints and a number of other factors. The overall objective of this strategy is to put in place a more effective means of implementing capacity development activities in order to respond to the sector needs in a more harmonised and well-coordinated approach while taking into account the overriding influences and requirements for change in the existing enabling environment. The results-based management approach has been mainstreamed into all levels of governance responsible for water management including the national, regional, district, and community levels of government. The capacity development and organisational improvement efforts done within the sector now pass through the results-based system. National officials have created a detailed and comprehensive manual and the GIZ has helped to develop and test a “toolkit” and a “workbook” to accompany the guidelines. The water sector results model is based on capability development as a planning and supervision focus, with clear requirements for baselines, continuous monitoring and adjusting mechanisms. Being two years into the programme, it is still “work-in-progress”, so it is too early to predict the resulting CD outcomes.

The EASP II in Nepal integrated a basic results-based system into its capacity development thrust, largely because the private sector and the financial institutions that participated or supported this programme worked on that basis. ESAP management noted that while it found it a rather straightforward affair to manage the “sector-outcomes” on the basis of results, they found it more complicated to use results-based approaches for capacity and capability development, as CD results were not clearly specified. The aBi (U-Growth Uganda) oversight and management functions rely on a continuous feedback of performance and corresponding improvements if required, based on the existence of monitoring systems to provide support to decision-making. A key aBi board member noted, “You cannot improve if you can’t step back and compare the results you are getting with those you wanted”. Within the health sectors in Mainland Tanzania and Zanzibar, interviewees spoke of having done capacity gap analysis and defined expected results, but were unable to follow-up on the analysis for various reasons.

With the exception of JWESSP Uganda, results-based CD management was not institutionalised or mainstreamed in the Danida supported interventions. As a consequence of not having defined CD results, learning from its use was not possible. That being said, it is important to note that results-based management was not a policy requirement for programming at the time when the majority of the interventions that the evaluation examined were formulated – even though ‘managing for results’ was introduced with the Paris Declaration and adopted in Danida’s guidelines in 2005 (the Logical Framework Approach was introduced in 1996 – having a focus on outputs). Respondents noted that they would have difficulty identifying what should be capacity and capability results for their sectors, but that it would seem possible to develop them over time and eventually arrive at manageable results.

It should also be noted that some Danida capability and capacity development interventions are recognised as having been successful without RBM approaches being applied, especially at the individual level. For example, the CD support to the Ugandan judiciary generated significant changes without applying an RBM approach. The scope of CD in that intervention was basically geared towards the individual level with a focus on improved quality of services (ref. Box 13).

Strategies and conditions for achieving CD outcomes (EQ10, EQ11, EQ12)

EQ10: Assuming that Danida were to consider responding to a request for CD support, under which circumstances, for which aspects of capacity and for which specific inputs and modalities would its support be appropriate and effective?

EQ11: What are the CD lessons learned which Danida could use to move forward working with new actors in development?

EQ12: What characterizes support to capacity development that is relatively more successful versus strategies and interventions that are relatively less successful?

Answer to EQ10: Assuming that the request relates to a long-term engagement, Danida should address all three levels of the CD framework (the enabling environment, the organisational level and the individual level (ref. Table 1, p.25) and ascertain that there exist an acceptable level of commitment and ownership. While all three levels may warrant support, the thrust of the support should be on the organisational level with elaborated outcomes that are conceived to enhance organisational performance. This comprehensive approach may not always be possible due to a great measure of uncertainty and complications. In such cases the PDIA concept could with advantage be applied during the initial implementation phase – and further on until the uncertainty is at a controllable level.

Answer to EQ11: The main lesson learned is that the private sector and CSOs can act in complementary roles to public service providers: 1) the private sector could augment the public sector’s capacity and provide additional capital through public-private partnerships (ref. Box 17); and 2) CSOs could assist in improving the quality and coverage of public services through accountability measures with a view to obtaining an appropriate balance between demand and supply.

Answer to EQ12: Successful CD is characterised by applying a comprehensive approach that aim at attaining higher levels of organisational performance by: conceiving interrelated CD interventions that address the enabling environment, the organisational level and individual level; and taking external and internal contextual factors and their potential change over the planned time span into consideration.

Ideal circumstances for Danida’s support to CD: There are many ways to examine the question of the most appropriate circumstances of the CD support in which Danida should engage. Overall, the Evaluation has not found sectors or functions that are particularly risky or for that matter particularly likely to be successful. Rather, the findings support the research of a number of other studies[21] dealing with the same issue that stress the complex nature of change and the need for a well thought-out strategy for achieving the outcomes sought, a comprehensive and yet flexible plan to achieve the objectives, and means to manage the change process.[22] The Evaluation also shows that CD thrusts worked best when they built on demonstrated “commitment and ownership” by stakeholders (ex. water sector in Uganda and TACAIDS in Tanzania). When any of these factors are absent or inadequate, the CD was either not likely to succeed (ex. PSAS component of U-Growth Uganda) or was likely to remain at a “process” level without developing clear results (ex. Ministry of Industry and Trade and Ministry of Health in Tanzania).

The Evaluation findings suggest that in order to be able to best support its country partners, Danida should allow for a variety of approaches and modalities, the selection of which reflects the specificities of the CD objectives sought related to the conceived change process. The objectives (expected outcomes), the strategies for bringing about change, the capabilities that are to be put into place, and the management of the CD process (to name a few) need to be thoroughly understood and agreed upon by key stakeholders. There are examples where this has not taken place (ex. health in Zanzibar, support to labour organisations in Tanzania, and support to the coordinating body for anti-corruption and governance (Directorate of Ethics/Inter-Agency Forum) in Uganda).

Lessons learned on new actors: The first lesson is that in specific cases the private sector could be mobilised to deliver services more quickly and with more stability than the public sector, ex. ESAP II, Nepal. The public sector could thus change its mode of operation from “doing” to “regulating and ensuring compliance”. Moreover, the private sector can bring its own investment capital to the table if the negotiating positions require it and profit margins enable it. The second lesson learned for new actors is that the “supply and demand” model can be adapted to go beyond a concept of “service delivery equilibrium” on how to develop the capacity and capability of both civil society and the public sector within a dynamic and uncertain environment. The model can be developed to provide scenarios that enable donors, civil society and the public service providers alike to negotiate how their strategies could be changed to improve service delivery (see Box 19). In this way, Danida works with both sides of the equation, and can bring in any number of stakeholders for analysis, scenario development, or change management.

Box 19: Local Government and Community Development Programme, Nepal – demand and supply

The most significant achievements of LGCDP I have been the re-establishment of links between state and citizens in the aftermath of the conflict, the reinforcement of participatory planning process in the absence of elected officials and the creation of citizens’ institutions across the country. The programme has promoted inclusive and participatory development and built government partnerships with CSOs and NGOs for citizens’ empowerment. The Local Governance Accountability Facility (LGAF) was established in 2009 to develop capacity among the communities to assess and enhance the transparency and accountability of local bodies. LGAF facilitated citizens’ engagement through CSOs in strengthening the downward accountability of local bodies, e.g. procurement rules, fund disbursement, etc. The extent to which women, children and disadvantaged groups have been included in governance processes was greatly enhanced by the establishment of community institutions like Citizens Awareness Centres and Ward Citizen Forums (ref. Focussed Evaluation 2012). A Strategic Partnerships with 13 CSOs was entered into in 2000 – in a new modality of support to CSOs in Nepal facilitating development of strategic plans that included institutional and capacity development that were followed up by annual work plans and budgets.

Successful and non-successful strategies for effective CD: The answer to what could constitute the means for improving the effectiveness of outcome-related capacity and capability change has largely been provided in the paragraphs above. To migrate to a change-oriented strategy, support should be directed towards the organisation’s expected performance. Based on its observations, the Evaluation finds that successful strategies for enhancing effective CD should among others include those presented in Box 22. Managing these requirements on an ongoing would be a challenge for a majority of public sector managers. An important point to consider in this regard is that the Evaluation clearly showed that the majority of project managers focussed their attention on the provision of inputs and the internal management of human resources, rather than CD overall and on outputs and outcomes.[23] Non-successful strategies for capacity/ capability effectiveness noted in this Evaluation (i.e. those that did not result in change at the organisational level) are also presented in Box 20.

Box 20: Successful and non-successful strategies for effective capacity development

Successful strategies for effective CD:

  • An organisational strategy that defines the anticipated level of capacity and identifies the players, their roles, their required capabilities;
  • A good understanding of what capabilities are required and what are the present capability gaps – generated form performance analysis;
  • A clear picture of the CD outcomes sought in terms of their qualitative and quantitative performance metrics in order to overcome identified problems;
  • A roadmap of a longer or shorter duration of the logic behind the proposed change process;
  • A mechanism to ensure on-going ownership and commitment to proceed with the change, and the necessary influence and leadership to create a space for change;
  • A management team that is not only technically qualified but is provided with the systems, tools and resources required and the delegated authority to act;
  • Dialogue and ongoing intelligence gathering to act on the enabling environment.

Non-successful strategies for CD:

  • Tacking on a CD component to a sector programme as an afterthought rather than as a core issue;
  • Focussing on individual capabilities without these being related to the organisation’s overall capacity and performance, ex. ACP II Uganda;
  • Engaging in CD without a clear view of finality or outcomes;
  • Assuming that national project managers are able to manage complex CD processes without facilitation assistance and mentoring;
  • Engaging in overly complex multi-donor processes that do not rely on subsidiarity and thus dilute the focus for change and disperse the limited absorption capacity.

Managing for CD outcomes (EQ13, EQ14)

EQ13: Given last decade’s focus on results-based management, how could Danida work with clearer definitions and reporting on results in CD?

EQ14: Based on evidence from Danida’s experience in CD, what conceptual elements should be built into the generic Theory of Change in order to improve the effectiveness of future CD support?

Answer of EQ13: Danida could expand on the CD definitions related to the organisational level and develop a generic results typology that will facilitate: the design of CD interventions; specification of results; and monitoring and reporting procedures that accounts for results. More CD expertise should be mobilised to get CD processes and results right.

Answer of EQ14: The basic elements of a ToC model should among others include: assessment of internal and external contextual factors; increased focus on higher-level outcomes; be multi-dimensional; hypotheses explaining links in the change process as required; multiple feedback loops enabling adjustment of the ToC if so required in consultation with stakeholders; and risk assessment and mitigation measures.

Results-based management: The Paris Declaration on Aid Effectiveness 2005 states that the “capacity to plan, manage, implement, and account for results is critical for achieving development objectives”. It urges developing countries to make CD a key goal of their national development strategies. Donors generally recognise that capacity cannot be imported as a turnkey operation or as a best case that worked elsewhere. Instead, it is the organisations themselves that must change, with donors and their experts acting as catalysts, facilitators, and brokers of knowledge and techniques, including how to manage the change process itself. Despite widespread agreement on these general principles, the results of efforts to develop capacity have persistently fallen short of expectations.[24]

Achievement of results has been a focal theme since the mid 1990s in Danida’s development assistance, which was manifested with the adoption of the Logical Framework Approach (LFA).[25] The Danida CD Guidelines (ROACH 2005) introduced “results-oriented approach to CD and change”. The results-oriented approach/results-based management was reiterated in the 2006 and 2011 CD guidelines. In 2011, Danida published a Framework for Managing for Development Results that is based on the OECD standard Managing for Development Results (MfDR, OECD Policy Brief 2009).[26] Correspondingly, ‘management for results’ was stated in the 2003-2009 programme management guidelines and was further articulated in the 2011-updated version.

This implies that focusing on results generally and CD specifically has been the intent of all Danida supported programmes included in the evaluation sample during formulation as well as implementation. The CD in the 2005 and 2006 guidelines relate to the organisation and generation of outputs, whereas the 2011 guidelines are broader by also relating to people and society as a whole (see Box 21), and ‘outcomes’ are included as part of the results chain. The focus on the organisation in the definitions is interesting, as CD interventions at the organisational level have not been emphasised to any great extent in the programmes included in the evaluation sample.

Box 21: CD definitions in Danida guidelines

ROACH 2005: Capacity is defined as the ability of an organisation to produce appropriate outputs.

Guidance Note on CD 2006: An organisation’s capacity is defined as the ability of the organisation to perform appropriate functions efficiently, effectively and sustainably in pursuit of organisational goals and outputs.

Addressing CD in Danish Development Cooperation 2011: The Guidelines refer to the OECD/DAC definition for capacity development, which is the ability of people, organisations and society as a whole to manage their affairs successfully (ref. Box 2 p.9)

Except for the Ugandan water sector programme (still under implementation until 2018), the Evaluation did not find any examples of “best cases” for RBM-based reporting. This is somewhat surprising, as the intent has been clearly stated in the CD guidelines – and also in the light of the overall emphasis on achievement of results in the programme management guidelines. Developing RBM-based analyses and reports, especially on those parts that refer to the reasons why progress has not been according to plan and those parts proposing modifications, need time and expertise to generate. Oversight functions need to be oriented less on resource re-allocation and more on outcome management than has been the case. The obvious difficulty for Danida and other donors of achieving of and reporting on CD results, demonstrate the inherent difficulties in launching CD and reform processes. While it has been relatively easy to apply RBM on tangible sector-outcomes and record these, it needs to be understood what the causes are for not addressing CD effectively – despite definitions and methodologies being in place.

Theory of Change: The Evaluation has shown that the focus of capacity/capability development needs to be at the organisational level, with all activities geared towards the achievement of a set of clear outcomes. Whatever logic model of change (ex. Theory of Change) is used it must incorporate the means to manage the process of change, not only the inputs and outputs but also outcomes, and how change is brought about to come from outputs to outcomes. While this may seem obvious, the experience of many donors shows that the ability of recipients and donors alike to manage this type of change is limited.[27]

The Evaluation notes that while all programmes had developed a logical diagram, most were at a relatively high abstraction level with few CD results or targets actually integrated into the documents. Much of the specificity lies in the inception reports or equivalent documents prepared by the executing bodies, but these were often not updated and did not serve as management tools per se. While respondents noted that Danida had shown certain flexibility in the execution of its programmes, managers interviewed tended to regard the documents as “work breakdown structures” that defined the levels of resources that could be applied, rather than a tool to: a) define and keep track of progress in the achievement of higher level results; b) identify and keep track of risks and critical assumptions; c) identify the relationship between inputs and expected results: outputs and outcomes; and d) help strategize on possible options and scenarios as the programme progress over time. The findings correspond well to the observation in the 2011 CD Guideline concerning the “missing middle” indicating a lack of specification of CD intermediary results. The degree to which a set of parameters has influenced the CD process of the 14 Danida supported interventions included in the DBR is shown in Table 11.

Table 11: Parameters influencing the CD processes – elements of a CD Theory of Change – DBR sample
Adaptation and flexibility
(to internal and external factors)
Concurrence of CD
and other factors
Demand from
recipient organisations
1. High 4 1. High 0 1. High 9
2. Medium 6 2. Medium 9 2. Medium 5
3. Low 1 3. Low 0 3. Low 0
4. Insufficient data 3 4. Insufficient data 5 4. Insufficient data 0
Focus on achieving
measurable results
Measurability of results
(capacity and performance)
 
1. High 1 1. High 0
2. Medium 11 2. Medium 5
3. Low 2 3. Low 9
4. Insufficient data 0 4. Insufficient data 0

Note: Desk-based review – scorecard results (Variable 13). The scores are derived from the DBR Scorecard based on the Evaluation’s assessment of available documentation.

The parameters[28] constitute essential elements of a CD Theory of Change model. The scores demonstrate: a) a high demand for CD from recipient organisations and a high emphasis on endogenous processes; b) satisfactory adaptation to contextual factors, stated flexibility to absorb contextual changes, and concurrence with external factors of importance for the CD process; and c) a satisfactory focus on achieving measurable results but a relatively low measurability of results. The parameters indicate that the potential for achieving CD results exist. Based on its observations and recent thinking, the Evaluation finds that a ToC model should among others include the elements shown in Box 22 (these elements are complementary to those mentioned for successful strategies (ref. EQ12).

Box 22: Elements to be included in a ToC model

  • Be based on assessments of internal and external contextual factors and how they evolve over time;
  • Be built on the basis of the results that need to be in place so that a higher-level or subsequent result can be achieved;
  • Be multi-dimensional when more than one organisation is engaged in the change process and reflect the institutional interactions in the organisational ecosystem;
  • Be based on hypotheses to explain the transition from outputs to outcomes and to impacts and take envisaged constraints into consideration;
  • Include multiple feedback loops that provide information on progress of achieving results as well as on the evolution of the enabling environment that would provide the basis for dialogue with decision-makers and stakeholders;
  • Incorporate a risk layer and associated mitigation measures;
  • Be designed so that inputs are allocated throughout the process; in fact, different inputs are required at various stages of the implementation and the lead-up to those inputs would more than likely require involvement of management;
  • Indicate not only how the process of change will be undertaken and managed, but also how the client environment will be prepared to deal with revised scenarios for reaching the outcomes and possibly revisions of the outcomes if they are not likely to be reached.

The above-mentioned elements are also reflected upon in the 2011 CD Guidelines. The application of the ToC models is now a formal Danida requirement for programme formulation and implementation, and as such the models and their use will be further developed as experience is gained.

4.4 Impact

EQ15: Are the current CD approaches an effective way to reach the poor (directly and/or indirectly)?

Answer to EQ15: The programmes in the evaluation sample do reach the poor, but CD approaches could be made more effective in reaching the poor in a number of ways: 1) through increased attention to organisational performance with a view to advancing effects that improve public service delivery of relevance for poor people; 2) public service could be delivered more effectively through partnerships with the private sector and civil society as appropriate; 3) the demand side could be enhanced through transparency and accountability measures that emphasise inclusive growth by targeting poor and disadvantaged groups; and 4) application of a simplified form of impact evaluations that could provide feedback during implementation to ascertain the degree to which the intended effects are achieved.

Equitable service delivery: The Danida CD interventions have aimed at reaching the poor through improvement of: 1) the enabling environment; 2) organisational performance; and 3) staff competence. Ideally, the three levels of CD should be mutually supportive to reach the optimal level of effectiveness. There is also a time aspect related to the three levels: improvement of the enabling environment is a long-term endeavour if new policies and legislation are required, whereas organisational improvements can be achieved relatively faster – depending on the complexity of the organisational ecosystem with many organisations as opposed to a single organisation. Enhancing staff’s competence can be done within a relatively short period, but will only be effective if the organisation accommodates and makes use of its staff’s increased competence. The change of national systems to provide equitable service to the population at large including poor people – as opposed to targeting selected communities in a specific geographical area – may take as long as 5-10 years or more, provided that there is a political will, ownership, readiness to change, and that needed financial and human resources can be made available as required.

Generally, Danida’s support has had a sector focus with explicit objectives of reaching the poor based on long-term engagements – and frequently in cooperation with other donors in order to achieve an appropriate magnitude of the support. In most cases there has not been an explicitly elaborated CD strategy, but nonetheless capacity has been implicitly developed through endogenous processes combined with various elements of CD related donor support. When tangible sector-outcomes that result in provision of services to which the poor has access (ref. Section 4.3), it is probable that these have contributed to poverty reduction. In countries and sectors where the enabling environment is relatively conducive with some measure of predictability the long-term and poverty oriented approach will be an effective way of reaching the poor, but could become even more effective if CD strategies are well articulated and implemented. In countries and sectors where the environment is not enabling, targeted short-term CD approaches may be more effective – and could be replaced with a longer-term engagement, if positive changes in the enabling environment are likely to occur and willingness to change has been demonstrated.

Reaching the poor: The Danida interventions included in the evaluation sample can be divided into three main categories: 1) overall governance, such as public sector reforms, public financial management, human rights and good governance, and anticorruption; 2) public services such as education and energy in Nepal, health in Tanzania, agriculture water and transport in Uganda; and 3) climate change and environment. The first category may provide benefits to poor people, but tangible benefits would take a long time to materialise. The public service programmes can reach poor people directly through improvement of the enabling environment and provision of direct services within the short- to the medium-term. Climate change and environmental management interventions will have both long- and short-term effects that affect poor people directly and indirectly.

Danida’s public service programmes generally include interventions that address the national and local government as well as communities. The support to local governments and communities for improved public service delivery (ex. health, education, employment, water, energy, transport, etc.) will generally reach poor people directly in those areas targeted for support. An imperative condition for providing stable public services to all segments of society including the poor is to generate sufficient domestic revenues on a predictable basis and having transparent and accountable PFM systems.

Two interventions supported by Danida succeeded in improving PFM in Nepal significantly and in increasing domestic revenues, which could lead to a more equitable distribution of public services (see Box 23).

Box 23: Improving PFM in Nepal leading to increase of domestic revenues

The Nepal Public Financial Management Support Multi-Donor Trust Fund (2011-2016) – WB is the lead donor: The most significant result was the introduction of the Treasury Single Account (TSA). The implementation started in 2009 and was completed in 2013 – resulting in 14,000 accounts of 4,500 spending units being reduced to 445 TSAs. The TSA is based on real time budget execution with reports being published daily for all public expenditures in the Nepal’s 75 districts. It is an online-based system linking the District Treasury Officers (DTCO) to the Financial Comptroller General’s Office where the central database is located with backup in the Government Integrated Data Centre. A Revenue Management Information System is piloted in Kathmandu Valley. The TSA has resulted in: ex-ante control of budgets increased; end of idle cash in the spending units; daily reconciliation of transactions with banks; and use of overdraft ceased. Overall this has resulted in increased confidence and credibility of the PFM. Extensive training has taken place of DTCOs, and regional and central level officers. Some 30 CSOs have been trained in overseeing local bodies’ budgets and social expenditures to enhance social accountability. Journalists have also been trained to enable them to report objectively on PFM matters. The PFM system has been immensely improved resulting in more effective and transparent transaction and increased upward and downward accountability.

The Revenue Administration Support (2013-2015) is supported with TA assistance from GIZ. In 2012, the Cabinet adopted the first strategic plan for the Inland Revenue Department (IRD) for the period 2012/13 to 2016/17, which set the framework for long-term decisions. Based on the strategic plan, the IRD drew up a reform plan for 2012/13 to 2014/15 to prioritise what actions were critical for the strategic plan including a training plan. IRD led the CD process, which created greater awareness among officials, motivated them, and generated a demand for more monitoring because staff knew they would be held accountable. CD was therefore demand-based. Leadership came from well-trained and technically qualified managers. IRD recognises that it is important to train its managers in how to operate as managers and to manage capacity. Substantial emphasis has been placed on information and communications technology. IRD has trained the private sector, taxpayers, and ministries to ensure compliance with and acceptance of the tax revenue system’s procedures. The improvement of the tax system has resulted in efficient tax collection and a steady increase of tax revenues. IRD has received years of donor support (WB, GIZ, Danida), mostly on technical training and systems development. The fact that donors have been involved for the long-term and letting IRD take the lead has enabled the development of a close partnership and has reduced risks.

Note: Danida provided its support through the Local Grant Authority. The overall purpose of the Local Grant Authority (LGA) is to provide a flexible and non-bureaucratic instrument, allowing for prompt and relevant response to development requirements by the Danish embassies in eligible countries, thereby contributing to better quality of Danish development cooperation. The embassies on which the LGA has been conferred have at their disposal a separate budget for projects and/or local consultancy services of up to DKK 5.0 million per project/activity without prior referral to the Ministry of Foreign Affairs.

4.5 Sustainability

The Theory of Change model does not include the ‘sustainability criteria’, but sustainability is the ‘raison d’être’ of CD. The thrust of sustainability is on maintaining the benefits derived from outcomes and impacts achieved during the programme period. The prospect of long-term ex-post impact from programme interventions depends to a large extent on the measures put in place to promote sustainability. The key elements for promoting sustainability of those services, products and regulations that have been established, improved, or expanded to cover a larger part of the population are:

  • Ownership has to a reasonable extent been created at all government levels engaged in supplying the public services;
     
  • The citizens’ demand for the public services is high and these are valued;
     
  • There are systems in place to ensure that the public services are efficiently and effectively delivered;
     
  • Capacity and capabilities have been adequately developed in the organisational ecosystem in order to cope with the established standards for service delivery;
     
  • There is a reliable flow of recurrent and development funds to finance the delivery of services.

One way of dealing with the sustainability aspects would be to incorporate these in the programme’s ‘exit strategy’ – assuming that there will not be a next phase – which should aim at maintaining the organisations’ achieved capacity and ideally further increase this in the ex-post period. This would require that preparation of the exit strategy is a well-defined task in the programme’s work plan. Generally, programme documents do not contain an exit strategy – although the 2009 and 2011 programme guidelines state that an exit strategy should be prepared when a programme reach the completion stage, or at an earlier stage. For learning purposes, it could be considered to undertake capacity and capability assessments of selected programmes two-three years after programme completion.

The way Danida and its TA (and other donors) supports reformers in driving the change process successfully through CD and in meeting the demand for public services in an accountable and transparent way may influence ownership positively – and thus increase the probability for sustainability. This being said, the reform process will also encounter constraints, as there will be opposition and hesitance to change for a number of reasons. A complementary role of the donors will be to facilitate coalitions among reformers and promote consistency and coherence in the support provided with a view to maintaining the space for reform, ex. the “Big Reforms Now” initiative in Tanzania that aims to support country-driven reforms within eight priority areas (including health and business).


[14] The selected parameters/factors are taken from the Approach Paper, Section 7 “Unfolding the ‘best fit’ and legitimacy focus areas: complexity of capacity adequacy of CD processes and donor responsiveness”.

[15] Based on the experience of the evaluators, the document authored by Paul Engel entitled “ Mechanisms that Promote Policy Coherence for Development” published by Askant Publishing, 2007, and the work of the European Centre for Development Policy Management on EU-Member States policy coherency as a backdrop.

[16] Ref. Malcolm S. Knowles. 1980. The modern Practice of Adult Education – From Pedagogy to Andragogy. Published by Cambridge, The Adult Education Company.

[17] DCF is a self-governing institution under Danida engaged in capacity development activities and development research.

[18] Previously, Danida’s Centre for Competence Development (DCCD) provided training for staff of the Ministry of Foreign Affairs. The DCCD was closed in 2007 and activities transferred to ministry’s Competence Centre. In 2011, the training activities were integrated in the ministry’s Human Resources Department.

[19] Universalia and IDRC. The Institutional and Organisational Assessment Model (OPA) – A comprehensive framework for Organisational Performance Assessment.

[20] Norad has studied this phenomenon in Tanzania: “Hunting for Per Diems….” March 2012. This document is discussed in the Tanzania Country Report. The per diem issue applies across donors, including Danida.

[21] Refer to the World Bank Institute’s comprehensive research library, the publications of Sida and the EU on capacity development.

[22] Refer to Danida’s 2011 Guidelines on CD and the World Bank Institute “Capacity Development Results Framework – A strategic and results-oriented approach to learning for capacity development”, 2009.

[23] This finding is based on the discussions with project managers and the reporting mechanisms used by them or Danida.

[24] Refer to a) OECD, “Survey on Monitoring the Paris Declaration – Overview of the results”, 2006, Paris; b) Independent Evaluation Group, World Bank “ Evaluation of World Bank Support for Capacity Development”, Washington, DC, 2007; c) World Bank Institute. “Developing Capacities in Countries: and WBI Annual Report 2012”.

[25] Danida. 1996. Logical Framework Approach: A Flexible Tool for Participatory Development.

[26] Danida. 2011. Danish Development Cooperation in a Results Perspective: Danida’s Framework for Managing for Development Results 2011-2014.

[27] Refer to “Danida. 2011. Addressing Capacity Development in Danish Development Cooperation” and literature reviews of Sida, WBI and EU for further details.

[28] The parameters in Table 11 are derived from the Approach Paper, Section 6: “A Theory of Change for Capacity Development” in which it is noted, “there is a broad consensus about the parameters that influence effectiveness and efficiency of CD processes”. The parameters constitute essential elements of a Theory of Change.

5 Hypotheses

The ToR/Scope of Services for the Evaluation defined four focus areas, which are translated into four corresponding “hypotheses”. These hypotheses were subjected to testing by the Evaluation through its evaluation approach, methodology and field evidence gathering. The following analysis briefly summarises the key conclusions that were made concerning the hypotheses. A more detailed analysis of the hypotheses is found in Annex J.

Hypothesis 1: Donor support to CD is (more) effective when it fits the drivers for and constraints to change

The analyses support this hypothesis by confirming that: the more an intervention is specifically designed to reflect the specific contexts and other realities of the problem it purports to resolve, and the more it is designed to deal with the drivers for, and constraints to, the achievement of its objectives, the greater are its chances of success.

In many ways Hypothesis 1 is not a hypothesis at all, but a statement of predictable outcomes based on a comprehensive praxis. Field experience by both the evaluators and the stakeholders met shows that it is not only very difficult to conceive of what would be a “best fit”, but it is a problem to agree on what the concept of “best” would represent in any particular context, so it has not been the experience of donors or recipients to accurately define what the “best fit” would be. The Evaluation nevertheless found that Danida has attempted to design its CD support in a way that tended to leverage the drivers (i.e. leading towards a “best fit”) for change, as Danida knew them. But there are many contributing factors that need to be taken into account.

Hypothesis 2: Donor support to CD is (more) effective when donors engage in dimensions of CD where external agencies are likely to be able to contribute (not too complex for outsider facilitation) and when donor involvement is found appropriate and legitimate

The Evaluation’s findings support this hypothesis by confirming that: donor engagement in CD is more likely to be effective when it contributes to warranted development outcomes and when the engagement is appropriate and legitimate.

The evidence from all three case countries suggests that donors are not restricted by complexity, or any other “systemic” factor from contributing to the resolution of a national constraint to development unless there is a political, policy or strategic reason (on the part of either the recipient or the donor) to not engage. The evidence also leads to the finding that it is not complexity that restricts the desire to contribute. Strictly speaking, there is really no such thing as a “lack of legitimacy”[29] in a bilateral relationship. Bilateral donors, and Denmark specifically, operate in “priority countries” under umbrella programme agreements of all types; they are therefore “legitimate”.

“Credibility” is another issue, and is reflected in the mutual trust between the donor and the national partners. Further, discussions with stakeholders in the three countries and results of the survey conducted suggests that the conditions under which a donor should engage are solely a factor of the extent to which it feels it can influence decisions that will be needed to generate expected outcomes and impacts (a perception of its credibility), the risks involved, and the expected value to both parties. That being said, the Evaluation finds that the hypothesis is more like conditional statement of causality. After all, in the absence of legitimacy and a meaningful expectation of being able to influence the achievement of successful results through the application of “appropriate” involvement, why should a donor contribute at all?

Interviews and survey results point to the finding that in the specific cases of Tanzania, Uganda and Nepal, Danida has been a trusted partner for decades in the sectors examined and its behaviour and commitment to CD corresponds to the perceived needs of its partner (as defined by the partner) as well as to its values, norms and sense of identity (to name a few). Evaluation results also clearly point to the fact that respondents are aware of, and appreciate the fact that Danida has been their “trusted partner” for many years and has let its support evolve with the changing conditions of the country and the sector organisations.

Hypothesis 3: Donor support to CD is (more) effective when one looks beyond the “supply-side” or “push” approaches that only work from the inside in public organisations, aiming also to foster broader accountability relations (the issue of so-called “supply and demand”)

The Evaluation’s findings support this hypothesis by confirming that: the Danida supported interventions have included support to the demand side through CD of external stakeholders dealing with oversight, accountability, good governance and transparency, but that the effect on public service delivery has not been significant – one reason being that the space for external stakeholders’ influence has been controlled by agents with power.

There is little direct evidence to show a causal link between Danida’s support to a specific “demand”, and the magnitude of any “supply” change. Although the Evaluation was able to identify instances where Danida support to NSAs resulted in a heightened ability to engage in advocacy, it was not able to qualify the influence of that increase in advocacy capacity (nor has any other evaluation or research that could be found). Nor did projects examined have any observable definitions (or targets) of what constituted the “ability to engage in advocacy”, or “strengthened” organisational ability.

Managing both sides of the supply-demand equilibrium would require capacity to be developed within both parties, and not only the capacity of the “demand”. The Evaluation found that referring to a “supply and demand” analogy, while simple and seemingly universal as a concept, likely leads to reductionism: the relationship at any time between the State and the People is a multi-layer and complex, and is never fixed, unlike the supply and demand intersect that is proposed in economics.

Hypothesis 4: Donor support to CD is (more) effective when it uses results sensibly to measure progress, correct course and learn

The Evaluation’s findings support this hypothesis by confirming that: CD outcomes would be more effective if results are appropriately defined in a way that leads to enhanced organisational performance. This would also facilitate: progress monitoring and provide justification – if need be – for correction of the CD process; and the organisations’ learning.

Almost every individual and agency interviewed (except for finance and budget-related ministries) in the course of the Evaluation indicated that their CD interventions were not results-based. Respondents noted that the management focus during the implementation of the project had been on the lower end of the results chain rather than outcomes and impact. There was consensus in respondents’ opinions that most of the CD actions that they had witnessed had little impact at the organisational level in terms of achieving expected outcomes.[30] Most respondents noted that their programmes might have generated better and more appropriate benefits if it had been structured better, and managed by results specifically.


[29] Refer to the Oxford dictionary definition of “legitimate”.

[30] There are exceptions to be sure; including the Ugandan prosecutors’ office for Anti-corruption Courts where the end result of Danida’s multi-year support is an organisation that has the human resource ability to execute its mandate providing it has political support for prosecutions.

6 Lessons learned

This chapter outlines lessons learned that Danida should consider in future formulation and implementation of CD support. These are also useful for executing agencies, consultants and technical advisers. The lessons learned are derived from the research and analysis in Chapter 4 and thus reiterate some of the major findings. The lessons learnt mentioned below are not necessarily adding new knowledge to the conventional wisdom concerning CD, but conform to the approaches sought as elaborated in Danida’s and other like-minded donors’ guidelines. The challenge remains to implement CD interventions so that they contribute more effectively to achieving development outcomes.

Effective aid

The lesson is: An increased attention to CD with a focus on organisational development could further promote recipient organisations’ performance and productivity and thus contribute to more effective development.

A well performing civil service at all levels of government (national, regional and local) – that provides services reflecting citizens’ needs – is the basis for citizens to recognise their government and respect the rule of law. An increased attention to CD could have the potential to enhance the recipient organisations’ performance and productivity for delivering public services. Despite the many obstacles with which recipient organisations are confronted, such as authority, financial resources, salaries, staffing levels, staff retention, etc., there are nonetheless examples on how organisations have been able to improve performance despite these difficulties. Danida has not fully exploited the potential for CD as outlined in its CD and TA guidelines, which is seen as a missed opportunity for enhancing the supported organisations’ operational basis and performance, which would otherwise have led to more effective aid.

Conceptualisation of CD interventions

The lesson is: CD works best when there is a demonstrated willingness and commitment among key stakeholders to change, and when national partners of the recipient organisations drive the CD process.

As the point of departure for CD processes, it is essential to map and continually update the level of the organisation’s capacity and associated capabilities in order to reconsider the tasks that potentially would lead to the achievement of the warranted level of performance. CD should encompass the organisation that is at the centre of the intervention as well as external organisations (public, private and CSOs) that are essential for the organisation’s overall performance (and by extension not only the targeted central agency). Preferably, all stakeholders should agree to and fully understand the push and pull factors that drive service delivery and its ties to politics and the various forces within civil society. Expectations of achieved changes to norms, standards and performance parameters for the organisation’s services and regulatory functions should be subjected to scenario-based analysis to ensure that they realistically can be achieved within the set timeframe and resource availability. Implementation of CD that requires behavioural changes – especially as regards the primary beneficiaries – takes a long time, even many years, so expectations of results need to be realistic.

Application of the ToC approach and RBM

The lesson is: ToC and RBM could be powerful tools for designing and implementing CD processes.

The ToC logic has (in recent years) been applied, as it was perceived to be a useful and effective way to manage change and the uncertainty that is attached to it. The ToC logic can be applied for long-term and comprehensive interventions that need to be adapted to a larger national reform processes, and it can also closely present the concrete actions and decisions that are required from the national partner. In dealing with the issue of how possible it is to prepare detailed plans for CD, the assumptions about how CD would likely progress should be made and the associated risk situation analysed. Data should be gathered to establish a CD baseline, and a CD needs analysis be conducted for outlining of the results-chain for the CD outcomes to be obtained. The stated results should be adequately specific to become the basis for RBM with targets set for the anticipated performance (i.e. results should be observable) for the various stages of the implementation process.[31] It should be anticipated that all stakeholders in a CD intervention do not necessarily share a common understanding of strategies, plans and concepts.

The ToC is could be a multi-dimensional, multi-layered and dynamic representation of how a series of actions, decisions, relationships, processes and other factors will likely combine, over time, to generate the results that are required. The ToC should incorporate actions at all three levels of the CD results framework. The ToC should also incorporate uncertainties and risks and illustrates where monitoring is required to validate hypotheses made to explain likely occurrences in the CD process. The dynamic nature of the ToC demands constant updating and supervision. When there is not enough information to enable a “comprehensive and thorough” ToC and RBM to be formulated, then Danida could design the intervention in such a way that the logic gets built up over two or three years, starting the implementation phase slowly, and adding precision to the model as time goes on.

Adjustment to changing context and partners’ ability

The lesson is: CD works best when flexibly managed within the evolving contexts of the organisations and the enabling environment, and when adapted to managers’ ability to lead the process. A step-wise approach should be applied if results are not well defined, contexts are uncertain or changes rapidly (“muddling through”).

Context analysis is not only important at the beginning of an intervention, but the likely way context may evolve over time is also an essential element of programme design. Mechanisms need to be incorporated into programme design that enable progress to be monitored against expected CD results, and then analysed regularly to assess adequacy of resource allocation, priorities and realism of set results. CD should be planned as an integral part of the programme (as opposed to a stand-alone thrust or a second thought) and should be specifically designed to facilitate the reform and change process that are the programme’s overall aim. When CD is not the focus of an intervention but an add-on, managers and decision-makers consider that it is a complement and may not place priority on it. A RBM framework has to reflect the ability of the partner to fully integrate it into the implementation process and then use it. It may take time to get to that point, but the experience of managers was that it is better to use a step-wise approach to getting to the point where RBM is comprehensive and useful, rather than not use it at all.

Technical assistance

The lesson is: TA works best when adapted to the demand of the national partner and complement national counterparts’ abilities, and when training provided reflects specific needs to close organisational capability gaps that constrain achievements of results.

The use of foreign TA should be considered carefully as it is generally seen by the recipient organisations as being very expensive and TA personnel is not easy to integrate in the public organisation. Good justification for TA should be provided in terms of complementary competence contributing to the reform and change process as regards knowledge, useful international experience, analytical and management skills, etc. TA advisers should preferably be provided in an advisory capacity with a well-defined relationship to the recipient organisation’s management that is meant to own and drive the process. An upcoming issue is whether TA advisers could have an intermediary role between the recipient organisation and the donor with a view to complementing the donor and reaching joint decisions on how best to utilise the donor support.

Formulating how technical advisers will be used for CD has always been difficult. An interesting report concerning TA in Tanzania deals with the demand and the supply for TA (see extract in Box 23). The subject of that research was on TAs generally and not necessarily the CD strategies, responsibilities and mandates of the technical advisers themselves. Although the report was done specifically for Tanzania, the lessons-learned and conclusions could be expanded to cover most countries if adequate adaptation is carried out.

Box 23: Supply and demand of TA in Tanzania

The demand for TA is uncoordinated and ad hoc. Although there is agreement amongst donors and Government that the Government should lead and coordinate TA, this is rarely implemented. In reality donors fill the gap, often directly determining where TA is to be provided or suggesting TA inputs to Government.

TA is usually supply led: In most cases TA is not clearly demand driven and as a result may not be well used. Some donors formulate their support through a review of government strategies and then propose support that, although aligned with the sector plan, has not been demanded by government. Many donors appreciate how some TA allows them increased access to information and in some cases can provide policy leverage or can fulfil a monitoring function.

Source: Tilley, H. 2014. “Supply and demand of technical assistance and lessons learnt. Country Perspective Note, Tanzania”: Paper prepared by Overseas Development Institute for International Health Partnership (IHP+).

Private sector and civil society

The lesson is: The private sector and CSOs have demonstrated that they effectively can contribute to public service delivery, and should therefore be incorporated in development programmes where appropriate in order to increase the scope and quality of services and for reaching the poor more effectively.

The private sector and CSOs have played a significant role in the Danida supported interventions. The private sector has especially contributed to the environmental, agricultural, business, energy, transport and health sectors in various forms of partnerships with the public sector. The private sector engagements have complemented public institutions’ service delivery capacity and in some sectors added much needed investment capital.[32] CSOs have contributed to downward and upward accountability and raised citizens’ awareness of their rights to appropriate public services. Although the involvement of CSOs so far have had limited effect on the scope and quality of public services, it is nonetheless part of a long-term process of enhancing citizens’ voice and rights and in ensuring accountability and transparency.

Gender and human rights

The lesson is: Promotion of human rights, gender equality and women’s rights have the potential to act as drivers of change and could thus add value to the change process – while also being essential aspects in themselves.

Gender equality and human rights have not featured markedly in the CD interventions examined in this Evaluation. Except for those programmes and components that explicitly deal with gender and human rights, these two thematic issues were not referred to in capacity development interventions in the evaluation sample. Donors and recipients are in quandary as to how to include and achieve the policy objectives of “human rights” and “gender” based approaches. The Danida HRBA Guidance Note[33] acknowledges that human rights standards and principles can act as drivers of change – provided that the partner has ownership and commitment the same would apply for gender equality. In the new country programme paradigm, actors and processes are invested in for their potential as drivers of change rather than just their implementation capacity. Norad has recently conducted an evaluation on its support to strengthen women’s and girls’ rights and gender equality with a focus on political empowerment, economic empowerment, sexual and reproductive health, and violence against women.[34]

Exit strategy

The lesson is: An exit strategy early on in a programme – or even as part of the formulation – would focus the attention on how best an organisation could maintain the required capabilities and capacity to operate and thus reduce its dependency on external assistance.

Many of the Danida programmes that were part of the Evaluation have gone through a number of five-year phases – either because the intervention was from the onset determined to be a long-term engagement (ex. health), or because it was decided that the outputs/outcomes of a previous phase could be or should be further developed. In such cases there would be no need to have an ‘exit strategy’ from the very beginning of the intervention. But generally (because the decision to proceed to a further phase is never taken at the beginning of a programme) each programme should have an exit strategy, and even more so with CD components. No interviewee was aware of a Danida exit strategy, and was not aware of exit strategies from any other donor either. A recent departure of the Global Fund from a Zanzibar pooled fund shows how unstable such funding can be and how much of a shock wave can be generated by an unprepared exit.

An ex-post evaluation at the time of exit or programme end, focusing on outcomes, impacts and sustainability could provide important insights to how the organisation could best continue the CD endeavours; ideally, donors should develop this skill and ability within organisations so that governments could internalize the results.


[31] See Danida 2011 CD Guidelines, Section 4.2, Identifying capacity results areas – filling the “missing middle”.

[32] The private health facilities in Tanzania, for example, have built, staffed and operated facilities without government support. In the same light, one could look at the aBi Trust that has mobilised and managed funds from non-national governmental sources.

[33] Danida. 2013. A Human Rights Based Approach to Denmark’s Development Cooperation.

[34] Norad. 2015. Evaluation of Norway’s support to women’s rights and gender equality in development cooperation.

7 Conclusions and recommendations

7.1 Overall findings and conclusions

The Danida supported interventions have been implemented in joint programmes with other donors (bilateral and multilateral) in highly complex environments. The thrust of Danida’s engagements has been at the sector level – generally with interventions at all administrative levels (national, regional, local) that involved interactions with line and sector ministries, local governments, civil society and the private sector. National poverty reduction strategies and sector legislation, policies and plans provided the directions for the support. The main characteristics of Danida’s support are:

  • Participation in multi-donor funded programmes;
     
  • Adoption or pursuance of the sector-wise approach;
     
  • Attention to external and internal contextual factors in programme design;
     
  • A comprehensive thrust on supporting the sector and its organisational ecosystem;
     
  • Realisation of the private sector’s and CSOs’ potential to complement the public sector;
     
  • Targeting the poor and disadvantaged groups.

What worked well in the achievement of CD results: Danida’s adherence to its programme management guidelines – as regards sector-wide approaches, ownership and partner-led implementation – has contributed to creating conducive environments for endogenous CD processes that in most cases increased organisational performance. The majority of the sector programmes in the evaluation sample have performed well and have experienced a significant increase in sector-outcomes in terms of coverage and quality, which is well documented in progress and other reports. Danida-supported interventions were to a large extent designed to reflect external and internal contextual factors in the overall programme design, which implicitly also included CD interventions. Danida has consistently demonstrated a commitment and a willingness to respond to the stated priorities and strategies of the priority countries and partner organisations – and is recognised as a trusted partner, largely because of its relative flexibility to adapt to changing contexts and its long-term commitment.

The Danida-supported programmes generally had few technical advisers and donor funding for CD was mainly used for training, equipment, and systems development. A major part of the work that led to increased organisational performance – in terms of generating sector-outcomes – has been undertaken by the organisations’ own staff. In some cases, the domestic revenue base for financing the sector’s recurrent expenditures was expanded by introducing user charges. An important feature in some programmes was the engagement of the private sector and civil society organisations (CSOs) acting in complementary roles to public service providers and thus contributing to the expansion of public service delivery capacity. In conclusion, it is reasonable to assume that the organisations’ internal capacity has grown by some measure through endogenous processes whether planned or unplanned.

What worked less well in the achievement of CD results: Changes in CD outcomes have not been tied to predefined results frameworks. While results-based management (RBM) has generally been applied and emphasised in Danish assistance for sector-outcomes, results-based CD in Danida programmes – as introduced in 2005 and building on the “Logical Framework Approach” (1996) – has to a large extent not been applied. Assuming that capacity grew simultaneously with the generation of sector-outcomes, it is essentially not possible to judge the actual extent to which the supported organisations’ capacity increases have been directly or indirectly influenced by Danida’s support. It is thus not possible to establish a direct link between sector-outcomes and CD outcomes. In the absence of direct evidence on CD results, the achievement of sector-outcomes is regarded as circumstantial evidence for CD results.

Although CD interventions have been included in programme documents and have been subject to progress monitoring and reviews, it is evident that a consistent and comprehensive approach to CD has generally not been applied. The Danida-supported programmes were not able to overcome many CD related organisational and enabling environment constraints – as opposed to what was the stated ambition in programme documents. An appropriate risk analysis of the CD context could have helped to highlight challenges so that these could be counteracted. Danida has always aligned itself to the strategies of the recipients, even if the expected results were not optimal and has in a few cases continued to support organisations that were not committed to change. The main thrust of CD interventions has been on the individual level and to some extent on policies and strategies related to the sectors’ enabling environment. The organisational level has only received limited attention, while arguable this should have been at the centre for enhancement of the partners’ performance – and for creating sustainable organisations.

It appears that Danida staff, technical advisers and consultants alike have not focussed on the strategic role that CD can play in bringing about developmental effects. Technical advisers posted in the partner organisations were often underutilised and were not adequately involved in facilitating organisational changes. This is in contrast to Danida’s TA guidelines as well as the preference as expressed by some partner organisations that Danida’s TA/CD support should support the organisations’ staff in playing a key role in programme implementation. The reduction of human resources in the Danish embassies and the increasing complexity of programme planning and implementation, also raise the question regarding the division of work between embassy staff and TA advisers and how this could be organised differently.

Danida’s capabilities to manage CD processes are in need of improvement. CD practices have not been as effective as they could have been, largely because of: a) under-specificity of results; b) inadequate CD management practices; c) ineffective oversight, project management and supervision; and d) inadequate systems to support CD. Danida needs to develop not only the capacity of target organisations, but also the capability of recipient country managers and Danida personnel (including TA advisers). Despite the renewed emphasis on CD, as expressed in the Paris Declaration (2005) and the Accra Agenda for Action (2008), Danida’s management has not been able to lift the CD ambitions into practical implementation. The Evaluation’s findings are in line with many published reports from leading development organisations such as the WB, ADB, EU and DFID.

A changing global development agenda: The global development agenda will undergo significant changes in the coming years. From a CD point of view, there will be a need to keep track of significant development trends and how these generally and for CD specifically will influence Danida’s future development assistance. The outcome document of the Addis Ababa Action Agenda (July 2015) for “Financing of Development” presents an ambitious financing framework that includes concrete policy commitments. The adoption of the post-2015 development agenda in New York in September 2015 and the new universal climate change agreement in Paris in December 2015 are two other essential events that will shape the future development agenda. The Danish Ministry of Foreign Affairs published “More Denmark in the World” in December 2014 presenting Denmark’s visions and foreign policy interests. The new Government that was sworn in in June 2015 may further elaborate the visions for Denmark’s development assistance.

A recent OECD survey (2015) provides insights into what partner countries anticipate will be their main development challenges within five to ten years, and how they expect their relations with donors to evolve to meet these challenges.[35] The survey finds that the demand for development cooperation will remain strong given the economic and environmental challenges that lie ahead. The 40 partner countries surveyed expect donors to shift to a more enabling role in coming years: providing vital finance in support of government-led sector programmes; delivering more and better technical assistance and policy support; and leveraging more private finance. Resources for development cooperation can be expected to be under heavy pressure from other priorities. There is therefore a need to focus on development effectiveness and on the sustainability of development interventions. Appropriate attention to CD could provide feasible options for addressing development challenges.

Overall conclusions: The findings above lead to the following conclusions:

  • Danida has, through its adherence to the programme management guidelines facilitated the creation of conducive environments for endogenous CD processes that have contributed to increased organisational performance in most of the Danida-supported interventions;
     
  • Support to CD was most successful when:
     
    • Internal and external contextual factors were taken into consideration;
       
    • There was strong ownership and commitment on the part of the organisations involved;
       
    • All three levels – the enabling environment, the organisational level and the individual level – were addressed simultaneously with specific attention to the organisational level in order to enhance organisational performance;
       
    • Wherever relevant, the private sector and the CSO were drawn into the implementation in a complementary way to enhance service delivery and coverage;
       
    • The demand side was addressed, e.g. through support for civil society calling for transparency and accountability in the provision of services.
  • While results-based management was emphasised as a Danida policy, most CD interventions were not results-based or were not managed as such;
     
  • Systematic attention to CD and organisational development could have promoted recipient organisations’ performance and productivity further and thus contributed to more effective development;
     
  • While CD and TA guidelines of good quality were available – and the intent of CD was well expressed in Danida documents generally – they have rarely been applied effectively during planning and implementation of development interventions;
     
  • Management at Danida headquarters and embassies could have engaged more effectively in CD during programme design and implementation.

7.2 Recommendations

Addressing capacity development in future

From a CD point of view, there is need to keep track of the global development trends and reflect on how these generally and for CD specifically will influence Danida’s development assistance. Ideally, leaving the supported organisations with adequate capacity to deliver their mandated public services and regulatory functions ought to be a cornerstone of the development assistance. Correspondingly, leaving the supported organisations with the agreed level of capacity to maintain and further develop their capacity is equally important, which similarly should be given adequate attention. It is evident that CD requires increased attention in order to bring about development effects more effectively.

The Evaluation recommends that:

  1. Danida should recognise the significance of CD and its potential contribution to development effects – consistently with its policy statements and guidelines. In that regard Danida should emphasise CD more prominently in its programmes and their management. Danida should – in priority countries – continue its practice of long-term engagement in line with national priorities and partner driven interventions, with a view to achieving sustainable public sector organisations and supportive enabling frameworks.
     
  2. Danida should consider if the Danida Fellowship Centre (DFC) could become the focal point for CD – with due reference to the 2014 CD Policy – keeping track on CD trends and strategies. For this to be effective, DFC should interact more closely with the Department for Technical Advisory Services.

Programming and integration of CD

Danida’s paradigm for programme management has shifted significantly – from the one that was in existence for the programmes under the Evaluation – with the introduction of “Guidelines for Country Programmes” (GCP) for Denmark’s priority countries. The first version of the GCP was issued July 2013 and updated in April 2015.

The overall framework remains unchanged, but a number of changes have been made:

  • The GCP includes a section on ‘Theory of Change’ – replacing the requirement that country-level programming should adhere to the five OECD/DAC criteria;
     
  • More guidance on monitoring of results reporting and a revised results framework, including the introduction of ‘Real-Time’ evaluation; and
     
  • More guidance and focus on fragile contexts and how flexibility can be applied in countries affected by fragility and conflicts.

While the GCP makes frequent references to CD, they could provide more overall direction and guidance on the design, planning and implementation of CD initiatives with due reference to the specific CD and TA guidelines. In short, whereas the GCP provides a sense of policy direction, it does not really provide practical support to those faced with the management of the CD “programme cycle”. CD is touched upon in several sections in the GCP, but the message does not come out as strongly as wanted if capacity is seen as a strong driver for change. CD is mentioned in connection with appraisal and reviews, but is not included in the list of analytical areas for the identification phase.

The current CD and TA guidelines still constitute a solid basis, but in view of the issues raised concerning CD and TA in this Evaluation there would be amble prospects for: a) updating the guidelines to be compatible to the new programme management paradigm; b) consolidating Danida’s processes and procedures supporting effective CD; and c) providing practical guidance to managers and practitioners. Successful CD is characterised by applying a comprehensive approach that aim at attaining higher levels of organisational performance by: a) conceiving interrelated CD interventions that address the enabling environment, the organisational level and individual level; and b) taking external and internal contextual factors and their potential change over the planned time span into consideration. RBM is emphasised for all development themes in Danida’s existing guidelines.

The Evaluation recommends that:

  1. The Guidelines for Country Programmes should – in connection with the next revision – emphasise CD more strongly as the cornerstone for development, and make due references to planning and operational issues as stated in the CD and TA guidelines. Danida should apply its change model (Theory of Change) in a way that integrates CD outcomes, which should be dynamic to reflect how the enabling environment must be managed as the intervention proceeds, and how organisational outcomes are achieved in a planned and systematic way. Danida should ensure that its interventions are clearly designed to reflect: the specific contexts and other realities of the planned intervention; as well as the conceived risks and how these could be mitigated. An experimental and learning approach should be applied for high-risk situations or where local contexts do not enable a full results-based approach. Danida should continue to support the demand side with a view to strengthen public organisations accountability.
     
  2. The CD and TA guidelines should be updated with an emphasis on their practical use during planning and implementation of programmes with an emphasis on CD outcomes and impact in public organisations and their potential cooperation with the private sector and CSOs. The TA guidelines should be updated to elaborate the role of technical advisers in capacity and capability development in the partner organisations and how TA advisers should interact with embassy staff. CD and TA guidelines could be combined in order to emphasise the interrelation between CD and TA.

Management of CD interventions

Management of CD interventions will require increased awareness and attention by Danida programme officers and national partners’ programme managers.

The Evaluation recommends that:

  1. Danida should assess what competences are required among its staff (including Danida advisers) at headquarters and embassies to address CD comprehensively in programme planning and implementation, and how CD is integrated in the overall programme design to ensure that CD receives adequate attention. Furthermore, Danida should consider what means could be applied to raise knowledge and competence among its staff, ex. through short courses, E-learning, etc. More generally, Danida needs to improve its knowledge management systems to the point where they are actually used as references by personnel.
     
  2. Danida should assess – consistently with its 2011 CD guidelines – what basic requirements and competences should exist in partner organisations to enable supervisors and programme officers to lead the CD process and manage the maintenance and further development of capacity and capabilities over the long-term – and what would be the means of developing such competences.
     
  3. Danida should develop a rapid assessment framework for CD that could be applied for appraisals, reviews and progress monitoring to ensure that CD is properly integrated in the programme design and adjusted to changing circumstances. The use of such framework could also be a significant source of learning as well as demonstrating Danida’s corporate accountability and assurance frameworks.

[35] OECD. 2015. Making Development Cooperation Fit for Future: A Survey of Partner Countries.


Colophon

Title:
Evaluation of Capacity Development in Danish Development Assistance

Subtitle:
Evaluation 2015.06

Summary:
Support for capacity development is a cornerstone in Danish development cooperation.

This evaluation assesses Denmark’s support to capacity development as an integral part of sector support to priority countries. The focus of the evaluation is on support to enhancing the capacity of public sector organisations’ performance as regards relevance, coverage and quality of services and regulations.

The evaluation concludes that Denmark has facilitated capacity development processes that have contributed to increased organisational performance in most of the supported interventions.

The evaluation finds that good and comprehensive guidelines for support to capacity development exists and suggest that if they had been applied more effectively during planning and implementation better results would have been forthcoming. In particular, the evaluation points to the lack of a systematic and comprehensive approach to planning and reporting on capacity development.

The evaluation has been carried out by Nordic Consulting Group, Denmark.

Notes:
Responsibility for content and presentations of findings and recommendations rests with the authors.

Publisher:
Ministry of Foreign Affairs of Denmark

Responsible institution:
Ministry of Foreign Affairs of Denmark

Copyright:
Ministry of Foreign Affairs of Denmark

Authors:
Nordic Consulting Group (NCG)
IRDC
DEGE Consult
Centre for Aid and Public Expenditure

Contributors:
Rosendahls - Schultz Grafisk (Digital version)

Language:
English

URL:
http://www.netpublikationer.dk/um/evaluation_2015_06/index.html

ISBN HTML:
978-87-7087-893-7

ISBN PDF:
978-87-7087-892-0

Version:
1.0

Version date:
2015-12-17

Publication standard no.:
2.0

Data formats:
html, gif, jpg, png, pdf, css

Publisher category:
Statslig

Top

This page forms part of the publication "Evaluation of Capacity Development in Danish Development Assistance – Evaluation 2015.06" as the Entire publication with graphics.
Version no. 1.0, 2015-12-17
Publication may be found at the address http://www.netpublikationer.dk/um/evaluation_2015_06/index.html