Royal danish ministry of foreign affairs - Go to the frontpage of um.dk   Publication  
 
 
     
 
 

Appendix 1 Framework and Stage 2 Terms of Reference

Framework Terms of Reference
for the
Multi-donor Evaluation of Support to Conflict Prevention
and Peacebuilding Activities in Southern Sudan

Final version

10 July 2009

1. Background

At the meeting of the OECD/DAC Network on Development Evaluation (February 2008), the Policy and Operations Evaluation Department (IOB) of the Netherlands Ministry of Foreign Affairs offered to lead a multi-donor evaluation of support to conflict prevention and peacebuilding activities in Southern Sudan. The Evaluation is one of a series of similar evaluations covering other countries (Sri Lanka, Haiti and the Democratic Republic of the Congo) and will be undertaken in 2009-2010.

As a first step IOB hosted a consultative meeting on 6 May 2009 at which evaluation departments of interested donors and international organisations and representatives from the Government of Southern Sudan discussed an approach paper stipulating the need for the Evaluation and providing its outline in general terms.[1] The meeting was attended by Belgium, Canada, Denmark, Germany, the Netherlands, Norway, the Government of Southern Sudan, Sweden - Sida, United Kingdom, UNHCR, UNICEF, UNOCHA, UNV, WFP, and the World Bank. Other agencies indicated an interest in the Evaluation but were not able to attend.

The participants ascertained that there is a demand for a comprehensive evaluation, which will provide insights in the challenges the Government of Southern Sudan and the donors and agencies engaged in Southern Sudan have been and are currently facing to implement the Comprehensive Peace Agreement signed in January 2005. Also, the Evaluation can provide lessons which may be useful for policies and programming in post-conflict situations and situations of fragility elsewhere. The focus and general approach of the Evaluation, its timing, and its governance structure were discussed. The current Terms of Reference for the Evaluation have been drawn up taking into account results of the discussion.

2. Southern Sudan a complex context in brief

Comprehensive Peace Agreement

9 January 2005 was a milestone date in Sudanese history. On this day the Comprehensive Peace Agreement (CPA) was signed in Nairobi, Kenya, which officially ended the war between the Government of Sudan (GoS) and the Sudan People´s Liberation Movement/Army (SPLM/A) that had waged for over twenty years. The establishment of the CPA offered hope and a unique window of opportunity for a country that has proven to be among the most volatile on the African continent. Among other things, the agreement provided for a Government of National Unity (GoNU) representing both the North and the South; an autonomous government for Southern Sudan; and a referendum in 2011, allowing the South to decide whether it will stay part of a unified Sudan or become an independent state.

The signing of the CPA constituted a major achievement. Nevertheless, Africa´s longest running civil war had left Sudan devastated. The 1983-2005 conflict had taken the lives of an estimated two million people and displaced almost four million more, while over half a million others fled to neighbouring countries. Unprecedented damage was done to the country´s physical and social infrastructure. Southern Sudan, covering an area of roughly 640,000 square kilometres, populated by an estimated eight to twelve million people, and characterised by a poor physical and economic infrastructure, particularly suffered the burden of warfare. Following its installation in Juba in July 2005, the newly created Government of Southern Sudan (GoSS) was faced with the challenge of having to kick-start large-scale reconstruction in a region that had already enjoyed little modern development prior to the outbreak of war.

Joint Assessment Mission

In an attempt to identify the priority needs in post-war Sudan, the World Bank and the UN established a Joint Assessment Mission (JAM) in 2004. With the full endorsement and participation of the GoS and the SPLM, a country-wide, year-long needs assessment was conducted and finalised in March 2005. For Southern Sudan, the JAM identified five areas needing special and immediate attention in order to improve the living standards of the population and make progress towards meeting the Millennium Development Goals (MDGs).

First of all, core public structures had to be created on every level of society in order to establish effective and legitimate governance. Secondly, there was a need to invest in the rehabilitation and construction of Southern Sudan´s infrastructure to enhance the region´s accessibility and overcome the isolation of its rural areas in particular. Thirdly, poor access to and delivery of basic social services such as water, sanitation, health care and education were a priority concern. Fourthly, transforming the agriculture sector and developing the private sector were considered crucial for Southern Sudan´s economic recovery. Finally, the reintegration of returning internally displaced persons (IDPs) and refugees in Southern Sudanese society posed an enormous challenge.
Not only did the JAM identify and prioritise key objectives for sustained peace, development and poverty eradication in Sudan; it also assessed and aggregated the estimated costs of their implementation. Taking into account the fact that the Sudanese absorptive capacity at the time of writing was extremely low, the JAM identified priorities for the short and the long run. In the short term, the JAM emphasised interventions that would provide ´quick wins´ and contribute to a tangible peace dividend. Longer-term development programmes would be strengthened by a massive emphasis on capacity-building activities and institutional development during the first two years. Consequently, the JAM provided detailed cost estimates for the short run (phase I, i.e. 2005-2007) and indicative cost estimates for the longer run (phase II, i.e. 2008-2011). For phase I, the estimated costs were US$ 3.6 billion, whereas for phase II, the estimated costs were US$ 4.8 billion. The shortfall for which external support was sought for phase I was estimated at US$ 1.4 billion.

Donor support to Southern Sudan

At the international conference held in Oslo in 2005 following the signing of the Comprehensive Peace Agreement, donors pledged US$ 4.5 billion to help rebuild Southern Sudan. Four years later, it is proving difficult to obtain adequate data on donor funding for Southern Sudan. The major drawbacks are the dispersed and fragmented nature of the information and the lack of a sound overview, specifically of funds provided for recovery and development.[2]

However, notwithstanding the difficulty to obtain a complete and accurate overview of donor support to Southern Sudan, it is safe to say that substantial amounts of money have been allocated to the region.[3] Throughout the civil war, the levels of international support to Southern Sudan had been substantial. As of 2005, with the CPA in place and the JAM documenting the needs for short- and longer-term development, the challenge put before the international donor community was to shift its assistance increasingly towards addressing development needs, notwithstanding the ever present humanitarian needs. A mix of flexible approaches and aid instruments was envisaged, striking a balance between humanitarian, recovery and development activities in order to support longer-term development goals while at the same time providing ´quick wins´ and contributing to a tangible peace dividend.

Donors´ funding strategies, policies and structures in Southern Sudan, however, seem to have been motivated not only by the challenges posed by the political and operational environment in the South, but also by their commitments to international agreements to harmonise, coordinate and align interventions in fragile situations.[4]As the expected inflows of aid after the CPA posed a major management challenge to the GoSS as well as to multilateral and bilateral development partners, aid coordination and harmonisation became the key principles for the international engagement in Southern Sudan. In light of this, the Joint Donor Office (JDO) was established in Juba, through which the Netherlands, Norway, Sweden and the UK, subsequently joined by Canada and Denmark, coordinate and pool their development assistance to Southern Sudan. Furthermore, a multitude of pooled funding mechanisms was established, the Multi-Donor Trust Fund-Southern Sudan (MDTF-S) administered by the World Bank, the Common Humanitarian Fund (CHF) managed by the United Nations Development Programme (UNDP) being the largest and most significant of them.[5]

Southern Sudan – current state of affairs

Despite a massive investment of the international community to address the priority needs the JAM had identified, improvement has been slow since 2005. With respect to successes and challenges regarding peacebuilding, information is fragmented and incomplete. However, it is generally perceived that the international community´s efforts have so far not (yet) succeeded to provide immediate peace dividends on the ground in Southern Sudan: basic service delivery is still largely dependent on the humanitarian aid channels and the international NGOs, and slow implementation is considered a major weakness in the international engagement after the CPA. But Southern Sudan faces other serious challenges as well. CPA implementation is far behind schedule and tensions with the North remain over issues such as the demarcation of the border and the contested „Three Areas´ (i.e. Abyei, Kordofan and Blue Nile) that were heavily disputed during the war, transparency regarding revenues in the oil sector and the pace of redeployment of troops. The period since 2005 has experienced a fragile peace and the region is furthermore plagued by insecurity which is, among other things, reportedly caused by localised banditry and conflicts over land and livestock, abuse by state agents, and the widespread and unregulated presence of small arms and light weapons. In the meantime, amidst dropping oil prices which have led to a severe financial crisis resulting in a shortage of resources to cover recurrent costs and investments in peace and development, Southern Sudan is preparing to take part in Sudan´s multi-party elections in February 2010 which will have a dynamic of their own, while the 2011 referendum on self-determination is approaching.

3. Rationale for the Evaluation

International assistance to Southern Sudan has been the subject of a large number of (project) evaluations and reviews as well as other types of evaluative studies. Donors and agencies commissioned such studies of specific programmes, projects and policies, as well as key aid instruments in use in Southern Sudan, including the MDTF-S and the CHF. These studies and reviews were triggered by the growing perception that the existing mix of interventions and funding mechanisms was not delivering results on the ground quickly enough to meet the huge needs and expectations of the Southern Sudanese people. Failure to address this ´recovery gap´ through the delivery of basic services and other expected ´peace dividends´ was considered to pose a serious risk to CPA implementation and thus to peace and stability throughout Sudan.

An inventory of available evaluations, evaluative studies and reviews made for the preparation of the approach paper for the Evaluation, however, shows that none of these studies and reviews presents a comprehensive and clear overview of the results of donor support to Southern Sudan since the signing of the CPA in 2005. Rather, they provide a partial and incomplete picture of the assistance provided, and generally lack information on the actual results of this assistance at field level (i.e. how did or does the assistance address the needs of the Southern Sudanese population). The findings of these studies are relevant for the project, programme or mechanism covered. But most of them do not take into account the influence of other interventions and lack assessments of the ways in which the different interventions are related and may have strengthened or possibly weakened each other.

In other words, there is an absence of comprehensive insight in the effects of international assistance on the security and development situation in Southern Sudan, which is necessary to assess overall progress made in the area of conflict prevention and peacebuilding. It is this ´evaluation or knowledge gap´ that underlines the need for a comprehensive multi-donor evaluation of support to conflict prevention and peacebuilding in Southern Sudan.

4. Objective of the Evaluation

As mentioned above, it is generally assumed that the international community´s efforts to support peacebuilding and to provide immediate peace dividends to the Southern Sudanese people have so far not been very successful. The Evaluation will test this assumption by assessing the extent of progress made and pointing out the factors driving success or failure. In this way the Evaluation will provide an important input into ongoing discussions and future policies and plans how to possibly improve the relevance, effectiveness, and – above all – the impact of the international engagement in peacebuilding processes in Southern Sudan in the run-up to 2011 and the post-2011 period.

The Evaluation will be comprehensive taking into account conflict prevention and peacebuilding activities directly and indirectly supported as part of humanitarian, reconstruction, development and diplomatic efforts in the period 2005-2009, and the ways in which these different actions have influenced each other. As such, the Evaluation is not only to produce information on the effects of the interventions thus far, but as a secondary objective, it will also shed light on the effectiveness of the comprehensive approach most development partners wish to pursue.

The main objective of the Evaluation is to provide insight in the effects of donor-supported conflict prevention and peacebuilding interventions in Southern Sudan and to provide lessons to be applied in ongoing and future donor involvement in the region (i.e. lessons learning).

It also serves an accountability purpose: since the signing of the CPA, large amounts of donor funding have been disbursed in Southern Sudan and the results of the Evaluation will provide insights into how these funds have been spent and to what effect.

Finally, the Evaluation is part of an ongoing process to pilot the working draft of the Guidance on evaluating conflict prevention and peacebuilding activities, developed by the OECD/DAC Network on Conflict, Peace and Development Cooperation and the OECD/DAC Network on Development Evaluation. This Evaluation and parallel evaluations in other conflict and post-conflict areas will provide feedback on the usefulness of this Guidance.

Although the Evaluation will focus on results accomplished, it is also expected to have a „forward- looking character´ in order to provide lessons to be taken into account in the post-2011 peacebuilding agenda in Southern Sudan, as well as provide broad lessons which may be of use in situations similar to those in Southern Sudan (i.e. states characterised by fragility and post-conflict conditions).[6]

Use of the Evaluation

It is intended that the evaluation results will be useful for a wide variety of stakeholders. The following primary users are identified:

  • the Government of Southern Sudan and its different administrative institutions;
  • the Southern Sudan Legislative Assembly;
  • the Government of National Unity and the Parliament;
  • on the part of the donors and international organisations: policy makers, aid administrators and
  • operational managers at headquarters and in the field;
  • parliaments of donor countries and governing bodies of international organisations;
  • other (non-government) organisations involved in the implementation of assistance in Southern Sudan;
  • the OECD/DAC;
  • the wider development community.

It is anticipated that the evaluation results will be widely disseminated through printed copies of the final report and through the posting of the final report on the websites of the respective donors and agencies commissioning the Evaluation as well as the website of the OECD/DAC. In addition, it is planned to organise a seminar in Juba (and possibly in Khartoum) once the Evaluation has been finalised to discuss its results and recommendations with a wide range of stakeholders. Individual donors and agencies engaged in the Evaluation will arrange their own dissemination activities which may include a management response on the evaluation results and recommendations. These donors and agencies may also organise and separately fund presentations, seminars, or workshops with participation of a representative of the Evaluation Team.

5. Defining conflict prevention and peacebuilding activities – scoping of the evaluation

At the consultative meeting mentioned in section 1, it was agreed that the Evaluation should not only cover conflict prevention and peacebuilding activities in the narrow sense (i.e. mediation and reconciliation), but should encompass the wide range of donor and agency support to Southern Sudan including its coherence, and thematic and sectoral focus. This implies coverage of support to productive sectors, social and physical infrastructure, social service delivery, capacity building and governance, etc.

This approach is in line with the „practical„ definition of conflict prevention and peacebuilding activities provided by the OECD /DAC Guidance on evaluating conflict prevention and peacebuilding activities (OECD 2008). The Guidance points out that there is considerable (theoretical) debate about what defines conflict prevention and peacebuilding. This, in turn, leads to difficulties to ascertain which activities, whether or not supported by donors and aid organisations, can be considered to directly or indirectly contribute to the prevention of conflict and the building of peace. It therefore provides a practical definition of four key categories of conflict prevention and peacebuilding strategy and action, which will also be adhered to in the Evaluation:

  1. „[I]nterventions that support the promotion of a culture of justice, truth and reconciliation, which can be critical in post-conflict regions in order to heal the wounds of conflict and reconnect society;
  2. [C]apacity building and promotion of good governance [which] are critical to human security, especially where states are unable or unwilling to deploy peaceful means to resolve conflict or sustainably and independently facilitate provision of key basic services;
  3. [C]onflict prevention and peacebuilding policies and actions often work to create incentives for systems that promote the peaceful resolution of conflict. Supporting reform of security and justice institutions – including the judiciary, penal, policing, parliaments, defence and military actors – is critical and should be seen as a long-term project to achieve democratic governance over security institutions while developing a wider justice and security system that upholds the rule of law and respect for the dignity of poor people;
  4. [S]ocioeconomic development and the policies to support it also matter, before, after and even during hostilities. Addressing structural violence and inequality is essential to reducing tensions and enhancing a society´s capacity to prevent violence – and is thus often a focus of conflict prevention work.´[7]

Socioeconomic development has a direct and indirect bearing on conflict prevention and peacebuilding and those involved in providing support to economic and social development through policies and actions need to take into account and be sensitive to the conflict situation or post-conflict conditions.

General scope of the Evaluation

The Evaluation will focus on donor support provided to conflict prevention and peacebuilding processes in Southern Sudan covering the period 2005-2009 (i.e. post-CPA). It will cover the four key categories of conflict prevention and peacebuilding strategy and action defined by the OECD/DAC Guidance (2008). It will in principle focus on the support provided and activities undertaken by the donors and agencies that are commissioning the Evaluation.[8] As is proposed by the OECD/DAC Guidance and because the definitive scope of the Evaluation cannot be determined in a detailed manner at this juncture, the Evaluation will follow a two staged approach. The detailed scope will be defined by taking into account the results of conflict (contextual) and policy analyses, as well as the insights regarding the support provided by donors and agencies (for details see section 6 below).

The advantage of this approach is that it provides the opportunity to determine the most relevant focus for the Evaluation based on insights into the current situation in Southern Sudan, an assessment of possible future developments, an overview of donor strategies and interventions as well as information on results achieved thus far obtained through secondary material (evaluations, reviews and other types of studies).

6. Approach and methodology – a two-stage approach

Challenges to conducting the Evaluation – the need for an incremental approach

The Evaluation will face numerous challenges, as both the interventions in question and the Evaluation take place within the context of a conflict (and its aftermath). This versatile (post-conflict) context has major implications for policy making and strategy, which have to be flexible to adapt to quickly changing circumstances. It also influences the relevance of different intervention modalities to be applied in time and space. Achievements will also vary due to changing circumstances. This context has an influence on the approach of the Evaluation as well. The Evaluators will have to deal with issues like a lack of baseline data, difficult access to respondents due to security constraints, a high turnover of staff in aid organisations creating a lack of institutional memory, and difficulties in assessing less visible political interventions. Furthermore, the implementation of the Evaluation may be challenged by changes in the security situation in Southern Sudan, or other events of a political nature, e.g. the elections currently planned in the first quarter of 2010. Changing security conditions may limit the possibility of conducting field work.

The OECD/DAC Guidance on Evaluating Conflict Prevention and Peacebuilding Activities proposes an incremental evaluation approach in order to deal with the above mentioned challenges. The Guidance states that an evaluation should include a form of conflict analysis to identify the key factors relating to conflict and the linkages between them, pointing to sources and dynamics of conflict (and conflict mitigation) as well as peace (and peacebuilding). A thorough and up-to-date understanding of the conflict is the first prerequisite for a conflict-sensitive evaluation process, as it provides the Evaluators with an opportunity to assess the actual results and possibly the impact of interventions in relation to a conflict analysis.[9]

Taking this into account, and adhering to the OECD/DAC Guidance, the Evaluation will apply a theory-based approach[10] with due attention for the „conflict-sensitivity´ of (donor and agency) policies and their support activities. The Evaluation will be implemented in two interrelated stages.

Stage 1 – Analytical analyses and scoping a ’sample-based’ evaluation

The first stage of the Evaluation consists of conducting analytical work of which the results will be used to determine the relevance of and to delineate the subsequent evaluation work. Stage 2 consists of a „sample-based´ evaluation covering interventions to be studied in detail at field level.

Stage 1 consists of the following four activities:

1. The conduct of a conflict analysis

The conflict analysis is to provide insight in the specific context in which the interventions have taken and are taking place. The method of analysis to be selected needs to be well-adapted to the post- conflict context and its related challenges, the scope of the Evaluation and the resources available. Key elements of the analysis should include aspects such as the profile of the conflict, its causes and potential for peace; actors; and dynamics and possible future trends. For reasons of comparison, the conflict analysis would also need to take into account the situation at the time the CPA was signed. The conflict analysis should build as much as possible on existing studies and reviews[11], combined with interviews with key stakeholders in (Southern) Sudan and at donor and agency headquarters.[12]

2. The conduct of a policy and strategy analysis

Second, an analysis of the policies and strategies of donors and international organisations providing support to Southern Sudan should be conducted to provide insight in the objectives and intervention strategies underlying donor and agency support to Southern Sudan.[13] These policies and strategies will be reviewed in terms of their relevance to the post-CPA context and its dynamics. This analysis will cover the policies and strategies of the donors and agencies commissioning the Evaluation, but may also include other actors. The results of the analysis will be taken into account when assessing the relevance of interventions as well as their coherence and the ways they are coordinated.

The policy analysis should be built on a review of relevant files, combined with interviews with key stakeholders both at headquarters and field level. For practical reasons, these interviews should be combined as much as possible with the interviews to be undertaken for the conflict analysis.

3. The conduct of an analysis of portfolios in support of conflict prevention and peacebuilding

Third, a comprehensive overview of donor and agency (direct and indirect) support to conflict prevention and peacebuilding in Southern Sudan needs to be created to serve to assess coherence of policies with the actual support provided. This overview will be established on the basis of the OECD/DAC CRS-codes commonly used to identify substantive categories of ODA.[14] The overview allows the Evaluation to identify major investments supported by donors and agencies as well as other activities which may be important in non-monetary terms.[15] Together with the other analyses to be conducted in Stage 1, this will enable the identification of the most relevant direct and indirect conflict prevention and peacebuilding interventions to be taken into account by a sample-based evaluation in Stage 2. To establish this overview, the Evaluation Team will have to use data on disbursements available from the Ministry of Finance and Economic Planning (MoFEP) of the Government of Southern Sudan, which operates an aid information system which captures donor investments in Southern Sudan. Another important source is information on activities funded through the above mentioned pooled funding arrangements. In addition, the donors and agencies involved in the Evaluation will be requested to provide information to the Evaluation Team on annual disbursements, including type of support (humanitarian aid, reconstruction aid, development aid), sectors supported and substantive information on activities supported. All types of donor funding (bilateral funding, multilateral funding and funding through NGOs) covering the period January 2005-July 2009 will have to be included.[16]

4. The conduct of an analysis of evaluations and research material

Finally, an analysis of evaluation reports, review reports and project/programme completion reports and other types of source material including academic and applied research needs to be conducted.

Source material for the above analyses

The core set of documents to be reviewed during Stage 1 consists primarily of policy statements, disbursement overviews and project/programme portfolios as well as evaluation and project/programme completion reports of the commissioning donors and agencies. These documents will be provided to the Evaluation Team by these agencies through or in coordination with the Chair of the Evaluation Management Group.[17]

To supplement that collection, the Evaluation Team will identify and collect additional documents. The Evaluation Team will maintain an integrated list of all documents collected. To facilitate summaries, comparisons and analysis, the Evaluation Team will categorise documents by commissioning agency, type of document, focus, methodology used, date, authors, principal content/findings, as well as other categories the Evaluation Team may deem important. To facilitate access to and use, the list should be organised in a database format that can be made available to other users upon the completion of the Evaluation.

Additional documents are likely to become available and to be used during the course of the Evaluation. The Final Report of the Evaluation should contain an overview of all sources used.

In summary, Stage 1 should build as much as possible on existing data sets, evaluative studies and reviews. It serves to provide an initial portrait of the various policies and their relation to the (post) conflict situation, a (substantive and financial) profile of the support provided (what are the major elements and are they connected?), and an initial analysis of results of the support provided. This portrait will subsequently be used to inform and shape the specific Terms of Reference for Stage 2, which will be built around a „sample-based´ evaluation at field level.

Stage 2 – ’Sample-based’ evaluation

The second stage of the Evaluation will involve an in-depth evaluation of conflict prevention and peacebuilding interventions supported in Southern Sudan with emphasis on field level assessment. The scope of the primary empirical data collection for the Evaluation through the conduct of field studies will be determined taking into account the results of Stage 1. The Evaluation Team will design the Terms of Reference for Stage 2. These ToR need to include a set of specific evaluation questions (see also section 7) and a research strategy and methodology (a purposeful sample of specific sectors, projects and programmes to be studied in detail, locations to be visited, data collection approaches and methods, et cetera). The Terms of Reference will be an integral part of the Stage 1 report (see also section 8 Products of the Evaluation).

7. Evaluation criteria and questions

The final evaluation criteria and detailed questions will be determined upon completion of the analyses in Stage 1 of the Evaluation. However, it is foreseen that the Evaluation will apply the common OECD/DAC criteria for evaluating development assistance. It will examine the relevance of the support provided, its effectiveness in terms of outputs, outcomes and impact (including the sustainability of the results) and its efficiency. In addition, the following particular aspects related to evaluating conflict prevention and peacebuilding activities will be taken into account: coherence, coordination, linkages, and coverage (see the OECD/DAC Guidance). Also it will be tried to ascertain the extent to which the support provided in this particular context has been able to take into account (follow) the Principles of the Paris Declaration on Development Effectiveness and the Principles of Good International Engagement in Fragile States.

This Framework Terms of Reference contains under each evaluation criteria a number of major questions. The questions below are preliminary and not exhaustive and will need to be further elaborated by the Evaluation Team during the course of the Evaluation. The Team may also formulate alternative questions on the basis of the outcome of their work during the first stage of the Evaluation.

Relevance

Was the support provided in line with the policy and procedures of the donors and agencies, with those of the GoSS, and the needs, priorities and rights of the affected populations as well as the dynamics of the post CPA-period?

The Evaluation will take into account the following issues:

  • At the level of policy development:
    • Interaction and consistency with the causes of conflict, key dynamics and driving factors, or key driving constituencies of the conflict (also taking into account the dynamics during the „post conflict situation´, i.e. following the signing of the CPA);
    • Interaction and consistency with post-conflict reconstruction and peacebuilding policy at the international level (e.g. OECD/DAC Principles of Good International Engagement in Fragile States), including responsiveness to new developments in such international policies;
    • Interaction and consistency with the policies and plans of the GoSS;
    • Interaction and consistency with the (changing) needs, priorities and rights of affected populations.
  • At the level of policy implementation:
    • Consistency of supported interventions with donor policies and examination of approaches and interventions in terms of conflict sensitivity and „do no harm´;
    • Provision and distribution of assistance based on (up-to-date) conflict analysis and assessment of needs, priorities and rights of affected populations;
    • Type of activities supported and modalities of implementation (channels, implementing partners, agreements);
    • Level of access secured to needy groups;
    • Flexibility of supported interventions / used instruments to adapt to changing circumstances affecting the needs and priorities of the population and the government.

  • In order to address the extent to which the developed policies and their implementation have taken into account the specific context in Southern Sudan with its longer-term and interconnected problems (connectedness) the following issues will be addressed:
    • Policies of donors and agencies and their collaboration to address the issue of linking relief, rehabilitation and development – addressing the gap between relief and development. The Evaluation will provide an opportunity to test the assumption that Southern Sudan is moving from relief to development and to better understand donor policies and how they have changed;
    • More particular in relation to the issue above: an analysis of decision-making processes to link humanitarian assistance, support for rehabilitation and development aid where appropriate (timeframes of assistance and the use of exit strategies);
    • The application of conflict analysis/analyses informing the choice and the design of interventions;
    • Institutional capacity building as part of the assistance provided, in order to create longer-term processes, structures and institutions for conflict prevention and peacebuilding in Southern Sudan.

Effectiveness

To what extent did the support provided achieve its purpose? And if it did not or not fully achieve its purpose, why not?

Issues to be addressed:

  • Realisation of the (changing) immediate needs of the affected populations on the one hand, and the Government of Southern Sudan in terms of institutional capacity building on the other (coverage and timeliness of support provided);
  • Provision and distribution of assistance taking into account gender and generation, including specific needs of women, children and the elderly;
  • Influence of and response to security issues and humanitarian access;
  • Attention to be paid to good and less effective practices.

Impact

What have been the wider effects of donor support to the implementation of the CPA in Southern Sudan?

Wider effects, also called impact, can be immediate and longer term, intended and unintended, as well as positive and negative. The Evaluation will try to establish the immediate and medium-term wider effects of the support provided.

The following issues will be addressed:

  • Effects of the assistance in terms of reducing the vulnerability of the affected populations and fostering preparedness and people's coping mechanisms;
  • Effects of the assistance in terms of livelihood development;
  • Effects of the assistance on government capacities and the development of the private sector;
  • Effects of the assistance in terms of reducing the chances of a relapse into conflict (i.e. addressing the root causes of conflict) and fostering preparedness of the Government of Southern Sudan and other stakeholders (e.g. civil society) to address the causes of conflict;
  • Effects of the assistance on the wider peacebuilding environment in Sudan.

Sustainability

To what extent are the accomplished results sustainable?

Issues to be addressed:

  • The effects of the present financial and fiscal conditions created inter alia by the (current) low(er) incomes from oil revenues on the capability of the Southern Sudanese government to maintain and/or increase current levels of service to the population;
  • The extent to which these financial and fiscal conditions have an effect on the current development planning of the GoSS;
  • The sufficiency of current and expected future funding levels of external support to ensure longer-term sustainability of the expanded systems of social services, governance and economic activity.

Efficiency

Were the financial resources and other inputs efficiently used to achieve results?

Issues to be addressed:

  • Aid management (funding mechanisms applied, programme and project cycle, staffing, tasks and responsibilities of ministry departments and embassies, inter-ministerial co-operation including – where appropriate – civil-military co-operation);
  • Criteria used in the selection of implementing partners (comparative advantage or other);
  • Organisation and costs of aid delivery at field level (diversion, security, creating access);
  • Use of monitoring of progress and achievements for programming, learning and accountability.

Coherence, coordination, linkages, and coverage

The Evaluation will pay special attention to four particular aspects related to the relevance, effectiveness and efficiency of the support provided, namely its coherence, coordination, linkages,and coverage.

Coherence

In this Evaluation coherence will be treated as comprehensively as possible, i.e. covering humanitarian, development, security and diplomatic efforts of the different actors. Assessment of coherence should focus on the extent to which different policies of actors were coherent. In other words, were humanitarian, development, security and diplomatic efforts of individual actors in line with each other? Also, were policies of different actors coherent, complementary or contradictory?

Issues to be addressed:

  • The extent of coherence between donor and agency policies and programming at field level and those of other actors;
  • Coherence of the different elements of support with the objectives and the process of implementation of the Comprehensive Peace Agreement (CPA);
  • Coherence with policies and interventions other than those aimed at supporting the implementation of the CPA (e.g. ongoing humanitarian support);
  • Possible effects of changing and diverging interests.

Coordination

The following issues will be addressed:

  • The effectiveness of coordination at policy and strategic levels and during implementation;
  • Involvement of donors in coordination mechanisms and processes;
  • Encouragement of operational partners to engage with coordination mechanisms and processes;
  • Trade-off between the need for coordination and local ownership (Principles of Good International Engagement in Fragile States), and the need for quick impact and peace dividend;
  • And more generally, the extent to which the support provided in this particular context has taken into account the principles of the Paris Declaration, i.e. ownership of the recipient government, alignment with policy and procedures, donor (and agency) harmonisation, managing for results, and mutual accountability.

Linkages

The following issues will be addressed:

  • The relative emphasis on and balance between the different types of support provided i.e. humanitarian, rehabilitation and reconstruction including security sector reform, protection and establishing the rule of law, and general socio-economic development;
  • The sequence of support in relation to the changing circumstances (appropriateness of interventions in time and space);
  • Links between specific peacebuilding interventions and longer-term development processes.

Coverage

The following issues will be addressed:

  • The inclusion or exclusion of certain geographical regions and its impact on the ability to reach all those in need;
  • Coverage in relation to the dynamics of conflict prevention and peacebuilding.

8. Products of the Evaluation

The Evaluation Team will prepare and submit several products on the dates specified in the projected time schedule presented in section 10 and will periodically report on progress and findings to the Evaluation Management Group and the Evaluation Steering Committee.

The Evaluation Team will produce the following outputs (deliverables):

  1. Inception report consisting of an updated evaluation work plan (draft and final report);
  2. Report of Stage 1 (conflict and policy analyses, overview and analysis of donor support, and analysis of evaluations, reviews and other documentary evidence) including a detailed design (ToR) of Stage 2 of the Evaluation (draft and final report);
  3. Presentations, seminars, and workshops, as appropriate;
  4. Standardized database of documents reviewed;
  5. Report of Stage 2 including an assessment of interventions at field level (draft and final report);
  6. Final report synthesising outcomes of Stage 1 and 2 (draft and final report);
  7. After completion of the Evaluation and on request, presentations, seminars, or workshops for particular donors and agencies (to be funded separately);
  8. Presentation of the evaluation results at a seminar in Juba (and possibly Khartoum).

9. Organisation of the Evaluation

The Evaluation´s governance structure should allow appropriate involvement, cooperation and ownership of the main stakeholders whilst safeguarding the independence, credibility and quality of the Evaluation, and ensure an effective and efficient evaluation process. Common to the practice of joint evaluations, three levels of governance will be applied:

Evaluation Steering Committee

To assure broad participation in the conception and oversight of the Evaluation, an Evaluation Steering Committee has been constituted, representing stakeholders with a strong interest in the Evaluation and actively participating in it.[18] The Steering Committee is co-chaired by the Director of the Policy and Operations Evaluation Department, Netherlands Ministry of Foreign Affairs and the Director Aid Co-ordination, Ministry of Finance and Economic Planning, Government of Southern Sudan. The Steering Committee will not engage in micro management, but will convene at critical junctures of the Evaluation for review, discussion and oversight. Its main tasks are to endorse the Framework Terms of Reference and the Terms of Reference for Stage 2 of the Evaluation, and review draft reports of the Evaluation regarding quality, credibility and clarity.

The co-chairs of the Steering Committee will take steps to inform the Government of National Unity (GoNU) about the Evaluation and explore possibilities for cooperation.

Evaluation Management Group

An Evaluation Management Group is responsible for the ongoing management of the Evaluation. The Management Group will directly oversee the work of the Evaluation Team contracted to execute the study, inform the Steering Committee about progress and prepare meetings of this Committee. An important role entails close, efficient and regular liaison with the Evaluation Team. The Evaluation Management Group consists of the Policy and Operations Evaluation Department, Netherlands Ministry of Foreign Affairs (Chair), the Evaluation Department of Danida (Ministry of Foreign Affairs of Denmark) and the Office of Evaluation (OEDE) of the World Food Programme. The Management Group is supported by experts from the Conflict Research Unit of the Netherlands Institute of International Relations „Clingendael´, which were involved in writing the approach paper for the Evaluation. The experts may be called upon to facilitate meetings, to play a catalytic role in bringing new and emerging issues to the attention of the Management Group and to provide advice on substantive issues and controversial debates. Furthermore, they will also be engaged in reviewing drafts of evaluation products and may provide direct feedback to the Evaluation Team. They will not be involved in the conduct of the actual Evaluation.

Reference Group in Southern Sudan

In order to maximise participation at the local level, provide advice, and give credibility, legitimacy and support to the Evaluation, the Ministry of Finance and Economic Planning of the GoSS has established a Reference Group. The Ministry of Finance and Economic Planning will also chair this Group. The Reference Group has a broad representation and involve institutions of the GoSS, development partners active in Southern Sudan including international and national NGOs or their umbrella organisation(s), and possible other interested parties such as researchers.

The role of the Reference Group is to review key documents of the Evaluation (the draft Framework Terms of Reference, the draft report of Stage 1 and the draft Final Report). It also serves as a „sounding board´ and „facilitator´ for the Evaluation Team during its work in Southern Sudan. The principal functions of the Reference Group are: i) to facilitate access to documents and personnel for the purpose of the evaluators; ii) receive, discuss and provide feedback on intermediate evaluation products; and iii) assist in the organisation of appropriate workshops or seminars during the course of Stage 2 of the Evaluation. The Reference Group will meet with the evaluators at critical points during Stage 1, as well during the fieldwork phase in Stage 2 (fieldwork inception and feedback of preliminary results).

Upon completion of the Evaluation a seminar will be held in Juba involving the Reference Group and other stakeholders to discuss the final report of the Evaluation.

10. Time schedule

It is anticipated that the Evaluation will start in November 2009 and be completed by August 2010, as indicated below.

Activity Completion date
Contract award to Evaluation Team 26 October 2009
Start of Stage 1 First week of November 2009
Submission of draft inception report to the Evaluation Management Group Fourth week of November 2009
Submission of draft report of Stage 1 to the Evaluation Steering Committee First week of January 2010
Steering Committee meeting to discuss draft report of Stage 1 Third week of January 2010 (date and place to be decided)
Start of Stage 2 Second week of February 2010
Fieldwork February-April 2010
Submission of draft fieldwork report to the Evaluation Steering Committee for review and comments (written procedure) Last week of April 2010
Preparation of final report May 2010
Submission of draft final report to the Evaluation Steering Committee First week of June 2010
Steering Committee meeting to discuss draft final report Last week of June 2010 (date and place to be decided)
Preparation final report and its clearance by the Steering Committee (written procedure) July 2010
Publication of final report August 2010
Dissemination seminar in Juba (and possibly Khartoum) Last week of September 2010 (date to be decided)

11. Evaluation Team

The Evaluation Team will be based in an established institution or consulting enterprise with a demonstrated track record in evaluation or evaluative research in complex conflict and peacebuilding contexts. It will consist of a group of evaluation professionals with experience of international development cooperation in complex conflict and peacebuilding contexts, led by a recognised and experienced expert. The organisation of the Evaluation is the responsibility of the Team and should be specified and explained clearly in the proposal.

The Evaluation Team will have to be composed in a way that it is able to undertake the tasks outlined in Stage 1 and the subsequent tasks to be undertaken in Stage 2.

Team composition

The Evaluation Team should consist of four to five international consultants as core team members and a number of associated team members for specific tasks. International consultants are defined as persons with an international background, e.g. with a substantial part of their professional experience from assignments in developing countries.

The team must contain:

  • Evaluation expertise to conduct a multi-donor evaluation under difficult circumstances including familiarity with all standard evaluation approaches, qualitative as well as quantitative methods of data collection, knowledge of OECD/DAC and other internationally agreed guidance and standards for evaluation;
  • Evaluation experience in complex conflict and peacebuilding contexts including capability to conduct field work;
  • Knowledge of evaluation design and methodology;
  • Peacebuilding expertise, i.e. theoretical knowledge and operational expertise of different peace processes on all tracks (diplomatic initiatives, civil society initiatives, grassroots groups, etc.), and knowledge of peacebuilding through development strategies (policies and programmes);
  • Human rights expertise including knowledge of internationally agreed standards and mechanisms, etc;
  • Security expertise i.e. knowledge of and experience with issues such as policing, the rule of law, security sector reform and disarmament, demobilisation and reintegration (preferably in the Southern Sudanese context);
  • Development expertise including (i) expertise on development policies and their effects on peacebuilding and state building, as well as conflict-sensitive operational strategies; and (ii) knowledge of and experience with aid and development effectiveness including the Paris Declaration and the current aid architecture;
  • Governance and fragile states expertise in particular institution building and strengthening of governance in complex and/or fragile situations;
  • Adequate knowledge of the situation in Sudan, and in Southern Sudan in particular;
  • Should have both sexes represented in the team and;
  • Also include one or more evaluators from Sudan and/or the African region.

Team qualifications

General qualifications of all core team members:

  • Advanced academic degree (minimum MA);
  • At least 10 years of experience with international development assistance;
  • Experience with evaluation and/or (evaluative) research in complex conflict and peacebuilding contexts including capability to conduct field work (three references);
  • Advanced knowledge of evaluation methodology;
  • Good writing and communication skills;
  • Fluency in English.

Specific qualifications covered by one or more core team members:

  • Peacebuilding expertise, i.e. theoretical knowledge and operational expertise of different peace processes on all tracks (diplomatic initiatives, civil society initiatives, grassroots groups, etc.), and knowledge of peacebuilding through development strategies (policies and programmes);
  • Security expertise i.e. knowledge of and experience with issues such as policing, the rule of law, security sector reform and disarmament, demobilisation and reintegration;
  • Development expertise including (i) expertise on development policies and their effects on peacebuilding and state building, as well as conflict-sensitive operational strategies; and (ii) knowledge of and experience with aid and development effectiveness including the Paris Declaration and the current aid architecture;
  • Governance and fragile states expertise in particular institution building and strengthening of governance in complex and/or fragile situations;
  • Adequate knowledge of the situation in Sudan, and in Southern Sudan in particular (at least two core team members);
  • Workshop facilitation.

Specific qualifications of the Team Leader:

  • At least 15 years of experience with international development assistance;
  • Evaluation experience in complex conflict and post-conflict contexts including situations requiring a wide variety of conflict prevention and peacebuilding activities;
  • At least three references as Team Leader for multidisciplinary evaluation teams;
  • At least three references of experience in evaluation of development assistance at policy and programme level;
  • Experience with conducting complex (joint) evaluations;
  • Excellent writing and communication skills;
  • Fluency in English.

Specific qualifications to be covered by one or more associated team members:

  • Knowledge of gender issues;
  • Human rights expertise including knowledge of internationally agreed standards and mechanisms, etc;
  • Knowledge of countries in fragile situations;
  • Knowledge of evaluation design and methodology;
  • Experience with literature search and analysis;
  • Good writing skills;
  • Fluency in English;
  • Adequate knowledge of the situation in Sudan, and in Southern Sudan in particular.[19]

It should be clearly stated which of the proposed team members cover which of the above qualifications.

12. Expected level of input of the Evaluation Team

The total expected (minimum) input amounts to some 923 person days. It is expected that the level of input for Stage 1 will be around 330 person days (core team including associated team members/research assistants). The expected level of input for Stage 2 is about 480 person days (core team and associated team members). The expected input for the synthesis phase amounts to some 105 person days (core team and associated team members). The input of the core team to disseminate the evaluation results is estimated at 8 person days. As different skills may be needed at different points in time, a high degree of flexibility is required. It is expected that the Team Leader or Deputy Team Leader will be available throughout the duration of the Evaluation i.e. November 2009-August 2010.

The total cost (fees and reimbursables) should not exceed Euro 765,000 (this amount includes a contingency of 5%).

References

1.Mariska van Beijnum and Jort Hemmer (March 2009). Approach paper for a multi-donor evaluation of conflict prevention and peacebuilding activities in Southern Sudan, 2005-2008, Conflict Research Unit, Clingendael Institute, The Hague.

NB. The Approach Paper is attached to the letter of invitation to Tenderers for information.

2. Organisation for Economic Cooperation and Development (2008). Guidance on evaluating conflict prevention and peacebuilding activities – Working draft for application period. Development Assistance Committee, Paris.

NB. The Guidance on evaluating conflict prevention and peacebuilding activitiescan be obtained at <http://www.oecd.org/dac/evaluationnetwork> accessing the publications, documents and guidance section.

Revised Stage 2 Terms of Reference

Multi-Donor Evaluation of Support to Conflict Prevention and Peacebuilding Activities in Southern Sudan, 2005 – 2009

16th February 2010

Introduction

The objectives for the evaluation as a whole are as follows:[20]

  • Through the use of standard OECD/DAC evaluation criteria, assess as systematically and objectively as possible the extent of progress made by the international community in supporting conflict prevention/peacebuilding and in providing peace dividends to the Southern Sudanese people. This will include pointing out the factors driving success or failure; and highlighting lessons accordingly.
  • Provide input into ongoing discussions and future policies/plans on how to improve the relevance, efficiency, effectiveness, and – above all – impact of the international engagement in peacebuilding processes in Southern Sudan in the run-up to 2011 and the post-2011 period.
  • Pilot the working draft of the Guidance on evaluating conflict prevention and peacebuilding activities (OECD/DAC Network on Conflict, Peace and Development Cooperation and the OECD/DAC Network on Development Evaluation).

The evaluation is independent and efforts have been made to ensure: (a) there is no conflict of interest between the chosen team and the tasks, (b) a rigorous procedure of triangulation and sourcing of evidence is in place, and (c) the evaluation structure (Steering Committee, Reference Group, Peer Group review) provides checks and balances to this effect.

From Stage 1 to Stage 2

The TORs for the Evaluation,[21] in line with the OECD/DAC Guidance,[22] suggest an incremental approach to the evaluation. There is a logical sequence in the thinking and presentation that entails:

  1. A Conflict Analysis that identifies the key factors relating to conflict and the linkages between them, pointing to sources and dynamics of conflict (and conflict mitigation) as well as peace (and peacebuilding). Such an analysis provides a starting point for assessing the extent to which conflict understanding and sensitivity has been (and will be) applied by donors at strategic and programmatic levels in Southern Sudan.
  2. A Donor Policy Analysisthat examines the objectives and intervention strategies underlying their support to Southern Sudan, and the extent to which they have applied a „conflict sensitive´ approach to these strategies and programmes. What was the framework for donors´ initial and subsequent approaches, and how has this evolved over time? To what extent has there been coherence and consistency of approach between donors and has this strengthened collective outcomes?
  3. An Aid Portfolio Analysis that provides the facts, figures and trends of donor support over the five year period (presented on an annual basis), enabling the evaluation to comment on the coherence of policies as well as the actual support provided. This overview will be established on the basis of the OECD/DAC CRS-codes commonly used to identify substantive categories of ODA; efforts will then be made to re-allocate these along the lines of the four CPPB categories.[23]
  4. Finally, an Analysis of Evaluation Reports, review reports and project/programme completion reports and other types of source material including academic and applied research. This provides an overview of the type and scope of evaluations to date, the key findings that emerge from these studies, and a comment on where gaps exist.

The combination of the above elements provides firstly a baseline for the evaluation, and secondly an opportunity for the team to extract several findings and assumptions to test in Stage 2. This will not be a comprehensive list, since inevitably others will emerge in the course of the field work, but rather a starting point and focus for the evaluation as it examines more closely various programmes to answer the key questions of the evaluation:[24]

  • Relevance: Was the support provided in line with the policy and procedures of the donors and agencies, with those of the GoSS, and the needs, priorities and rights of the affected populations as well as the dynamics of the post CPA-period?
  • Effectiveness: To what extent did the support provided achieve its purpose? If it did not (or only partially) achieve its purpose, why was this?
  • Impact: What have been the wider effects of donor support in supporting a climate of peace in Southern Sudan and to the implementation of the CPA in Southern Sudan?
  • Sustainability: To what extent are the accomplished results sustainable?
  • Efficiency: Were the financial resources and other inputs efficiently used to achieve results?
  • Coherence: Were humanitarian, development, security and diplomatic efforts of individual actors in line with each other? Were policies of different actors coherent, complementary or contradictory?
  • Coordination and linkages: Were the coordination mechanisms chosen by donors effective? To what extent was there adherence to the Paris Declaration Principles? What was the relative emphasis on and balance between the different types of support provided (humanitarian, rehabilitation and reconstruction), and was the sequencing of intervention appropriate?

Annex 1 presents the Evaluation Matrix, which sets out the TOR questions plus supplementary and more detailed questions developed by the evaluation team. Based on this, we have developed a „reporting template´ (Annex 2) which provides the specific questions and focus for the fieldwork teams. Most importantly, this reporting template gives the fieldwork teams a common structure for the Stage 2 reporting, as well as a basis from which they can develop checklists of questions to apply to each programme being assessed. These checklists will be developed by the fieldwork teams for interviews and focus group discussions, as they will be tailored to the context and programme being assessed.

The TOR (page 31) states that Stage 2 is an, “in-depth evaluation of conflict prevention and peacebuilding interventions supported in Southern Sudan with emphasis on field level assessment”. Thus, Stage 2 takes the entire team to Southern Sudan to field test the arising assumptions from Stage 1, derive greater in-depth knowledge of specific activities funded by aid donors, and evaluate these through a Conflict Prevention/Peacebuilding „lens´. The sample of activities will not be representative in a strictly quantitative sense, but will be a purposive and indicative selection of activities from which broader lessons will be drawn. Following this, a Final Report will compile and summarise findings from the field work, Stage 1 literature and further in-depth discussion with stakeholders at all levels.

General Themes – overriding narrative

Given that several thousand interventions are potentially covered by this evaluation over the five-year period (2005-2009), the rationale for selection is deliberately based on fulfilling the overall TOR questions, while also „ground-truthing´ and testing assumptions that emerge from the Stage 1 findings. This approach does not attempt to cover all types of programmes or geographic areas per se, but rather the projects and locations that can best provide evidence to answer a set of evaluation assumptions drawn from the Stage 1 analysis. Each team will follow up on the Stage 1 preliminary set of findings and assumptions specific to their expertise (outlined in the sections below). In addition to these, we have drawn some a priori assumptions from Stage 1. These form a „narrative´ that runs across all sectors to be covered in Stage 2, and which will be tested in the field and revisited when the Final Report is drafted.

  1. Underlying much of the discourse among donors is an assumption that peacebuilding equates with central state building [Stage 1 report, p2]. The state building agenda may have been at the expense of an appropriate concentration of resources (financial and other) on issues of poverty and marginalization in the population at large. It could also be that donors have wilfully avoided or de-emphasized questions of legitimacy of the state itself in Southern Sudan. If so, what implications does this have for peace in a highly volatile political climate? Central to the issue of state building in Sudan is that of the rule of law (the fact that nobody can be placed above the law) has gathered a large amount of resources. Justice and security services are required in outlying areas as well as in the cities. We will examine how the rule of law (or its absence) has affected the wider population. Similarly we will review how decentralisation has taken place, and whether this has contributed to a presence and effectiveness of the state at the local level. This will in particular be tested by Team 1 (see below).


  2. The emerging findings from Stage 1 [Stage 1 report, p25] show that there has been a lack of joint diplomatic and developmental approach between donors, as well as a disjuncture between the two. This division between politics and aid derives from the traditional separation of the two areas within ministry structures but also from the difficulty of merging and harmonising donor countries´ political relationships with Sudan. In Southern Sudan it means that there has been a failure to engage with fundamental political issues, particularly at local levels, and to design aid programmes that help mitigate rather than exacerbate conflict. This applies particularly to conflicts related to land and natural resources. Experience in other putatively “post-conflict” contexts suggests that lack of political cohesion among leaders and their political factions can lead to renewed conflict, although power-sharing is hardly sufficient for a sustainable peace. There is a notable absence of an overall framework to deal with such problems. Significant areas of sampling to identify evidence concerning this hypothesis will be collected in areas relating to donor coordination, to SPLA reform, and to the natural resource governance sector. This will be covered by teams 3, 5 and 6.


  3. A comprehensive strategic plan for recovery and development has been very late in coming despite the fact that a number of assessments were made in advance of and after the signing of the CPA (e.g. the Joint Assessment Mission), and despite the UN drawing up annual work plans since 2005. This is particularly apparent in Southern Sudan where the government has been working to a budget sector planning approach (through the Budget Sector Working Groups, BSWG) strongly supported by the international community [Stage 1 report, p31]. The result has been some ten budget sector plans for 2008 to 2010. As government institutions struggle to fulfil a wide range of obligations, decision-making is more aligned to operational planning concerns than overarching strategic ones, perhaps informed by a general sense that everything is needed which means that nothing is particularly prioritised. Of particular relevance is the scale and cost of infrastructure programmes needed for all areas. Infrastructure issues will be covered by team 3.


  4. Civil strife is part of a pattern of violence where the Sudanese state (both North and South) -as a vehicle for special interest groups - has played a major role [Stage 1 report, p15, for instance]. The legitimacy of the state in Southern Sudan is not just the establishment of state institutions, but also the building of trust and respect for those institutions. There is a danger that without appropriate checks and balances towards the apparatus of state (judiciary, police, etc), predatory modes of behavior becomes part of the problem rather than the solution. These areas of focus will also be complemented by a review of the notion of capacity building and political space. This hypothesis will particularly be tested by teams 1, 3 and 4.


  5. The country suffers from the combined effects of two sets of crises that are closely interrelated: (a) a crisis of governance, and (b) a livelihoods crisis [Stage 1 report, p8]. The complexity of Southern Sudan should have led planning processes and assistance organisations to incorporate conflict sensitive approaches regardless of whether they are directly addressing conflict issues in their work. This seems not generally to have been the case so far [Stage 1 report, p16]. Team 2 will concentrate on the activities of socio-economic programmes in outlying areas.


  6. Despite the existence of a mandate which authorises UN peacekeeping troops in Southern Sudan „to protect civilians under imminent threat of physical violence´, a narrow interpretation and a paucity of troops have combined to confine the blue helmets to monitoring the implementation of the military aspect of the CPA only. One consequence is that local communities remain largely unprotected.

Rationale for choice of programmes examined in Stage 2

The aid portfolio analysis undertaken in Stage 1 included hundreds of composite programmes, pooled fund programmes and individual projects undertaken over a five year period. It would be impossible for the evaluation team to adequately examine a statistically representative sample from these; therefore, we have chosen to select a purposive sample informed by:

  1. Preliminary discussions with major stakeholders over what, in their opinions, constituted the key drivers of conflict and factors in conflict management in Southern Sudan and their net effect for a „peace dividend´. These were outlined in the evaluation Inception Report (page 17-18) and included infrastructure (particularly roads), demining, security sector reform and judiciary, SPLA transformation/reform, social sector (particularly health and education), and reintegration/return of IDPs and refugees (including land issues).
  2. Central issues emerging from the Stage 1 literature. In addition to the above, these included the added value (or not) of programmes under pooled funds, security and governance at local levels, challenges of decentralised government and capacity, ownership and responsibility in service delivery, local peacebuilding and Rule of Law.
  3. The extensive professional experience and knowledge of the team itself in Southern Sudan over the period before and since the CPA signing, as well as of lessons learned from a number of other post-conflict and fragile state settings. The team are able to draw on years of research and field experience of the country, and in themselves represent a unique resource and pool of knowledge.

In so far as the heart of the analysis is the interplay between the four CPPB categories – socioeconomic development, good governance, reform of justice and security institutions, and culture of justice, truth and reconciliation – a selection of programmes will be taken from each one of these. However, the choice will not be determined by the proportionality of donor support to each. Some 80% of funding has gone to the first (socio-economic), yet it could be that relatively much smaller community security programmes, for instance, have proportionally greater impact on peacebuilding.

Notwithstanding logistics and time constraints, the purposive sample of programmes chosen is not ad hoc. Conflict mapping in Southern Sudan suggests certain „flashpoints´ over time – often area and/or tribally specific. Donors themselves have been aware of this. Hence, there has been a concentration of resources in conflict-prone areas such as Jonglei[25] and Upper Nile. Likewise, patterns of migration and return have determined where and when a greater percentage of social service resources are allocated.

Two final variants are taken into account – horizontal programming (the extent to which a „set´ of interrelated programmes was chosen in one area) and vertical programming (the extent to which a programme addressed relationships and challenges from community to local and State levels, and therefore how sustainable it became). The former presents an interesting counterfactual: what differences can be seen between areas where efforts have been made towards programme saturation and those where programmes have been far more ad hoc? There will be obvious methodological constraints here relating to scale of implementation, different funding sources and the timing of interventions which means that impact measurement is not yet possible.

Methods

There are essentially two frames of reference for Stage 2: the Evaluation Matrix (Annex 1) and the CPPB Guidelines.[26] The Evaluation Matrix provides the overall framework for the evaluation by making a clear link to the TOR (MoFA 2009). Based on this, the evaluation team has developed a reporting template (Annex 2) that gives a consistent structure for the Stage 2 fieldwork reports. Based on their own professional judgement, the field teams will develop specific checklists of questions for undertaking key informant interviews and focus group discussions. Additional questions specific to the sectors/programmes under review will be added by the consultants in the course of their field work.

As set out in the previous section, the assumptions drawn from the Stage 1 Report will be ground tested; they are not exhaustive, but they provide a „cornerstone´ for an evaluation that otherwise risks being either too general or too fragmented as it examines a huge number of inputs and programmes over a five-year span. While the selection of interventions to visit is based primarily on testing these assumptions/ hypotheses, other variables were also taken into account, namely:

  1. The interplay and overlap of CPPB categories: Socio-Economic Development, Good Governance, Reform of Justice and Security Institutions, and Culture of Justice, Truth and Reconciliation.
  2. Geographical variance in Southern Sudan – the importance of context, place, and time.
  3. The „clustering´ of activities and the extent to which they were interrelated, adequately sequenced and had outcomes greater than the sum of their parts.

There will be other variables – access and logistics, whether the programmes are still „active´, etc – but it is important for the teams to look at the various sectors as a whole, then to use specific field examples to attain context-specific knowledge and test broader assumptions. Key questions will include: What difference has the absence or presence of such services made in terms of conflict prevention and peace building? To what extent has vertical and horizontal sectoral coordination been possible, and what are the linkages with other relevant programmes such as those aimed at developing capacity at the State level and below?

Stage 2 will not be a comprehensive evaluation of each activity under review; indeed, in some cases this will already have been commissioned elsewhere. Rather, it is an opportunity to review the activity using the specific tools and questions outlined above, cross-referencing these with findings drawn from the project/ programme literature. However, we note in the Stage 1 Report that there have been very few cross-sectoral evaluations and that even the agency-specific monitoring and evaluation reports often do not address their own objectives in relation to peacebuilding and conflict prevention.

Mixed methods will be used at field level:

  • Review of project documentation, including any additional evaluations which have become available since the Stage 1 report was submitted.
  • Interviews with relevant government officials, donors, NGOs, funding mechanism managers at Juba, state, county and payam levels. As far as possible these will be arranged in advance.
  • Focus group discussions in the project areas with communities (men and women separately, when possible – disaggregating further as appropriate – e.g. returnees/host communities; tribe x/tribe y, etc).
  • Direct observation combined with interviews at project sites.
  • Triangulationof information obtained from all sources.

Data collection/analysis

The teams will benefit from having experienced local consultants who will where necessary be able to follow up on some issues not covered during the field work. At the same time, we have research assistants in Juba throughout Stage 2 able to track down additional project literature and interview project staff. There will be constant discussion and information exchange between the teams both during and after the field work, and each team member will present their preliminary findings to key stakeholders before departing Juba/Khartoum.

In general, the teams will spend some initial time in Juba, conducting interviews, gathering monitoring and other information, as well as confirming logistical arrangements. The teams will then spend 7-10 days outside Juba working with project staff, visiting locations and conducting focus group discussions and interviews. After the field visits, the teams will spend additional time in Juba, to follow-up on „information gaps´, conducting interviews, sharing findings with the other teams, and debriefing as appropriate.

It is important to emphasize that the teams are not conducting evaluations of individual projects per se, but rather gathering evidence in relation to broader assumptions derived from Stage 1 and analysis of CPPB interventions that is in line with the overall TOR. To this end, the site visits have been „clustered´ around specific sectors. The teams can split up to cover additional project sites, and they will make use of existing knowledge and monitoring information before visiting the sites. The focus is thus more on „ground truthing´ existing knowledge (from the Stage 1 literature review, annual and evaluation reports, etc) and testing the emerging assumptions – rather than conducting a series of individual project evaluations.

In the following sections we outline the areas and activities to be further developed by the four core teams. The arrangements are still unfolding, so certain details will change, mainly as a result of logistical constraints.

Team composition, themes and geographical locations

There are 6 teams, with the 4 core teams having three consultants each. Each team has a set of specific sector responsibilities and locations. This means that they will cover the sector as a whole in Southern Sudan, and with ground-tested examples in particular locations.

Team 1. Rule of Law, Decentralised government, Local peacebuilding
  Site visits: Upper Nile, Jonglei
Team 2. Recovery & reintegration, Socio-economic development, basic services, livelihoods, pooled funds, NGO projects.
  Site visits: Lakes, Warrap, Northern Bahr el Ghazal
Team 3. Infrastructure, Local Governance, Land, Farmer-pastoralist conflicts.
  Site visits: Eastern and Western Equatoria
Team 4. Urban-Rural issues, Demining, Capacity Building overall strategy.
  Site visits: Central Equatoria
Team 5. SPLA reform programme, Donor Coordination, (and Reference Group).
  Site visits: Juba
Team 6. Oil Issues, GoNU-GOSS analysis, 3-Areas analysis, Donors not covered in Juba.
  Site visits: Khartoum

Figure 1 summarises timings, agencies, etc. The broader analysis will include interviews and document collection at Juba/ Khartoum/capitals level – each core team will spend about a third of their time in Juba itself.

Teams 1-4 are the „core´ teams who will travel to a total of eight States. We have attempted here to cover the most important CPPB categories (and sub-categories), accepting that the crosscutting issues (gender, environment, capacity building) will be included in each assessment. In addition, there is a gender specialist who, as well as joining one team, will review all inputs and conduct a series of separate investigations in Juba.

Risks

The activities and locations chosen below are subject to change on almost a daily basis as the team acquires greater knowledge, but most importantly due to logistics and security constraints.[27]It is extremely difficult to make final arrangements without having a team member on the ground, so some plans will be subject to last minute change. In particular, confirmation of flights to State capitals is uncertain, but we have a full-time logistics officer in place to assist the teams.

Visiting activities on the ground depends on the goodwill and assistance of the implementing agency. There has been a great deal of forward planning in this respect, including the understanding that, as an independent evaluation, the team will choose its own list of informants without undue interference.

However, the selection and location of „beneficiary´ focus groups will depend on project staff to arrange this in advance. If there is a level of bias, it is likely to be urban/ rural and (probably) gender-related, as well as tilted toward the best examples of programmes. Every effort will be made to overcome these, such as (for example) arranging separate meetings for women and speaking with local observers who are not only beneficiaries of programmes.

Projects underway will be easier to review than those which have closed. Methodologically, one way around this – and in addressing the programme „cluster´ issues as opposed to specific projects – is to arrange several „general population´ focus groups and ask generic questions based on changes in access to services, security, justice and broader governance over the five year period. Standard PRA techniques might be employed (before-after wealth ranking, perceptual comparisons), but the precise nature and dynamics of these meetings cannot be known in advance. It will therefore be necessary for the teams to be highly flexible in the methods used, accepting that these will not be consistent across each sector/area.

Finally, there is the known constraint that access to individuals, particularly government officials, will be confounded by every day demands on their time and the added intensity of activities around the forthcoming April elections.

Click to see the table: Figure 1: Summary of field teams''

TEAM 1

Team 1 will focus on three issues, in Upper Nile and Jonglei, with particular programmes as follows:

  • Rule of Law.UNDP Institutional Support to the Judiciary, Support to Police and Prison Service, Promoting Access to Justice and fostering a culture of human rights, DFID UK Justice Sector Programme, PACT Promotion of Community Security Power Sharing (decentralised government). UNDP Local Government Recovery Program. Support to the States[28]
  • Peace-building.PACT Early Warning Posts, Enhancing Peace and Community Stability, People to People Peace-building, Roads for youth and peace, Water for recovery and peace programme
  • Crosscutting.UNDP Good Governance and Equity in Political Participation

Competition over natural resources combined with widespread ownership of small arms is fuelling violence between Southern Sudan´s many tribes. Traditional flashpoints have also become more dangerous as jobless youths are disaffected by the lack of development in the South. Because the livelihood base of civilians has been frequently targeted, local communities are vulnerable to manipulation and easily co-opted by armed groups. As a result civilians are armed to protect themselves and both the spread of small arms and communal conflicts has increased. For example, despite the gains recently made by CSAC, the UN has failed to develop a strategy for engagement in the Jonglei crisis, including how to support arms control and disarmament exercises and mobilize the necessary expertise and capacity.

Given the huge number of small arms in circulation, the GOSS has understandably concentrated on civilian disarmament. But as long as communities do not have confidence in government security forces to provide for their safety or to address their grievances, civilian disarmament – whether peaceful or forcible – will remain ineffectual.

The Southern Sudan Police Service (SSPS) was created in 2005, and its capacity remains extremely limited. Its rank and file comprises former SPLA soldiers; they are overwhelmingly illiterate and lack appropriate training. With the SSPS being poorly equipped, in some areas civilians are often better armed than the police. The SPLA is thus often relied upon to intervene in cases of localised insecurity, yet their discipline (as well as that of the police) is a cause for concern.

The absence of, or mistrust towards, rule-of-law institutions (courts, prisons) means that communities are more likely to resort to violence to resolve disputes. There is a tension between the state role in formal Rule of Law and traditional forms of justice. Indeed, the state is often not viewed as a legitimate actor in this sphere.

Local peace building initiatives are often more successful when linked to the provision of basic services and tangible „peace dividend´ resources. The most effective programmes are those that have promoted linkages with the wider governance and security environment, including building capacities in local government. Related to this is the possibility that there has been too much focus on the central institutions of GoSS and not enough on building up democracy in States (and perhaps Counties). Many programmes have been unrealistic, with faulty assumptions and over-ambitious objectives, suggesting that donor policies and programmes have not made adequate use of the contextual research and analysis in which they have invested. This, and the failure to take a synchronized and strategic approach to capacity building across levels of government, has undermined effectiveness and sustainability.

TEAM 2

Team 2 will focus on the contribution of socioeconomic-related projects and programmes on conflict prevention and peace building in Southern Sudan with a particular focus on basic services (health, education, water and sanitation) and livelihoods support implemented within the context of broader reintegration and recovery programmes and strategies. Three States will be covered (the projects are still to be confirmed):

Northern Bahr el Ghazal  
Tearfund CIDA-funded: Omdurman, Aweil East. 3-year Integrated nutrition, food security, health, water and sanitation programme.
Canadian Food Grains Bank/World Relief Canada – 3 year nutrition and food security programme (Health component handed over to IRC in January 2009, who is funded by USAID through MSI)
   
IRC USAID funded health
Save the Children SRF- funded - Aweil Town East, Agricultural training support, income generating scheme and peace building initiatives
   
Mercy Corps EC-funded: Integrated Food Security (Aweil)
Concern BSF-funded health – Nyamlel
   
Lakes  
VSF-B EC-funded: Livestock and integrated rural Development (Rumbek)
Oxfam Livelihoods programme (Cuibet)
GTZ Livestock Production Marketing (Rumbek)
DOR BSF-funded education
   
Warrap  
WVI USAID-funded: Socioeconomic Development – Kuajok, Luanyaker, Tonj; SRF funded agricultural training & support
ADRA Danida-funded: Health and Water (Twic).
VSF-G EC/UNDP – RRP (Kuajok – ended).
AMA BSF-funded education and health IRD
SRF-funded agricultural cooperatives, small business development, Gogrial West

Sectoral and geographic focus

Within each of the three contiguous states, the team will look at relevant work funded through bilateral and pooled funding mechanisms and by different types of implementers – national and international NGOs, UN agencies and private contractors.

In line with the findings from the Stage 1 report, we will look at and compare the rationales, relevance, effectiveness and efficiency of bilateral approaches (USAID and to some extent the EC) to funding basic services and livelihoods support, with multilateral support provided through pooled funding mechanisms (for example, the MDTF, BSF, SRF, and CHF). Evaluations to date suggest that with the exception of the BSF, other pooled funds have had slow and cumbersome bureaucratic procedures that appear to have undermined the efficiency, effectiveness and impact of programmes. Questions to be explored are whether alternative aid instruments might have been more appropriate and what impact the choice of management agent (UN, private contractor, bilateral donor) and selection procedures (appointed or competitive bidding) has had on performance and delivery. The degree to which effective and strategic coordination has taken place between fund decision-makers as well as field-level project implementers will also be examined at Juba, State and county levels

The discourse of „post-conflict recovery´ on which the aid architecture was based will be critically appraised. In a context where continuing insecurity, humanitarian need and political fragility are the defining features, the „relief to recovery´ rhetoric may be of questionable relevance. There is also a lack of clarity around what constitutes „ownership´ and how important it is – and to whom - in the Southern Sudan context. The team will explore further the concept of government „ownership´ of the projects and programmes implemented – what it means and how important it is to different stakeholders (donors, central and state governments, communities etc), given the donor emphasis on state-building. Other crosscutting issues arising from the Stage 1 report - such as whether approaches to decentralization, GoSS capacity building (for example, USAID BRIDGE, UNDP Support to States) and sustainability in relation to basic services have been strategic and effective will also be reviewed. We will also explore the degree to which the development of road infrastructure has been prioritized and its role in facilitating service delivery.

Rationale for State selection

Northern Bahr el Ghazal is historically one of the most neglected areas of Southern Sudan. According to the GoSS Statistical Yearbook published in 2009, Northern Bahr el Ghazal received 500,000 returnees from 2004-2009 – more than twice as many as received by any other State. These large numbers of returning IDPs have created pressure on the already poor communities and scarce basic services in the State. All three States have some of the worst child immunization and stunting rates, and lowest primary completion and secondary school attendance rates in Southern Sudan.[29]

From the mid-80´s until 2002 the northern border areas of both Northern Bahr el Ghazal and Warrap States were subject to attacks and raids, executed by GoS-sponsored Popular Defense Force militia, („murahaleen )’.The consequences were physically and psychologically devastating. In addition to the killings, cattle looting, and widespread displacement, women and children were often raped and/or abducted.[30] Some of these abductees have been reunited with families and communities after years in captivity, but many remain missing.[31]Many of the Dinka clans affected still harbor intense animosity towards the Baggara tribes who perpetrated these raids.[32]

Lakes State, while not directly affected by such raids, has suffered from outbreaks of conflict over land and cattle between Dinka pastoralists and more settled farmers from minority tribes. Some senior GoSS officials at both national and state level, whose cattle and kinsmen have been affected, are accused of encouraging retaliatory raids. Likewise, inter-tribal conflict has also erupted in Tonj County in Warrap State, one of the areas Team 2 plans to visit. Lakes State is also important in that from around 2000 until 2005, Rumbek, was the de facto capital of Southern Sudan. Although the infrastructure established during that period has helped it to retain its status as a meeting and training point as well as a significant political „centre´, the decision to move the capital to Juba may have resulted in the redirection of resources – potential „peace dividends´ – away from Lakes. This hypothesis will be explored further.

TEAM 3

Team 3 will focus on three issues in Eastern and Western Equatoria, with particular programmes as follows:

  • Physical infrastructure.
    1. WFP: The Southern Sudan emergency road repair and mine clearance project (target was 3,000 km of roads between 2004 and 2009). USAID: Re-engineering and resurfacing the major transport route from Juba to Nimule (a border crossing to Uganda).
    2. Water for Recovery and Peace Programme (WRAPP – USAID/OFDA through Pact);
    3. USAID: Sudan Accelerated Infrastructure Program, a partnership between the Government of Southern Sudan, UNOPS and USAID, which supports recovery in Southern Sudan.

  • Land issues. USAID land tenure laws programme, EC Technical, Legal and Constitutional Advice to Land Commission, ,Property Rights Resource Governance Program (SPRRGP)


  • Local Government.UNDP/CRS/PACT: The three agencies are working together to support a 3-year (first-phase) local government capacity building programme covering a full range of policy and legal development, recovery planning systems, infrastructure, intergovernmental relations, finance, training and civic engagement (LGCRP).


  • Gender Issues in relation to land, water, and local government (e.g. GGEPP)

The Equatorial states have been a major locus for intra-South conflict such as between and among pastoralists and farmers over land and water, such as manifested by cattle rustling, and among tribal groups. Over 70% of the South Sudanese population is dependent on livestock for their livelihoods and competition over grazing land and watering points for cattle is a major source of tension. These states continue to experience insecurity arising from these conflicts as well as ethnic militias, enlisted forces, and bandits. In addition, Western Equatoria was affected by cross-border influx and raids by LRA fighters from the DRC and Uganda. Team 3 will not assess impacts of donor programmes on such overt violence. Rather, it will examine whether the above programmes mitigated these conflicts through increasing the wider availability of vital natural resources and community facilities and thus reducing grievances that fuel the conflicts, and/or through creating local structures that incorporate into peaceful channels tensions and disputes over resources and struggles for control over them.

With less than half of Southern Sudan´s 7,500km road network estimated to be open year round, rehabilitating the region´s roads is an indisputable priority. Roads and other infrastructure were to provide one of the visible „peace dividends´ by meeting local needs for transport and better delivery of services, stimulating trade and commerce, and fostering greater South-South and wider economic integration. The various programmes were also to provide employment opportunities to displaced, returning young men and other disadvantaged people. There are less than 50 km of tarmac roads in the entire region, concentrated in the capital, Juba. During the long rainy seasons many rural locations are unreachable by road or air for weeks at a time. The evaluation will assess the relative merits of short-term upgrading of roads (WFP) as opposed to more permanent road construction that would take longer to complete (MDTF). WFP´s special operation in southern Sudan for emergency road repair and mine clearance has so far rebuilt approximately 2,500 km of roads. The main strategic road links connect (a) Kenya from Lokichoggio to Kapoeta, Juba and east of the Nile to Bor heading towards Malakal and (b) Uganda from Yei northwest towards Rumbek and Wau and north to Wunrok. They also have supported the deployment of the UN Mission in Sudan (UNMIS) and the return of southern Sudanese IDPs and refugees over newly demined routes.

The team will assess whether the sequencing and geographical location of road construction was appropriate. It will also investigate whether the new roads have substantially increased the volume and kinds of trade, commerce and outside investment yielding employment opportunities, as has been reported at least for Juba. The claim is that better roads stimulate economic growth; for example, with roads linked to the River Nile, the cost of food production and delivery will be reduced, and farmers´ access to markets will grow. In addition, the delivery costs of food aid will decrease and eventually the need for large quantities of aid to support people in Southern Sudan should diminish. There is already evidence to show, for instance, that the opening of routes from Uganda into Sudan through the town of Yei has resulted in the price of basic items such as maize, sugar, soap and fuel decreasing by about 20 percent. Because new roads and other physical infrastructure are so visible as improvements, the team also needs to ask whether they have affected public attitudes and they attribute any change to the peace process. Another relevant impact question is whether roads have increased or decreased security.

Land is fundamental to the way in which the Sudanese conflicts and humanitarian crises have evolved for it has been the focus of localised conflict over land and water between competing ethnic groups. The evaluation will first examine what patterns of use and ownership have emerged, and what kinds of land issues have arisen, over the last five years. It will then look at the policies realized through the efforts of the Land Commission and impacts of the recently passed Land Act. The basic question from a conflict perspective is the extent that new policies and programmes have made local tensions over the allocation of land and land tenure more subject to regulation through agreed-on laws and procedures, and thus peacefully; whether land has become used as patronage by competing factions and politicians; and whether they have stemmed attempts at land-grabbing by well-placed individuals.

In Southern Sudan, WASH services can also become politicised if they are perceived to favour particular sections of the population based on political, social, religious or ethnic grounds. WASH delivery has in some areas become part of cross-sector state-building activity, when water provision through water points are used as an incentive for less tangible state-building activities such as capacity-building of local administrations. PACT developed a cross-sector project that used a combined strategy of providing roads, wells and support to the police force, targeting youth in order to manage localised conflicts. At the same time, there often can be trade-offs for donors between the political imperative to rapidly increase coverage and the need to initiate policy and institutional reforms.

Given the apparent emphasis on strengthening central government, issues of local government will be reviewed from the angle of whether the provision of new services and infrastructure has at least begun to make local governments to be perceived to be legitimate governing entities. From a conflict perspective in southern Sudan, the value of more effective, legitimate local governments would be to instil interest-based loyalties that are localized but multi-ethnic. This allows them to become a focal point for the orderly reconciliation of competing interests and to thwart the higher-level political or ethnic appeals that encourage violence. Relevant indicators here are the extent to which local government officials been actively engaged in planning and service delivery of programmes, and whether government structures are becoming more instrumental in whatever improvements have been made.

The cross-cutting gender issues being pursued by the Gender Specialist on Team 3 can be illustrated with regard to water supply. Evidence suggests that including both women and men in the planning and management of schemes increases sustainability. In Southern Sudan men are often absent from their communities for extended periods of time; therefore, female involvement in scheme planning and management is particularly important. The team will examine whether such involvement is occurring in all the above programmes and others

TEAM 4

Team 4 will focus two issues: approaches to overall capacity development in the South, and demining. The geographic area is Central Equatorial State, including the capital Juba as well as rural regions.

  • Demining. (CIDA), UNMAS: Mine Victim Assistance and Demining Programme; UNDP Sudan Mine Action Capacity Building and Development Project.


  • Overall capacity building strategy. USAID: Bridge Programme

Southern Sudan has a wide cross-section of experienced demining operators, both non-governmental and commercial, but the sector´s indigenous management structure is still in its infancy. With the UN Mine Action Service (UNMAS) focusing its efforts on supporting an increased the pace of demining, one of the main tasks of the UNDP has been to assist the Southern Sudan Demining Commission (SSDC) to become the de factoas well as the de jurecoordinating body for the south. In June 2006, the Southern Sudan Demining Commission (SSDC) was established by a Southern Sudan presidential decree. UNDP´s work in this area is supported by the governments of the United States, the Netherlands, Italy, Sweden and Canada. UNDP activities are undertaken in close coordination with national authorities, United Nations Mine Action Service (UNMAS), UN Children´s Fund (UNICEF), UN High Commission for Refugees (UNHCR) and the World Food Programme (WFP).

The well-resourced demining programme is one of the few where clear evidence of positive impact on conflict and peacebuilding is reported. Assessment will be made of how and why donors prioritised geographic areas, how collaboration with national authorities took place and to what extend diverging mandates have had an effect on the implementation and effects of demining projects. Some problems have recently emerged; for instance, between the UN Mine Action Office´s direct operational mandate and national authorities on priority setting.

The low capacity of state institutions was indentified in the Conflict Analysis (Stage 1) as one of the factors instability in Southern Sudan. Evaluations consistently mention the lack of focus on developing government capacity from the State level down. For example, irregular and unpredictable financial transfers to the States (which has improved in 2009) and down to counties, as well as low budgeting and financial capacity and accountability across all levels of GoSS, have undermined the sustainability of basic service provision. The consensus is that the scale of support for capacity building and capacity provision provided by the donor community fell short of what would be required to establish functioning Government at regional, state and local level. What assistance there was has been fragmented and lacking in overall strategy. This is starting to change now with the introduction of a large three-year USAID programme which will build GoSS capacity to deliver basic services at all levels and a UNDP programme focused on building State-level capacity.

The evaluation will look at what the missed opportunities in capacity development were within a new government that (unusually) was very open to such assistance. Would capacity development per se have offset some of the widely reported structural impediments to recovery in Southern Sudan? Conversely, and for example, would greater investment in the private sector have created its own momentum in attracting more able Sudanese back to the country?

TEAM 5

The Team Leader (in Juba) will be responsible for coordinating team meetings on the ground, reporting to the Ministry of Finance (MOFEP) and the Reference Group, and reporting/introducing the evaluation to various Juba-level sector and general inter-agency meetings. In addition, a particular area of focus will be SPLA reform.

  • US State Dept and UK Government (through Adam Smith Institute, and funded through the Global Conflict Prevention Pool) Support to Security Sector Reform (SSR) and SPLA transformation - training in basic command and staff skills, internal security, counter-insurgency, close protection, medical and communications skills, etc. SSR work includes support to legislative work and capacity within Ministry of SPLA Affairs.

Following the signing of the CPA, important steps were taken to turn the Sudan People´s Liberation Army (SPLA), a rebel group, into a professional army and to develop a civilian police service[33]. This has involved the uneasy integration of formerly hostile southern militias into the ranks of the SPLA. The implementation of security sector reform has been challenging largely because the SPLA and SAF were not genuinely committed to downsizing their militaries.

The CPA defers decisions on major issues until the end of the six-year interim period and the Southern Sudan referendum in 2011. This includes the decision on the ultimate shape of the armed forces, giving the impression that the agreement is more of a ceasefire allowing both parties to consolidate their positions and seek security and political gains during the interim period. The political environment in general was not conducive as both SPLA and SAF were not genuinely committed to downsizing their military[34].

Our review of evaluation literature Stage 1 shows that no serious attempt was made to agree on a common policy framework for engagement in Southern Sudan and genuinely link interventions such as the SPLA transformation, the DDR programme and interventions in other rule of law areas, particularly the police. Instead, activities were pursued bilaterally. Security sector reforms were, for example, dropped from JDT´s priorities.

The roadmap for the transformation of the SPLA into a professional army was provided by the SPLA White Paper on Defence adopted in June 2008. This paved the way for the SPLA Act adopted by the Southern Sudan Legislative Assembly in February 2009. Yet the professionalization of the SPLA has been slow and has encountered some setbacks. For instance, the UN has confirmed that the SPLA continues to employ child soldiers in contravention of international conventions[35]. There are numerous reports of SPLA personnel using excessive force and committing human rights violations when dealing with the civilian population, particularly during disarmament operations[36].

The United States has provided direct bilateral funding (around $60 million per year) to support restructuring of the SPLA. In addition, the United States has provided around $250 million annually to support the UN Mission in Sudan (UNMIS), including its efforts to create a police force in the South. Thus far, SSR in southern Sudan has yielded mixed results.

The UK is providing support through a number of different programmes: (1) Support to GOSS in developing an effective security decision making architecture with capacity to provide effective[37] ; (2) Support to the Defence White Paper process (completed)[38]; (3) Support to the SPLA Transformation process; (4) Support to the Ministry of SPLA Affairs; (5) Parliamentary oversight of defence and security. Most of the funding is through the (DFID/FCO/MoD co-administered) Conflict Prevention Pool.

TEAM 6

Team 6 will focus on the „view from the North´ encompassing some of the broader aid and political issues not covered elsewhere, and the way these have impacted on the potential for peace in Southern Sudan. The team will travel to Khartoum.

Team 6 will specifically focus on several key issues that came out of the Phase I analysis with overriding importance to peace in Sudan:

  • Oil Issues: an examination of the manner in which oil has dominated North-South relations since the signing of the CPA, what the key areas of contention are, and progress towards their resolution.
  • The 3-Areas (Abyei, Southern Kordofan/Nuba Mountains and Blue Nile). How unresolved issues in the 3-areas have curtailed progress in the CPA and to what extent these will remain conflict „flashpoints´
  • In addition, Team 6 will supplement the analysis of donor policy with inclusion of those donors without representation in Juba, including Arab countries, India and China.

The Stage 1 report underscored the fact that CPA implementation in the whole of Sudan is of principal importance to peace and stability in the South. The Stage 1 analysis specifically refers to an extensive conflict analysis conducted in 2008 for DFID, focused on the Three Areas (Abyei, Southern Kordofan and Blue Nile).[39] The lack of progress towards border demarcation and the continuing uncertainty with regard to the status of the Three Areas is cause for increased tension, if not conflict. Key problems included uncertainty about the future, lack of a peace dividend, increasing problems related to return of displaced people and lack of employment for militarized young men. Conflict could be triggered before the referendum on the status of Southern Sudan. A referendum on the status of Abyei is due in 2010, and the process of „Popular Consultations´ on the status of Southern Kordofan and Blue Nile States is also due to take place in 2010.

Issues around oil and wealth sharing have dominated the CPA. With 75% of Sudan´s current production coming for the South, these remain highly contentious issues and central to any negotiated outcome post-2011. Oil revenues significantly influence the political landscape in Southern Sudan. 95% of revenues of the GoSS come from oil, providing GoSS with a budget greater than that of Kenya. The oil factor may have undermined any efforts to widen the political settlement. Instead, various actors are jockeying for position and access to resources derived from oil revenues. Even if aid focuses on widening the political settlement beyond the political elite its influence compared to oil wealth is limited.

Team 6 will frame its analysis around the work of the Assessment and Evaluation Commission (AEC), the commission that monitors and supports the implementation of CPA. The members of the AEC are representatives from the Government of National Unity (three from the National Congress Party and three from the Sudan People's Liberation Movement); representatives from the Intergovernmental Authority on Development (IGAD) (Kenya and Ethiopia); and representatives from Italy, the Netherlands, Norway, the United Kingdom and the United States, that witnessed the peace negotiations leading to the creation of the CPA. The African Union, the Arab League, the European Union and the United Nations have observer status. THE AEC has 4 major working groups, each focusing on a protocol in the CPA: Power Sharing, Wealth Sharing, Security Arrangements and the Three Areas (Abyei, Southern Kordofan and Blue Nile).

The interviews conducted by Team 6 will be geared towards a better understanding of:

  • The above-mentioned issues/areas of contention (specifically focusing on oil issues, and the Three Areas);
  • The strategy and policies of AEC with regard to the issues;
  • Progress towards their resolution since the signing of the CPA, and specifically the role played by AEC in this regard;
  • The alignment with government policies;
  • The current situation/remaining issues of contention.

Deliverables

The Stage 2 report will comprise a 20-page overview of key findings and conclusions from the six field studies. The six studies will be attached as separate annexes, containing detailed description and evaluation evidence. The timeline for delivery is:

5 April All teams will have returned from the field.
Mid-April Team meeting to review draft report findings
1 May Submission of draft Stage 2 report to Steering Committee plus a draft annotated Contents for the Final Report
Mid-May Following SC comments, team meeting to prepare draft for final Synthesis Report
2nd week June Submission of draft Final Report to Steering Committee
End-June Steering Committee meeting
July Revision of Final Report (following comments) and submission of final draft

Annex 1: Southern Sudan CPPB Evaluation Matrix

The Evaluation Matrix sets out the overall evaluation questions as set out in the TOR (MoFA 2009), as well as more specific questions developed by the evaluation team. It provides an overarching framework for the evaluation, so that the team can keep in mind the TOR questions and the link to the more specific questions that are being addressed at any one moment in time (such as during the fieldwork). The reporting template for the Stage 2 fieldwork (Annex 2) has been developed from this Evaluation Matrix, so as to ensure a consistent approach to the evaluation. The Evaluation Matrix is designed not to become a „straight jacket´ for the team (who may need to explore lines of enquiry as they emerge) but rather a tool to ensure the evaluation fulfils the requirements of the TOR.

Click to see the table: The Evaluation Matrix ''

Annex 2: Fieldwork Reporting Template

The template below provides an initial set of headings and sub-headings against which the field teams will report. The purpose of the template is to provide a consistent structure for the Stage 2 reporting, as well as making the link to the TOR questions (as set out in the Evaluation Matrix). The questions are provided as guidance, and will be used by the field teams to develop specific checklists of questions that can be asked in interviews or during focus group discussions – as this will vary from programme to programme and by different contexts.

Click to see the table: Annex 2: Fieldwork Reporting Template''

[1] Approach paper for a multi-donor evaluation of conflict prevention and peacebuilding activities in Southern Sudan, 2005-2008, Mariska van Beijnum and Jort Hemmer, Conflict Research Unit, Clingendael Institute, The Hague, 27 March 2009 (Prepared for and in collaboration with the Policy and Operations Evaluation Department (IOB) of the Netherlands Ministry of Foreign Affairs).

[2] The OECD/DAC Aid Statistics for instance provide a general picture of the net ODA receipts for Sudan as a whole. In the period 2005-2007, these amounted to almost US$ 6 billion. Unfortunately, the OECD/DAC statistics do not differentiate between receipts that were allocated to Southern Sudan and those that went to other parts of the country.

[3] For example, the OECD/DAC Aid Statistics provides a general picture of the net ODA receipts for Sudan as a whole amounting to US$ 5,985,000,000 in the period 2005-2007. The Financial Tracking System operated by UN OCHA, shows a total of US$ 915 million on humanitarian and early recovery funding in Southern Sudan, excluding bilateral funding and direct funding by international NGOs.

[4] E.g. the Paris Declaration on Aid Effectiveness, the OECD/DAC Principles for Good International Engagement in Fragile States, the Good Humanitarian Donorship Initiative, and the UN reform process.

[5] Other major pooled funds that have been established to channel donor support to Southern Sudan are the Strategic Partnership Agreement (SPA) between the UK, the Netherlands and Denmark on the one hand, and UNDP on the other; the Basic Services Fund (BSF); the Capacity Building Trust Fund (CBTF); the EU Recovery and Rehabilitation Programme (RRP); and the Sudan Recovery Fund for Southern Sudan (SRF-SS).

[6] An evaluation of the JAM appears to be planned in 2011. The results of the current evaluation may be useful for that particular evaluation.

[7] Organisation of Economic Cooperation and Development, 2008, p. 17.

[8] Belgium, Canada, Denmark, Germany, the Netherlands, Norway, the Government of Southern Sudan, Sweden – Sida, United Kingdom, UNDP, UNHCR, UNICEF, UNOCHA, UNV, WFP, and the World Bank. During the course of the Evaluation it may be useful to also include the support and activities of additional donors and agencies.

[9] Ideally, such a conflict analysis should include a baseline analysis performed during the planning stage of the intervention(s), as well as updates and conflict monitoring (over time). For comparison, a „current´ or updated analysis at the time of the evaluation is needed. The Evaluators will have to review and take into account existing conflict analyses made by donors, agencies and others and updates of the (post)conflict situation in establishing the conflict analysis for the purpose of this Evaluation. Most likely only partial and very specific analyses will be available and have to be reviewed and synthesised by the Evaluation Team. It is important that similarities and differences in individual conflict analyses are highlighted in the Evaluation, since they will influence donor and agency policies and the theory behind these.

[10] The Evaluation should look into the implicit or explicit theory or model underlying the respective donor and agency policies and interventions aiming to produce the intended outputs, outcomes and impacts in terms of conflict prevention and peacebuilding.

[11] Currently, the development NGO PACT Sudan is commissioning a conflict assessment in the framework of its South Sudan Peace Fund (SSFP) funded by the Department of International Development of the United Kingdom (DfID). This analysis will be completed in October 2009 and will serve as a useful input for the Evaluation.

[12] The conflict analysis includes features of a political economy analysis, which focuses on the wider context of the aid interventions, the various actors involved and the interests they may pursue (see also policy and strategy analysis).

[13] One may distinguish various types of interventions which play a role in conflict prevention and peacebuilding in Southern Sudan; for instance humanitarian and development interventions which have different motivations and are considered relevant at different moments in time and space. Stage 1 of this Evaluation will include mapping these different types of interventions.

[14] It should be noted that the inventory or overview needs to include all types of support to Southern Sudan and will not be restricted to support for conflict prevention and peacebuilding activities in the narrow sense as described under DAC Code 152 „Conflict Prevention and Resolution, Peace and Security´.

[15] The Evaluation will not only have to cover activities that are major in terms of their monetary value („follow the money´), but should also include specific sectors and/or issues important to conflict prevention and peacebuilding which may have been given lesser attention as donors find them difficult to engage with and/or lack appropriate funding instruments to address them.

[16] Obtaining such information could be a challenge since not all donors may be able to provide disbursement figures and programme information which differentiate between Southern Sudan and other parts of the country.

[17] For the governance of the Evaluation see section 9 Organisation of the Evaluation.

[18] The current Steering Committee consists of Belgium, Canada, Denmark, Germany, the Netherlands, Norway, the Government of Southern Sudan, Sweden - Sida, United Kingdom, UNDP, UNHCR, UNICEF, UNOCHA, UNV, WFP, and the World Bank. Other donors and agencies may wish to join.

[19] In view of the multitude of (local) languages in Southern Sudan, it is expected that the Evaluation Team will employ interpreters during its field work activities.

[20] Pages 22-23 of Framework Terms of Reference for the Multi-donor Evaluation of Support to Conflict Prevention and Peacebuilding in Southern Sudan, Final Version, 10th July 2009, Policy and Operations Evaluation Department, Ministry of Foreign Affairs, The Netherlands (MoFA 2009).

[21] Pages 27-30 of the TOR (MoFA 2009).

[22] OECD /DAC Guidance on evaluating conflict prevention and peacebuilding activities (OECD 2008)

[23] Under Stage 1, the evaluation team has produced a „database´ of all intervention in Southern Sudan. This currently records 2,189 interventions, over the period 2005-2009.

[24] Pages 32-37 of the TOR (MoFA 2009).

[25] In one attack in a village in Jonglei state in August 2009, some 161 people were killed, most of them women and children.

[26] OECD/DAC ibid

[27] The teams will be subject to the security advice and instructions of UNMIS.

[28] The Stage 1 report stated: „UNDP initially started with a substantive focus on decentralized service delivery and support to the capacity development of local governments. However, upon request of GoSS, a shift of funding was made away from local government to central level capacity development. In early 2009, an internal UNDP assessment of the situation in Southern Sudan was undertaken by the staff of the office. Two main conclusions came out of this: (i) the shift of focus to central level capacity development in 2005 had been too extensive and that enhanced focus was needed on developing the capacity of local governments to perform their duties. According to UNDP this step should have been taken already two years ago, but will now be implemented in 2009 and 2010.´

[29] GOSS Statistical Yearbook 2009, pp. 30, 34, 62.

[30] Report of the International Eminent Person Group on Slavery and Abduction, 22 May 2002.
http://www.state.gov/documents/organization/11951.pdf

[31] Sudan Abductee Database Project, Rift Valley Institute, 2005. http://www.riftvalley.net/index.php?view=abductee

[32] These clans also resent the failure of the „awlaad Garang´ who they feel failed to prioritize the deployment of sufficient SPLA troops to protect them from these attacks, although this has been mitigated somewhat by the appointment of Salva Kiir to the presidency as he comes from Warab State.

[33] The UN Secretary-General, acknowledging that CPA implementation depends on successful integration of former SPLA combatants into professional military, police and other uniformed services, called for UN to coordinate security and justice sector projects more comprehensively. United Nations Security Council, Report of the Secretary-General on the Sudan, S/2008/267, para. 58, April 22, 2008, http://www.un.org/Docs/journal/asp/ws.asp?m=S/2008/267

[34] Saferworld (2008) Developing Integrated Approaches to Post-Conflict Security and Recovery: A Case Study of Integrated DDR in Sudan.

[35] Report of the Secretary-General to the Security Council on Children and Armed Conflict.
A/63/785-S/2009/158. March 26. New York: United Nations.

[36] Gagnon, Georgette (2009). Southern Sudan: Protect Civilians, Improve Rule of Law. New York: Human Rights Watch. February 12. Available at:
http://www.hrw.org/en/news/2009/02/12/southern-sudan-protect-civiliansimprove-rule-of-law.

[37] This includes helping to develop and implement the second phase of Police and Justice programme for period 2009-12.

[38] This included strengthening HR, logistics, procurement, training systems and information management.

[39] Vaux, T, S Pantuliano and S Srinivasan (2008) Stability and Development in the Three Areas,DFID




This page forms part of the publication 'Aiding the Peace - A Multi-donor Evaluation of Support to Confict Prevention and Peacebuilding Activities in Southern Sudan 2005–2010' as chapter 13 of 15
Version 1.0. 28-04-2011
Publication may be found at the address http://www.netpublikationer.dk/um/11006/index.htm

 

 
 
 
 
  The Ministry of Foreign Affairs © | www.um.dk