Executive Summary

The Secondary Education Support Programme (SESP) came into effect in 2003 with a total basket of USD 74.8 million. The seven-year programme is implemented by the Ministry of Education (MoE) in Nepal and supported by the Asian Development Bank and Danida.

The SESP policy objective is to expand quality secondary education suitable for the need of national development. In addition, the three main intermediate objectives of SESP are to i) improve the quality and relevance of public secondary schooling; ii) improve access to public secondary schooling with a particular emphasis on girls and students from poor and disadvantaged groups and districts; and iii) develop the institutional capacity and management of central and district educational institutions and public secondary schools based upon a decentralised system of planning and management. The programme has four components, each with a separate output:

1. Increased equitable access to an improved learning environment, especially for educationally disadvantaged groups, ethnic minorities and girls;
    
2. An improved and more relevant curriculum, technically improved assessment and accessible instructional materials;
    
3. An integrated system supporting teacher education, development and management; and
    
4. Improved institutional capacity in the school sector based on a system of decentralised planning and management.

Some of the activities under component 1, notably construction of new classrooms and provision of scholarships, were to be implemented with greater intensity in ten so-called Programme Intensive Districts (PID) where the need was assessed to be the greatest.

During the May 2008 technical review it was agreed between the MoE and its development partners to conduct a joint evaluation of the SESP in its final year of implementation before it is going to be integrated into the framework of the School Sector Reform from 2009 onwards. The overall purpose of the evaluation is to provide information about the outcomes and document early signs of impact of the SESP that Ministry of Education, the development partners and other education stakeholders can use for improving the policy framework and further the design of the on-going School Sector Reform (SSR).

This report is the main output delivered by the evaluation team contracted by Danida to carry out the joint evaluation. The team includes experts from COWI A/S in Denmark and sector experts from Nepal. The joint evaluation relied on a combination of quantitative and qualitative methodologies. In addition to data analysis, document review and interviews with key stakeholders in Kathmandu the joint evaluation involved fieldwork, including participatory focus group discussions based on the Most Significant Change approach. To evaluate whether the programme intensive activities in the ten PID districts in the western part of Nepal have made a significant contribution compared to similar districts with no programme intensive activities, three sets of PID and non-PID districts were selected. The selection was made to ensure comparability in as many ways as possible except for the districts’ inclusion or exclusion from the programme’s intensive activities. In each district, the team had detailed interactions with schools, district administration and other key stakeholders. A summary of the key findings of the joint evaluation are presented in the following.

Relevance of the SESP is evident at a policy level given that the SESP objectives are basically a rewording of four objectives from the Secondary Education Development Plan. Moreover, education is one of the four priority areas of Nepal’s poverty reduction strategy paper and is an important recipient of government funding. The original motive of SESP to be implemented in closer collaboration with local authorities is also in tandem with the Government of Nepal’s (GoN) overall decentraliation strategy. Moreover, the three intermediate objectives are all accepted by interviewed stakeholders as relevant and the specific benefits delivered at the district level have been appreciated and utilised by the beneficiaries.

There is coherence with programmes such as Education for All and its predecessor with their primary focus on improving the access and equity, quality and relevance, and institutional efficiency of the primary education sub-sector. Moreover, given the tendency of the majority of development partners and organisations in Nepal to focus on primary education, the choice of Danida and ADB to focus on secondary education is highly relevant, especially in view of the sub-sector’s importance for strengthening productivity and private sector development through the education of young people with skills relevant for the needs of the market.

After a slow start, the efficiency of the programme and the performance of the GoN improved. Outputs have largely been delivered according to plan which is a remarkable achievement given the conflict and ensuing tensions in Nepal. The performance of the donors is largely satisfactory in terms of making resources available on time and development partners have demonstrated commitment by agreeing to align programme management more closely to national procedures.

The effectiveness of the programme is satisfactory in terms of improving secondary enrolment and in institutionalising a bottom-up planning modality. In some cases progress has even been impressive such as increases in lower secondary enrolment, in girls’ enrolment and in Secondary Leaving Certificate (SLC) pass rates. Progress in enhancing access and equity in enrolment has improved significantly especially for lower secondary. The evidence suggests that scholarships have been instrumental in boosting enrolment for girls and marginalised groups. The schools constructed in the PIDs can also explain part of the increase in enrolment. Moreover, the construction and rehabilitation of school buildings and other infrastructure is, despite concerns about quality, consistently quoted by stakeholders in the districts as one of the most significant changes created by SESP.

The very significant increase in SLC pass rates is encouraging and an indication of improvements in the quality of secondary education. However, a more comprehensive assessment of progress towards targets reveals a number of areas where progress has been more modest. Given the validity and reliability problems related to SLC pass rates and the fact that other quality indicators do not mirror the positive developments in SLC data, the overall assessment is that quality has improved less dramatically than suggested by the SLC rates. For example, an assessment of grade 8 students carried out in 2008 does not support the notion of a dramatic increase in quality. Hence, while part of the increase in the SLC rate is likely to reflect real improvements to the curriculum, to teacher education programmes, to student assessment and to the learning environment, the very significant increase in SLC rates can arguably also be explained by poor reliability of data, continued cheating and pressurising of teachers by parents in district and national examinations, and the adoption of a more liberal assessment and promotion policy. The overall trend is positive nevertheless: An improvement in the quality of education can be detected at the school level as further documented by the district field visits undertaken by the evaluation team.

The improvement in quality is notable since the significant increase in access has clearly put additional strains on a school system that was already struggling at the outset to deliver quality education relevant to the needs of the nation. Despite the positive trend, the overall effectiveness of the teacher education component has been reduced by a number of factors including lack of resource material at the schools, inadequate teacher allocations and a marked mismatch between the training situation and the actual conditions in most classrooms (e.g. introduction of teaching methods that do not take into account the very high number of students in classrooms). Teachers have been faced with increased workloads and are left with little time to prepare for and organise classroom activities encouraged in the training. A key outstanding challenge is the need to increase the proportion of female teachers and teachers from minority groups especially in secondary school.

The programme has not had an adequate vision and strategy for building the capacity of the implementing partners, but different entities have still received relevant and much appreciated inputs. The overall impression remains however that progress in capacity development is limited and that many of the interventions are either too limited in scope or the effectiveness is being diluted by high staff turnover – especially at district level.

The sustainability of the gains made in access will need to be carefully monitored in the coming years. A key issue to follow is the retention rates and achievements of girls and students from disadvantaged groups. A key outstanding challenge is the need to increase the proportion of female teachers and teachers from minority groups, especially in secondary school. Sustainability also depends on the capacity of the education management system to deliver. Hence, a key lesson learned is that a programme with the scope and ambition of SESP needs to be supported by a strong capacity development plan. At the school level, School Management Committees (SMC) are an interesting platform on which increased participation and accountability can be built. Current capacity development efforts need to be enhanced to enable the SMC to fulfil its envisaged role and take full ownership of the various outputs created by SESP. A notable sign of this commitment would be the development of maintenance plans for the works constructed through SESP. Currently such maintenance plans exist in very limited numbers, which may be an indication of limited local level ownership to the programme.

Finally, at the impact level, it is notable that quality appears to be improving although not at the rate suggested by the increase in SLC pass rates. As the assessment against Vision 2012 shows, it is unlikely that the current rate of progress will allow Nepal to arrive at the Vision by 2012. There is need for increased focus on monitoring of quality of education, further revision of curriculum, a more coherent teacher education programme and an improved system for deployment and management of teachers. Moreover, measures for capacity development at all levels of the system need to de designed and rolled out in a more coherent and informed way. However, the Vision remains relevant as an ambitious target that may inspire future reform efforts in the Nepalese education sector.

The assessment of the SESP against the evaluation criteria is summarised in the table below. Given that only early signs of impact can be detected at present, impact assessments have not been included in the table.

Summary of the SESP Joint Evaluation

Legend: ----- very high; ---- high; --- medium; -- low; - very low

The joint evaluation has identified a number of strategic recommendations, which could contribute to policy development in view of the on-going School Sector Reform. The primary responsibility for taking these recommendations forward lies with the GoN through the MoE and its various implementing entities notably the Department of Education (DoE), the Curriculum Development Centre (CDC), the National Centre for Education Development (NCED), the Office of the Controller of Examinations (OCE) and the Janak Educational Materials Centre (JEMC). However, Nepal’s development partners and the civil society also have a key responsibility in engaging in dialogue with the GoN on how best to sequence the reform and prioritise the external funding being made available to the sector.

1. The strategies and methods employed to increase quality further need to be adjusted to better cope with the increasing student population in secondary education. There is a need to develop a better and more nuanced understanding of quality and its components. It is important that such a definition is fully owned by the GoN, and there is consequently a need to encourage the work already going on under the leaderships of the MoE to come up with a definition of quality in education in consultation with key stakeholders. Similarly, the MoE needs to define strategies to roll this out to schools and classrooms. The GoN needs to factor in processes such as student and teacher performance incentives and the actual conditions facing teachers in the classrooms when rolling out some of the specific recommendations mentioned in this report, such as further revision of the curriculum and the development of a more coherent teacher education programme.
    
2. In terms of the actual strategies, there is, in coherence with the decentralisation process, a need for a more contextualised approach with greater utilisation of local resources and increased flexibility in design of buildings and classrooms so that designs from DoE can be adapted to local conditions. Another important aspect is to strengthen the capacity at local level for increased accountability for school performance. The school-mapping exercise is critical in this context.
    
3. Strategies for promoting equity and inclusion of disadvantaged groups need to go beyond the access issue and, in the case of children with disabilities, the physical access within the school area has to be strongly enforced. The strategy should focus on developing a learning environment which is free of discrimination and which facilitates the retention and performance of students from disadvantaged groups.
    
4. The potential for collaboration with NGOs, international NGOs and the private sector should be better utilised. As a starting point, representatives could participate more fully in implementation of the recently launched School Sector Reform programme as well as in the forthcoming reviews.
    
5. Finally, as a crosscutting recommendation, future reforms and their underlying rationale need to be more clearly communicated to key stakeholders in the school system (e.g. teachers) to have a real effect and impact at district and school levels.

In addition more specific recommendations are summarised in the table below. The recommendations argue inter alia for the continued investment in infrastructure, scholarships, recruitment of new teachers, teacher follow-up training and increased focus on improvement in school management. All of these interventions are needed to sustain the gains of current investments.

For the bulk of the specific recommendations, the primary responsibility lies, as for the strategic recommendations GoN but again development partners have a key role to play as do representatives from the civil society. The onus is on the GoN to facilitate this process.

Some of the specific recommendations can be implemented immediately, while others require a long-term approach. Moreover, some of the issues may be outside the direct control of the education system, but require immediate action such as the need to reactivate the teacher deployment system. The lack of teachers has been identified as one of the main stumbling blocks for making further progress in terms of improving quality. Secondly, it seems important that the effect of reducing scholarship amounts be assessed as a matter of priority. Significant funds are spent on scholarships and it will be important to ensure early on in SSR implementation whether this practice needs to be corrected in order to maximise the value added of the programme. Similarly, the DoE should follow up immediately on the quality issues identified by the evaluation team to ensure that funds are spent as efficiently and effectively as possible at the district and school levels.

Summary of the SESP Joint Evaluation