7 Quality

One of the intermediate objectives of the SESP is to improve the quality and relevance of public secondary schooling. As pointed out in the Core Document, this is to be achieved through curriculum reform, reforms of the assessment and examination system, and through teacher education interventions.

This chapter assesses the relevance of the intermediate objective (Section 7.1), the overall effectiveness measured as progress towards the intermediate objective (Section 7.2), and efficiency and effectiveness of the various strategies launched under SESP to increase quality and relevance (Sections 7.3 to 7.6). The assessment is summarised in Section 7.7 including an overall assessment of the sustainability of the benefits produced. Finally, Section 7.8 includes specific recommendations for the consideration of the GoN and its partners.

7.1 Relevance

SESP is clearly relevant in the sense that it has addressed many of the factors believed to influence the quality of education, such as improving teachers’ training and experience, curriculum reform, providing better classrooms, availability of learning materials and improved management of schools. The overall approach to curriculum development in SESP seeks to take a wide range of factors into account. These include the syllabi, the teaching-learning materials, the assessment systems, and the teachers’ qualifications. Such a comprehensive approach is vital for educational development. Also, it seems that the priorities and intended practices of the new curriculum are supported by both teachers and students.

Secondly, the overall rationale for having a clear focus on quality is highly pertinent, since this area has been on the policy agenda in Nepal for at least a decade. The interventions proposed by SESP are well argued and clearly target areas in need of support. The high number of untrained teachers and the long tradition of rote-based learning are two clear indications that there is a need for coherent interventions to improve the quality of education.

Several of the inputs to be provided under the SESP consultancy package were designed to support activities related to curriculum development, assessment and teacher education. As pointed out in Pinz (2008), many of the original ToR for this consultancy, which were framed in 2002, “were not relevant at the time of delivery”. Hence, the delay in fielding these inputs has meant that the relevance of some of the interventions has decreased.

7.2 Progress Towards Target

Pass Rates
The performance targets for the quality dimension of SESP were to raise and sustain measurable improvements in educational outcomes as evidenced by grade 8 and SLC pass rates (refer to Box 7.1 below). The exact target for these increases was never established. It was underlined, however, that the improvements in educational outcomes should specifically be achieved for girls and students from traditionally disadvantaged groups.

Chart 7.1 below presents the overall development in SLC pass rates as well as the pass rate for girls. The increase since 2002-03 is staggering: The pass rate for the overall population has doubled, and the rate for girls has increased more than five-fold. The overall increase in SLC pass rates coincides with a marginal improvement in transition and promotion rates.

Chart 7-1 - SLC Pass Rates (Country vs. Girls)

Total Source: GoN, 2008e.

The increase in the pass rates can be explained by many factors: Reports from the case study districts suggest that frequent testing of students in secondary education takes place. For grade 10 students, a “send up” exam is carried out to determine whether a student is eligible for appearing for SLC examination or not. This has been practiced in order to prepare students for SLC exams and thus increase the possibility of better SLC pass rates in schools. Other reported causes of increased pass rates include organisation of extra preparatory (coaching) classes for the examinees, use of ’private school’ textbooks, and testing based on the grade 10 curriculum only.

Table 7.1 below shows the results of comparing the relative change in SLC pass rates over the period 2005-07 for PIDs to the relative change in non-PIDs at national and regional levels and by focused comparisons. The table illustrates the difference in percentage points between PIDs and non-PIDs by the final year of the comparisons, 2007. The number of plusses or minuses indicates the scope of the difference. If the difference between the two scores is less than five percentage points the difference is not treated as substantial.

Table 7.1 Indexed SLC Pass Rates, 2005-07 Trends for PIDs Compared to non-PIDs

Source: Own calculations based on OCE.
Legend: LSS Lower secondary school; SS Secondary school; ++ PIDs scoring >15 percentage points higher than non-PIDs by 2007; + PIDs scoring 6-15 percentage points higher than non-PIDs by 2007; = Difference between PIDs and non-PID no greater the 5 percentage points by 2007; - non-PIDs scoring 6-15 percentage points higher than PIDs by 2007; - - non-PIDs scoring >15 percentage points higher than PIDs by 2007;

A clear difference between national level and PIDs in favour of the PIDs. While SLC pass rates for the nation as such have doubled, they have increased dramatically from 17.7 per cent in 2002/03 to 53.9 per cent in 2007/08 for the ten PIDs. Despite this increase, the PIDs are still behind the country average in absolute terms.

The increase in SLC pass rates for girls in the PIDs has, like the national average, increased by almost a factor of five, but from a much lower base. Girls’ pass rate stood at 3.85 per cent in 2002-03, and it has increased to 18.3 per cent in 2007-08. It is, however, worth noting that the current level of 18.3 per cent represents a significant drop from the 2006-07 level of 28.78 per cent.

The comparison at regional level by contrast does not reveal a consistent difference between PIDs and non-PIDs. Only the PIDs in the mid-western region have, on average, outperformed the rest of the districts in the region. The results at regional level cannot, however, refute the hypothesis that PIDs have fared better than comparable non-PIDs.

The validity of the results from the western region is limited, given that the data only reflects the performance of one single PID compared to 15 non-PIDs, Rupandehi, which started out from a relatively higher point of departure than all the other PIDs (the only PID with an SLC pass rate above 10 per cent in 2002/03).

For the far-western region, the results are somewhat inconclusive with a small underperformance of girls in the PIDs. The results from the far-western region are arguably influenced by the relatively poor performance of Kailali, which, as the district in the region with the highest population, demonstrated the lowest pass rates for girls compared to the rest of the PIDs in 2007. Over the entire span of the SESP intervention, the pass rates for girls in Kailali has only doubled, while for many of the other PIDs, it has increased by a factor of five, six and in some cases even more.

The focused comparison confirms that PIDs have outperformed their comparators by a considerable margin. The fact that the PIDs start from a nominally lower point of departure explains part of this, but the overall increase in SLC rates in the PIDs nevertheless remains remarkable, and would arguably not have been achieved without the PID intervention. It is likely that the increased focus on SLC pass rates created by SESP has contributed to boosting awareness and performance in the districts for this particular indicator.

Data are not available to follow similar trends for disadvantaged groups. It has, however, been reported through the district case studies that students from the traditionally marginalised communities do well in such exams but data does not allow for an assessment of change over time. As an example, the DEO Planning Officer from Bardiya reports that more Tharu students now pass with higher marks in the SLC than before. Similarly, the National Federation of the Disabled Nepal reports remarkable improvements in terms of internal efficiency and SLC performance for disabled people. Apparently, more than 60 per cent of visually impaired students sitting for SLC exams now pass, and more than 20 per cent of the students with hearing disabilities. Accordingly, this somewhat dispersed evidence point in the same direction, i.e. that SLC pass rates have also increased for disadvantaged groups during the period of SESP implementation.

The validity and reliability of pass rates as an indicator of the quality of education have, however, been widely discussed during the evaluation team’s interactions with stakeholders in Nepal. The very significant increases in SLC pass rates for the nation, and in particular for the PIDs, lend further support to the hypothesis that data reliability and/or reliability is limited. Some PIDs have seen pass rates for girls increase by a factor of 10 – in some cases even 15.

The change in the test form mentioned in Box 7.1 has, according to the DEO in Doti, contributed to the increase in performance in that particular district. Moreover, it has been reported through the case studies that some of the interventions related to student assessment and examination (refer to Section 7.4 below) have had a positive impact on the way SLC examinations are being conducted. For example, specification grids, usually for exams taken by the schools and the resource centres, are reportedly also used in SLC examinations. Stakeholders have also pointed to a greater variety in the structure of SLC questions which may enable a fairer assessment of the students.

These changes notwithstanding, a high level of cheating and pressurising of teachers by parents in district and national examinations is still reported. PABSON and a number of teachers and DEO staff interviewed by the evaluation team are frustrated and critical of the SLC exam monitoring system and the perceived pressure to demonstrate good SLC pass rates. However, there is no indication that cheating and other malpractice has increased over the period of SESP intervention, but it is likely that the inclusion of SLC pass rates as major performance target in the SESP has increased attention and awareness in the districts of the need to demonstrate an increase in SLC pass rates. For example, according to a majority of teachers, parents and RPs interviewed in the case study districts, a more liberal assessment and promotion policy has been introduced in recent years.

Moreover, the assessment of grade 8 students carried out in 2008 does not support the notion of a dramatic increase in quality. Although longitudinal data is not available to assess changes over time, the assessment indicates that student performance is still disappointing despite the improvements in SLC pass rates. According to the assessment, overall performance of the 2,640 participating students was below expectations for each subject (Nepali, English, Mathematics, Social Studies, Science, Health and Population). Students had only achieved half of the envisaged skills. Importantly, the study establishes that, in comparison with previously conducted studies with much smaller target population and scope, there has been little progress in achievement in Science and English, hardly any in Mathematics and regress in Nepali. This conclusion should, however, be treated with some caution, since it is not based on directly comparable baseline data.

In conclusion, the very positive trends in SLC rates are most likely indicative of an improvement in the quality of education. For example, the fact that girls’ SLC pass rates have increased more than boys’ pass rates is largely expected given the extra focus on providing scholarships and other targeted inputs to girls. However, given the validity and reliability problems related to SLC pass rates and the fact that other quality indicators do not mirror the positive developments in SLC data, the overall assessment is that quality has improved less dramatically than suggested by the SLC rates.

Didactic and Teaching Methods
The notion that quality has improved, albeit less than SLC rates suggest, is confirmed by the evidence from the case study districts that indicates marginal, positive changes in the softer aspects of the learning environment, notably the awareness and, to some extent, behaviour of teachers in the classroom.

District case studies indicate increased awareness among teachers about new teaching methods. Students interviewed in the districts have consistently reported that teacher behaviour is starting to change. Students in a number of schools have mentioned that teachers’ attendance has improved and classes are held regularly, indicating better teacher management. The students have also pointed to improvements in the teaching styles of teachers, away from the traditional lecture method towards more interactive methods, including more discussion and better two-way communication with the teachers. It has also been reported that a culture of sharing experiences and good practices between teachers has increasingly taken root. Moreover, the majority of stakeholders interviewed at district level point to a reduction in the use of corporal punishment, although there are still exceptions to this trend, as evidenced by one of the district case studies.

The EFA joint evaluation similarly shows that teachers and resource persons interviewed at district level point to “child-centred” learning methods being adopted. The EFA joint evaluation also reports that students find that “teaching has become more interesting”. However, it is possible that students’ assessment of change over the five-year period of the SESP implementation is somewhat biased by the fact that the students themselves, in the course of the implementation, have advanced to higher grades where the style of teaching is expected to be more advanced, irrespective of the changes introduced through the SESP. In other words, it may be difficult for students to assess change in teacher behaviour over time at a given class level, since they move from one grade level to the next during that time.

The above indications of improvement in behaviour need to be considered further together with the statements made by several teachers and staff at Education Training Centres, indicating that very little of the teacher training is being practiced in the classroom. This issue is further elaborated in Section 7.5. In conclusion, the overall trend is positive, but not as positive as the significant increases in SLC pass rates and the positive statements by students would suggest.

As for quality, the evidence of changes in inclusive practices at school level is also somewhat ambiguous. Tangible improvements have, as further elaborated in Section 7.3 below, been made to the teaching materials and the curriculum. However, some of the evidence presented by stakeholders to the evaluation team suggests continued discriminatory behaviour at school level. Students from minority groups in Bardiya, for example, mentioned that it can sometimes be difficult to understand some of the Nepali words used by the teachers. Moreover, despite the achievements in terms of increased Dalit enrolment reported in the previous chapter, the NFDN and FEDO still consider that the learning environment is not conducive for Dalit enrolment. The organisations have acknowledged some improvement in the use of teaching materials and methods at schools, but they also point out that quality remains poor. According to these organisations there is continuous discrimination in the treatment of Dalit students and “very little knowledge and practice of social inclusion” at school level. NGOs interviewed and NGOs in some districts visited by the evaluation pointed out that the weaker students (who often have a Dalit background) often sit at the back of the classroom and therefore get less teacher attention. In summary, some of the tangible changes made to the textbooks and curriculum do not appear to have been enacted as intended in the classroom.

Despite these concerns, the majority of teachers, trainers and district officials interviewed by the evaluation team consider the extent of discriminatory practice in schools to be very limited. While this may be taken at face value as a sign of an improved learning environment, it may also be an indication of how little the interviewees are aware of or willing to analyse education issues from a discrimination and social inclusion point of view. Some of the words used during the interviews indicate that this may be the case. It is also notable that there is no method or agreed approach in the schools to assess and measure the extent of discriminatory practice by teachers and other figures of authority in the schools.

Accordingly, the evidence suggests that quality is improving overall, but not at a rate that is high enough to cater for the massive increase in access created by SESP. Arguably, the SESP has not focused enough on creating an environment that caters for disadvantaged groups, but a more detailed assessment of this particular area would be required to confirm this hypothesis. However, it is noted that inclusion strategies do not form a major part of the Core Document.

7.3 Curriculum

Efficiency
The purpose of the National Curriculum Framework (NCF) was to provide the overall framework for development of appropriate, grade-specific curricula by setting out a broad outline of the overall intentions of the curriculum. The NCF has been implemented with considerable delay, but has now been approved by the GoN through the National Curriculum and Evaluation Council.

The initiative to develop the NCF started in 2004, but according to CDC, the initiative only truly started in 2006, when a new executive director was appointed. The development of the NCF, especially the decision on which themes to include in the framework, appears to have been influenced by political factors. Accordingly, the major political changes that took place during SESP implementation have clearly also affected the preparation of the NCF. The 2008 status report observes that technical backstopping provided to CDC has been inadequate.

Although approved, the framework is still being fine-tuned to take into account various challenges such as the need for an integrated curriculum that includes Higher Secondary School.

Through SESP, the curriculum for grades 9-10 has also been revised. In total, a change in 15 subjects was implemented with emphasis on gender and inclusion. Stakeholders from the national Dalit organisations confirmed that they had been engaged in 2004 to review some textbooks (through ESAT) and as a result, some discriminatory words and stories had been changed. However, they also claim that many teaching materials and books still need to be revised.

Curriculum revision for lower secondary levels is still in process. As for the NCF, the revision of the grade 9-10 curriculum was done with some delay, apparently as a result of the time needed to agree on the selection of common subject matters for students.

Effectiveness
According to some stakeholders consulted by the evaluation team, the NCF has not been communicated in an effective manner. The NCF was disseminated in 40,000 copies to schools but, as pointed out by CDC, “most teachers do not open it” – arguably because it is perceived as having limited relevance for their daily work. None of the teachers interviewed during the district case studies have referred to the NCF. Accordingly, the overall assessment is that NCF has not yet served its purpose.

At local level, the new curriculum for grades 9-10 has been positively received by the teachers and the students in the case study districts visited. The new curriculum is better sequenced across levels and subjects, more activity oriented, there is adequate provision for making references to local contexts, and it encourages students to be investigative and creative. For example, teachers in one school in Doti mentioned that the curriculum now matches the respective grades better, but still needs to be more vocational or skills-development oriented. Moreover, the quality of textbooks has also improved as they now contain more illustrations. Teachers in turn have indicated that the new curriculum, besides providing a good transition from one level to the next, has also established coherence among the different subjects at the same level. As an example, it was mentioned that statistics is now taught at same level in economics, maths and population. Likewise, teachers in one school in Kailali have noted that there is much greater focus on conversation skills in English. Further, they also found improvements with respect to the inclusion of description of different groups and communities in social studies textbooks.

Despite these improvements, stakeholders interviewed at schools in Jumla and Rasuwa were of the opinion that students left the school system without the required qualifications to take up a “real job” requiring a minimum of vocational skills. In Rasuwa, the need for more emphasis on tourism, a dominant sector in that particular district, was emphasised. Hence, while the overall assessment at district level is that the curriculum has improved in terms of coherence across grades, style of presentation, and sensitivity to marginalised groups, there is some indication, based on evidence from only two out of six districts visited, that the actual subjects being taught could be better targeted towards the specific needs of the labour market.

The assessment of the revised curriculum is more mixed at the central level. On the one hand, DoE points to the revision as one of the main achievements of the programme. PABSON further notes that a more practical approach is used through the revised curriculum, and TUN emphasises that the revised curriculum increasingly focuses on life skills and has become more inclusive, also mentioning disabled students. On the other hand, some key stakeholders refer to the revision as mainly a “repackaging” of the curriculum and its focus on life skills is questioned.

Stakeholders from the NGO community have pointed out that CDC could have done more to involve stakeholders in the curriculum revision to make the contents as practical and as relevant to the regional context as possible. Moreover, some NGOs interviewed suggest that the secondary curriculum could include basic information about Nepal’s laws and human rights aspects, with a view to developing future citizens and making them aware of their rights and obligations as citizens. CDC for its part points out that curriculum reform was not a major part of the programme. CDC claims that “it was seen merely as a technical matter” and considers that there was inadequate provision for capacity development on textbook writing and curriculum development. The overall assessment is that SESP has been successful in creating a more relevant curriculum, but there is still ample scope for further revising the curriculum to focus on analytical skills and student-centred learning, as well as inclusive and non-discriminatory teaching. Furthermore, there is scope for consulting a wider group of stakeholders in developing a curriculum that is “better matched” to developing future “nation developers”. At the same time, a more strategic approach within the programme to capacity development could have facilitated the work of CDC and other implementing entities.

SESP further included a separate set of activities to promote local contents curricula. None of the stakeholders consulted by the evaluation team centrally and in the districts can, however, point to any successful examples of local contents curricula for secondary school. This is clearly an area where progress has been less than satisfactory. CDC confirms that there is a demand locally for such curricula. Teachers, SMC and parents in districts agree that the curriculum has to be more practically oriented, more “life-skills oriented” and more suited to the regional and local context.

The SESP Core Document refers to the establishment of ad hoc district level fora for incorporation of local curricula. The existence of such fora was reported in the majority of districts visited by the evaluation team. However, it was reported that this forum or curriculum committee (consisting of DEO as Chair, DDC, SMC, development organizations, etc.) had not done any significant work, except for holding occasional meetings and sending reports to the CDC. Interviews at district level suggest that adequate expertise has not been available locally to develop such curricula. This is also confirmed by the 2008 status report, and appears to have been the key constraint for this particular intervention. It further seems that a more thorough situational analysis would have been useful to document local conditions in various parts of the country, acknowledging the role of teachers and local communities without leaving them with the responsibility to develop parts of the curriculum themselves.

7.4 Student Assessment and Examination

Efficiency
The SESP Core Document defines the need to “improve the quality and efficiency of student assessment and examinations”. The specific interventions include upgrading of 8th grade exams, consolidation of quality gains in SLC exams, and the establishment of a secure printing press.

Specific outputs include a handbook on student assessment (including a new specification grid), a training package that has been provided to head examiners, and the construction of 36 marking/exam halls. Overall progress has been satisfactory, but the OCE points out that the scope of training has been too limited to ensure effective rollout. In addition, the longer-term behavioural effect of the training has been limited by the frequent transfer of DEO staff. Strikes in some districts have created implementation obstacles, just as the late delivery of TA has reduced the overall effectiveness of the input.

To further improve the examination environment, the SESP has provided substantial funds for the procurement and installation of a secure printing press for JEMC to print SLC exam forms, rather than ordering them from India. The procurement of the press was significantly delayed, but a Japanese company was eventually selected through a second tender, and the press has now been installed and is operational in a newly constructed and guarded building. It is not immediately clear why such a task must rest with the public sector. However, the JEMC claims that the investment will be paid back to the GoN as if JEMC had operated as a commercial entity. It should be noted, though, that the market on which JEMC operates is price-regulated. Alternatively, the assignment could have been contracted out to a private enterprise. The argument for printing these forms in Nepal rather than in India seems to be based more on principles of self-sufficiency than on efficiency and value-for-money considerations.

Effectiveness
Notwithstanding these reservations, the evidence from the case study districts suggests that teachers are aware of the need to use specification grids for better student assessment. Likewise, the teachers in the case study districts feel that the new evaluation or student assessment methods, which are based on a specification grid, enable the teachers to conduct a more comprehensive and fair assessment of the students. Teachers from Doti and Dadeldhura have reported that they now focus more on a continuous system of assessment of learning, comprising unit tests, quarterly and final exams. Further, according to the students, there is now more variety in the structure type of questions in test papers, resulting in a reduction in failure rates. As mentioned in Section 7.2 it is likely that these changes also have contributed to the very significant increases in SLC pass rates. The OCE, on the other hand, reports that teacher awareness of changes in the exam system is limited, and it is possible that more could have been done to communicate the changes so that the overall effect could have been increased.

In summary, the fact that the teachers interviewed by the evaluation demonstrate awareness and support for the new initiatives suggest that the overall effect of the initiative has been satisfactory, at least with respect to the districts visited by the evaluation team. The activities under SESP have, however, not explicitly targeted the reported widespread malpractice and cheating, and it is possible that overall effectiveness could have been improved if these issues had also been addressed by the programme. In this context, the OCE mentions that it is essential for the DEOs to ensure effective enforcement and regulation of examinations at the local level.

With respect to the secure printing press, the effectiveness of the investment depends on the degree to which GoN is able to maximise the utilisation of the press. If it is only used for printing SLC exam papers, it is estimated that it will only be operational for 25 days a year. It is obvious that such an expensive machine should be used to the maximum of its capacity.

7.5 Teacher Education and Development

Efficiency
The overall intention is to improve the quality and efficiency of teacher education and promote effective teaching in the classrooms. Two types of training are provided: The first is ten months of in-service training provided to teachers who have not had pre-service training. This is provided by the various ETCs across the country. This training is given in three phases or modules based on a revised curriculum. Lower secondary and secondary teachers spend a total of two months at the ETC. The second type of training is provided by the Resource Centres (RC). This is more of a short course (between three and seven days), mainly demand-based subject teacher training at the lower secondary and secondary levels. Moreover, management training is provided to head teachers and the construction of a number of Lead Resource Centres (LRC) is anticipated. At the central level, the activities also include development of training materials and grants to the Faculty of Education for competency-based pre-service training.

As a result of the programme, a high proportion of teachers have taken the ten-month in-service training. In the districts visited, for example, all previously untrained teachers will have been trained by November 2009. NCED reports that 97 per cent of the total force has now been trained. Hence performance is far ahead of target according to the NCED. Data for the 2008/09 Flash I Report establish that in community schools, 54.8 per cent of teachers in lower secondary are now fully trained. For secondary school, the proportion of fully trained teachers is significantly higher at 79.4 per cent (GoN, 2008g: 41). Although this figure does not correspond fully with the NCED data, it is clear that SESP has contributed significantly to increasing the number of trained teachers, especially in secondary school, while lower secondary seems to have been under-prioritised. Similarly, community-funded teachers report that they are not on the priority list of training.

Some schools have been reluctant to send their teachers away for such period. The insurgency arguably also had an effect. For example, training centres have not functioned as envisaged. Recruitment of female teacher and teachers from disadvantaged groups has also been a challenge. NCED reports that female trainers “do not come to our centres” because of mobility problems. However, a relatively high proportion of the very few female teachers at lower secondary and secondary are reported to be fully trained – in fact the share of fully trained teachers is higher for women at both lower secondary and secondary levels. For secondary school, 82.9 per cent of the few female teachers are fully trained.

Effectiveness
There are no longitudinal data available to assess changes in the capacity of teachers over the time of SESP implementation to assess the overall effectiveness of the teacher education interventions. The majority of teachers interviewed by the evaluation feel that the training has improved their cognitive skills, increased content knowledge and oriented them to various ways of student-centred, child-friendly and activity-based teaching methods. However, feedback from the case study districts consistently indicate that while teachers are increasingly aware of new teaching methods, the training has not been applied as intended in the classrooms. Hence, although students, as reported above, point to a change in the behaviour of teachers and the teacher-student interactions, it is clear that the change in behaviour would have been more significant if teachers had been able to fully apply their skills in the classroom.

The perceived lack of effect may, as elaborated below, reflect the quality of trainers, the discrepancy between the training situation and the poor conditions teachers face in the classrooms, lack of follow-up and support mechanisms on the job, a weak performance culture in the schools and the continued inability of the system to provide quality materials in a timely manner to facilitate teaching.

The ten-month teacher training was delivered through a cascade system using a training of trainers approach. As the Status Report 2008 points out, the number of subject-specific trainers is limited. In many cases, training centres had to rely on roster trainers. According to NCED, the purpose of such rosters is to “pull the best experts from the local market”. However, given a remuneration of only NRP 200 per session (less than DKK 15), it is questionable whether ETCs will be able to attract the best candidates to the roster. In theory, the teachers on the roster should be the best. However, the evaluation consistently heard complaints about the quality of the courses, as the teacher trainers were no more qualified or knowledgeable about the subject taught than the trainees. Many therefore regard the in-service training as having limited value. Accordingly, the evidence consistently indicates that the training could have been more effective if better skilled trainers had been available to deliver the training.

The training is typically aimed at teaching a class of 30-40 students, meaning that the training does not resemble a real classroom situation with limited materials to facilitate the teaching and overcrowded classrooms. The student-teacher ratio (STR) has increased slightly for lower secondary and is currently at 69 students per teacher. In the Terai, the STR is at 88, higher than for any other eco-zone. Moreover, there is no provision for practice teaching in the lower secondary and secondary teacher training. Teachers interviewed by the evaluation team report that they have not been able to implement everything they have learned from training, such as the use of lesson plans, teaching materials and activity-based teaching, mainly due to a large number of students and a heavy workload.

Although some mechanisms are in place to provide on-site professional support to the school teachers in order to translate training into enhanced student learning outcomes, these measures are assessed to be inadequate by teachers and teacher trainers interviewed by the evaluation team. They point to the need for giving teachers the necessary on-the-job training and guidance in the classroom. This is confirmed by the GoN (2008) and by the Teachers’ Union of Nepal (TUN).

In addition, a mindset and performance challenge may be at play: Some of the central stakeholders consulted indicate that teachers do not prepare properly, and the MoE reports that teacher absenteeism is still a major issue (refer to aide memoire from the May 2008 Joint Annual Review Meeting). The hypothesis of a poor performance culture in the schools is strengthened by the notion that the design of the teacher promotion system does not clearly reward teachers for using new techniques and methods. The particular dynamics and performance incentives of the teachers depend on their nature of employment. There are, as pointed out by the TUN and AIN, more than ten different categories of teachers. This makes it increasingly difficult to design a performance-based system, as the various types of teachers respond to different incentives and depend on different bodies for their continued employment.

Finally, there is no evidence to support that the readiness of the system to provide teaching materials and textbooks to the schools has improved. Access to school books and teaching aids presenting new methods are a precondition for effective teaching of new methods. However, as consistently claimed by teachers interviewed by the evaluation, such teaching aids are rarely available, and textbooks continue to arrive with delay or in inadequate quantities. This was also cited as a problem in the aide memoire from the May 2008 SESP technical meeting. As far as teaching materials are concerned, it is essential they support the educational priorities of the reform with its focus on student activity and learner-centeredness. In a situation with large classes and relatively poor teacher qualifications, the materials themselves play an even more significant part. They must encourage students to become involved in a wide range of investigative activities and encourage small-group interaction.

It is outside the scope of the joint evaluation to assess the efficiency and effectiveness of the school distribution system. However, JEMC headquarters and the regional distribution centre in Nepalgunj indicated to the evaluation team that the system has become more efficient because of increasing printing capacity and closer follow-up throughout the distribution system. They also report, however, that external events such as the strikes and the insurgency have continued to impact on the effectiveness of the distribution system.

7.6 Teacher Recruitment

Efficiency
The core document recognises the need to encourage women to join the teaching profession. It does so by providing specific funds for in-service training, provision of residential allowances and construction of women’s hostels.

It is questionable whether this represents an efficient use of resources given that other factors have a significant bearing on willingness and ability of female teachers to work at secondary schools. District case studies show that it is difficult to attract women candidates to lower secondary and secondary posts in rural areas. Head teachers and district education officers explained that it was difficult to get qualified applicants, although it is mentioned in the vacancy note that female, Dalit and Janajati applicants would be preferred. It is generally believed that the educated women prefer to work in urban areas and are hesitant to travel to remote districts, as they still face societal and family barriers to working in such areas away from their families.

Similarly, the provision of residential allowances was not assessed by stakeholders interviewed by the evaluation team as sufficient to attract female teachers. Finally, it is notable that provision for women’s hostels had been dropped due to design flaws and unavailability of land as elaborated in Section 6.3.

Effectiveness
Overall, the effectiveness of any drive to boost teacher recruitment is reduced by the fact that the teacher deployment system has not been operational in Nepal for more than ten years. As a result, schools and districts throughout the country have been forced to recruit large number of temporary teachers, such as teachers recruited out of the so-called relief quota (locally referred to as “Rahat” teachers”) and teachers recruited by the community. It was reported through the district case studies that Rahat and community employment of teachers has increased the possibility for employing women, Dalit, and Janajati candidates to teacher positions. However, no data are available to assess or substantiate this change.

The share of female teachers at lower secondary had increased from 18.6 per cent in 2007-08 to 24.7 per cent in 2008/09 and for secondary, the share had increased from 11.5 per cent to 15.9 per cent (GoN, 2008e). Despite these achievements, the situation is particularly critical at secondary, where GPI is 0.19 – and even lower at 0.04 in the mountain eco-zone.

The importance of recruiting female teachers for boosting girls’ enrolment and retention is underlined by the EFA joint evaluation. It would seem that more progress has been made in this area in the primary school – greatly facilitated by a larger pool of qualified female primary school teachers. The need for increasing the number of female teachers at the higher classes remains as urgent and relevant as ever.

For secondary school, the share of Dalit teachers had decreased slightly from 3 per cent to 2.3 per cent, which in turn reflects that only few Dalit candidates with a bachelor’s degrees are available. The trend is similar for Janajati teachers: A decrease from 11.2 per cent in 2007-08 to 10.9 per cent in 2008-09 for lower secondary, and a decrease from
9.4 per cent in 2007-08 to 7.2 per cent in 2008-09 for secondary. Despite the increase in relative terms, it is worth noting that the recruitment had increased in absolute terms.

7.7 Assessment

The interventions adopted by the SESP to improve the quality of education have clearly all been relevant and have been met with strong demand.

The efficiency is assessed to be medium. For the curriculum development activities, the process leading to the development of the NCF does not seem to be efficient. The technical assistance (TA) was provided with delay and in inadequate measures which in turn impacted the overall efficiency. Similarly, for the teacher education component, it seems that the use of teacher trainers with higher skills (combined with other factors such as provision of teaching education materials and on-site support for professional development of the teachers) could have led to an overall better use of resources. With these observations in mind, it should be noted that the SESP has been able to deliver most of the outputs within the agreed time according to budget.

The overall quality has increased during the period of SESP implementation, measured by the increase in the SLC pass rate. The qualitative evidence from the district case studies confirms that the quality of education is improving, but also indicates that the change is less significant than what is suggested by the very high increases in SLC pass rates. There is still a long way to go before students’ learning and achievements are significantly improved. The indications of changes in teacher behaviour in the classroom is a notable step in the right direction, but it is clear that the SESP interventions could have been more effective in a different and more facilitating context. For example, the additional challenge of awareness and sensitisation of the social discrimination practices in the community and in the classroom have not been addressed sufficiently. Moreover, the curriculum needs to match better the demands of preparing future citizens of Nepal.

The approach to improving quality has mainly been input-output based and has not adequately targeted factors such as the performance incentives of teachers and students, the inadequate number of subject teachers available to provide quality education to a rapidly increasing student population, inadequate capacity in the implementation system and the need for continuous professional support to teachers. Moreover, it seems that the contextual constraints on the new curriculum have not been taken sufficiently into account. If a new curriculum is to change dominant teaching-learning practices in significant ways, it needs to be built on thorough analyses of the context for which it is intended, and it should consider how instruction is to be conducted and supported to acknowledge the role of contextual constraints.

It is generally agreed that one of the most significant factors for any educational reform to succeed is the education and professional development of the teachers. However, teacher education initiatives within SESP have not addressed some of the key problems faced by teachers, such as how to organise participatory and learner-oriented instruction in situations with very large classes, with few resources and weak infrastructure and in a situation with a relatively weak educational background for many teachers. In a situation with large numbers of teachers who are not educated specifically for the profession and lack subject matter expertise, there is an obvious need to ensure that minimum requirements are met, and therefore to offer short courses for those who are already in the profession. However, in the longer run there is an obvious need to develop more coherent teacher education programmes with a strong professional orientation. This assessment builds on the recognition that teaching is a profession that requires continuous reflection in action (Schön, 1983). It should not primarily be seen as a technical task where pre-developed academic knowledge is used to solve pre-specified educational problems. Rather, it requires teachers to make on-the-spot decisions with regard to the contents, the students, and especially the students when working with the contents (Skott, 2004).

The assessment of the need for a more coherent teacher education programme further draws on the National Research Council (2001) in which teaching proficiency in relation to mathematics is specified. “Such proficiency requires (1) conceptual understanding of the relevant core knowledge; (2) fluency in carrying out instructional routines; (3) strategic competence in planning instruction and in solving problems that arise in instruction; (4) adaptive reasoning, i.e. the ability to reflect on and learn from teaching experience; and (5) a productive disposition towards both the subject and its teaching and learning, meaning that they are able to phrase new and relevant questions in these fields and to develop qualified answers to these questions”. These five strands of teaching proficiency are interwoven and between them, they are considered as a fundamental prerequisite for quality instruction. This calls for the need to situate the education of prospective and practising teachers in close proximity to classroom instruction, both when it relates to the content itself and when it involves the students.

Moreover, programmes for professional development of teachers need to be in line with the emphases and priorities of the education sector as a whole and to take into consideration the constraints imposed on classroom teaching and student learning by factors such as student-teacher-ratios, the quality and availability of teaching-learning materials, physical conditions etc. At the same time, the mentality and performance culture of the teachers may also need to be addressed more explicitly. If changes are to be made through the teacher education programmes, an attitudinal change among the teachers also needs to be promoted. It is important, for example, that the MoE system encourages performance and change.^[4]

One of the intentions of the SESP was furthermore to allow for a more practical approach to the contents, one that takes local conditions seriously. In practice, this has led to a situation in which parts of the curriculum is decided at the district level. The evaluation has found no evidence that such local development has taken place. This is hardly surprising. It is not a simple task to develop high-quality curricula with the materials needed to support it, and it seems somewhat naïve to expect local communities to be able to be in charge of such development. Rather than expecting them to do it on their own, it should be recognised that they are in need of massive support in order to complete the task successfully, or it should be the obligation of the central level to develop curricula that are sensitive also to local educational needs. In either case, it should be acknowledged that it is a task that requires considerable resources.

Further action is required on behalf of the MoE and its implementing partners if some of the key results delivered under SESP are to become sustainable. For the revised curriculum and the new teaching methods to be properly institutionalised, it is important that factors such as scarcity of teachers and lack of performance incentives in the teaching profession are addressed. More generally, it is vital that the gains made in terms of increased access and enrolment are mirrored by corresponding gains in quality and improvements to the learning environment so that retention rates can be increased. Gains made in terms of access are only truly sustainable if they are mirrored by similar increases in quality.

7.8 Specific Recommendations

1. Collection of longitudinal data on student achievement
   SLC pass rates have been adopted by SESP as the key proxy indicator for whether the educational system improves the quality of education. The joint evaluation concludes that a more comprehensive measure is needed to assess student performance. Similarly, the EFA joint evaluation points to the need for a set of school quality indicators to be used for a more comprehensive assessment of progress (Norad 2008: 20).
   
   In order to substantially improve the information available to policymakers as to whether the education system delivers value for money, it is recommended that the GoN initiate a process whereby the collection of subject-wise longitudinal data on student achievement is defined and incorporated in the SSR. The data should preferably be quantified to allow for a consistent assessment of change over time. It must be emphasised that the recommendation is not only to develop and distribute more standardised tests nationwide. The data collection should also involve more qualitative aspects of students’ progress in some of the key subjects, including the students’ development of conceptual understanding, their proficiency in applying their knowledge in real world contexts, and their ability to solve problems within the domains of the different subjects. Such studies could be implemented on a sample basis in part to respond to any issues raised by the findings of the quantitative longitudinal data collection on student achievement. Information from such qualitative studies may feed into a continued curriculum development process. This is not to undermine the relevance of SLC pass rates, but it is clear that they cannot stand alone.
    
2. Further revision of the secondary curriculum
   Despite progress made under the SESP, there is still a need for the development of a “better matched” curriculum and for building the subsequent capacity to implement it. Issues still to be elaborated include addressing social inclusion and discriminatory aspects, which have to be dealt with more strongly. Secondly, the evaluation endorses the proposal to give further emphasis in the secondary curriculum to making students aware of their rights and responsibilities under Nepal’s laws, with a particular focus on the human rights aspects.
    
3. Improve the character and quality of existing teacher education programmes
   It requires a long-term effort and commitment to change entirely the dominant approach to teacher education, including educating teacher educators with the relevant background, developing suitable teaching-learning materials for teacher education, and changing the belief that the best educational background of a teacher is a strong academic background in the subject(s) in question. However, the three recommendations below may also be introduced, even if the ambition is merely to improve the character and quality of existing short-term courses for practising teachers. It is recommended that teacher education programmes are characterised by the following three principles:
    
   1. Modelling good teaching. This means initiating teaching-learning practices in teacher education that resemble situations envisaged in the prospective or practising teachers’ own classrooms.
       
   2. Strengthening the relation between the academic content of instruction in teacher education with the content as taught by the prospective teachers upon graduation. This means that teacher education programmes should – whenever possible – attempt to take the contents as taught in school as a starting point for the development of the participating teachers’ own academic development. Student teachers may then discuss subject matter questions and problems that arise from classroom instruction and deal with them at significantly higher and qualitatively different academic levels than what is expected from their future students.
       
   3. Discussing students’ work and classroom interaction. Teacher education programmes should include a significant element of school-based instruction and discussion of transcripts or examples of classroom interaction or students’ solutions to tasks. Also, and equally important, the student teachers should consider what options the teacher has to promote student learning in each of the examples dealt with.
       
4. Efficient usage of the secure printing press
   To maximise the value from the significant investment in the secure printing press, the GoN should allow the JEMC to use the press to its maximum for purposes within JEMC’s mandate.
    
5. Teacher deployment and management
   Finally, it seems vitally important that the GoN reactivate the teacher deployment system, so that the student-teacher-ratio can be substantially improved with a particular focus on subject matter teachers at the lower secondary and secondary level. Strategies of how to recruit female teachers and teachers from disadvantaged groups should be given particular attention in this context. A more fundamental issue to be addressed is the performance culture of the teachers. The current system does not adequately reward performance and innovative behaviour. MoE and the Teacher Service Commission should be tasked to critically review how the current staff appraisal system can be revised to better reward performance and application of student-centred techniques, for example.

[4] The proposals from the National Research Council in relation to mathematics are in line with many more general suggestions for teacher education. Clandinin and Conelly, for instance, have written extensively about teachers and their qualifications, and in a recent article, Clandinin has developed their argument further (Clandinin 2009). She suggests focusing on “teacher knowledge” rather than “knowledge for teachers”. The latter refers to pieces of knowledge that teachers are expected to acquire independently of context and carry with them into their classrooms. In contrast, teacher knowledge is embedded in teachers’ lives and gained from experience in relevant contexts, and Clandinin worries that “Teacher knowledge, personal practical knowledge, the experiential knowledge teachers construct and reconstruct in life and work contexts and find expression in their practices are given little or no attention in policy statements designed to reform curriculum” (p.7). The recommendations above are meant to remedy this situation.