Appendix 1: Terms of Reference
Evaluation of the Farmer Field School Approach in ASPS II, Bangladesh
Farmer Field Schools (FFS) is a very popular extension and education approach worldwide. 78 countries are implementing this method (Braun et al, 2006), although in different forms and with varying focus depending on e.g. the national context. The Farmers Field School approach started in Indonesia. In Bangladesh, the method was first used in the early 1990’s through a UNDP funded, FAO implemented Integrated Pest Management (IPM) programme.
Denmark has supported the agricultural sector in Bangladesh for three decades. The support has included FFS activities since the mid 90’s, and in the ongoing Agriculture Sector Programme Support II (ASPS II), FFS activities are a substantial part of the implementation approach.
The ASPS II supports the government of Bangladesh to implement the Poverty Reduction Strategy Paper (PRSP). Major issues in the PRSP are pro-poor economic growth as well as improved human nutrition. In relation to agriculture these were envisaged to be achieved through diversification and by providing support to crop, livestock and fish production. One of the important aspects of poverty is the high rate of malnutrition caused by limited access to diversified food. ASPS II focuses on promotion of food diversification for the poor. The ASPS is in accordance with national policies and strategies, including the National Agricultural Policy, National Livestock Development Policy, the National Fishery Policy, and the New Agricultural Policy.
ASPS II was initiated in 2006 and during 2010 was given an extension up to June 2013. This will be followed by a third phase of support. The overall budget of ASPS II is DKK 610 million of which DKK 531 million is Danida contribution while DKK 79 million is contributed by the Government of Bangladesh. The budget allocation for FFS related activities in ASPS II amounts to DKK 93.2 million excluding international and national adviser support and capital investment.
The ASPS II comprises three components: 1) Agricultural Extension, 2) Regional Fisheries and Livestock Development, and 3) Rural Roads and Market Access, which involve three ministries. Ministry of Agriculture (MoA) has responsibility for the crop production sub-sector, the Ministry of Fisheries and Livestock (MoFL) has responsibility for the fisheries and livestock sub-sectors and the Ministry of Local Government, Rural Development and Cooperatives (LGRD&C) has the responsibility for the rural roads, and marketing facilities.
The FFS approach is utilised in all the three components of the ASPS II, however with variations: The Agriculture Extension Component comprises 20 fixed sessions, covering integrated crop management of rice as well as homestead activities, health and nutrition sessions and sessions on club formation. The target group for the 20 sessions is gender specific. Under the Regional Fisheries and Livestock Development Component farmers and fishers may choose from a menu of 38 modules in aquaculture, poultry rearing (chickens and ducks), small ruminant (goat and sheep) rearing, cattle fattening, dairying, homestead gardening (winter and summer vegetables, tree crops), and nutrition, including health and sanitation and social issues. The 38 modules are not targeted from a gender specific point of view, but the selection of the target group is ensuring a gender balance. The FFS approach has been utilized in Regional Fisheries and Livestock Development Component since 2007 only. In both components FFS is based on season-long training that takes place in the field or by the fish pond, covering the different stages of the crop (crop calendar), livestock or aquaculture system and giving opportunities for experiential learning. The institutional and financial structures differ between the components (ministries), which may have influence on the sustainability of the results and impact in general.
In September 2010, the Embassy in Dhaka suggested that the Danida Evaluation Department (EVAL) should initiate an evaluation of the Farmers Field School approach as applied in ASPS II. The evaluation should ensure documentation on lessons learned and provide inputs for preparation of the third phase of the programme during which continued support to the FFS approach is being considered. It is expected that the evaluation will in particular provide information on the level to which the FFS approach is contributing to increased income and food security at household level and as well as to women’s involvement in development processes in Bangladesh. In a wider perspective the proposed evaluation of the FFS approach is also relevant in terms of assessing the extent to which the approach used in Bangladesh may underpin the Danish Development policy goals of supporting economic development and promoting gender equality (cf. Denmark’s new development policy strategy “Freedom from Poverty – Freedom to change”, 2010).
The evaluation will be conducted as a joint exercise with the Ministry of Agriculture, Extension department representing Bangladesh in the management group for the evaluation. A reference group comprising key stakeholders, such as Ministry of Fisheries and livestock will also be established. Efforts will also be made to involve other development partners (bilateral and multilaterals) in the evaluation process e.g. during country field work and/or evaluation workshops.
2. Scope of Evaluation
The FFS approach is today used worldwide, but whereas the approach originally was used as a method for extension work related to Integrated Pest Management in Indonesia, it is today used for a wide range of thematic issues in very many different settings. The FFS approach may even have different major objectives – from an interest in purely agricultural technical improvements to being an entry point for including social and community development into agricultural development. This expansion of the use of the approach is likely to continue.
In Bangladesh, the FFS approach differs even within the various components of the ASPS II. The point of departure was extension work related to Integrated Pest Management in rice, which was later transferred to other crop production activities in ASPS I.
In ASPS II, the approach formed the basis for extension work related to integrated crop production, and the FFS approach is now further developed under the Regional Fisheries and Livestock Development Component. The development of the approach has been in terms of technical issues as well as different institutional and financial frameworks. These various approaches have developed over a period of some years and have not yet been evaluated in terms of their technical effectiveness and cost efficiency or their influence on the sustainability of the outcome of the support. In order to get the best picture (pro’s and con’s) of the extension approach, the evaluation will focus on the results (including longer term effects and, to the extent possible, also impacts) from the use of FFS in ASPS
II. Where reliable and relevant data exist from ASPS I and other programmes and projects in Bangladesh, such data will be made use of. A literature study of results of other programmes/projects in the region (and from Bangladesh supported by of donors) will be undertaken as a pre-study to the evaluation.
The FFS approach utilized in the Rural Roads and Market Access Component (carried out as part of the Regional Fisheries and Livestock Development Component) will not be evaluated separately, but as a part of the evaluation of the Regional Fisheries and Livestock Development Component.
3. Main Purpose
The main purpose of the evaluation is:
- To analyse and to document – in a gender perspective – the results and the lessons learned from using the Farmer Field School approach in the ASPS II in Bangladesh.
The outcome of the evaluation will feed into the preparation of the expected ASPS III.
4. Evaluation Focus
The analytical part of the evaluation needs to be context specific and in particular to take into account differences in the way the FFS is practiced in different components of the ASPS.
In meeting the objective of the evaluation the emphasis will be on the following focus areas:
- The training mode for improved production: The extent to which the training approach is useful for various types of agricultural and livestock production systems (agriculture, horticulture, poultry, ruminant livestock and aquaculture) in various contexts. The assessment will include the production as a whole as well as various specific aspects of the production (integrated pest management, soil fertility, pond preparation, modes of housing livestock, feed and fodder, etc.)
- The FFS approach and its possible effects on access to production inputs and services including credit and marketing: The extent to which the group formation under FFS may have facilitated access to credit, common procurement and marketing, and future extension services, including access to services from both the public and the private sector.
- Intra-household relationship: Extent to which women’s participation in the training (fully or partly), has influenced the social relationships at household level, including women’s status, their ability and confidence to make decisions and their greater adaptability in the face of challenges and opportunities (see also e).
- Other income generating activities: Extent to which group formation within FFS has facilitated other joint income generating activities among the group members.
- Improved livelihood: Extent to which the FFS approach has influenced the livelihood of households: economically and in terms of household nutrition, as well as in terms of resilience to negative changes and ability to take advantage of opportunities.
- Institutional Arrangements: Extent to which the funding, monitoring and other institutional arrangements used by the components have been appropriate and efficient, and have contributed to the success and sustainability of the FFS approach.
5. Specific Evaluation Criteria and Questions
The OECD/DAC evaluation criteria (relevance, efficiency, effectiveness, impacts, and sustainability) will be applied. In order to address these criteria a list of possible questions are provided in the table below. The inception report should include an elaboration of these questions, and how they will be addressed and investigated.
Table 4.1 Proposed Key Evaluation Questions in relation to the OECD/DAC Criteria
||General relevance of the evaluation criteria (Danida)
||Specific questions in the Evaluation of the FFS approach in Bangladesh|
||- “The extent to which the objectives of a development intervention are consistent with beneficiaries’ requirement, country needs, global priorities and partners’ and donors’ policies”.
||- To what extent is the FFS approach relevant vis-á-vis a) Denmark’s policy/strategies for development cooperation in general, and Danish cooperation with Bangladesh, in particular, b) national policies and strategies for economic development and food security in Bangladesh.|
- To what extent have the technologies discussed in the FFS sessions proved to be relevant and adaptable to the needs of the participating (male and female) farmers?
- To what extent do the Ministry of Agriculture, the Ministry of Fisheries and Livestock or other agricultural programmes use FFS approach(es) outside the ASPS II, and which other methods – and for what reason – are utilised in Bangladesh?
- What are the advantages and disadvantages of various training approaches for different sub-sectors and different target groups?
||- “A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results”.
||- Have the activities been implemented as planned (including time wise) and have allocated resources been put to good use?|
- To what extent have the intended beneficiaries (gender disaggregated data) participated in the various modules and sessions?
- To what extent have the funding, monitoring and other institutional arrangements been appropriate and efficient?
- How do the costs per FFS participant compare with costs of other extension programmes?
- Cost/efficiency: What are the total costs involved per household per FFS compared to increased household benefits?
||- “The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance”.
||- Have activities carried out led to the intended outputs and outcomes of the programme in terms of increased income and food security (or are they on track to do so, considering the time frame), including the gender perspective?|
- How many, and who, within the household have participated in FFS and which activities have they participated in?
- To what extent has the FFS approach had limitations (including gender aspects) and for what reason?
- To what extent has the FFS approach influenced participants (women/men) and others in the community (women/men) to improve access to production inputs and services?
- To what extent has the implementation of FFS facilitated identification of markets, improved marketing methods and/ or influenced sales prices?
- To what extent has the FFS led to the formation of sustainable groups, and if and where this is the case how have these groups benefited the members and the community more widely?
- To what extent are groups formed as a result of FFS activity an avenue for provision of services to farmers?
||- “The positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended”.
||- To what extend has the involved households experienced an increase in income, and food security?|
- To what extent have other household activities been influenced by the FFS approach and what has been the result?
- To what extent has there been a change in family member’s status within the family?
- Has the FFS formation resulted in other group activities, and if so have all previous FFS group members participated? And if not, why not?
- Have FFS inspired other groups to form, and if so, which (economic) activities have these undertaken, and what are the outcome of these activities?
- Have FFS forums, field days and other activities such as cross visits lead to replication of technical or other concepts learned during the FFS by members of the surrounding community
- To what extent has the FFS approaches resulted in change in income and income distribution at intra, and at inter household level?
- To what extent has the FFS approach influenced the nutritional status (and possible health status) of family members and in the community, in general?
- To what extent has the FFS influenced other developments, like education and awareness?
- Are there any other unplanned impacts of the FFS activities from the point of view of the participants or implementers?
||- “The continuation of benefits from a development intervention after major development assistance has been completed. Probability of long-term benefits. There silience to risk of the net benefit flows over time”.
||- How many FFS have been established under ASPSII, how many have been phased-out (according to plan and prior to planned)? How many farmers/HHs have been reached?|
- How many of the programme initiated FFS group structures continued, in which form (Farmers Associations, CBO, clubs, others), for which type of activities, and with what level of programme support if any (including national/ international adviser cost)?
- Have there been constraints in terms of replication and/or continuation of group activities?
- Which institutions have been used to promote FFS in Bangladesh and are such institutions likely to be able to continue after Danish support has been terminated?
6. Approach and Methodology
Evaluations of Danish development activities are carried out in accordance with the OECD/DAC Evaluation Quality Standards which requires, inter alia, that a sound methodology for all evaluations be used and explained in the evaluation report. The purpose of the methodology, and the basis on which its soundness is assessed, is to produce reliable data that allow for valid evaluative judgments that are useful for learning and making decisions (MFA, 2006, p.66). The validity of the individual evaluation however will depend on the data available.
The proposed evaluation of the FFS can be considered a ’thematic’ evaluation, i.e. with a focus on how, when, and in which context FFS is an appropriate extension method in Bangladesh and how, when and in which context this approach may lead to other activities influencing the livelihood on village level in Bangladesh. The FFS evaluation will be a real-time ’learning evaluation’ since the support to FFS is ongoing and since the evaluation will provide inputs for the planning of further support.
The evaluation will build on existing documentation as well as on primary data, to be collected by the evaluation team. In early 2010 a Joint Technical Review of the ASPS II was completed. The review highlights – inter alia – the overall status on the programme institutional setting, financial management aspects and monitoring and reporting, as well as aspects of these on component and sub-component level. One of the recommendations of the joint technical review is to look into the FFS and community approach utilised in the components. Impact studies are taking place as part of the regular monitoring of the programme. In 2010, impact studies on development objectives of the Regional Fisheries and Livestock component have taken place, and an impact study of the Agricultural Extension components is expected finalised in early 2011.
The methodological approach of the evaluation of the FFSs is expected to include the following main elements:
- Desk study of programme documents, including technical review reports, impact studies, progress reports, PPO’s, etc.
- Desk study of relevant national statistics (including health statistics) of Bangladesh.
- Stakeholder analysis through structured interviews with: a) focal groups and individual participants and non-participants of FFSs, b) relevant ministry staff, c) embassy staff d) programme staff, e) selected CBOs, regarding the outcome within the relevant focus areas.
- Survey and/or in-depth qualitative and quantitative analysis of XX selected cases of FFSs in the three geographically areas North West Bangladesh, Greater Barisal, and Greater Noakhali).
- Presentation of preliminary findings in Bangladesh (validation workshop).
A thorough context analysis (including analysis of institutional issues related to the utilization of the FFS) and programme theory based evaluation thinking will constitute important elements of the analytical framework for the evaluation (to be developed by the evaluation team). The analytical framework for the evaluation must also ensure proper consideration of gender issues to facilitate the specific analysis of the gender aspects of the FFS approach.
The methodology must ensure that the question of attribution versus contribution is addressed in a systematic manner. If data allows, use of matching techniques to identify comparison groups (with and without FFS interventions) should be included. Data triangulation and validation of information shall be considered in all aspects of the evaluation.
The evaluation team shall comment on and develop the analytical framework and the methodology further as part of their proposal (bid) and as part of the inception phase. Proposals for improvements/consolidation of the suggested methodology will be welcomed.
To ensure proper use of available data and a realistic design of the evaluation, it is suggested that the inception phase includes a pre-visit to Bangladesh by selected members of the evaluation team. The visit should include an assessment of available data and identify the sites to be visited during the main field work.
7. Time Table
It is the intention that a contract for the evaluation of the Farmer Field School approach can be signed by the end of February 2011 and, that a final draft report can be ready by September 2011.
8. Composition and qualification of the Evaluation Team
The Evaluation Team shall consist of international and regional/local consultants with experience in evaluation of development assistance (i.e. evaluations that conform to the DAC evaluation definition).
The organisation of the team’s work is the responsibility of the consultant and should be specified and explained clearly in the tender. The Team Leader should be an international consultant. The Team Leader is responsible for the team’s reporting to and communication with the Evaluation Management, and for the organisation of the work of the team. The Team Leader will participate in the Evaluation Reference Groups’ meetings and other meetings as required.
The full text of the ToR can be found at http://www.evaluation.dk.
 For example Danish environmental support to FFS in Cambodia has reached more than 80.000 farmers 2000-05.
 Evaluation Guidelines, Ministry of Foreign Affairs/Danida, October 2006 p. 66: Inception: Methodological considerations.
 Only evaluations according to OECD/DAC Guidelines carried out for evaluation clients such as evaluation departments of developments organisations are accepted.
 ’International consultants’ are defined as persons with an international background, i.e. a degree from an internationally recognised university and professional experience from assignments within developing and developed countries.
This page forms part of the publication 'Evaluation of the Farmer Field School Approach in the Agriculture Sector Programme Support Phase II, Bangladesh' as chapter 13 of 14
Version 1.0. 22-12-2011
Publication may be found at the address http://www.netpublikationer.dk/um/11112/index.htm