Royal danish ministry of foreign affairs - Go to the frontpage of um.dk   Publication  
 
 
     
 
 

Annex B Evaluation Matrix

Relevance

The criterion of Relevance relates to the extent to which the objectives of an intervention are consistent with beneficiaries’ requirements, needs, overall priorities and partners’ and donors’ policies, strategies etc.

Evaluation Question Proposed Indicators Means of Verification
(source/method)
1. Is the composition of the portfolio
of Danish support relevant in view
of partner country policies and
strategies?
Level of coherence with
country policies and
strategies (examples)
Document review
(Burkina and Tanzania)

SWOT analysis workshop/
focus groups (Burkina
and Tanzania)
2. Is the composition of the portfolio
of Danish support relevant in view
of Danida policies and strategies?
Level of coherence
with Danida policies and
strategies (examples),
at formulation and
currently
Document review
3. Are the objectives of specific projects/
networks/centres/studies
relevant in view of knowledge gaps
and/or information needs and,
where relevant, research capacity
needs within the substance area
and countries addressed?
Evidence of no
duplication/overlap

Indication that relevant
knowledge/information
is not already available/
existing
Grant Application Forms

Key stakeholder interviews

SWOT analysis workshop/
focus groups (Burkina and
Tanzania)
4. How and to what degree is the
relevance of the overall portfolio
of projects and mix of modalities
continuously ensured (e.g. by
responding to changing Danish
or partner country priorities etc.)?
Changes in portfolio
composition in response
to policy changes
Document review
5. How appropriate is the approach
for formulating research themes
which guide FFU annual calls for
applications. Are the mechanisms
fit for purpose – in terms of ensuring
relevancy, particularly in terms
of usefulness (to partners and/or
Danida)?
Evidence of compliance with
criteria for participation,
demand-driven, gender
sensitive (for example)
Review of procedures
and application process
6. How appropriate and relevant
are the different channels and
modalities, individually and
compositely? Where relevant,
this assessment should be made
in view of the principles set forth
inter alia the Paris Declaration
on Aid Effectiveness ((particularly
alignment and ownership), and
in terms ensuring that research
is demand-driven.
Share of projects being
led by non-Danish
organisations
Online survey (all countries)

SWOT analysis workshop/
focus groups (Burkina and
Tanzania)

Document review
7. How appropriate are the different
strategies for ensuring and
enhancing relevance to end users,
both in terms of content, partner
involvement, dissemination etc.?
Number of projects
conducting end-of-project
events

Level of interaction
with non-research
stakeholders/partners

Number of projects
relevant to end-users

Inclusiveness of dissemination
options
Online survey

Project site visits (Burkina
and Tanzania)

SWOT analysis workshop/
focus groups (Burkina and
Tanzania)

Key stakeholder interviews

Effectiveness

The criterion of Effectiveness relates to the extent to which the intervention’s objectives were achieved, or are expected to be achieved taking into account their relative importance.

 In the present case, this links to assessing the delivered outputs and, to the degree possible, outcomes – in terms of research/knowledge produced, disseminated, put to use etc. and research capacity built – for individual projects and to the degree possible also at portfolio level.

Evaluation Question Proposed Indicators Means of Verification
(source/method)
8. How appropriate/plausible
are assumed causal pathways from
research and/or capacity building
outputs over behavioural outcomes
to wider impacts?

(Previously Q13)
Deviations in implementation
of original plans
Document review

Key stakeholder interviews
(Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Online survey
9. To what extent have planned outputs
and objectives of individual
research projects, special studies,
centres or networks been
achieved?
Level of compliance with
indicators
Document review (Annual
and Completion Reports
for selected projects)

Key stakeholder interviews
(Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Online survey
10. What is the quality level
of produced research?
Number of publications
and citations in Peer review
Journals
Bibliometric analysis
11. What is the nature of the
collaboration established between
Danish and developing country
research institutions? Has Danida
support been decisive in establishing
this collaboration? Might
different forms of collaboration
have emerged in the absence
of Danida support?
Roles and responsibilities
of collaborators

Number of collaborations
that would potentially
have been established
without Danida support
Key stakeholder interviews
(Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Online survey
12. Has sufficient attention has
been given to building research
capacity (at the level of individuals/
institutions/country where
relevant), and have such efforts
have been successful? (Was
capacity built as intended?)
Performance/capacity
improved within targeted
institutions
Key stakeholder interviews
(Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Online survey
13. Have sufficient efforts been
made to disseminate research
findings, and have findings been
communicated in a way that
promotes use in partner countries,
internationally and/or in Danida?
Number of projects with
an appropriate dissemination
strategy/action plan
implemented
Document review

Key stakeholder interviews
((Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Online survey
14. How and to what degree have
research findings been put
to use – internationally, in Danida
programmes and/or by national
partners?
Evidence (examples) on
use of research findings
in follow-up activities
(research, production,
marketing etc.)
Document review (second
and third phase projects)

Key stakeholder interviews
(Selected projects in
Burkina, Tanzania, DK
and selected non-field
countries)

Danida staff (agriculture)

Online survey
15. How adequate are the monitoring
systems and follow up requirements
imposed by Danida?
Performance-based release
of funds

Level of compliance
with reporting formats
(projects)
Document review
(progress/completion
reports, accounts)

DFC/Danida financial/
monitoring reports

Key stakeholder interviews
16. How (if at all) is coherence
and synergy between different
channels and modalities
(competitive grants, PRCP
project grants, research centres,
networks, BSU) ensured?
Existence of mechanisms
in place for ensuring
interaction, sharing of
experiences/information/
learning
Key stakeholder interviews

Efficiency

The criterion of Efficiency can be seen as a measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results.

Evaluation Question Proposed Indicators Means of Verification
(source/method)
17. What is the level of resources
employed in the process of
administering and monitoring
support, and is this appropriate?
This will include a comparison
between different channels/
modalities.

(two questions)
Level of budgets and
human resources allocated
to administration and
monitoring tasks

Timely reporting (monitoring
and financial reports)

Timely feed-back and
follow up on progress
(reports)
DFC/Danida financial and
administrative reports

Key stakeholder interviews
18. Has division of labour between
the FFU, UFT, the Danida Fellowship
Centre, the embassies
and partner institutions been
appropriate – for individual modalities
and in terms of
the mix of modalities?
Level of duplication/
omission of tasks
Task descriptions/ToR/
guidelines

Key stakeholder interviews
19. Have projects been undertaken
as planned, i.e. followed their
time schedule, using the resources
planned, delivering outputs as
planned?
Level of compliance with
budgets/time schedules/
indicators
Document review

Key stakeholder interviews

Online survey
20. Has sufficient attention has been
given to ensuring harmonisation
with other donors’ support?
Information on multiple
funding included in
application forms

Number of joint/collaborative
donor initiatives
Guidelines for grant
application

Document review
(including annual reports
from relevant institutions)

Impact and Sustainability

Impacts and sustainability can be seen as being interrelated in the sense that impacts relates to the wider and longer-term effects, and sustainability to whether effects and achievements will be sustained over time

Evaluation Question Proposed Indicators Means of Verification
(source/method)
21. What differences has the
cooperation, capacity building
and/or research projects made
to the participating researchers
and the participating institutions?
Changes in rank/position/
career development
(researchers)
Changes in performance
of the institutions (activity
levels, partnerships,
funding, networks)
Document review
(including annual reports
from institutions)

Key stakeholder interviews

Online survey
22. What differences have the findings
and/or the application of the
results had with the field of
research/the sector/the local
end users?
Level of changes observed
(new initiatives started
following the project)
Document review
(including annual reports
from institutions)

Key stakeholder interviews

Online survey

Project site visits
23. Who has benefitted from the
improved research capacity
(direct beneficiaries such as
those researchers who have been
trained, as well as the indirect
beneficiaries, such as their
students) and how?
Number of direct
beneficiaries

List of indirect
beneficiaries

Nature/type of benefits
Document review

Key stakeholder interviews

Online survey
24. Who has benefitted from the
application of the findings
to help solve development
problems and how?

(linked to Q13)
List/type of beneficiaries
and functions
Document review

Key stakeholder interviews

Online survey
 


This page forms part of the publication 'Evaluation of Danida supported Research on Agriculture and Natural Resource Management 2006-2011' as chapter 16 of 16
Version 1.0. 09-09-2013
Publication may be found at the address http://www.netpublikationer.dk/um/11214/index.htm

 

 
 
 
 
  Ministry of Foreign Affairs of Denmark © | www.um.dk