IFAD’S RESULTS MEASUREMENT FRAMEWORK
Kevin Cleaver, Associate Vice President, Programmes
Gary Howe, Director, Strategic Planning
April 2010
1
Content of presentation
2

Results areas and indicators that IFAD uses in
managing its operations, resource mobilization,
human resources, risks and efficiency

Measurement and reporting of results

Baseline results

Targets for 8th Replenishment
IFAD uses its loans and grants for
Improve basic foods
and staples
Include cash crops:
exports are growing
Integrate livestock to
match rising demand
3
RMF 2007-09: A snapshot of achievements compared to
the 2005 Independent Evaluation of IFAD
RESULTS
% of projects designed in
1995-2001 period rated
satisfactory by IEE (2005)
Latest results (RIDE, for
Dec 2009 Board)(% of
projects satisfactory)
Medium-term (2011) target
established by IFAD’s Board (%
of projects rated satisfactory)
61%
ARRI – 82%
PCR – 87%
80%
37%
ARRI – 91%
PCR – 80%
70%
25-55%
ARRI – 100%
PCR – 71%
65%
40%
ARRI – 73%
PCR – 75%
80%
n.a. (ARRI)
69%
70%
Satisfactory overall at entry (effectiveness,
poverty impact, sustainability, innovation)
-
100%
90%
% of country programs satisfactory for
adherence to aide effectiveness agenda
-
96%
90%
Average time from project approval to
effectiveness (months)
18
13.7
n.a.
Average time from project approval to first
disbursement (months)
n.a.
20
n.a.
Project effectiveness at completion
Satisfactory impact on poverty at completion
Satisfactory impact in learning, scaling up
and/or knowledge management at completion
Sustainable at completion
Satisfactory country strategy impact on food
security, income empowerment
4
Level 1: Macro outcomes
Level 2: IFAD Country programme and project outcomes
Level 3: IFAD Country programs and project outputs
Level 4: IFAD Country programme and project design
and implementation support
Level 5: IFAD’s Institutional management & efficiency
5
Attribution to IFAD
Strengthens
Outcome more relevant to IFAD
Objectives
IFAD’s second Results Measurement Framework:
the Results Chain
RMF in the context of IFAD: Operating Model
6
Level 1: Macro and sectoral Outcomes
Baseline figures and global targets that IFAD will monitor
Baseline
year
Baseline
value
2012
target
2005
26
21
2002 - 2004
17
10
1.3. MDG 1: Children under 5, underweight % 1/
2005
27
17
1.4. Crop production index (1999-2001 = 100) 2/
2006
112.4
Tracked
1.5. Agricultural value added (annual % growth) 2/
2004
4.1
Tracked
Indicator
1.1. MDG 1: Population living on less than a $1.25 a day (%)
1/
1.2. MDG 1: Prevalence of under-nourishment in population (%) 1/
Source:
1/. United Nations, The Millennium Development Goals Report, 2008
2/. World Bank, World Development Indicators, 2009
7
Level 2: IFAD Country programmes & project
outcomes

Project performance measured at Project start-up and
at completion by:




8
Relevance – consistency of the project objectives with the
priorities of poor rural people
Effectiveness – how well projects perform in delivering against
their objectives; and
Efficiency – how economically resources are converted into
results.
Sustainability, the net benefits sustained beyond the
implementation period
Level 2: IFAD Country programmes & project outcomes
contd …



9
Replication and scaling up, to add value to successful
innovations and
Gender equality and women’s empowerment, integration of
women’s as well as men’s concerns, so that women and men
benefit equally.
Poverty impact - the changes that have been perceived to have
occurred in the lives of poor rural people.
 Food security, availability, access to food, and stability of
access

Physical and financial assets, for income and crisis coping
 Empowerment, as an end in itself, as well as a means of
reducing poverty
Level 1: IFAD Country programmes & project outcomes:
2009 Baseline Values and 2012 Targets
Chart 1: Country Programme and Project Outcomes:
Targets for 2012 and Achievements in 2009
PCR 2008-09 - Baseline
RMF target for 2012, PCR
ARRI 2009
% of projects moderately satisfactory
or better
100%
80%
60%
40%
20%
0%
10
Relevance
Effectiveness
Efficiency
Rural poverty
impact
Sustainability
Innovation,
learning and
replication
Gender equity
PCR 2008-09 - Baseline
94%
84%
65%
83%
75%
71%
76%
RMF target for 2012, PCR
90%
90%
75%
90%
75%
75%
80%
ARRI 2009
91%
82%
55%
91%
73%
100%
Level 2: Data Sources and Reporting

Data obtained from the project completion reports and project
evaluations (summarised in ARRI)

Evaluation instruments use a six-point scale.
6= highly satisfactory,
5 = satisfactory
4= moderately satisfactory,
3= moderately unsatisfactory
2= unsatisfactory,
1 = highly unsatisfactory

Broadly, ratings of 4 and above are taken as acceptable and 3 or
below as unsatisfactory

RMF targets are set mostly as percentage of projects rated 4 or above
at start-up determined by QA findings and at completion
11
Level 3: Project Outputs



At the third level, RMF reports on outputs
Project outputs are reported on an absolute basis
such as hectares of land brought under irrigation,
number of active savers and borrowers.
Key outputs are aggregated and reported



12
against IFAD’s six strategic areas
using the RIMS system
reporting on a sex-disaggregated basis, where
applicable
Level 3: Project outputs against: Baseline Values and
Targets
Indicator
3.1 People receiving services from IFAD-supported projects
Baseline Baseline
year
value
Target
2012
2007
29 million
60 million
2007
57:43
50:50
3.2 Area under constructed/rehabilitated irrigation schemes (ha)
2008
470
thousand
3.3 Common Property Resources (CPR) land under improved
management practices (ha)
2008
3.86
million
3.4 People trained in crop production practices/technologies
Male : Female ratio
2008
1.72
million
50:50
3.5 People trained in livestock production practices/technologies
Male : Female ratio
2008
1.07
million
35:65
Male : Female ratio
13
Level 3: Project outputs against:
Baseline Values and Targets contd …
Indicator
3.6
Active borrowers (Number)
Baseline
year
2008
Male : Female ratio
3.7
Savers (Number)
Baseline Value
4.35 million
52:48
2008
Male : Female ratio
5.44 million
51:49
3.8
Roads constructed/rehabilitated (‘000’ km)
2008
15 thousand
3.9
Marketing groups formed/strengthened (Number)
2008
25 thousand
2008
162 thousand
3.10 People trained in business and entrepreneurship (Number)
Male : Female ratio
53:47
3.11 Enterprises accessing facilitated non-financial services (No)
2008
19 thousand
3.12 People trained in community management topics (No)
2008
672 thousand
38:62
Male : Female ratio
3.13 Village/community action plans prepared (in '000)
14
2008
24 thousand
Target
Level 3: Data Sources and Reporting

RIMS is IFAD’s trademark definition of key impact
indicators, which it suggests be included in project M&E
systems

RIMS operates at three levels of the results chain:
outputs, outcomes and impact, based on the data from
the Project M & E system

By providing common definition of indicators, RIMS
permits aggregation of outputs across all IFAD projects,
as presented in previous tables
15
Results and Impact Management System (RIMS)
Results and Impact Management System (RIMS)

About 40 IFAD projects have established baselines for
impact assessment using special RIMS surveys

RIMS was developed prior to the development of the self
and independent evaluation systems: indicators are
therefore not fully aligned

Management will review RIMS in order to bring about full
coherence among results system

RIMS helps define indicators to be included in M&E
systems; it does not substitute M&E system.
16
Level 4: Country Programme and project management


17
At the fourth level IFAD measures the performance of country
programmes and projects at entry and during implementation
The main focus is on:

Quality at entry of country programmes for income, food security,
empowerment of women and men

Adherence of aid effectiveness agenda in terms of country
ownership, alignment and harmonisation

Quality at entry of projects for effectiveness, poverty impact,
sustainability, & innovation, learning & scaling up

Better implementation support, measured by time elapsed for Board
approval to 1st disbursement, % of problem projects and pro-activity,
time-overrun, and delay in processing withdrawal applications etc.
Level 4: IFAD Country Programme: Baseline and Targets
Source
Baseline
year
Baseline
value (%)
2012
target
(%)
QA
2008/09
>90
90
Client
survey
2009
96
90
4.3 Effectiveness
QA
2008/09
93
90
4.4 Rural poverty impact among the target group, such as
(a) physical and financial assets, (b) food security,
(c) empowerment, and (d) gender equality
QA
2008/09
91
90
4.5 Sustainability of benefits
QA
200809
81
90
4.6 Innovation, learning &/or scaling up
QA
2008/09
86
90
Indicator
% of country programmes rated satisfactory or better for:
4.1 Contribution to increasing the incomes, improving the food
security, and empowering poor rural women & men
4.2 Adherence to aid effectiveness agenda
% of projects rated satisfactory or better at entry for:
18
Level 4: Project Implementation Support
Indicator
Source
Baseline
year
Baseline
value
2012
target
Better Implementation Support
4.7
% of on-going projects actually receiving
international co-financing
PPMS
2008
61
65
4.8
Average time from project approval to first
disbursement (months)
PPMS
2008
20
14
4.9
% of problem projects in which major corrective
actions are taken (Pro-activity index)
Div PPR
2008
63
75
4.10 Percentage of projects for which IFAD performance
rated 4 or better
ARRI
2008
64
75
4.11 % of problem projects in the ongoing portfolio
PPMS
2008-09
19
15
4.12 Percentage of time overrun for average project
PPMS
2008-09
22
20
4.13 Time taken for processing withdrawal applications
(directly supervised)
WATS
2008
35
-10%
over
2009
19
Level 4: Data Sources and Reporting
How undertaken:

The QA at entry rating is done independently by the consultants

The client survey, which reports on the perception of in-country
stakeholders (‘clients’) comprising of representatives of
governments, donor agencies and civil societies, undertaken
annually, ensures anonymity of the respondents

Non-rated absolute numbers, such as average time elapsed, cofinancing levels are generated through PPMS and WATS.

For day to day management monitoring in real time, Operations
Dashboard has been developed. Brings data from different
systems in a user-friendly interface.
20
Level 4: Data Sources and Reporting
21
(cont’d)
Level 5: Institutional management and efficiency

Level 4 deals with what IFAD does operationally to achieve field level
impact through its country programmes – in projects and at the macrolevel of the MDGs

Level 5 addresses how well IFAD manages its resources to support
these operations – specifically, how well:





22
It focuses its workforce on the operational area
It engages the commitment of its workforce
It creates a diverse staff drawing on experience and capabilities
It achieves efficiency in expenditures
It manages risks
Level 5: Baseline and targets
Indicator
5.1
% achieved of replenishment pledges
Source
Baseline
year
Baseline
value
(%)
2012
target
(%)
Peoplesoft
Financials
2008
93
100
Global staff
survey
2008
70%
75%
5.3 % of workforce in programmes (operations)
Peoplesoft HR
2008
56%
65%
5.4 % of workforce from Lists B and C Member States
Peoplesoft HR
2008
33%
Tracked
5.5 % of women in P5 posts and above
Peoplesoft HR
2008
30%
35%
HR
2007
141
100
Peoplesoft
Financials
2008
US$90
Tracked
ROL
2008
76%
20%
POW&B
2008
16.30%
13.5%
5.2 Staff engagement index (% staff positively engaged
in IFAD objectives)
5.6 Average time to fill professional vacancies (days)
5.7 Cost per payslip (US$)
5.8 % of high-priority internal audit recommendations
that are overdue
5.9 Budgeted expenses per US$1 of loan and grant
commitments
23
Managing to achieve results and impact

The RMF targets are not just a basis for reporting, they are the points of
orientation for the management of IFAD

Since 2007, IFAD has operated a comprehensive corporate planning and
performance management system (CPPMS) that concentrates on the
elements of performance directly managed by IFAD.

The CPPMS is IFAD’s system for managing to achieve the results.


It establishes objectives for all managers

It focuses on what managers can actually manage and be directly
accountable for
Focuses on Levels 4 and 5 – on what IFAD directly manages, on the activities
that lead to development and institutional efficiency results
24

Project development and implementation support

The institutional platform for project-related work
The results management matrix - 1


At the corporate level, management to achieve results revolves around
targets and performance indicators for four activity clusters:

Cluster 1. Country programme development and implementation

Cluster 2. High-level policy dialogue, resource mobilization and strategic
communication

Cluster 3. Corporate management, reform and administration

Cluster 4. Support to Members’ governance activities
Each of these groups has sub-clusters with specific targets (what we call
Corporate Management Results – CMRs), e.g. Cluster 1 has:

25
CMR 1 Better country programmes; CMR2 Better project design; and
CMR3 Better project implementation support
The results management matrix - 2

Each CMR includes an objective the achievement of which will
contribute to better performance against the RMF targets

Each CMR has a basis for measuring performance - a number of Key
Performance Indicators, e.g.,

CMR 1: Adherence to aid effectiveness agenda

CMR 2: % of projects rated satisfactory or better at entry for
effectiveness

26
CMR 3: % of problem projects in the ongoing portfolio
The results management process

Each year every department prepares a plan organizing all activities around the CMRs
they directly contribute to (according to function) – according to corporate level priorities

Every division prepares their own plan indicating their planned contribution to
achievement of the corporate and departmental CMRs

Every individual prepares a work plan organizing their contribution to achievement of
divisional plans

Performance is reviewed on a quarterly basis on corporate, departmental and divisional
basis in performance conversations covering every department and every division,
focused on:


progress towards results

risks that need to be dealt with in achieving results

resource management issues
The objective is identification of management actions (including financial and human
resource deployment) that need to be taken in real time to keep performance on track
27
The performance conversation process
- By 2nd week after close of qtr
- Division Directors & staff
- Review performance and risks
- By 3rd week after close of qtr
- Department Heads & Division Directors
- Review performance and risks
(department risk registers)
- By 4th week after close of qtr
- Operations Management Committee &
Executive Management Committee
- Results, resources and risks report
(corporate risk register)
28
Anchorage in day-to-day management systems - 1

This system is not designed to produce reports to the Executive Board.
The reports on progress (RIDE) is produced from the information that
IFAD needs and generates to manage itself for results on a day-to-day
basis.

The key to the effective functioning of managing for results is accurate
information on outputs and inputs. The Management Information
System is being constantly upgraded:
29

The CPPMS brings together all plans, KPIs and risk reports within the
Peoplesoft system

Budget and financial information is drawn from the Peoplesoft financial
and budget systems

Loan and grant information is drawn from the Loan and Grant System (to
be replaced in 2011)
Anchorage in day-to-day management systems - 2

Project performance and supervision data are drawn from the PPMS,
supplemented by:

Project Status Reports On-line

PMD Management Dashboard (reports)

Human resource management data are drawn from the Peoplesoft
HR database, supplemented for reporting and analysis by a new HR
dashboard

Although all of these systems contribute to tracking performance
against RMF, none are specifically for that purpose. They are all
dedicated to normal processes of outcome, output and input tracking.
30
Bringing efficiency under scrutiny

The results-based management system has been focused on
development impact, and the monitoring system in this area is very
robust – contributing to a major up-turn in IFAD’s project performance

There are still issues in management for development results –
particularly data quality

The key issues for further elaboration lie in the efficiency area

31
IFAD has been raising its overall efficiency – and has actually cut
administrative budgets in real terms

It needs the efficiency drive to be based on detailed and accurate
understanding of work load and volumes – to set more realistic targets
and performance indicators

This work is under way, and we expect more comprehensive indicators
to be in place by the end of the 2nd Quarter 2010