- Increase fontDecrease font Text size
Mid Term Review of the Resource Allocation Framework (RAF)
Mid Term Review of the Resource Allocation Framework (RAF)
In September 2005 the GEF Council adopted the Resource Allocation Framework as a new system for allocating financial resources to recipient countries. The RAF was endorsed as part of a package of "institutional reforms" of the GEF-4 Replenishment.
The main objectives of the RAF are to:
Increase impact of GEF funding on the global environment; Improve on the existing GEF country-driven approach and partnerships; and Increase the predictability and transparency of GEF funding allocationsFrequently Asked Questions about RAF [ FAQ]
Main Documents
The terms of reference for the Mid-Term Review were approved by the GEF Council in November 2007. To ensure that all legitimate concerns are taken into account when designing and developing this review, the TOR were based on extensive comments from Council members and GEF Focal Points.
Key information about the Mid Term Review of the RAF can be found in the 2-page flyer in: [English] [French] [Russian]
Publication
The Mid Term Review of the RAF was presented to the GEF Council in November 2008.
Technical Papers
The final report was based on the following technical papers:
1. Context and Methodology
2. Design of the RAF
3. RAF Allocations and Utilization
4. Implementation of the RAF
5. Delphi Study of the GEF RAF Benefits and Indices
6. Effects of the RAF on the Small Grants Programme
7. Stakeholder Survey Annex
8. Comparative Review of PBA Systems
2. Design of the RAF
3. RAF Allocations and Utilization
4. Implementation of the RAF
5. Delphi Study of the GEF RAF Benefits and Indices
6. Effects of the RAF on the Small Grants Programme
7. Stakeholder Survey Annex
8. Comparative Review of PBA Systems
Statistical Annex #1 on simulations
Statistical Annex #2 on allocations and portfolio
Statistical Annex #3 on quintiles
Statistical Annex #4 on classification
Statistical Annex #2 on allocations and portfolio
Statistical Annex #3 on quintiles
Statistical Annex #4 on classification
Frequently Asked Questions on the RAF Mid-Term Review
On the review design and methodology• Is there a difference between an evaluation and a review? Not in this case. It is called a mid-term review of the RAF to reflect the language of the GEF Council decision, but it is a regular evaluation following normal evaluation policies and practices.
• What will be the structure of the final MTR report? A draft outline is available in the terms of reference on the internet. The MTR will consider elements of RAF context, design and mostly implementation; changes in roles and responsibilities; changes in the pipeline and projects.
Timing of the review• Is it not too early to evaluate the RAF? It is too early to assess the full and longer-term impact of the RAF and the projects approved under RAF funds. There are, however, many aspects that can already be evaluated, including the effects on roles, pipeline development and approval, partners, funding allocation, etc. Any mid-term evaluation will normally assess if original design is still relevant, if implementation is progressing well, and if any changes are needed to reach the objectives. The RAF will also be subject to an evaluation at the end of GEF-4.
• Is it feasible to apply the results of the MTR to the second half of RAF implementation in view of the short time-frame of RAF implementation and that the GEFSEC needs to prepare changes for November 2008? The GEF Evaluation Office will be interacting with the GEF Secretariat based on the preliminary data collected well before the report is issued. The EO intends to issue the MTR report in October 2008 together with a paper with recommendations for GEFSec on how to fine-tune RAF implementation for the second period.
• How will the MTR ensure accurate results of the MTR when only a limited number of countries have been able to access GEF resources in GEF-4 by now? If countries have not been able to access resources then the MTR would bring this point forward before GEF Council and partners so that they can understand the reasons for this and take appropriate actions. Countries will be expected to help identify the issues of concern, and the MTR can help to share the results with Council.
On results• Will the MTR look at impact? Yes. But impact of the introduction of the RAF itself, not of the ultimate impact on the global environment (of project funded by the RAF). It is too soon to assess the latter one, though we may get some preliminary indications from the changes to the pipeline. The RAF, as a new system of allocating resources, has other effects on the GEF, and these will be reviewed.
• How will the MTR look at cost-effectiveness of the RAF? This is challenging. No comprehensive assessment of GEF cost effectiveness before the RAF exists; and the ultimate results of the RAF are of course not yet ready. The MTR will provide a tentative assessment of the actual costs, which can be considered together with the emerging results.
On recommendations and use of the review• How will the MTR change the RAF? Will the MTR recommend changes to the RAF? It is too soon to say, and will depend on both the MTR recommendations and the decisions by Council. As with all evaluations, the GEF management will be expected to provide a “management response” (by the CEO/GEF Secretariat in coordination with the Agencies) to the GEF Council. The Council will make decisions when the MTR is presented in November 2008. MTRs normally make recommendations to improve implementation.
• Will the MTR influence the reallocation of resources at RAF mid-term point? This is not clear yet. The MTR may have recommendations concerning the reallocation of resources but it will be up to Council to make a decision on this topic. The process and timing for reallocation is managed by the GEF Secretariat. The mid-point of the RAF period is 30 June 2008. The MTR will be presented to the November 2008 Council. It is likely that fundamental changes (if any) would be recommended only after a full evaluation at the end of GEF-4.
• Will the evaluation be addressing the question of expanding the RAF to other Focal Areas (in addition to climate change and biodiversity) in the future if feasible? The review will focus on the areas covered by the RAF – climate change and biodiversity, but we will consider possible spillover effects to other focal areas as well. The MTR would also cover lessons learnt of RAF in these two areas. At the time of its review of the MTR, the GEF Council will expect a GEF Secretariat summary of the technical progress with indicators associated with the other Focal Areas. Consequent to the review of indicators, Council will have to address the feasibility of expanding the RAF to other Focal Areas.
On comparison• Will the MTR compare the “old” system with the new system under RAF? Yes. This implies analyzing historical commitments; roles and responsibilities (what are partners doing differently now); the pipeline and the project cycle. However, several policies and practices have changed (unrelated to the RAF) such as the project cycle. It may therefore not be possible to attribute all system changes to the RAF.
• How will the RAF be compared to other performance-based allocation systems (PBAs) when the results are not yet known? The MTR will compare the design, context, parameters and implementation arrangements. The different PBAs do not all have the same objectives in any case, and so would not have the same results. The GEF performance indices have common elements with other PBAs, but the global environment benefits indices are seen as unique, so a full comparison is not possible. Instead, the assessment of the benefit indices will look at whether they are using the best available scientific data.
On assessment of RAF indices• Will the MTR look at the origin of the RAF, the formulas used for calculation etc.? Yes. The MTR will look at indicators and indices used by the RAF, and consider “what if” scenarios via simulations. The MTR team has the underlying data used for the RAF indices and will look at it in varied ways.
• What is a Delphi approach (and what will it mean to assess the indices)? A Delphi approach (or method) is a useful tool for assessing qualitative information for which there is no hard data. It is often used in the private sector for forecasting. It involves a peer assessment by subject matter experts, who together debate and reach a consensus on the quality of the subject under review. In order to ensure that the conclusion is representative, large groups of experts are involved, from a broad range of standpoints and backgrounds.
• Will the MTR prepare new indicators for the indices? No. it is not appropriate for an evaluation to dictate design. This is a managerial responsibility. The MTR may point to strengths and weaknesses of existing indicators, and possible alternatives.
• Will the MTR look at countries’ policies and institutions for the environment? Yes and no. The MTR will analyze the indices for the performance component, the indicators behind it, the process to develop data and possible alternatives. It is not feasible to assess the actual policies and institutions of the (161) RAF countries in one review.
• Will the MTR review allocations for countries that may fall under group allocation for BD but still be rich in biodiversity? Yes. The MTR will analyze allocation for many different types of countries, including LDCs, and on SIDs. Information in the MTR will be broken-down by region/individual countries.
On modalities and portfolio• Will the MTR look at the changes in numbers of medium-size projects and full-size projects? Yes. This involves analyzing number of approved and pipelined projects, average amounts, timeliness and distribution over countries, Agencies and Operational Programs. The MTR will use the data of the Joint Evaluation of the GEF activity cycle (that has these data for about 2000 projects) as a baseline. However, if few projects have been approved or pipelined under GEF-4, we may not have sufficient data for a statistically significant trend.
• Will the MTR look at adaptation and NAPA follow-up funding in relation to RAF allocations? Adaptation funding from the LDCF and SCCF has been allocated outside the country’s RAF allocation, and will therefore not be covered by the RAF MTR. However, the RAF will consider the issue of adaptation indicators, as well as whether countries intend to use their RAF allocation for adaptation-related projects.
• Will the MTR look at technology transfer among countries, and if this is supported through regional and national projects? The MTR will not look at technology transfer as a specific scope, as this is not one of the RAF immediate objectives. It will how ever, look at the regional projects and the project pipeline, and will address it if raised as an issue there.
• Will the MTR look at changes in the roles and relationships between Agencies? Yes. Some qualitative data will be available from the pipeline and portfolio analysis, complemented by qualitative feedback. Some changes in roles and relationships will not be related to the RAF, such as the new cycle and the policies on Implementing and Executing Agencies and their comparative advantages.
• Will the review take account of how various GEF Secretariat decisions played a role in delays at the beginning of GEF-4? Yes. The MTR team is developing a detailed timeline of events, decisions, instructions and other communications from the Secretariat on the RAF. The timeline will also include delays at different levels and at various points, starting with the decision to implement the RAF itself.
• Will the MTR take account of differences across countries regarding the level of preparation for PIFs? Yes. These differences can vary depending on characteristics of the project pipeline, types and levels of country capacity, and so on.
• Will the MTR take into account the differing nature of climate change and biodiversity programs? Yes. The different nature of index calculations, baseline scenarios, and so on could cause differences in timing of submission of PIFs across Focal Areas. The MTR will also look at historical focal area-related constraints. Also, programmatic approaches play a role in the timing to approval.
• Will the MTR consider “strategic replacement,” or switching of projects from one focal area to another? Yes. The focus of the MTR is on climate change and biodiversity, but we need to analyze the overall effect of RAF on other areas as well. This includes multifocal area projects.
On delivery and timeframes• How will the MTR address the delivery and (under-)utilization of RAF funds so far? The MTR includes several questions to country (and Agency) stakeholders regarding the issue of utilization of funds, phrased in an open-ended way to allow respondents to raise the issues of concern. Some countries have already used the 50% of their individual country allocation; for others, the MTR will seek feedback on why countries have not utilized their RAF funds. Answers to this question will be differentiate according to the type of country (individual country allocation, countries in the group, LDCs, SIDs, etc.)
• Apart from disbursement rates, how will the MTF consider the effects of the RAF on quality of projects? This is a very challenging issue to address. We can utilize various trends in data on operational programs to analyze information on project quality, but to address this question comprehensively we would need to look at relevant data for each project. There are no baseline data, however, on substantive quality of projects. In addition, substantive priorities for projects differed for the GEF-3 and GEF-4 replenishment periods. We will analyze some trends from existing data, but will also rely upon perceptions of quality based on our interviews.
On stakeholder involvement in the review• How will the MTR involve national stakeholders? Mainly through the regular Sub-regional Workshops, and National Dialogues or Constituency meetings organized under the GEF Country Support Programme between March and August 2008 (http://www.gefcountrysupport.org/). The EO expects to interact with Focal Points during the plenary session, in working groups, through country/constituency interviews and through individual focal point interviews. In terms of topics for discussion, the EO is interested in learning about what results Focal Points are expecting, their suggestions on the methodology of the review, and also to find out from them how they see the RAF in relation to other reforms undertaken by the GEF in GEF-4.
• How will the MTR involve NGOs? The EO has established a partnership with the NGO network to ensure NGO and civil society feedback. Mechanisms will involve direct inputs from the network, participation of regional NGO focal points in sub-regional workshops, surveys, select interviews at country level and international NGOs, analysis of NGO involvement in the pipeline, and debate at the NGO dialogue workshop in April 2008.
• Will my answers be confidential? Yes. Following our Office’s standard practices all information received (no matter how) will be treated confidentially. The MTR has set up an email account (RAFevaluation@TheGEF.org) that will be read only by GEF EO staff involved in the MTR. Individuals or institutions will not be directly quoted in the report, unless GEF EO receives permission.
• Will surveys be tailored to different stakeholders? Yes, to the extent possible. Some questions will remain the same so that we have enough answers to analyze statistically. All stakeholders can also express their views in survey comment fields.
The RAF Mid-Term Review Team
The team members leading the evaluation of the Resource Allocation Framework are:
Siv Tokle (Norway, MTR task manager) has over 21 years of development experience. After joining the GEF in 2003, she led evaluations of the GEF Activity cycle and modalities; the experience of the Executing Agencies; and developed the GEF M&E Policy. She covers thematic responsibilities for climate change, ozone and POPs in the EO, and led the last program study of climate change in the GEF.She has worked as Evaluation Advisor in UNDP, where she was responsible for revising the UNDP monitoring and evaluation framework. She has managed many UNDP evaluations. She also worked as a Policy Advisor in the office of the UNDP Administrator, where she covered inter-agency harmonization and program / project management procedures. Ms. Tokle has been stationed for UNDP and UNIDO in the Democratic Republic of Congo, Guinea Bissau, Mali and Austria. She has also worked in the private sector in Norway and India. She holds a Masters in Economics and Business Administration (specialization in international economics and strategic planning).
James L. Fremming (USA, management expert) has been active in international development and social science research for more than fifteen years as a performance management specialist, evaluator, strategic planner and trainer. His diverse sectoral background includes advanced training in political science and substantial experience in environment, democratic governance and equitable economic development. His consultancy work includes Indicator development assistance to major field conservation projects, including EcoGovernance in the Philippines and EPIQ in Indonesia; Design and delivery of training workshops on performance-based evaluation and monitoring for the World Bank Institute), USAID, and other organizations; and Leadership of sector assessments or performance management teams for USAID in Bangladesh, Jordan, Macedonia, Nigeria and Ukraine. Mr. Fremming has taught political science at three major universities, designed and managed congressionally sponsored program evaluations at the U.S. Government Accountability Office, and served public and private sector clients in the US and 20 other countries as an evaluation consultant. He holds Masters degrees in Political Science; and in International Studies. Gary M. Costello (USA, environmental expert) has over 25 years of experience in the fields of environment, natural resources management, biodiversity, biosafety, ecotourism, economics, agriculture and sustainable land management, and institutional development. He has consulted to the World Bank/IFC, GEF, USAID, Asian Development Bank, IFAD, Harvard Institute for International Development, Inter-American Foundation, International Center for Ocean Development, IDRC, Canadian Hunger Foundation and University of Ghana, including the recent Medium Size Project (MSP) Program Evaluation for the WB GEF Coordination Unit. Expertise includes overseeing as many as 20 individual technical assistance and research projects with budgets as large as $30 million and a staff of 20 project managers. He has conducted numerous assignments for evaluations, supervision reports, implementation completion reports, project support, and country Environmental Management Reviews in East, South and Southeast Asia, East and West Africa, Latin America, and the Caribbean. He holds a Masters degree in International Development; in Social Science; and in Public Administration (Concentration in Natural Resource and Environmental Economics). Kenneth Watson (Canada, economist) has been involved in the design and evaluation of performance-based resource allocation systems (PBAS) for seven years. He wrote the summary technical paper on PBAS for the first inter-IFI experts meeting in Manila in 2003; and he has contributed to PBAS analysis at IFAD, the Inter-American Development Bank, the Asian Development Bank, the Caribbean Development Bank and the World Bank IDA. He led the consulting team for the evaluation of the Asian Development Fund VI-VIII, and the evaluation of the Special Development Fund of the CDB, Cycle 5. His doctorate is from Harvard University. Divya Nair (India), GEF EO Evaluation Analyst since 2005, where she led the statistical work for the evaluation of the GEF activity cycle and modalities; conducted fieldwork and analysis for the evaluation of impact on biodiversity in Africa; and is currently working on the evaluation of the catalytic role of the GEF. She has worked on issues of international governance and its effects on the poor, with the International Food Policy Research Institute (IFPRI) in Washington DC, the International Fund for Agriculture and Development (IFAD) in Rome and the Indian Council for Research on International Economic Relations (ICRIER) in New Delhi. She holds a Master in Public Affairs from Princeton University, with a specialization in International Development. As recipient of the British Commonwealth (Chevening) Scholarship, she has completed another MA in International Law and Politics at the University of Hull UK. Yukui Zhou (China/Australia, statistical analyst) has more than 30 years of experience in statistics, survey analysis and data modeling. He has a Ph.D. in Applied Math/Theoretical Physics and postgraduate diplomas in Statistics and Business Systems. Dr. Zhou has served as Assistant director and Analyst for the Australian Bureau of Statistics; consulted on a number of issues including for the U.S. Bureau of Labor Statistics on the commodity and services systems of CPI (price indexes); for the U.S. Bureau of Transportation of Statistics on the Air Travel Price Index system; for the U.S. Bureau of Economic Analysis; and has also worked as researcher in mathematics. Florentina Mulaj (Kosovo) has worked as a consultant on Investment & Policy Promotion for the IFC Foreign Investment Advisory Services in Washington, D.C.; as analyst on Political Economy & Energy in the World Bank Kosovo Office; and as analyst on Management Reform in the Executive Office of the UN Secretary-General in New York. She holds a Master in Public Affairs from Princeton University on International Relations & U.S.-European Transatlantic Cooperation, with Certificate in Science, Technology, and Environmental Policy (STEP), and a B.A. in Economics & International Relations. Shaista Ahmed (USA) has worked as a senior research assistant at the Federal Reserve Board of Governors in Washington, D.C. where she conducted research on anti-trust regulation, efficacy of banking structures, and merger and acquisitions of banks and thrifts. She also examined issues pertaining to children with disabilities and assisted in compiling a qualitative evaluation of country-specific initiatives to advance their social inclusion at the UNICEF Innocenti Research Center. She holds a Master in Public Affairs from Princeton University with a specialization in Economics and Public Policy and a B.A. in Economics and Political Science.The World Perspectives, Inc. (WPI) is a Washington D.C. based consulting firm undertaking the independent Delphi study of the RAF for the midterm review. The description of WPI and the team members leading each of the three areas of the Delphi study are:
World Perspectives, Inc. (WPI), founded in 1980, provides worldwide market intelligence, analysis and consulting services covering both markets and public sector policies. These services are utilized by major international commercial businesses, technology providers, financial institutions, multilateral development banks, investment funds, government agencies and non-profit organizations in the agri-food and natural resource sectors. WPI brings together professionals with extensive industry background and high-level government policy and program management experience, complemented by a global network of consultants with targeted sectoral and geographic expertise, to provide broad-based services for our clients.
WPI is well known for its daily market and policy report, Ag Perspectives, which is used around the world by government officials, producers, trading companies and most of the major multinational companies and cooperatives engaged in agriculture or natural resource policy. The company also provides a wide range of consulting services to these entities covering market and policy analysis, investment analysis, audit and evaluation, feasibility studies and strategic planning support.
Performance: WPI team member Steve Dewhurst served as Director of Budget and Program Analysis (OBPA) for the U.S. Department of Agriculture. During his many years of with the Department he served as a key advisor to 12 cabinet officers in developing, presenting, and executing USDA’s $110+ billion annual budget. He is highly regarded for his understanding of policy, programs and budgets. He is a three time recipient of the Presidential Rank Award for outstanding government service. Biodiversity: WPI consultant Dr. Erik Lichtenberg is a renowned expert in agriculture and environment, land use, and natural resource economics. He is Professor, Department of Agricultural and Resource Economics at the University of Maryland. Dr. Lichtenberg has researched, written and spoken extensively on performance based environmental systems. Climate Change: Dr. Mohan Munasinghe, contributing as a consultant to WPI for this study, presently serves as Chairman, Munasinghe Institute for Development, Colombo; Vice Chair, UN Intergovernmental Panel on Climate Change (IPCC), Geneva; Honorary Senior Advisor to the Government of Sri Lanka; and Visiting Professor, United Nations University, Tokyo. Until recently, he was Senior Advisor/Manager, World Bank, covering sustainable development, environmental policy, water, energy and transport; and Chancellor, International Water Academy, Oslo. After working over 35 years in the international arena, he is recognized as a leading world authority on environment and development, with field experience in climate change adaptation and mitigation, disaster management, economics, environment, energy, telecommunications, transport, urban infrastructure, and water resources projects in most developing countries.
Dr. Munasinghe has won a number of international prizes and honors for research and applications. Most notably, as IPCC Vice Chair, he was co-winner of the 2007 Nobel Peace Prize with other IPCC Team Members and Al Gore, for efforts to build up and disseminate greater knowledge about climate change, and to lay the foundations for the measures that are needed to counteract such change.