Showing 1-26 of 26 results

Records of the Office of Operations Evaluation Série organique

Policies and Procedures

This series consists of records related to the development of policies and procedures for the OED. Policies and procedures for OED programs and work products were developed in intra-OED correspondence, staff meetings, retreats, committees, and working groups and via Bank-wide task forces, committees, issuances, and operational manuals. The majority of the series consists of records created and received by the OED Directors-General (OEDDG) and OED Directors (OEDDR) that relate to the development of Department policies and procedures. Most of these guidelines relate to the creation of OED work products, such as audit reports, impact evaluation reports, Country Assistance Reviews and Evaluations, and special evaluation studies.

The series also consists of records documenting meetings of the Department Management Team (DMT) as well as records related to Departmental Management Team retreats.

Bound Reports and Studies Maintained by the DGO's Reference Library

The series consists of three sets of leather bound volumes that were maintained in the Office of the Director-General, Operations Evaluation (DGO). Volumes are titled: Annual Reports - Annual Reviews - JAC Reports - JAC Subcommittee Reports (October 1971 - November 1995); Project Evaluation - Project Performance Audit Reports (PPARs) - Project Completion Reports (October 1972 - August 1994); and Special Studies - Impact Evaluation Reports (July 1973 - August 1994).

Records of the Director, Agriculture and Human Development Division (OEDD1)

Series consists of records created and received by the Director of OED's Agriculture and Human Development Division (OEDD1). Specifically, the series contains records gathered by OEDD1 Director Graham Donaldson, including reports, correspondence, intra-OED memoranda, agendas and minutes of meetings, and seminar outlines. The records generally relate to the extension of OED's influence and the dissemination and use of OED products throughout the Bank, in international organizations, and in other countries. Among the topics covered are: a review of OED reports and advice regarding their publication; policy regarding the dissemination of OED findings and information; the establishment of operations evaluation programs in other countries and in the Inter-American Development Bank; the Bank's coordination with countries and other organizations doing post evaluations of projects fully or partially funded by the Bank; Bank-provided training on program monitoring and evaluation; and the Evaluation Capability Development Program. Included are a number of requests from Bank staff and from non-Bank organizations for specific OED reports. Two large folders on the Joint Audit Committee (JAC) include: intra-OED memoranda regarding the Department's responses to specific JAC inquiries and requests; agenda and minutes of JAC meetings; the OED Director's requests for Donaldson's comments on JAC reports; OED's proposals for the JAC work program; and OED comments on the Report of the Working Group on Project Completion Reports and on the Report of the Task Force on Dissemination and Utilization of OED Findings.

Chronological File of Director-General Robert Picciotto

This series contains copies of the letters, memoranda, and electronic messages Robert Picciotto sent from September 1992 to June 2001, the period during which he was Director-General, Operations Evaluation (DGO).

Picciotto's files for September 1992 to December 1996 contain only copies of outgoing correspondence; copies of attachments to his outgoing correspondence are not routinely included until 1996. Beginning in January 1997, incoming letters and electronic messages are filed with Picciotto's replies,and his handwritten responses are often written on incoming correspondence. His correspondence is with: President Wolfensohn; OED staff; officials in other organizations and countries who were responsible for performance evaluations or development assistance; officials in countries in which OED was conducting a Country Assistance Review (CAR) or Evaluation (CAE); editors of scholarly journals; members of the Committee on Development Effectiveness (CODE) and its predecessor, the Joint Audit Committee; and members of the Evaluation Cooperation Group (ECG). His correspondence with OED staff primarily concerns: OED work programs, evaluation process issues and specific evaluation projects; planning for OED seminars and workshops and for the annual OED staff retreat; the deliberations of the Evaluation Cooperation Group (ECG) which Picciotto chaired; efforts to establish an International Development Evaluation Association; and the establishment of the Comprehensive Development Framework (CDF) Evaluation Steering Committee. Specific records include: Picciotto's replies to applicants for Bank positions; his comments on draft Bank publications or directives; his response to a report regarding the Bank from the Allan Meltzer Commission; his input for President Wolfensohn's speeches or articles; copies of his own articles; and a draft of the OED booklet, Evaluation in the Bank and IFC. This series also includes nine letters and memoranda, dated July 1992 - August 1992, which Picciotto sent while still Vice President of Corporate Planning and Budgeting (CPBVP).

Correspondence of the Directors-General, Operations Evaluation with Bilateral and International Organizations

The series consists of letters, memoranda, and messages between the DGO and the heads of evaluation units of bilateral and international organizations. The series documents primarily the formulation, implementation, and impact of evaluations within these bilateral and international organizations. Some of the correspondence is with the Special Advisor and Assistant to the DGO and the Director, Operations Evaluation Department (OEDDR). The series also contains copies of studies, publications, and reports of audits and evaluations sent to DGO. A copy of a February 15, 1980 memorandum from President Robert S. McNamara to DGO Mervyn L. Weiner addresses Bank support for upgrading national audit institutions. With this memorandum is correspondence describing implementation actions taken by the Bank and cooperation with the International Organization of Supreme Audit Institutions (INTOSA).

Director-General Robert Picciotto's Files on the Independent Evaluation of the Pilot Phase of the Global Environment Facility (GEF)

The series consists of records related to the Global Environment Facility (GEF). The GEF was a three-year pilot program begun in 1991 to provide grants and low interest loans to developing countries for programs to relieve pressures on global ecosystems. The Facility was a co-operative venture among national governments, the World Bank, the United Nations Development Program (UNDP) and the United Nations Environment Program (UNEP). In April 1992, GEF participants agreed that its structure should be modified, and at a subsequent meeting in December of that year in Abidjan the participants requested an independent evaluation of the Pilot Phase.

Correspondence regarding discussions held within the Bank regarding the GEF independent evaluation of the Pilot Phase are part of the series, including minutes of a January 25, 1993, meeting on the topic convened by the Chief, Infrastructure and Energy Division (OEDD3) at which ENVGE (Global Environment Facility Administration) was represented. Drafts of the Terms of Reference (TOR) to be presented at a GEF meeting in Rome on March 4-5, 1993, are part of the series. Drafts of the Inception Memorandum for the independent evaluation, the Inception Report, and the Terms of Reference prepared after the Rome meeting are also part of the series. Most of these records were prepared following a March 12, 1993, meeting attended by the Deputy Director of UNDP's Central Evaluation Office (CEO), the Chief of OEDD3, and an OEDD3 consultant (John Malone). Establishment of an Independent Panel of Experts for the GEF evaluation was discussed at this meeting. Copies of letters of invitation extended to individuals selected to serve on the panel are part of the series.

Memoranda, paper copies of electronic and facsimile messages, correspondence, newspaper articles, and news releases in the series document the work of the Independent Panel of Experts (Independent Evaluation Panel [IEP]) and the three Evaluation Managers (Robert Picciotto - DGO, World Bank, Gus Edgren - UNDP, and Nay Htun -UNEP) responsible for conduct of the evaluation. Also documented are the managers' contacts with the Chairperson of IEP (Dr. Alvaro Umata), the principal evaluators (Jim Kelly - UNDP, Andrea Matte-Bakers, and Stjepan Keckes - UNEP, John Malone - World Bank), and the Evaluation Staff Coordinator (W. Haven North - World Bank).

The series includes: the final Inception Memorandum; work plan outline and budget for the evaluation; agendas, discussion papers, and minutes of meetings of the IEP and of GEF participants; copies of the questionnaires distributed to GEF participants as part of the evaluation; a draft Project Evaluation Form (PEF) consisting of a summary of questionnaire responses; and the interim and final reports of the evaluation. A copy of Global Environment Facility-The Pilot Phase and Beyond, Working Paper Series, Number I, May 1992, contains the text on the restructuring of the GEF approved by participating governments on April 30, 1992. Also in the series is the July 20, 1993, Back-to-Office report of Hideki Mori (OEDD3), of a GEF evaluation mission to Japan, Thailand, and Lao People's Democratic Republic from June 16 to July 5, 1993.

Special Evaluation Studies

The special evaluation studies build on the primary assessments covered by Completion and Audit Reports and Impact Evaluations. Those primary evaluation products for specific projects generate data, findings, and lessons that support the special evaluation studies which examine the impact of a number of projects and their implications for the World Bank's policies, practices, and procedures. The special evaluation studies cover topics and issues raised by the Executive Directors and by management and operations staff as well those emerging from audit work. The studies focus on countries, sectors, Bank processes and policies, and broad themes.

  • Sector Studies- Address the effectiveness of Bank programs on sector policies and on project selection, design, and implementation. Many of the sector studies compare experiences across countries (examples include studies of human resource development, Bank experience in the Education Sector, and rural water supplies), whereas others are in-depth reviews of particular sectors in individual countries (e.g., educational development in Korea, population policies and programs in Kenya).

  • Process- Studies Evaluate the Bank's business processes from a development perspective. These studies examine how well the Bank applies its policies and evaluate the efficiency and effectiveness of Bank business practices. Examples include: studies of: poverty assessments; Environmental Impact Assessments and Action Plans; and Project Appraisals.

  • Policy Reviews- Examples of topics covered by such studies include: experience with agricultural research; fiscal management; financial sector reform; and experience in large dams.

  • Thematic Studies- Examine broad themes that encompass multiple sectors and countries and themes of special emphasis for the Bank, such as: resettlement policy; gender; anti-corruption; and poverty alleviation.

The content of the files for the special studies varies but the more complete files contain: drafts and final copies of an Initiating Memorandum (IM), a Study Design Paper, and an Approach Paper; comments on these initial documents from OED staff and from members of the Committee on Development Effectiveness (CODE) or its predecessor, the Joint Audit Committee (JAC); Terms of Reference for the OED staff members and/or consultants assigned to the study; copies of previously issued Bank and OED reports, studies, and audits relating to the study topic; Back-to-Office Reports from study missions; intra-OED memoranda regarding methodology, schedules, and progress on the study; various drafts of the study with comments on the drafts from OED staff, Bank staff in other units, representatives of other development and aid organizations, and officials in client countries; drafts of the DGO's comments to CODE (or JAC) regarding the evaluation; minutes of CODE (or JAC) meetings at which the study was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; and summaries of discussions and minutes of meetings of the Executive Directors at which the study was discussed; copies of the OED Precis and Fast Track Brief relating to the study; and copies of letters transmitting the final report to interested parties.

Country Assistance Evaluations

The series consists of Country Assistance Reviews (CARs), Country Assistance Notes (CANs), Country Assistance Evaluations (CAEs), and related records created by the OED. Country Assistance Reviews (CARs) were launched in 1994 when they were endorsed by the Joint Audit Committee (JAC) Subcommittee on OED Reports. OED's first CAR focused on Ghana and was completed late in FY 95. The CARs were, in essence, countrywide impact evaluations that concentrated on the overall impact and development effectiveness of the Bank's whole program of assistance (for both lending and non-lending activities) to a country over a number of years, usually a decade. They evaluated the Bank's overall assistance program to a country in the same way that performance audits evaluated individual projects. CARs processing schedules were eventually timed and designed so as to be useful inputs into the preparatory work for the corresponding Country Assistance Strategy by both Bank staff and government officials and to provide the ExecutiveBoard with an up-to-date, independent evaluation of past results of Bank assistance one to three months in advance of the scheduled CAS consideration.

Country Assistance Notes (CANs) were introduced in 1997 as a way to speed the process of country assistance evaluations. CANs were meant to be mini-CARs and thus were more limited in the scope and/or time frame covered. For example, CARs covered all relevant sectors of a country, whereas the CANs were more selective. The rationale for introducing the CANs was to achieve greater coverage of countries in the short term. After the introduction of CANs, CARs were limited to the largest borrowers, in total or per capita, or to outliers in terms of performance and GDP per capita. CANs were produced for countries undergoing a second evaluation and for which a CAR had previously been produced. The length of both the CARs and CANs changed over time. The first CARs were typically about 100 pages; later CARs were limited to 15 to 30 pages of main report. The CANs were shortened to 5 to 15 pages. As time went by, OED softened the distinction between CARs and CANs and by the late 1990s, the country-wide evaluations were simply referred to as Country Assistance Evaluations (CAEs), but OED did generate mini CAEs that were the equivalent of CANs.

CARs, CANs, and CAEs were based largely on a review of OED and Bank Reports available in Washington: Staff Appraisal Reports; Performance Audit and Implementation Completion Reports; Country Assistance Program/Strategy Papers; Country Briefs; Country Economic Memoranda; sector reports; Country Risk Assessments; management briefs for the Annual Meetings and for high-level field visits; and project and general country files. They also relied on direct interviews with key current and former government officials, other relevant national and/or association representatives, donor representatives, and current and former Bank staff members. CANs did not usually involve an OED mission to the client country and personal contact with other donors, whereas CARs may have involved more than one mission and increasingly included evaluation participants from other donor agencies. CARs, CANs, and CAEs were all prepared in close consultation with the Regional operational staff.

The content of the files for the country assistance evaluations varies somewhat but the more complete files contain: copies of the previous OED and Bank studies and reports for the country; Terms of Reference for the OED staff members and/or consultants assigned to the project team; drafts of the Approach Paper and Study Design Paper; memoranda and correspondence regarding the team's travel plans if a mission to the client country was required; a Back-to-Office Report regarding the mission; intra-OED correspondence regarding the format for the evaluation; various drafts of the evaluation; minutes of One-Stop Review Meetings at which the top OED managers evaluated progress on or reviewed a draft of the evaluation; comments on the drafts from OED and Regional staff, officials in the client country, and representatives from other aid/lending organization; drafts of the DGO's comments to CODE regarding the evaluation; minutes of CODE meetings at which the evaluation was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; summaries of the Executive Directors' discussion of the report and copies of the chairman's published comments; drafts of the management response to the evaluation; a draft of the OED Fast Track Brief and Precis regarding the evaluation; the final, published (grey cover) evaluation; and letters transmitting the final, published evaluation to officials in the client country. A few files also contain questionnaires completed by stakeholders, paper copies made from computer-assisted presentations, and copies of records used in workshops held for various stakeholders.

All of the country assistance evaluations in this series were generated by the OED's Country Evaluation and Regional Relations Division/Group (OEDCR) and its predecessor, the Country Policy, Industry and Finance Division (OEDD2). The series contains reports and related working files for many countries, including: the Russian Federation, Bulgaria, Lithuania, Paraguay, El Salvador, Brazil, Peru, Haiti, the Dominican Republic, Guatemala, Chile, Guinea, Niger, Eritrea, Zambia, Lesotho, Jordan, Mongolia, India, Zambia, Yemen, Tunisia, Nepal, Pakistan, Costa Rica, Malawi, Kenya, Thailand, Cameroon, Zimbabwe, Uruguay, Croatia, Uganda, Ghana, Cote D'Ivoire, Cambodia, Mozambique, West Bank and Gaza, Papua New Guinea, Philippines, Ecuador, Tanzania, India, Poland, Ecuador, Argentina, Chile, Maldives, Kazakhstan, Indonesia, Morocco, Mexico, Albania, Azerbaijan, Bangladesh, Burkina Faso, Egypt, Jamaica, Sri Lanka, Tgo, Ukraine, Brazil, Armenia, Belarus, Bolivia, Nepal, China, and Turkey. Of special note in this series is the extensive documentation for the CAR for India completed by a team headed by Gianni Zanini and issued in 2001. In addition to the usual documentation generatedfor a country assistance evaluation, the files for the India CAR include an extensive collection of Bank reports on India, materials from workshops organized by OED and the South Asia Region from March to May 2000, and twenty or more background papers prepared by OED staff and consultants. Also of note are the voluminous files for the Russian Federation CAR which, in addition to the usual CAE records, include schedules and briefing papers for President Wolfensohn's visits to Moscow in May 1998 and February2000.

Subject Files of the Directors-General, Operations Evaluation

This series consists of subject files containing: copies of the Director-General's outgoing correspondence; and the reports, articles, minutes, agenda of meetings, and other records received by the DGO. Much of the series relates to routine OED administrative matters but the files cover a broad range of topics: the 1993 review of OED's processes and structure (filed under OED Process Review); changes in the Operational Directives affecting OED (filed under Operational Directives); the establishment of an independent Inspection Panel in the Bank (filed under Operations Inspections in the Bank); the establishment of the OED's Publication Committee; Failed States and UN/World Bank cooperation in post-conflict reconstruction; and rules of conduct for OED staff. Specific documents include: summaries, drafted by OED attendees, of discussions of OED-related topics covered in Executive Board meetings (filed under Board of Executive Directors); invitations, trip itineraries, copies of presentations made by OED staff, agenda, and other documents regarding conferences and seminars attended by OED personnel (filed under Conferences and Seminars); Apex reports for OED; copies of statements and presentations made by the Director-General (filed under Presentations) agenda, articles, and other documents relating to the 1994 Joint DAC/UNDP/World Bank Seminar on Technical Cooperation; the program packages provided to staff members prior to 1993 and 1995 OED staff retreats; and copies of notes summarizing the weekly meetings the Director-General had with the OED managers, November 1989 - May 1992 and September 1992 - June 1995 (filed under Management Team Meetings). Filed under Miscellaneous are personal notes sent to the Director-General and much of his correspondence with job applicants and other non-Bank personnel. The files for the Task Force on Portfolio Management contain copies of memoranda regarding the changing mandate of OED.

Director-General Robert Picciotto's Files Relating Primarily to Committees, Commissions, Task Forces, and Boards on which he served

The series consists of files compiled by DGO Robert Picciotto that relate to topics not directly connected with the work of OED; most of the files concern committees, commissions, task forces, or boards on which Picciotto served. The majority of files concerns the Bank-wide Social Development Task Force (SDTF) which was established early in 1996 as an acknowledgement of the social dimensions of development in project design and to take a considered look at the various initiatives and approaches underway onsocial development and to make recommendations on further actions to advance [that] work. Picciotto served as chair of the Task Force's Satellite Group 9 on Bank Instruments of Lending. Picciotto's Task Force records include: agenda for meetings and retreats; copies of and comments on the drafts of the Group 9 report; correspondence with other members of Group 9 and with other Task Force members; and copies of input from the other Task Force groups.

Related to the SDTF are records filed under Adapting Lending Products and Adaptable Lending Products. Picciotto was a member of the New Products Steering Committee which was organized to implement the recommendations of the SDTF. That led to the Adaptable Lending Group which first met in March 1997 to create a new line of lending products recommended for Bank-wide adaptation to implement SDTF goals. The Executive Directors approved the Adaptable Lending Instruments in September 1997. The files contain Picciotto's correspondence regarding the development of the new lending products.

Filed under Project Implementation Commission - Brazil are Picciotto's records, dated July 1999 - August 1996, for the Brazil/World Bank Commission on Improving the Developmental Effectiveness of World Bank Financial Projects. Picciotto was a member of the joint Bank/Brazil Commission which was created to improve the Bank's portfolio in three sectors (education, rural poverty, and health) in the Northeast Region of Brazil and in the environment sector across Brazil. Picciotto's recordsinclude: correspondence with Commission members; Terms of Reference for consultants assigned to work with the Commission; minutes of Commission meetings; intra-OED correspondence regarding audits of Brazilian projects and the establishment of new projects in Brazil; draft outlines of the Commission's final report; drafts of the Executive Summary of the report; and comments regarding drafts of the final report. Drafts of the entire report are not in the files.

Picciotto served on the Joint Fund-Bank Committee on Finance and Development which was the advisory board for Finance and Development (F and D), the quarterly publication of the International Monetary Fund and the Bank. His Board records (filed under Internal Correspondence - Finance and Development) date from December 1987 when he was Vice President for Corporate Planning and Budgeting. Included are: his correspondence with the editor of F and D; agenda and minutes of Board meetings; copies of book reviews he provided for publication; resumes of job applicants for positions with the publication; articles submitted for publication which were forwarded to Picciotto for comment; and a copy of an article, A New Project Cycle for the World Bank? which he and Rachel Weaving submitted for publication in October 1994.

In addition to board, task force, and committee records, this series includes files on several other non-OED topics. Filed under Socratic Seminar Files are Picciotto's records regarding his coordination work for a television program, Banking on the Poor, which was part of a public television series The Fred Friendly Seminars. Picciotto had the lead in planning for the October 1, 1994, program which took place in Madrid, Spain, at the time of the Bank's annual meeting. It was the third in a series sponsored by the Bank to examine the role of international finance in fighting poverty. The program followed the Socratic Seminar format, created in the 1970s by Fred Friendly and Stuart Sucherman. Picciotto's records are dated May to October 1994 and include concept papers and e-mails regarding logistical planning for the program.

This series also includes copies of June 1996 - June 1998 correspondence (filed under Knowledge Center - Management) Picciotto received regarding the Knowledge Management Board that was established in July 1997 as well as copies of his comments on proposals for knowledge management within the Bank.

Correspondence Files of the Directors-General, Operations Evaluation, Relating to the Establishment of and Participation in DAC and OECD

The series consists of correspondence files containing communications between the DGO and representatives of the Organization for Economic Cooperation and Development (OECD). The correspondence relates to aid evaluation and the promotion of evaluation functions in client countries. This exchange culminated on December 1, 1980, in DGO Mervyn Weiner's nomination of Shiv Kapur, OEDDR, as the Bank's correspondent for DAC's work in aid effectiveness. Following Kapur's selection as DGO, Yukinori Watanabe, OEDDR,was nominated on October 15, 1984, as the Bank's representative to DAC's Expert Group on Evaluation.

Correspondence and other documents for the period 1981 - 1995 relate to the establishment of DAC's Expert Group on Aid Coordination (later Evaluation) which held its first meeting in Paris on March 8 and 9, 1983, and include meeting schedules and agenda, meeting summaries, reports, seminar papers and other correspondence of the Directors-General who served as the Bank's representatives at high level meetings of the DAC Expert Group on Aid Evaluation during this period. Topics include: primary health care; technical cooperation; non-governmental organizations (NGOs); the economic environment; gender issues; public support for aid; and aid effectiveness. Correspondence regarding DGO Robert Picciotto's participation in the Third Annual Conference on Environmentally Sustainable Development held in October 1995 is also included.

Subject files covering the period 1984 - 1989 consist primarily of issue papers and proposals prepared for discussions at DAC meetings and seminars as well as correspondence dealing with issues of coordination, evaluation and the environment. Subjects covered in the records relate to institution building, health, the environmental impact of development projects and the Economic Development Institute (EDI) evaluation function. Documents prepared for a November 1988 meeting with NGOs are also in the files.

Transparencies for Presentations Made by the Directors-General, Operations Evaluation

Series consists of transparencies and/or transparency master (hard) copies prepared for presentations made by the Director-General, Operations Evaluation (DGO), within the Bank and to external groups. Most of the presentations dealt with the role played by OED in supporting the Bank's development agenda. Some of the transparencies are accompanied by copies of speeches or presentation outlines.

Annual Reviews and Annual Reports

The series consists of records created and received by the Directors, Operation Evaluation Department (OEDDR), and the Directors-General, Operations Evaluation (DGO), which relate to the creation of Annual Reviews of OED evaluations as well as Annual Reports on Operations Evaluation (AROE).

In September 1975, the Operations Evaluation Department (OED) began issuing a report known as the Annual Review of Project Performance Audit Results (ARPPARs) as a means of disseminating the results of the project performance audits it conducted. This review presented in summary the major findings of the audit reports for completed projects and the Bank's response to them. The findings of the ARPPARs were discussed by the Executive Board's Joint Audit Committee (JAC) and JAC's findings were then reviewed by the full Board. By July 1981, a selective system of performance auditing was implemented in OED in keeping with the increase in the number of projects funded. In 1986, the report was renamed the Annual Review of Project Performance Results (ARPPR), and by 1989, the ARPPR was replaced by the Annual Review of Evaluation Results (ARER) which provided a comprehensive summary of evaluation findings. By 1997, the name of the report had changed once again to the Annual Review of Development Effectiveness (ARDE). In the ARDE, OED used the outcomes of evaluations of completed operations to provide a longer view of performance trends. Records in this series that relate to these types of Annual Reviews include drafts, correspondence, memoranda, press releases, comments, and meeting minutes.

The first Annual Report on Operations Evaluation (AROE) covering the period July 1975 - June 1976 was published in October 1976. The main purpose of the report was to indicate the status of evaluation work in the Bank and to comment on the main findings of evaluation activities. Subsequent reports reviewed the independent evaluation work performed within OED and the self-evaluation conducted by the Bank's operating departments (for example, the AROE for 1985 covered OED evaluation activities and evaluations conducted within the International Finance Corporation [IFC], the Economic Development Institute [EDI], and the Bank's economics and research staff). The format of the AROE was determined by the DGO and the plan for developing the report was made in consultation with OED. Drafts of the report were circulated to Bank managers for comments and the final version was submitted to JAC for consideration by the Bank's Board of Executive Directors in conjunction with JAC's own report on evaluation and the ARPPAR (later ARER and ARDE). Records in this series that relate to the creation of AROEs include correspondence, memoranda, initiating notes and outlines, drafts, comments, and meeting minutes.

Records of the Director, Country Policy, Industry and Finance Division (OEDD2)

The series consists of records created and received by the Director of OED's Country Policy, Industry and Finance Division (OEDD2). Records relate to Initial Executive Project Summaries (IEPS), adjustment lending, and other aspects of OEDD2's work. A very small number of records were added to the files by the Sector and Thematic Evaluation (OEDST) and Country Evaluation and Regional Relations (OEDCR) Divisions.

The series includes records containing comments that OEDD2 staff made on Initial Executive Project Summaries (IEPS) and on other project-related records generated by the Regions, including Final Executive Project Summaries (FEPS), Initiating Memoranda (IM), and Project Concept Documents. The record on which the comments were made is not always in the file, but in some cases the file does contain minutes from the OEDD2 staff review meeting at which the record was discussed.

Also contained in this series are files relating to adjustment lending. These include: copies of papers presented on various aspects of adjustment lending; memos on issues to be covered in adjustment audits; copies of statements made before the Board on structural adjustment lending; and memos on performance indicators for adjustment lending. Also included are background files for the FY 93 OED study of adjustment lending in Sub-Saharan Africa and the FY 96 OED report on The Social Impact of Adjustment Lending.

Also part of the series is OED's response to an Upstream Activities initiative encouraged by Bank President Wolfensohn toexpand OED's real time involvement in initial executive project summaries (IEPS) and initiating memoranda (IMs) for adjustment lending.

The series also includes files documenting activities associated with annual reviews and major studies undertaken by OEDD2 primarily between the years 1992 and 1996. A file containing records used in the 1991 annual review of performance results of 49 transport projects evaluated by OED and a file containing planning documents and drafts of the 1995 study of industrial restructuring are contained in the series. Also part of the series is files created to document activities associated with country assistance and other reviews. Files related to studies on OED methodology and procedures, conferences and seminars in which OED participated, and OED replies to studies and reports prepared in other Bank units are also included.

The series also includes project performance files containing draft and printed copies of a series of sectoral and project performance indicators study reports. This series of reports was developed as part of follow-up to the November 1992 Wappenhans Report (Report of the Task Force on Portfolio Management). Audit files were created for copies of comments received in OEDD2 in 1995 from reviewers of evaluative memoranda for Implementation Completion Reports (ICRs) and Performance Audit Reports (PARs) prepared in the division. These copies were retained as part of an initiative to incorporate findings or issues raised in the comments into divisional ledgers.

Chronological Files of Directors-General Mervyn L. Weiner and Shiv S. Kapur

This series contains the chronological files of Mervyn L. Weiner for May 4,1982 to June 11,1984 and of Shiv. S. Kapur for June 27,1984 to February 28,1986. Weiner's file consists of: carbon copies of the letters,memoranda,and telex messages he sent to officials in aid organizations and in development banks,evaluation officials in other organizations,President Clausen,the Bank's regional vice presidents,OED staff,staff in the Bank's Personnel Management Department,and job applicants. Most of his file consists of: courtesy correspondence such as replies to invitations to speak; thank you notes for articles and books received; memoranda regarding travel plans and arrangements; post-travel notes to officials in host countries; notes to friends and colleagues regarding his retirement plans; and replies to inquiries regarding employment in OED or requests for OED publications. Some are more substantive and include proposed speech material for President Clausen,comments on OED draft reports,memoranda regarding OED staffing issues,and letters to Bank country directors regarding his observations from a recent trip.

Kapur's file consists of electrostatic copies of his letters,memoranda,and telex messages sent to OED staff,evaluation officers in other organizations and countries,officials in client countries,regional Bank staff,and job applicants. Much of his correspondence is substantive and includes: copies of his oral presentations to the Joint Audit Committee and to the Executive Directors; his comments on draft OEDproducts; and memoranda regarding arrangements for OED seminars,the OED work program and budget,and OED personnel issues. The remainder of his correspondence is routine in nature and concerns requests for Bank publications or employment,his annual leave plans,or his appreciation for assistance while on official travel.

Files of the Directors-General, Operations Evaluation, Documenting Relations with United Nations Organizations

This series consists of records that document the Director-General's coordination with units and personnel within the United Nations (UN) that were responsible for evaluations. The correspondence in the files is primarily between UN evaluation personnel and the Director-General or his special advisors, and it concerns evaluation methodology and strategy, an exchange of evaluation personnel between the institutions, joint Bank and UN evaluation efforts, and funding for evaluations. Among the files for the United Nations Development Programme (UNDP) are reports, correspondence, and minutes of meetings of the Inter-Agency Working Group on Evaluation (IAWG), which was established in January 1984 to provide a forum for discussion of evaluation issues by the UNDP and the executing agency partners and to promote simplification and harmonization of evaluation reporting practices among the UNDP and the executing agencies. An advisor to the DGO was a member of the Group. The files also contain UN publications and reports regarding evaluation methods and procedures or specific evaluations.

Director-General Robert Picciotto's Working Files for OED Studies, Reports, and Reviews

This series contains the working files that Director-General Robert Picciotto and his assistant, Pablo Guerrero, maintained for OED's special evaluation studies, Annual Reviews (known as the Annual Review of Evaluation Results [ARER] until 1997 when they became the Annual Review of Development Effectiveness [ARDE]), impact evaluation reports, annual Process Reviews of Annual Reports on Portfolio Performance (ARPP), Country Assistance Reviews, and Country Assistance Notes. Also included are files for OED'sannual policy ledgers which compiled OED recommendations from its reports and management responses to those recommendations. Most of the documents in these files are presumably duplicated elsewhere among the OED records relating to Annual Reviews and to the creation of evaluation products, but many of the intra-OED memoranda and e-mail exchanges are originals, and a number of the documents are heavily annotated by DGO Picciotto. The records in the files relating to OED's 1993 - 1997 annual ARPPs are uniqueto this series.

The files on any given OED product generally contain: multiple drafts of the study, report, or review; intra-OED correspondence and e-mail exchanges; minutes of meetings of the Committee on Development Effectiveness (CODE) and of the Executive Directors at which the product was discussed; drafts of management responses to the OED product; comments on draft and final versions of the product; copies of OED presentation slides, agenda, and other documents from OED workshops related to a study; and clippings regarding OED issuances and Bank policies. Files for evaluation studies may also contain Terms of Reference for consultants assigned to the study, initiating memoranda, design and approach papers, and a copy of the printed study.

Files of the Directors-General, Operations Evaluation, Documenting the Early History of OED

This series consists of files compiled by the Director-General containing his copies of reports and correspondence documenting the early history of the Operations Evaluation Department. Filed under Operations Evaluation History are copies of documents dated 1969 - 1974 that established the evaluation function in the Bank and the responsibilities of the OED and the Director-General. Included are copies of the DGO's correspondence with President Robert McNamara, the President's memoranda and circulars regarding OED and the evaluation function, and a 1974 General Accounting Office (GAO) proposal regarding Operations Evaluation. Filed under GAO Audits are copies compiled in June 1973 of documents dated 1967, 1969, and 1973 regarding audits of several international and government institutions in response to a bill to revise the functions of the Comptroller General of the U.S (the head of GAO). Filed under U.S. Congressional Staff Report on International Finance Institutions (IFIs) and DGO's Commentary are copies of documents dated 1975 - 1981 regarding a 1979 investigation of International Finance Institutions by the U.S. House Committee on Appropriations. DGO Mervyn L. Weiner reviewed a draft of the investigative report and prepared the Bank's response. That file also includes a 1981 GAO review of operations evaluation in the World Bank and Weiner's response to recommendations from that review. A file labeled OED 50: 'Learning About Development - Major Evaluation Findings' contains the DGO's copy of the draft of abooklet on OED prepared in connection with the Bank's 50th anniversary and OED's 20th anniversary. Filed with the draft are background materials used by Stokes Tolbert, one of the two consultants who drafted the booklet. Filed under Operation and Impact of the Bank's Evaluation System are various drafts of the 1989 OED booklet, Improving the World Bank's Performance: the Role of Operations Evaluation. Also in that file are copies of the Operational Directives chapters that concerned OED and copies of OED staff and DGO memoranda, articles, and presentations about OED and the Bank's evaluation system, 1975 - 1989.

Records of the Directors-General, Operations Evaluation, Regarding the Evaluation Capacity Development Program

This series contains correspondence and other documents regarding: the development of monitoring and evaluation systems in member countries; the evaluation of specific Bank-funded projects; and the formal Evaluation Capability Development Program (ECDP), which was established in OED in 1987 and was renamed the Evaluation Capacity Development Program in 1993. The main tasks of ECDP were to: strengthen networks of evaluators in developing countries; assess countries' interest in evaluation and their evaluation capacity-building requirements; organize evaluation seminars and workshops; provide short-term training within OED for client staff; and organize pilot programs in a small number of countries. The files generally contain: correspondence; reports from evaluation missions; Terms of Reference for special studies or missions regarding project evaluations; studies, reports, and articles regarding ex-post evaluations or evaluation capabilities; agenda and other planning documents for monitoring and evaluation seminars and workshops; and schedules and other records regarding visits of staff from client countries to OED for monitoring and evaluation training. Of special note is the extensive documentation filed under India: Narmada which OED compiled while working on an evaluation of the Narmada project. Most of the correspondence in the series is with the Director-General or with Robert van der Lugt and Pablo Guerrero, the two special advisors who coordinated ECDP within OED.

Correspondence of the Directors-General with the International Finance Corporation (IFC)

The series consists of correspondence primarily between DGO Yves Rovani and IFC Chief Evaluation Officer Walter I. Cohn, and relates to the DGO's responsibility for oversight of IFC's operations evaluation activities. An Operations Evalution Unit (OEU) was established within the Development Department of IFC in 1984. Included are drafts of IFC operations evaluation work programs for fiscal years 1986-1993 forwarded by Cohn to Rovani and to his successor Robert Picciotto for review and comment. Accompanying many of the work programs are proposals for special studies. Draft approach papers for those studies, also forwarded to DGO for review, include those prepared for the following studies: IFC's experience in the tourism sector (1989); performance of IFC's oil and gas exploration program (1991); successful IFC investments in Sub-Saharan Africa (1991); lending for industrial technology development (1992); IFC's market assessments (1992); and IFC's experience in the privatization of State-owned enterprises (1992). Comments prepared by the Country Policy, Industry and Finance Division (OEDD2) on some of the approach papers are included.

Also among the correspondence are comments from the DGOs to Cohn on OEU work programs and use of resources along with notes from the first meeting (July 16, 1986) at which the role of DGO in operations evaluation in IFC was discussed. Also included are: minutes of special meetings of the Portfolio Committee at which IFC evaluation studies were discussed; a list of IFC project completion reports for fiscal year 1982 to fiscal year 1986; a copy of a May 20, 1987, memo distributed to all IFC high level staff on new procedures for project completion reports; and Cohn's February 21, 1991, memorandum copied to Rovani in which he responds extensively to issues raised at an earlier Joint Audit Committee (JAC) Sub-Committee on Operations Evaluation meeting regarding IFC's evaluation capacity, IFC evaluation work, and the relationship between IFC and DGO and OED. In this memo he also responds to issues surrounding a report prepared by OEU on IFC's experience in the agricultural production sector.

A small amount of miscellaneous correspondence arranged by subject (1983 - 1995) includes: formal and informal memoranda and other correspondence between the Director, OEU, Walter I. Cohn and DGO's Mervyn Weiner, Yves Rovani, and Robert Picciotto; approach papers for studies forwarded for approval to DGO before being sent to the JAC, copies of IFC policy papers, and draft IFC evaluation reports, workprograms, and budgets. Also included are an Annual Portfolio Evaluation Report for FY 1983 and an updated mandate statement for evaluation in the Bank and IFC that reflects findings of the March 1995 report of the review of evaluation in IFC.

Subject files (1980 - 1995) found with the correspondence include the report of IFC's first evaluation of a segment of its portfolio of investments, IFC Hotel Portfolio Review, forwarded by DGO Mervyn Wiener to the Joint Audit Committee (JAC) for review on May 11, 1981. Also included are reports of IFC evaluations in the agricultural, mining, and petrochemical sectors completed between 1983 and 1994 and in the cement and textile industries sectors between 1983 and 1989. There are also files containing correspondence and reports relating to: evaluations of successful investments in Sub-Saharan Africa (1994); support for small and medium scale enterprises (1995); investment assessments performed by OED (1992 - 1994); IFC's Five Year Program for fiscal years 1985 - 1989; and the review of evaluation in IFC (1994 - 1995).

A small amount of miscellaneous IFC and OEU reports (1979 - 1989), some of which are annotated, forwarded to DGOs Mervyn Weiner, Yves Rovani, and Robert Picciotto are in the series. A number of these reports were distributed to the Board of Directors including three confidential Project Completion Reports (PCRs) on three IFC investments in Guatemala (1979), Brazil (1979), and Indonesia (1979). These three reports were distributed in January 1980 to the Board based on a decision made at an October 30, 1979, Board meeting. Each director was to receive a copy to indicate the type of PCR IFC had been preparing and to illustrate why caution was necessary when disseminating information contained in the reports. Other IFC reports included are: Annual Portfolio Evaluation Reports for fiscal years 1984 and 1986 - 1988; a report entitled Deterioration in IFC's Portfolio: An Analysis (March 1985); and a report prepared by a consultant entitled IFC in Sub-Saharan Africa, Proposals for a New Approach (February 1987).

Records of the Director, Operations Evaluation Department

The records in this series contain the reports, correspondence, and studies compiled by the Directors of OED (OEDDR). The Director is selected by the Director-General, Operations Evaluation (DGO), and is responsible for the overall management of the OED. Principal functions include:

  • assisting the Director-General in making periodic assessments of the adequacy and effectiveness of the operations evaluation system in light of the institutional objectives of the World Bank;

  • carrying out reviews of the Bank's completion reports and other self-evaluations, performance audits on selected completed projects, impact evaluations, and evaluation studies focusing on operational programs, policies, strategies, and processes;

  • assisting member countries to develop their own operations evaluation capacities; and

  • disseminating evaluation findings both within the Bank and to the wider development community.

The series contains the records of Christopher R. Willoughby, who was the first Director of OED. Primarily consisting of memoranda, these records document the early development of the operations evaluation function and of the Operations Evaluation Department in the 1970s. Also contained in this file are records of subsequent OED Directors from the 1980s and 1990s. These records relate to a variety of topics.

Impact Evaluation Reports

The series primarily consists of Impact Evaluation Reports (IERs) and the working files produced during the creation of those reports. The series also consists of a number of special studies produced by the OED. IERs assess the performance of operations at full development, some five to ten years after the close of disbursements on the Bank's loan. The evaluations are selective; only a small percentage of projects undergo an impact evaluation. These evaluations provide a second look at a project by analyzing the long-term effects - intended or unintended - on people, institutions, and the physical environment. The evaluations assess projects against the goals that were stated at appraisal and also against a broad set of criteria that relate to social dynamics, income distributions, effects on women and families, institutional development, sustainability, and the environment. This series consists of IERs from the very beginning of their production in 1979 and through into the 1990s.

The content of the files for impact evaluations varies somewhat but the more complete files contain: copies of the Completion Reports, Audit Reports, and other OED-generated records relating to the project; non-Bank background reports and publications relating to the project; Terms of Reference for the OED staff members and/or consultants assigned to evaluation; drafts of the Approach Paper and Study Design Paper; memoranda and correspondence regarding the team's travel plans if a mission was required; a Back-to-Office Report regarding the mission; intra-OED correspondence regarding the format for the evaluation; various drafts of the evaluation; comments on the drafts from OED and Regional staff, officials in the client country, and representatives from other aid/lending organizations; drafts of comments by the Director-General, Operations, (DGO) to the Committee on Development Effectiveness (CODE) or its predecessor, the Joint Audit Committee (JAC) regarding evaluations; minutes of CODE or JAC meetings at which the evaluation was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; and summaries of the Executive Directors' discussion of the evaluation and copies of the Board chairman's published comments. Many of the impact evaluations relied on field surveys to gather opinions of stakeholders. The raw survey data is not included in an evaluation file, but sample survey questionnaires and data derived from the surveys may be in the file.

The series contains the first IER produced by theOED, referred to as a Project Impact Evaluation Report (PIER). The report was for the Mexico Third Irrigation Project (1978 - 1979) which focused on irrigation rehabilitation programs in irrigation districts Nos. 17 (Region Lagunera) and 23 (San Juan del Ro). The evaluation was undertaken by Chief Evaluation Officer John Malone about five years after the performance audit was conducted for the project. Other early impact evaluation reports in this series are for: Kenya's First Smallholder Agricultural Credit Program (1980); the Kenya First Livestock Development Project (1981); the Roseires Irrigation Project in the Sudan (1980); the Burundi-First Arabica Coffee Improvement Project (1980); the Malaysian Muda and Kemubu Irrigation Projects (1981); the Atlantico Irrigation Project in Colombia (1982); and the San Lorenzo Irrigation and Land Settlement Projects in Peru (1982). In addition, records related to an IER conducted jointly with the National Economic and Development Authority (NEDA) and the Central Bank of the Philippines on the Philippines Second Rural Credit Project are included. The report was issued in 1983 after an extensive evaluation that involved a survey of beneficiaries in different regions. Also issued in 1983 was an IER for the Indonesia Irrigation Rehabilitation Project.

The series contains many other IERs and IER working files produced through the mid to late 1980s and 1990s. Agriculture, urban, education, transport, industry and transmigration projects are represented. Of note, one IER covers three income generating projects for refugees in Pakistan for which the Bank acted as executive agency on behalf of the United Nations High Commissioner for Refugees (UNHCR). Also contained are background materials on Indonesia that OED's Agriculture and Human Development Division (OEDD1) gathered from a wide variety of sources in preparation for a 1993 impact evaluation covering loans for three transmigration projects in Indonesia. The files include the Project Completion Reports for two of the loans (Transmigration II [L1707] and III [L 2248]) and Terms of Reference for the impact evaluation but no other documents regarding the impact evaluation.

The series also includes a number of special process and thematic studies. Reports and related working papers include: a 1987 and 1988 OED special study entitled Management of Renewable Resources in Agricultural Operations; two transport sector evaluation studies entitled The Transport Sector in Mexico: An Evaluation (1998) and Transport in China: An Evaluation of Bank Assistance (1999); a copy of a 1998 paper (The Industrial Organization of Corruption: Why Corruption Hurts More in Africa than in Asia) prepared by Sector and Thematic Evaluation Division (OEDST) staff members Antti Talvitie and Binyam Reja, based on their participation in an OED evaluation of five transport projects in Indonesia; background records for a 1996 Infrastructure and Energy Division (OEDD3) process study, Effectiveness of Environmental Assessments and National Environmental Action Plans (EA/NEAP Study), which evaluated the Bank's use of project-level environmental assessments (EAs) and highlighted the differences between countries that had an approved National Environmental Action Plan (NEAP) and those that did not; working files for the 1995 Country Policy, Industry and Finance Division (OEDD2) study The Social Impact of Adjustment Operations: An Overview; and special impact evaluation studies covering a range of agriculture topics, including Harvesting the Waters (1980s), The World Bank and Structural Adjustment in Agriculture; Reforming Agriculture: The World Bank Goes to Market, Irrigation O & M [Operation and Maintenance] and System Performance in Southeast Asia (1996), Natural Resource Management in Bolivia: 30 Years of Experience (1993), and Dynamics of Rural Development in Northeast Brazil: New Lessons from Old Project (1991), as well as one file for a special process study entitled Monitoring and Evaluation Plans in Staff Appraisal Reports Issued in Fiscal Year 1995.

Also in this series are correspondence, reports, and publications that evaluator Christian Polti gathered while OEDD1 was working on two impact evaluation studies. The first study deals with the World Bank's experience with irrigation development and examined irrigation projects in Morocco, Mexico, the Philippines, and Thailand. The second study deals with cotton development programs in Cote d'Ivoire, Burkina Faso, and Togo.

Completion Reports and Audit Reports

The series consists of records related to the project monitoring and evaluation function of the Operations Evaluation Department (OED). The majority of the records in the series are final audit reports for individual, Bank-sponsored projects and records related to the creation of those reports. Reports date from the beginning of OED and its project evaluation activities in 1973. OED-authored reports include Project Performance Audit Reports (PPARs) and Performance Audit Reports (PARs). Reports completed byoperating departments and submitted to the OED include Project Completion Reports (PCRs), Project Completion Notes (PCNs), Implementation Completion Reports (ICRs), and Implementation Completion Notes (ICNs). The PCRs can be characterized as either reports that required and received an audit by the OED or as pass through PCRs which OED division chiefs determined did not require an audit. In the case of the latter, Notes of Record are sometimes included to explain why an audit was not required. The series also consists of background materials and working files created or collected during the preparation of audit reports. These records include: correspondence; memoranda; staff appraisal reports; reports and recommendations sent to the Executive Directors for approval of the loans or credits; loan or credit agreements that were executed; annotated audit drafts; project progress reports; and field notes. In some cases, working files related to audits and report preparation are those accumulated by specific evaluation officers in OED.

The types of records that accompany the final evaluation reports vary, as procedural requirements changed over the years. OED evaluating officers created files for all completion reports and audit reports for which they were assigned responsibility. Most of the pre-1982 working files containing PCRs and PPARs reflect procedures put in place by the Bank between 1973 and 1979. The time frame for completion of PCRs after final disbursements of loans / credits varied as did the extent ofthe involvement of operating departments and clients in their preparation. In the mid-1970s, the PCR was to be prepared within six months of final disbursement by the Bank's Regional Vice-Presidency of the country in which the project took place. Information for the PCR was gathered from project files, supervision reports, and discussions in the field between Bank and Government officials associated with the project. However, before 1976 the majority of the PPARs prepared by OED evaluating officers were for projects for which no PCR had been prepared.

In 1976, procedural changes required all PCRs to be sent through the Director-General, Operations Evaluation (DGO), without OED comment to the Executive Board along with the PPAR. In addition to the PCR, the PPAR working files from 1976 onward include: the Terms of Reference for the OED audit mission; the Back-to-Office Report from the mission; PPAR drafts; comments on the drafts from Bank officials in the Region, Government officials and representatives of businesses or institutions involved in the project; and copies of transmittal letters accompanying the PPAR. Loan or Credit Agreements, Staff Appraisal Reports, and Reports and Recommendations by the Bank President for loans and credits, most of which are of earlier date than the other records in the file, are included in some PCR and PPAR files. By the end of 1979, OED conducted abbreviated audit reviews for about half of the PCRs it received. The working files for these passed-through PCRs usually contain: a memorandum addressed to the Regional Vice President indicating that based on the OED review of the PCR an audit would not be performed; comments on the PCR; a final draft of the PCR sent for printing by OED; a copy of the printed PCR forwarded to the Executive Board; and copies of the transmittal letters used in the OED distribution of the printed PCR.

At the beginning of fiscal year 1983, a new system of selective project performance auditing was introduced which required that 40 to 50 percent of projects be audited. PCRs were selected for audit according to whether they were for: large, complex, or innovative projects; projects an Executive Director proposed for performance audit; projects with high lending priority; projects which were first in a given sector / sub-sector for a country or last in a series; or a series of projects subject to a combined audit. OED division chiefs were given responsibility for determining the projects to be audited and for making decisions on pass-through PCRs. Some files for projects which OED did not audit contain: a Summary Note explaining the reasons for the pass-through decision and evaluating the quality of the PCR; a copy of the draft PCR; comments regarding the draft PCR from the client country; a Pass-Through PCR Unit Cost Sheet tracking OED staff time allocated to the project; memoranda and letters transmitting a final copy of the PCR to the Executive Directors, the Bank President, and officials of the client country; and a final copy of the PCR. Found in other working files for post-fiscal year 1983 is the Note of Record, which was prepared by the OED evaluating officer after review of the PCR with the division chief. It contained the reasons for the pass-through decision. The Note of Record assessed the project as described in the PCR in terms of its objectives, scope, implementation, costs, and sustainability. It was distributed only to the Director of OED (OEDDR), the OED Library, the working file, and the OED annual review coordinator. Also found in some workingfiles with the Note of Record is the Project Information Form (PIF) used to input data about the project into OED's database for use in annual reviews and studies. Results for all PCRs were reported to the Executive Board.

New guidelines for completing PCRs were put into place effective July 1, 1989. Most of the PCRs in the OED working files after this date have a different format from earlier prepared PCRs. Parts I and II of the report were prepared by the Regional Office and Part III was completed by the client. Some working files include Project Completion Notes (PCNs) which were prepared in lieu of PCRs for projects for which: the loan was never signed by the client; the Bank withdrew the loan; disbursement was very fast or ahead of schedule; no PCR was ever completed; or for which there were no implementation issues. The decision to create the PCN was based on a review of the Staff Appraisal Report prepared before the loan or credit was granted, internal Bank memoranda, minutes of Board discussions, supervision reports, and interviews with staff. The PPAR was carried out on the basis of the recommendation for audit made by OED at the time of review of the PCR. The contents of the PPAR working file remained largely unchanged from earlier periods but some PPARS for cluster audits (i.e. audits of multiple projects) were included.

By 1994, operations staff (also called Regional staff) evaluated all completed operations for which they were responsible and OED audited a representative sample of completed operations. For last supervision missions departing after July 1, 1994, Regional staff was required to prepare an Implementation Completion Report (ICR) or Implementation Completion Note (ICN) similar to the earlier PCN. Issues encountered during implementation, achievements, and the economic rate of return, were among the factors covered in the ICR. Contributions from client agencies (whole reports or comments) were attached to the ICR. Co-financiers were invited to participate in evaluations in which they provided funding. All ICRs were to be completed within six months of the final loan / credit disbursement. By 1997, about 25 percent of completed projects were audited by OED. Like the earlier PPARs, Project Audit Reports (PARs) were field-based and incorporated the views of client countries and major stakeholders. Projects were subject to audit if they were: large and complex; innovative or had unusual features; controversial, with issues unresolved during the ICR evaluation; first in a series or expected to offer especially strong lessons; suitable to be analyzed together with similar projects; useful for providing building blocks for sector or country evaluation study; or requested by Executive Directors.

The ICR, PCR, PCN, and ICN working files for the mid-1990s and later generally contain the printed report, a PIF and control sheet, drafts of the DGO's evaluative memorandum regarding the report or note, a copy of a memorandum from OEDDR transmitting the draft evaluative memorandum to the relevant Country Director, and comments from the Bank's Regional staff on the draft. Some files contain the OED-ICR-Review-Evaluation Summary which recommended for or against an audit of the project. Projects were evaluated for outcome, sustainability, institution development, Bank performance, and client country performance based on information in the ICR and PIF. Correspondence with Regional and other Bank staff and with officials in client countries regarding their comments on the draft report can also be found in the files. Some files contain copies of intra-OED electronic messages commenting on the draft evaluative summary and on the review process. A few of the files contain background documents records such as Staff Appraisal Reports (as was the case for PCR files for earlier periods).

In addition to draft and final copies of the reports, the PAR files for this period typically include: Initiating Memoranda; Terms of Reference (TOR) addressed to OED evaluators designated to travel to the field to conduct the audit; letters to Government officials and others related to the project announcing the mission to develop the report; Back-to-Office Reports from the audit mission; and comments on the draft audit report. Comments from the OED audit panel and an Audit Panel Review form can be found in some files along with drafts of the DGO's evaluative memorandum, a PIF and cover sheet, and copies of letters transmitting the final printed copy of the PAR to officials in the client country. Some of the PAR files contain copies of related background materials such as the Staff Appraisal Report and the Loan Agreement.

The review and audit activities of regional operations and the OED were conducted for the projects of all sectors. These include:

*Education- Primary; secondary; tertiary; vocational; technical

  • Health and nutrition Specific disease and HIV/AIDS prevention and management; population projects

  • Water Sewarage and drainage; sanitation; water supply; water pollution control; flood control

  • Agriculture Commodity production; livestock development; fisheries development; research; support services; marketing

  • Human resources Extension and training; social sector management; science and technology

  • Rural development Rural roads; rural electrification

  • Urban development Urban engineering; air pollution; urban management

  • Telecommunications General sector

  • Energy and power Power generation; refinery engineering; energy efficiency; oil field development; gas transport and distribution; oil, thermal, and gas exploration and development; pipeline rehabilitation

  • Transport Navigation; railways; highways; rural roads; aviation; ports

  • Finance and trade Private sector finance projects; financial rehabilitation projects; economic management and public enterprise; technology development projects; small and medium enterprise projects; development banks; export development; shipping; imports program; debt and debt service

  • Industry Foundry modernization; small-scale industry; fertilizer production; mining and other extractive sector projects; pulp and paper engineering; public enterprise assistance; natural resource management; forestry; off-shore development

  • Infrastructure Housing; construction projects

  • Reconstruction and emergency Earthquake and flood recovery; drought relief

  • Tourism General sector

  • Structural adjustment Public sector reform loans; adjustment loans and credits; trade policy adjustment

  • Client capacity building Technical assistance loans; central government administration projects; planning support

Evaluations of non-sector and multi-sector projects are also included.

The series also consists of evaluative reports and related materials created by OED staff that are not PARs or PPARs and are not related specifically to individual projects. These include: staff appraisal reports initially developed for the projects; reports and recommendations sent to the Executive Directors for approval of the loans or credits; materials related to a study by Carl Jayarajah entitled Overview Study: World Bank Support for Small and Medium Industry; a draft of correspondence related to a proposed Review of Lessons Learned in Railway Projects (RLLRP) for the OED Lessons & Practices publication; an approach paper, drafts, and background materials for Review of the World Bank Experience with Electric Power Generating Facilities (1985); a draft study outline, Terms of Reference, and supporting materials for the 1987 - 1989 study Columbia: An Evaluation of Bank Lending to the Colombia Power Sector between 1970 and 1986 along with the March 1991 final study report; the approach paper and draft of the 1988 - 1989 country case study World Bank Support for Rural Roads Maintenance - Philippine Case Study; a draft of Senior Evaluator Jan de Weille's March 1992 report Annual Review, Project Performance Results for 1991 - Transport; the approach paper and drafts of the June 30, 1994, grey cover sector review (Report 13291) Rural Electrification in Asia: A Review of Bank Experience; records related to a study of the Bank's involvement in Mexico's power sector over four decades; background materials for Study of Adjustment Lending in Sub-Saharan Africa (Oct. 1996 - July 1997) including an approach paper, preliminary draft, draft speech prepared for the Director-General, Operations Evaluation, and copies of related reports; the report Case Study of World Bank Activities in the Health Sector in Mali accompanied by a memorandum from OEDST inviting the DGO and other OED personnel to a meeting review regarding the study; working files of evaluator Basil Coukis which include annotated copies of PCRs and PPARs as well as background materials such as loan guarantee agreements, staff appraisal reports developed for loan or credit applications, progress reports, background correspondence, and field notes used in the preparation of evaluations; and two reports concerning trade policy reform for Mexico, Mexico: Trade Policy Reform, World Bank's Support for Trade Liberalization Program by Yalcin M. Baran (Sept. 15, 1989) and Overview Study: World Support for Small and Medium Industry issued by the Projects Department of Latin America and the Caribbean Regional Office (June 24, 1986). Finally, the series consists of records related to the OED's problematic review of the Thirteenth Indian Railway Project (Credit 582-IN). The files contain the April 1979 PCR, various versions (dating from 1981 to 1986) of the PPAR, and correspondence regarding the project. The first and the subsequent draft PPARs for this project were controversial and found to be inadequate for various reasons. The controversy led President Clausen to appoint a special audit team for the Railway Project and to request a reevaluation of OED internal procedures. Materials related to the subsequent review as well as to the OED evaluation are included; the latter includes: intra-OED correspondence; correspondence with Transportation Department staff, the South Asia Regional Office, and Indian government officials; and minutes of a Joint Audit Committee meeting and Executive Directors' meeting.

Publications

This series contains documentation regarding the drafting and publication of booklets aimed at explaining the evaluation function in the World Bank to non-Bank audiences. The early editions of this booklet were titled Operation Evaluation: World Bank Standards and Procedures. By 1993, the title had changed to Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation.

The sub-series contains records documenting the drafting and publication of the booklets. Included are: intra-OED correspondence; multiple drafts of the booklets; comments on the drafts; and agenda, minutes, and summaries of meetings of the Joint Audit Committee and of the Board of Executive Directors at which drafts were discussed. The sub-series also contains draft and/or published copies (in English and French) of various booklet editions (1976, 1979, 1984 [draft only], 1993).

General Subject Files of the Directors-General, Operations Evaluation

This series consists primarily of reports, studies, and other issuances sent to the Director-General either for his comments or as part of a general distribution to high-level managers in the Bank. Filed with these issuances are comments and related correspondence of the Director-General and his special advisers and comments that the OED staff provided the DGO. The documents from the Kapur and Rovani eras are primarily originals, many with handwritten comments from the Directors-General. The Picciotto-eradocuments are primarily electrostatic copies. Key topics covered by the files include: mechanisms for integrating OED findings into the policy formation process of the Bank (filed under Dissemination and Feedback); evaluation criteria used in appraising projects (filed under Methodology); OEDs involvement in the monitoring and evaluation of NGO programs and in providing evaluation guidance to NGOs (filed under NGOs); OED guidelines for evaluating environmental projects (filed under Environment); planning for the 1995 G-7 Summit in Halifax (filed under G-7 Summit); the role of gender in OED work (filed under Women in Development); DGO/OED comments on the Annual Report on Portfolio Performance (ARPP) and on the disconnect between the ARPP and Completion Report/Audit Report ratings (filed under Annual Report on Portfolio Performance); and planning for the 1994 Annual Review of Evaluation Results (filed under Annual Review). Filed under Development Committee Task Force on Multilateral Development Banks (MDBs) are replies to questions and copies of documents DGO Picciottos adviser Pablo Guerrero provided to the Task Force secretary. The MDB Task Force files also contain Guerreros comments on draft background studies submitted to the Task Force. Filed under Main Complex Rehabilitation Project (MCRP) are records regarding a 1993 Bank inquiry into cost overruns for an improvement program to the Banks headquarters complex at 1818 H Street NW. That file includes copies of Robert Picciottos 1993 memoranda outlining the input he had into the funding required for the project while he was Vice President of Corporate Planning and Budget. The files also include Picciottos comments on a draft of the S. Guhan chapter, The Banks Project Lending in South Asia, 1971-1990, in Volume II of the Brookings history of the World Bank.

Shared Records of the Director-General, Operations Evaluation (DGO), and the Director, Operations Evaluation Department (OEDDR)

Series consists of files of both the Director-General, Operations Evaluation (DGO), and the Director, Operations Evaluation Department (OEDDR). The records document OEDDR and DGO interaction with: staff in other parts of the Bank; U.S. officials in the General Accounting Office, the U.S. Agency for International Development, and the Department of the Treasury who were interested in the evaluation function within the Bank; and officials in industrialized nations, NGOs, and international aid organizations who were responsible for, or interested in, monitoring and evaluation processes.