Fonds consists of records related to activities undertaken by the Operations Evaluation Department in support of its function as the Bank's independent operations evaluator. The majority of the records in this fonds are working files for project evaluations (Project Performance Audit Reports [PPARs] or Performance Audit Reports [PARs]) produced by the OED; these files are generally accompanied by Project Completion Reports (PCRs) or Implementation Completion Reports (ICRs) that were produced by operations departments and submitted to the OED. Working files for other OED evaluation products include: Impact Evaluation Reports (IERs); Country Assistance Reviews (CARs); Country Assistance Notes (CANs); Country Assistance Evaluations (CAEs); and special evaluation studies. Records related to annual reviews of Department evaluations are also included as are records related to Department annual reports. Publication files for OED periodicals, OED Processes, and Lessons & Practices are also included.

The fonds also consists of records created and received by the Department's Directors-General (DGO), Directors (OEDDR), and division directors. These records generally relate to: policy and procedure development; reporting; Department, Bank, and external projects, committees, and commissions; communications and dissemination of reports; operations management; and Department administration.

The records in this series contain the reports, correspondence, and studies compiled by the Directors of OED (OEDDR). The Director is selected by the Director-General, Operations Evaluation (DGO), and is responsible for the overall management of the OED. Principal functions include:

• assisting the Director-General in making periodic assessments of the adequacy and effectiveness of the operations evaluation system in light of the institutional objectives of the World Bank;

• carrying out reviews of the Bank's completion reports and other self-evaluations, performance audits on selected completed projects, impact evaluations, and evaluation studies focusing on operational programs, policies, strategies, and processes;

• assisting member countries to develop their own operations evaluation capacities; and

• disseminating evaluation findings both within the Bank and to the wider development community.

The series contains the records of Christopher R. Willoughby, who was the first Director of OED. Primarily consisting of memoranda, these records document the early development of the operations evaluation function and of the Operations Evaluation Department in the 1970s. Also contained in this file are records of subsequent OED Directors from the 1980s and 1990s. These records relate to a variety of topics.

This sub-series is a collection of memoranda and related records compiled by Christopher R. Willoughby who was among the first employees of the Operations Evaluation Unit and was appointed the first director of the Operations Evaluation Department in July 1973.

• The first set of files consists of memoranda (dated November 1970 - November 1972; February - April 1973; and February - August 1975) sent to President Robert McNamara from Christopher R. Willoughby and also John Adler, who was director when the Operation Evaluation Unit was established in 1970. The memoranda concern the early workings of Operations Evaluation including: the decision to begin with an in-depth look at the Bank's work in one country, Colombia, and in one sector, electrical power; the development of the format for these first two reports; progress reports on the work of the unit and on the development of future work programs; and the relationship between evaluation and supervision. Some of the original memoranda to McNamara were returned to Operations Evaluation marked with a red stamp indicating President has seen. One long memorandum for the record summarizes the January 1971 visit with the Colombia Operations Evaluation team of Dr. Lauchlin Currie who headed the Bank's General Survey Mission to Colombia in 1949. Included in this first set of files are handwritten notes and memoranda for the record summarizing meetings Willoughby and others had with McNamara.

The second set of files consists of memoranda (dated April 1970 - February 1976) sent and received by Willoughby and his copies of key memoranda documenting the development of the operations evaluation function and the changing position of the Operations Evaluation staff within the Bank. Included are Willoughby's handwritten drafts and final copies of the key issuances affecting Operations Evaluation and his notes from key meetings. One folder contains Willoughby's drafts of a September 29, 1975 memoranda from Mohamed Shoaib, Vice President, Operations Evaluation Division, outlining the history of the evaluation function from 1971 to 1975.

This sub-series contains reports, correspondence, and studies compiled by the Directors of OED (OEDDR). Most of the files contain comments from OED staff members and/or the director on draft documents from other Bank employees. Filed under General Comments on Review of Field Staff Benefits are OED comments on various Bank documents, 1995-1998. Key topics covered by other files include: OED staff views on the dissemination and use of OED findings (filed under Dissemination); ratings of the Narmada and Kedung Ombo Completion Reports (filed under Narmada); reports of the Joint Audit Committee Informal Subcommittee on OED Reports; a select review of OED by the Internal Auditing Department planned for 1996; plans for an OED study of NGOs; OED collaboration with the Economic Development Institute (now WBI); joint OED/Operations Policy Department reports on the quality of economic analysis in staff appraisal reports for projects approved 1993 - 1997 (filed under Economic Analysis, Econ III and Econ IV and Economic Analysis: Correspondence); a 1996 proposal for simplified project documentation for investment operations (filed under Simplified Documentation); and OED staff retreats for December 15-17, 1987, November 29-December 1, 1989, June 26-28, 1991, and September 8-10, 1993. These retreat files contain agenda and schedules, copies of documents discussed at the retreats, copies of presentations, and post-retreat evaluation forms.

Filed under Portfolio Management are: memoranda regarding funding arrangements for OED studies and on borrowers' interest in ex-post evaluations; OED staff comments on the draft Wapenhans Report and on the draft chapter concerning OED; and copies of minutes of the Task Force Steering Committee on which Eberhard Kopp served. Filed under Streamlining Bank Procedures are memoranda and correspondence Kopp compiled in 1985 and 1986 before he became OEDDR Director and while he was a member of Working Group #3 of the Task Force on Streamlining Bank Procedures.

Series consists of files of both the Director-General, Operations Evaluation (DGO), and the Director, Operations Evaluation Department (OEDDR). The records document OEDDR and DGO interaction with: staff in other parts of the Bank; U.S. officials in the General Accounting Office, the U.S. Agency for International Development, and the Department of the Treasury who were interested in the evaluation function within the Bank; and officials in industrialized nations, NGOs, and international aid organizations who were responsible for, or interested in, monitoring and evaluation processes.

This sub-series contains correspondence and other records exchanged between OED staff and officials located in industrialized countries who were responsible for, or interested in, monitoring and evaluation processes. Included is correspondence with officials in the Netherlands, Germany, Canada, Australia, Sweden, Denmark, France, Japan, Saudi Arabia, Ireland, Greece, Austria, the United Kingdom, and the United States. Most of the correspondence is with the Director of OED, the Director-General, Operations Evaluation (DGO), and the DGO's advisors, Robert van der Lugt and Pablo Guerrero, who were responsible for developing evaluation capacity in member countries. The correspondence concerns: requests for information regarding OED's evaluation policies and procedures or assistance in conducting an evaluation; exchanges of evaluation methodologies; international seminars on evaluation; visits of OED staff to other countries to conduct an evaluation or to attend an evaluation conference or seminar; and visits toOED of evaluation officials from abroad. Included are copies of other countries' studies, reports, and manuals regarding evaluations and evaluation methods.

The U.S. correspondence is with officials in the U. S. Agency for International Development (AID), the General Accounting Office (GAO), and the Department of Treasury; most are copies of records that are presumably located elsewhere in OED and Bank records. The US AID correspondence primarily concerns exchanges of policy papers, studies and reports onevaluation guidelines. The GAO correspondence relates to 1977, 1981, and 1991 congressionally required GAO reviews of the independent evaluation system of the Bank and a 1996 GAO review of the overall role of the World Bank in a post-Cold War environment.

The earliest GAO file contains documents assembled by OED Director Christopher R. Willoughby in 1973 and 1974 following release of the 1973 GAO report on U.S. participation in World Bank. Included in the file are a number of documents from Bank PresidentRobert McNamara, Vice President Mohamed Shoaib, the U.S. Executive Director Charles O. Sethness, and Willoughby, discussing the audit and evaluation functions in the Bank and proposed changes in the structure and function of Operations Evaluation.

The Department of Treasury file concerns the department's 1971-1973 interest in the evaluation function of the Bank and Treasury's support of the GAO recommendation for the establishment of an independent review body to evaluate the Bank's activities.

The files in this sub-series contain documents maintained by both DGO and OEDDR, and much of the content of this series is duplicated elsewhere, especially in the files for OED studies and process reviews.

Most of the files relate to the work of OED, but some, which concern non-OED matters, contain records from other Bank units that were forwarded to the DGO/OEDDR for comment or as a result of a general distribution to high-level Bank staff. Most of the files fall under one of two general headings: 1) Management and Oversight and 2) External Affairs. The Management and Oversight files concern issues involving the President, Executive Directors, members of the Committee on Development Effectiveness (CODE), and Bank Vice Presidents. Included are: correspondence with CODE members; copies of reports from OED that went to CODE and to the Executive Directors; intra-OED correspondence regarding preparation for meetings with CODE members and the Executive Directors; OED's response to a draft of the 1996 audit of OED by the Internal Auditing Department; agenda and minutes of OVP (Operations, Vice Presidents) meetings and retreats; intra-OED correspondence regarding the DGO's comments on records and topics to be discussed at OVP meetings; correspondence with Managing Directors and copies of records to be discussed with the Directors; files documenting OED preparations for briefings and/or briefing papers provided to President Wolfensohn on his arrival at the Bank and prior to some of his overseas trips; OED's contributions to the observance of the 50th anniversary of the Bank and comments from the DGO and OED staff on draft chapters of the Brookings Institution's history of the Bank; and agenda and background documents relating to the 1995 OED management team retreats. Under Clusters are files coordinating work on a study (or a broad area of interest) assigned to a specific OED Division (i.e., OEDCM, OEDCR, and OEDST).

Under External Affairs are files concerning the DGO's and OEDDR's interaction with, or reports regarding: IFC and MIGA staff; international aid organizations; evaluation organizations; the US General Accounting Office; the World Commission on Large Dams; NGOs; the Multilateral Development Banks; the members of the DAC Expert Group on Aid Evaluation (filed under OECD-Development Assistance Committee); and other Bank units and departments. Filed under Liaison with other OED Units - OEDDR is a chronological file of correspondence, 1988 - 1996, between the DGO and OEDDR or between OEDDR and other Bank units.

This series consists of records related to the development of policies and procedures for the OED. Policies and procedures for OED programs and work products were developed in intra-OED correspondence, staff meetings, retreats, committees, and working groups and via Bank-wide task forces, committees, issuances, and operational manuals. The majority of the series consists of records created and received by the OED Directors-General (OEDDG) and OED Directors (OEDDR) that relate to the development of Department policies and procedures. Most of these guidelines relate to the creation of OED work products, such as audit reports, impact evaluation reports, Country Assistance Reviews and Evaluations, and special evaluation studies.

The series also consists of records documenting meetings of the Department Management Team (DMT) as well as records related to Departmental Management Team retreats.

The files in this sub-series, compiled by the Directors of OED (OEDDR), outline the policies and procedures affecting OED programs and work products. Included in the first segment (1970 - 1994) are files concerning: the Project Completion Report / Project Performance Audit Report (PCR/PPAR) evaluation system and the pass-through procedures for PCRs that began in 1982; the dissemination and use of OED findings; the various revisions of the Procedural Guidelines for Operations Evaluation Staff and of the chapters in the Operational Manual regarding Project Completion Report / Implementation Completion Report (PCR/ICR) guidelines and project supervision in the Bank; the development and use of the Project Information Form (PIF); the development of OED's evaluation seminars; and the role of environmental issues in Bank evaluations. One file (Lessons Learnt from Project Performance Audit Reports - Agriculture) contains the OEDDR's copies of memoranda dated December 1980 to July 1986 from the Assistant Director ofthe Agriculture and Rural Development Department (AGR) to AGR division chiefs outlining the lessons learned from reviews of Agricultural PCRs and PPARs. Another file (Operations Evaluation Department - General) contains copies of key documents in the history of OED from 1970 to 1988.

Much of the second segment (1985 - 2001) contains documentation regarding the Bankwide ICR Process Review Working Group which was co-chaired by OED Director Elizabeth McCallister and was established in December 1997 to make the ICR process more efficient, effective, and timely. The records for the working group include agenda and minutes of meetings, various drafts of the group's final report, comments on the drafts, and intra-OED correspondence on the workings of the group. Other files contain: correspondence concerning changes in the PIF and instructions for completing the form; intra-OED correspondence regarding the discrepancy between OED and Regional project ratings (PPAR-Project Ratings); internal OED e-mail discussions of the ICR review process and the development of the ICR Evaluation Summary form; intra-OED correspondence regarding specific ICR reviews; draft ICR guidelines and comments on them; revisions to the OED handbook, Procedural Guidelines for Operations Evaluation Staff (Guidelines - Amendments); the drafts of a style manual standardizing the format for OED reports; and a proposal for dealing with the workflow problems associated with ICR bunching, i.e. the receipt of the majority of ICRs during the months of May, June, and July. Filed under PPAR - General is a copy of July 1995 Performance Audit Guidance, which was prepared by the OED Audit Guidance Working Group in 1995.

This sub-series contains correspondence and reports of the DGO that established policies and procedures regarding OED's programs and work products. Most of the records were compiled by Robert Picciotto, but a few early files contain correspondence of Directors-General Shiv Kapur and Yves Rovani. Much of the DGO's correspondence is with the Director of OED and OED staff or with Pablo Guerrero, special advisor and assistant to the DGO. The files contain information regarding: OED procedures and criteria for reviewing Project Completion Reports (PCRs) and Implementation Completion Reports (ICRs) (under ratings and PCR review process); evaluation capacity training for OED staff, other Bank staff, and external Bank partners (under training); procedures and standards for Country Assistance Reviews (CARs) and Country Assistance Notes (CANs) (under country assistance work); OED special studies and OED impact evaluations; the format and content of new OED issuances and tools designed to disseminate lessons learned from OED's evaluation work (under Lessons & Practices, Evaluation in the World Bank [WB] booklet, Policy Ledger, and Evaluation Text Base); revisions of the Project Information Form (PIF) and instructions for using it; the disclosure and publication of evaluation results; and the PCR (and later the ICR) backlog in each region. Filed under Initial Executive Project Summaries (IEPS) are OED procedures for reviewing IEPSs for proposed operations and copies of OED comments on specific IEPSs and responses from the Regions regarding the OED comments.

Much of this sub-series consists of the DGO's copies of records from a number of working groups and task forces that affected the work of OED. Particularly extensive are the records regarding the Task Force on Evaluation Capacity Development (ECD) in Developing Countries which was established in 1992 to design mechanisms to assist member countries in enhancing their post evaluation capacities. Pablo Guerrero was the DGO/OED member on the Task Force. A file labeled PCR Task Force contains copies of records relating to three groups established to review the PCR process: 1) the Committee on the Revision of the PCR, Audit, and Pass-Through Procedures which met in 1986; 2) the Working Group on Project Completion Reports which was established in January 1988 and sent its report to the Joint Audit Committee (JAC) in January 1989; and 3) the joint OED/Operations Review Panel on Project Completion Reporting which was established in November 1990 and sent its report to JAC in June 1991. Other groups whose work is documented in these files are the Bankwide ICR Process Review Working Group, which met in 1998 to make the ICR process more efficient, effective, and timely, and the Working Group on Evaluation Dissemination, which was established in fiscal year 1994 to review the Bank's experience in incorporating evaluation results into new operations. It issued its final report in June 1996.

This sub-series contains records documenting Departmental Management Team retreats for: July 1996; March, October, and November 1998; October 2000; and the retreats for the entire DGO/OED staff for October 1997, October 1998, January 1999, and January 2000. The content of the files for each retreat varies, but the more complete files include: intra-OED correspondence planning the retreat; copies of invitations sent to non-OED retreat speakers; handwritten notes documenting the retreat proceedings; retreat agenda; copies of articles provided as background reading and of documents to be discussed at the retreat; paper copies made from computer-assisted presentations; and post-retreat issuances summarizing specific retreat sessions and containing typed versions of the flip charts generated during retreat sessions.

This sub-series contains records documenting the weekly meetings of the Departmental Management Team (DMT) which consisted of the Director of OED, the directors of the OED Departments, and some support staff. The files generally contain handwritten notes of the meeting's proceedings, meeting agenda, copies of documents discussed at the meetings, and typed minutes of the preceding meeting. The typed minutes are columnar in format and for each topic covered show a summary of the discussion, the follow-up action required, the staff member responsible for the action, and the expected date of completion for the action. The files for many of the 2002 meetings do not contain minutes.

This sub-series contains records documenting departmental staff meetings which were attended by all DGO/OED managers, regular OED staff, OED consultants, and temporary OED staff. The files contain: agendas; copies of documents discussed at the meetings; paper copies made from computer-assisted presentations; and handwritten notes of the meeting proceedings.

The series consists of records created and received by the Directors, Operation Evaluation Department (OEDDR), and the Directors-General, Operations Evaluation (DGO), which relate to the creation of Annual Reviews of OED evaluations as well as Annual Reports on Operations Evaluation (AROE).

In September 1975, the Operations Evaluation Department (OED) began issuing a report known as the Annual Review of Project Performance Audit Results (ARPPARs) as a means of disseminating the results of the project performance audits it conducted. This review presented in summary the major findings of the audit reports for completed projects and the Bank's response to them. The findings of the ARPPARs were discussed by the Executive Board's Joint Audit Committee (JAC) and JAC's findings were then reviewed by the full Board. By July 1981, a selective system of performance auditing was implemented in OED in keeping with the increase in the number of projects funded. In 1986, the report was renamed the Annual Review of Project Performance Results (ARPPR), and by 1989, the ARPPR was replaced by the Annual Review of Evaluation Results (ARER) which provided a comprehensive summary of evaluation findings. By 1997, the name of the report had changed once again to the Annual Review of Development Effectiveness (ARDE). In the ARDE, OED used the outcomes of evaluations of completed operations to provide a longer view of performance trends. Records in this series that relate to these types of Annual Reviews include drafts, correspondence, memoranda, press releases, comments, and meeting minutes.

The first Annual Report on Operations Evaluation (AROE) covering the period July 1975 - June 1976 was published in October 1976. The main purpose of the report was to indicate the status of evaluation work in the Bank and to comment on the main findings of evaluation activities. Subsequent reports reviewed the independent evaluation work performed within OED and the self-evaluation conducted by the Bank's operating departments (for example, the AROE for 1985 covered OED evaluation activities and evaluations conducted within the International Finance Corporation [IFC], the Economic Development Institute [EDI], and the Bank's economics and research staff). The format of the AROE was determined by the DGO and the plan for developing the report was made in consultation with OED. Drafts of the report were circulated to Bank managers for comments and the final version was submitted to JAC for consideration by the Bank's Board of Executive Directors in conjunction with JAC's own report on evaluation and the ARPPAR (later ARER and ARDE). Records in this series that relate to the creation of AROEs include correspondence, memoranda, initiating notes and outlines, drafts, comments, and meeting minutes.

This sub-series contains the working files of the Directors, Operations Evaluation Department (OEDDR), relating to the drafting of the 1st - 16th (1975 - 1990), and 18th - 20th (1992 - 1994) Annual Review of Evaluation Results (ARER). The content of these files varies somewhat but a typical file contains: correspondence among the OED division chiefs and among members of the Annual Review team regarding the format and content of the Review; OEDDR's correspondence with other Bank units requesting their inputfor the Review; a draft of the Review; comments on the draft from the Director-General (DGO), OED staff, and other Bank officials; OED replies to the comments; copies of the comments the DGO made to the Executive Directors at the Board meeting at which the Review was discussed; minutes of meetings of the Joint Audit Committee at which the Review was discussed; and summaries of discussions of the Review at meetings of the Joint Audit Committee and of the Executive Directors.

Filed at the beginning of the sub-series are notes of a December 1982 meeting held to discuss the publishing of OED impact evaluation reports and edited versions of the Annual Reviews for readers outside the Bank. A copy of an article from Consulting Engineer, September 1978, discussing the first Annual Review and the organization of OED is found in the file for the third Annual Review.

This sub-series contains the working files of the Directors-General, Operations Evaluation (DGO), documenting the drafting of the Annual Review of Evaluation Results (ARER) for 1981 - 1993. The files for 1981 - 1984 primarily contain correspondence with OED staff members and with staff in the Agriculture and Rural Development Department concerning input for the agricultural section of the report. The later files concern input for the entire report.

Most of the documents found in this series are duplicatedin WB IBRD/IDA OPE-04-01 (Annual Review Working Files of the Directors, Operations Evaluation Department [OEDDR]), but a few are unique to this series. In some cases, the DGO corresponded directly with the President or with other Bank Vice Presidents regarding the Annual Review without copying OEDDR or forwarding copies of the related replies to OEDDR. In other cases, his special assistants answered Annual Review inquiries, and their correspondence is not duplicated in the OEDDR files.

This sub-series contains the working files of the Director, Operations Evaluation Department (OEDDR), documenting the drafting of the Annual Reports on Operations Evaluation (AROE) for 1983 - 1985. The files primarily contain electrostatic copies of the correspondence of Alexander F. Kirk, chief of OEDD3, the unit assigned responsibility for compiling the report. Included are: memoranda from the sectors, regions, and other Bank units containing information for possible inclusion in the reports; intra-OED memoranda regarding the content and format of the reports; drafts of the reports and comments on the drafts; copies of the statements the Director-General, Operations Evaluation (DGO), made regarding the reports to the Executive Directors; and minutes of the meetings of the Executive Directors at which the AROE and the Annual Review of Project Performance Results were discussed. Also included is information regarding the input OED provided to the Bank's Annual Report.

This sub-series contains the working files of the Directors-General, Operations Evaluation (DGO), that relate to the drafting of the Annual Report on Operations Evaluation (AROE). The files generally contain: copies of the DGO's correspondence with other Bank units (the Regional units, the Sectors, the Economic Development Institute, and IFC's evaluation unit) regarding their contributions to the report; draft initiating notes and outlines for the report; notes to the OED Management Team and other intra-OED memoranda regarding the report; comments on drafts of the report; copies of the draft management response to the report; printing requests for the report; copies of the DGO's statement to the Executive Directors regarding the report; summaries of discussions of the report at meetings of the Executive Directors; and post-publication correspondence regarding the report. Files for reports generated under DGOs Kapur and Picciotto consist almost exclusively of electrostatic copies; files for DGO Rovani containoriginal documents.

This sub-series contains the shared working files of the Directors-General, Operations Evaluation (DGO), and the Directors, Operations Evaluation Department (OEDDR), for the Annual Reviews of Development Effectiveness (ARDE) for the years 1994 - 2000 and for the Annual Reports on Operations Evaluation (AROE) for 1993 - 1999. Most of the records in the sub-series are electrostatic copies of DGO Robert Picciotto's correspondence and copies of various drafts of the reports that he received, but there are alsosome original records from the OED Directors. The sub-series also includes an OEDDR-only file for the AROE for 1998.

The AROE files generally contain: draft initiating notes and outlines for the report; intra-OED memoranda regarding the report drafting process; comments on drafts of the report; copies of the draft management response to the report; and copies of the DGO's statement to the Executive Directors regarding both the AROE and the ARDE.

The ARDE files generally include: intra-OED memoranda regarding the report's format and content; drafts of the approach paper and design paper; comments on the draft report from Bank staff; copies of statements regarding the report from the DGO, the chair of CODE, and various Executive Directors; paper copies of slides used in computer-assisted presentations regarding the report; press releases; and copies of OED Reach, OED's Fast Track Brief, and OED Precis containing articles regarding ARDE. The files for ARDE 2000 include several items of 2002 OED correspondenceregarding the approach paper for the 2002 report.

This series contains documentation regarding the drafting and publication of booklets aimed at explaining the evaluation function in the World Bank to non-Bank audiences. The early editions of this booklet were titled Operation Evaluation: World Bank Standards and Procedures. By 1993, the title had changed to Assessing Development Effectiveness: Evaluation in the World Bank and the International Finance Corporation.

The sub-series contains records documenting the drafting and publication of the booklets. Included are: intra-OED correspondence; multiple drafts of the booklets; comments on the drafts; and agenda, minutes, and summaries of meetings of the Joint Audit Committee and of the Board of Executive Directors at which drafts were discussed. The sub-series also contains draft and/or published copies (in English and French) of various booklet editions (1976, 1979, 1984 [draft only], 1993).

The series consists of records related to the project monitoring and evaluation function of the Operations Evaluation Department (OED). The majority of the records in the series are final audit reports for individual, Bank-sponsored projects and records related to the creation of those reports. Reports date from the beginning of OED and its project evaluation activities in 1973. OED-authored reports include Project Performance Audit Reports (PPARs) and Performance Audit Reports (PARs). Reports completed byoperating departments and submitted to the OED include Project Completion Reports (PCRs), Project Completion Notes (PCNs), Implementation Completion Reports (ICRs), and Implementation Completion Notes (ICNs). The PCRs can be characterized as either reports that required and received an audit by the OED or as pass through PCRs which OED division chiefs determined did not require an audit. In the case of the latter, Notes of Record are sometimes included to explain why an audit was not required. The series also consists of background materials and working files created or collected during the preparation of audit reports. These records include: correspondence; memoranda; staff appraisal reports; reports and recommendations sent to the Executive Directors for approval of the loans or credits; loan or credit agreements that were executed; annotated audit drafts; project progress reports; and field notes. In some cases, working files related to audits and report preparation are those accumulated by specific evaluation officers in OED.

The types of records that accompany the final evaluation reports vary, as procedural requirements changed over the years. OED evaluating officers created files for all completion reports and audit reports for which they were assigned responsibility. Most of the pre-1982 working files containing PCRs and PPARs reflect procedures put in place by the Bank between 1973 and 1979. The time frame for completion of PCRs after final disbursements of loans / credits varied as did the extent ofthe involvement of operating departments and clients in their preparation. In the mid-1970s, the PCR was to be prepared within six months of final disbursement by the Bank's Regional Vice-Presidency of the country in which the project took place. Information for the PCR was gathered from project files, supervision reports, and discussions in the field between Bank and Government officials associated with the project. However, before 1976 the majority of the PPARs prepared by OED evaluating officers were for projects for which no PCR had been prepared.

In 1976, procedural changes required all PCRs to be sent through the Director-General, Operations Evaluation (DGO), without OED comment to the Executive Board along with the PPAR. In addition to the PCR, the PPAR working files from 1976 onward include: the Terms of Reference for the OED audit mission; the Back-to-Office Report from the mission; PPAR drafts; comments on the drafts from Bank officials in the Region, Government officials and representatives of businesses or institutions involved in the project; and copies of transmittal letters accompanying the PPAR. Loan or Credit Agreements, Staff Appraisal Reports, and Reports and Recommendations by the Bank President for loans and credits, most of which are of earlier date than the other records in the file, are included in some PCR and PPAR files. By the end of 1979, OED conducted abbreviated audit reviews for about half of the PCRs it received. The working files for these passed-through PCRs usually contain: a memorandum addressed to the Regional Vice President indicating that based on the OED review of the PCR an audit would not be performed; comments on the PCR; a final draft of the PCR sent for printing by OED; a copy of the printed PCR forwarded to the Executive Board; and copies of the transmittal letters used in the OED distribution of the printed PCR.

At the beginning of fiscal year 1983, a new system of selective project performance auditing was introduced which required that 40 to 50 percent of projects be audited. PCRs were selected for audit according to whether they were for: large, complex, or innovative projects; projects an Executive Director proposed for performance audit; projects with high lending priority; projects which were first in a given sector / sub-sector for a country or last in a series; or a series of projects subject to a combined audit. OED division chiefs were given responsibility for determining the projects to be audited and for making decisions on pass-through PCRs. Some files for projects which OED did not audit contain: a Summary Note explaining the reasons for the pass-through decision and evaluating the quality of the PCR; a copy of the draft PCR; comments regarding the draft PCR from the client country; a Pass-Through PCR Unit Cost Sheet tracking OED staff time allocated to the project; memoranda and letters transmitting a final copy of the PCR to the Executive Directors, the Bank President, and officials of the client country; and a final copy of the PCR. Found in other working files for post-fiscal year 1983 is the Note of Record, which was prepared by the OED evaluating officer after review of the PCR with the division chief. It contained the reasons for the pass-through decision. The Note of Record assessed the project as described in the PCR in terms of its objectives, scope, implementation, costs, and sustainability. It was distributed only to the Director of OED (OEDDR), the OED Library, the working file, and the OED annual review coordinator. Also found in some workingfiles with the Note of Record is the Project Information Form (PIF) used to input data about the project into OED's database for use in annual reviews and studies. Results for all PCRs were reported to the Executive Board.

New guidelines for completing PCRs were put into place effective July 1, 1989. Most of the PCRs in the OED working files after this date have a different format from earlier prepared PCRs. Parts I and II of the report were prepared by the Regional Office and Part III was completed by the client. Some working files include Project Completion Notes (PCNs) which were prepared in lieu of PCRs for projects for which: the loan was never signed by the client; the Bank withdrew the loan; disbursement was very fast or ahead of schedule; no PCR was ever completed; or for which there were no implementation issues. The decision to create the PCN was based on a review of the Staff Appraisal Report prepared before the loan or credit was granted, internal Bank memoranda, minutes of Board discussions, supervision reports, and interviews with staff. The PPAR was carried out on the basis of the recommendation for audit made by OED at the time of review of the PCR. The contents of the PPAR working file remained largely unchanged from earlier periods but some PPARS for cluster audits (i.e. audits of multiple projects) were included.

By 1994, operations staff (also called Regional staff) evaluated all completed operations for which they were responsible and OED audited a representative sample of completed operations. For last supervision missions departing after July 1, 1994, Regional staff was required to prepare an Implementation Completion Report (ICR) or Implementation Completion Note (ICN) similar to the earlier PCN. Issues encountered during implementation, achievements, and the economic rate of return, were among the factors covered in the ICR. Contributions from client agencies (whole reports or comments) were attached to the ICR. Co-financiers were invited to participate in evaluations in which they provided funding. All ICRs were to be completed within six months of the final loan / credit disbursement. By 1997, about 25 percent of completed projects were audited by OED. Like the earlier PPARs, Project Audit Reports (PARs) were field-based and incorporated the views of client countries and major stakeholders. Projects were subject to audit if they were: large and complex; innovative or had unusual features; controversial, with issues unresolved during the ICR evaluation; first in a series or expected to offer especially strong lessons; suitable to be analyzed together with similar projects; useful for providing building blocks for sector or country evaluation study; or requested by Executive Directors.

The ICR, PCR, PCN, and ICN working files for the mid-1990s and later generally contain the printed report, a PIF and control sheet, drafts of the DGO's evaluative memorandum regarding the report or note, a copy of a memorandum from OEDDR transmitting the draft evaluative memorandum to the relevant Country Director, and comments from the Bank's Regional staff on the draft. Some files contain the OED-ICR-Review-Evaluation Summary which recommended for or against an audit of the project. Projects were evaluated for outcome, sustainability, institution development, Bank performance, and client country performance based on information in the ICR and PIF. Correspondence with Regional and other Bank staff and with officials in client countries regarding their comments on the draft report can also be found in the files. Some files contain copies of intra-OED electronic messages commenting on the draft evaluative summary and on the review process. A few of the files contain background documents records such as Staff Appraisal Reports (as was the case for PCR files for earlier periods).

In addition to draft and final copies of the reports, the PAR files for this period typically include: Initiating Memoranda; Terms of Reference (TOR) addressed to OED evaluators designated to travel to the field to conduct the audit; letters to Government officials and others related to the project announcing the mission to develop the report; Back-to-Office Reports from the audit mission; and comments on the draft audit report. Comments from the OED audit panel and an Audit Panel Review form can be found in some files along with drafts of the DGO's evaluative memorandum, a PIF and cover sheet, and copies of letters transmitting the final printed copy of the PAR to officials in the client country. Some of the PAR files contain copies of related background materials such as the Staff Appraisal Report and the Loan Agreement.

The review and audit activities of regional operations and the OED were conducted for the projects of all sectors. These include:

*Education- Primary; secondary; tertiary; vocational; technical

• Health and nutrition Specific disease and HIV/AIDS prevention and management; population projects

• Water Sewarage and drainage; sanitation; water supply; water pollution control; flood control

• Agriculture Commodity production; livestock development; fisheries development; research; support services; marketing

• Human resources Extension and training; social sector management; science and technology

• Rural development Rural roads; rural electrification

• Urban development Urban engineering; air pollution; urban management

• Telecommunications General sector

• Energy and power Power generation; refinery engineering; energy efficiency; oil field development; gas transport and distribution; oil, thermal, and gas exploration and development; pipeline rehabilitation

• Transport Navigation; railways; highways; rural roads; aviation; ports

• Finance and trade Private sector finance projects; financial rehabilitation projects; economic management and public enterprise; technology development projects; small and medium enterprise projects; development banks; export development; shipping; imports program; debt and debt service

• Industry Foundry modernization; small-scale industry; fertilizer production; mining and other extractive sector projects; pulp and paper engineering; public enterprise assistance; natural resource management; forestry; off-shore development

• Infrastructure Housing; construction projects

• Reconstruction and emergency Earthquake and flood recovery; drought relief

• Tourism General sector

• Structural adjustment Public sector reform loans; adjustment loans and credits; trade policy adjustment

• Client capacity building Technical assistance loans; central government administration projects; planning support

Evaluations of non-sector and multi-sector projects are also included.

The series also consists of evaluative reports and related materials created by OED staff that are not PARs or PPARs and are not related specifically to individual projects. These include: staff appraisal reports initially developed for the projects; reports and recommendations sent to the Executive Directors for approval of the loans or credits; materials related to a study by Carl Jayarajah entitled Overview Study: World Bank Support for Small and Medium Industry; a draft of correspondence related to a proposed Review of Lessons Learned in Railway Projects (RLLRP) for the OED Lessons & Practices publication; an approach paper, drafts, and background materials for Review of the World Bank Experience with Electric Power Generating Facilities (1985); a draft study outline, Terms of Reference, and supporting materials for the 1987 - 1989 study Columbia: An Evaluation of Bank Lending to the Colombia Power Sector between 1970 and 1986 along with the March 1991 final study report; the approach paper and draft of the 1988 - 1989 country case study World Bank Support for Rural Roads Maintenance - Philippine Case Study; a draft of Senior Evaluator Jan de Weille's March 1992 report Annual Review, Project Performance Results for 1991 - Transport; the approach paper and drafts of the June 30, 1994, grey cover sector review (Report 13291) Rural Electrification in Asia: A Review of Bank Experience; records related to a study of the Bank's involvement in Mexico's power sector over four decades; background materials for Study of Adjustment Lending in Sub-Saharan Africa (Oct. 1996 - July 1997) including an approach paper, preliminary draft, draft speech prepared for the Director-General, Operations Evaluation, and copies of related reports; the report Case Study of World Bank Activities in the Health Sector in Mali accompanied by a memorandum from OEDST inviting the DGO and other OED personnel to a meeting review regarding the study; working files of evaluator Basil Coukis which include annotated copies of PCRs and PPARs as well as background materials such as loan guarantee agreements, staff appraisal reports developed for loan or credit applications, progress reports, background correspondence, and field notes used in the preparation of evaluations; and two reports concerning trade policy reform for Mexico, Mexico: Trade Policy Reform, World Bank's Support for Trade Liberalization Program by Yalcin M. Baran (Sept. 15, 1989) and Overview Study: World Support for Small and Medium Industry issued by the Projects Department of Latin America and the Caribbean Regional Office (June 24, 1986). Finally, the series consists of records related to the OED's problematic review of the Thirteenth Indian Railway Project (Credit 582-IN). The files contain the April 1979 PCR, various versions (dating from 1981 to 1986) of the PPAR, and correspondence regarding the project. The first and the subsequent draft PPARs for this project were controversial and found to be inadequate for various reasons. The controversy led President Clausen to appoint a special audit team for the Railway Project and to request a reevaluation of OED internal procedures. Materials related to the subsequent review as well as to the OED evaluation are included; the latter includes: intra-OED correspondence; correspondence with Transportation Department staff, the South Asia Regional Office, and Indian government officials; and minutes of a Joint Audit Committee meeting and Executive Directors' meeting.

The series primarily consists of Impact Evaluation Reports (IERs) and the working files produced during the creation of those reports. The series also consists of a number of special studies produced by the OED. IERs assess the performance of operations at full development, some five to ten years after the close of disbursements on the Bank's loan. The evaluations are selective; only a small percentage of projects undergo an impact evaluation. These evaluations provide a second look at a project by analyzing the long-term effects - intended or unintended - on people, institutions, and the physical environment. The evaluations assess projects against the goals that were stated at appraisal and also against a broad set of criteria that relate to social dynamics, income distributions, effects on women and families, institutional development, sustainability, and the environment. This series consists of IERs from the very beginning of their production in 1979 and through into the 1990s.

The content of the files for impact evaluations varies somewhat but the more complete files contain: copies of the Completion Reports, Audit Reports, and other OED-generated records relating to the project; non-Bank background reports and publications relating to the project; Terms of Reference for the OED staff members and/or consultants assigned to evaluation; drafts of the Approach Paper and Study Design Paper; memoranda and correspondence regarding the team's travel plans if a mission was required; a Back-to-Office Report regarding the mission; intra-OED correspondence regarding the format for the evaluation; various drafts of the evaluation; comments on the drafts from OED and Regional staff, officials in the client country, and representatives from other aid/lending organizations; drafts of comments by the Director-General, Operations, (DGO) to the Committee on Development Effectiveness (CODE) or its predecessor, the Joint Audit Committee (JAC) regarding evaluations; minutes of CODE or JAC meetings at which the evaluation was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; and summaries of the Executive Directors' discussion of the evaluation and copies of the Board chairman's published comments. Many of the impact evaluations relied on field surveys to gather opinions of stakeholders. The raw survey data is not included in an evaluation file, but sample survey questionnaires and data derived from the surveys may be in the file.

The series contains the first IER produced by theOED, referred to as a Project Impact Evaluation Report (PIER). The report was for the Mexico Third Irrigation Project (1978 - 1979) which focused on irrigation rehabilitation programs in irrigation districts Nos. 17 (Region Lagunera) and 23 (San Juan del Ro). The evaluation was undertaken by Chief Evaluation Officer John Malone about five years after the performance audit was conducted for the project. Other early impact evaluation reports in this series are for: Kenya's First Smallholder Agricultural Credit Program (1980); the Kenya First Livestock Development Project (1981); the Roseires Irrigation Project in the Sudan (1980); the Burundi-First Arabica Coffee Improvement Project (1980); the Malaysian Muda and Kemubu Irrigation Projects (1981); the Atlantico Irrigation Project in Colombia (1982); and the San Lorenzo Irrigation and Land Settlement Projects in Peru (1982). In addition, records related to an IER conducted jointly with the National Economic and Development Authority (NEDA) and the Central Bank of the Philippines on the Philippines Second Rural Credit Project are included. The report was issued in 1983 after an extensive evaluation that involved a survey of beneficiaries in different regions. Also issued in 1983 was an IER for the Indonesia Irrigation Rehabilitation Project.

The series contains many other IERs and IER working files produced through the mid to late 1980s and 1990s. Agriculture, urban, education, transport, industry and transmigration projects are represented. Of note, one IER covers three income generating projects for refugees in Pakistan for which the Bank acted as executive agency on behalf of the United Nations High Commissioner for Refugees (UNHCR). Also contained are background materials on Indonesia that OED's Agriculture and Human Development Division (OEDD1) gathered from a wide variety of sources in preparation for a 1993 impact evaluation covering loans for three transmigration projects in Indonesia. The files include the Project Completion Reports for two of the loans (Transmigration II [L1707] and III [L 2248]) and Terms of Reference for the impact evaluation but no other documents regarding the impact evaluation.

The series also includes a number of special process and thematic studies. Reports and related working papers include: a 1987 and 1988 OED special study entitled Management of Renewable Resources in Agricultural Operations; two transport sector evaluation studies entitled The Transport Sector in Mexico: An Evaluation (1998) and Transport in China: An Evaluation of Bank Assistance (1999); a copy of a 1998 paper (The Industrial Organization of Corruption: Why Corruption Hurts More in Africa than in Asia) prepared by Sector and Thematic Evaluation Division (OEDST) staff members Antti Talvitie and Binyam Reja, based on their participation in an OED evaluation of five transport projects in Indonesia; background records for a 1996 Infrastructure and Energy Division (OEDD3) process study, Effectiveness of Environmental Assessments and National Environmental Action Plans (EA/NEAP Study), which evaluated the Bank's use of project-level environmental assessments (EAs) and highlighted the differences between countries that had an approved National Environmental Action Plan (NEAP) and those that did not; working files for the 1995 Country Policy, Industry and Finance Division (OEDD2) study The Social Impact of Adjustment Operations: An Overview; and special impact evaluation studies covering a range of agriculture topics, including Harvesting the Waters (1980s), The World Bank and Structural Adjustment in Agriculture; Reforming Agriculture: The World Bank Goes to Market, Irrigation O & M [Operation and Maintenance] and System Performance in Southeast Asia (1996), Natural Resource Management in Bolivia: 30 Years of Experience (1993), and Dynamics of Rural Development in Northeast Brazil: New Lessons from Old Project (1991), as well as one file for a special process study entitled Monitoring and Evaluation Plans in Staff Appraisal Reports Issued in Fiscal Year 1995.

Also in this series are correspondence, reports, and publications that evaluator Christian Polti gathered while OEDD1 was working on two impact evaluation studies. The first study deals with the World Bank's experience with irrigation development and examined irrigation projects in Morocco, Mexico, the Philippines, and Thailand. The second study deals with cotton development programs in Cote d'Ivoire, Burkina Faso, and Togo.

The series consists of Country Assistance Reviews (CARs), Country Assistance Notes (CANs), Country Assistance Evaluations (CAEs), and related records created by the OED. Country Assistance Reviews (CARs) were launched in 1994 when they were endorsed by the Joint Audit Committee (JAC) Subcommittee on OED Reports. OED's first CAR focused on Ghana and was completed late in FY 95. The CARs were, in essence, countrywide impact evaluations that concentrated on the overall impact and development effectiveness of the Bank's whole program of assistance (for both lending and non-lending activities) to a country over a number of years, usually a decade. They evaluated the Bank's overall assistance program to a country in the same way that performance audits evaluated individual projects. CARs processing schedules were eventually timed and designed so as to be useful inputs into the preparatory work for the corresponding Country Assistance Strategy by both Bank staff and government officials and to provide the ExecutiveBoard with an up-to-date, independent evaluation of past results of Bank assistance one to three months in advance of the scheduled CAS consideration.

Country Assistance Notes (CANs) were introduced in 1997 as a way to speed the process of country assistance evaluations. CANs were meant to be mini-CARs and thus were more limited in the scope and/or time frame covered. For example, CARs covered all relevant sectors of a country, whereas the CANs were more selective. The rationale for introducing the CANs was to achieve greater coverage of countries in the short term. After the introduction of CANs, CARs were limited to the largest borrowers, in total or per capita, or to outliers in terms of performance and GDP per capita. CANs were produced for countries undergoing a second evaluation and for which a CAR had previously been produced. The length of both the CARs and CANs changed over time. The first CARs were typically about 100 pages; later CARs were limited to 15 to 30 pages of main report. The CANs were shortened to 5 to 15 pages. As time went by, OED softened the distinction between CARs and CANs and by the late 1990s, the country-wide evaluations were simply referred to as Country Assistance Evaluations (CAEs), but OED did generate mini CAEs that were the equivalent of CANs.

CARs, CANs, and CAEs were based largely on a review of OED and Bank Reports available in Washington: Staff Appraisal Reports; Performance Audit and Implementation Completion Reports; Country Assistance Program/Strategy Papers; Country Briefs; Country Economic Memoranda; sector reports; Country Risk Assessments; management briefs for the Annual Meetings and for high-level field visits; and project and general country files. They also relied on direct interviews with key current and former government officials, other relevant national and/or association representatives, donor representatives, and current and former Bank staff members. CANs did not usually involve an OED mission to the client country and personal contact with other donors, whereas CARs may have involved more than one mission and increasingly included evaluation participants from other donor agencies. CARs, CANs, and CAEs were all prepared in close consultation with the Regional operational staff.

The content of the files for the country assistance evaluations varies somewhat but the more complete files contain: copies of the previous OED and Bank studies and reports for the country; Terms of Reference for the OED staff members and/or consultants assigned to the project team; drafts of the Approach Paper and Study Design Paper; memoranda and correspondence regarding the team's travel plans if a mission to the client country was required; a Back-to-Office Report regarding the mission; intra-OED correspondence regarding the format for the evaluation; various drafts of the evaluation; minutes of One-Stop Review Meetings at which the top OED managers evaluated progress on or reviewed a draft of the evaluation; comments on the drafts from OED and Regional staff, officials in the client country, and representatives from other aid/lending organization; drafts of the DGO's comments to CODE regarding the evaluation; minutes of CODE meetings at which the evaluation was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; summaries of the Executive Directors' discussion of the report and copies of the chairman's published comments; drafts of the management response to the evaluation; a draft of the OED Fast Track Brief and Precis regarding the evaluation; the final, published (grey cover) evaluation; and letters transmitting the final, published evaluation to officials in the client country. A few files also contain questionnaires completed by stakeholders, paper copies made from computer-assisted presentations, and copies of records used in workshops held for various stakeholders.

All of the country assistance evaluations in this series were generated by the OED's Country Evaluation and Regional Relations Division/Group (OEDCR) and its predecessor, the Country Policy, Industry and Finance Division (OEDD2). The series contains reports and related working files for many countries, including: the Russian Federation, Bulgaria, Lithuania, Paraguay, El Salvador, Brazil, Peru, Haiti, the Dominican Republic, Guatemala, Chile, Guinea, Niger, Eritrea, Zambia, Lesotho, Jordan, Mongolia, India, Zambia, Yemen, Tunisia, Nepal, Pakistan, Costa Rica, Malawi, Kenya, Thailand, Cameroon, Zimbabwe, Uruguay, Croatia, Uganda, Ghana, Cote D'Ivoire, Cambodia, Mozambique, West Bank and Gaza, Papua New Guinea, Philippines, Ecuador, Tanzania, India, Poland, Ecuador, Argentina, Chile, Maldives, Kazakhstan, Indonesia, Morocco, Mexico, Albania, Azerbaijan, Bangladesh, Burkina Faso, Egypt, Jamaica, Sri Lanka, Tgo, Ukraine, Brazil, Armenia, Belarus, Bolivia, Nepal, China, and Turkey. Of special note in this series is the extensive documentation for the CAR for India completed by a team headed by Gianni Zanini and issued in 2001. In addition to the usual documentation generatedfor a country assistance evaluation, the files for the India CAR include an extensive collection of Bank reports on India, materials from workshops organized by OED and the South Asia Region from March to May 2000, and twenty or more background papers prepared by OED staff and consultants. Also of note are the voluminous files for the Russian Federation CAR which, in addition to the usual CAE records, include schedules and briefing papers for President Wolfensohn's visits to Moscow in May 1998 and February2000.

The special evaluation studies build on the primary assessments covered by Completion and Audit Reports and Impact Evaluations. Those primary evaluation products for specific projects generate data, findings, and lessons that support the special evaluation studies which examine the impact of a number of projects and their implications for the World Bank's policies, practices, and procedures. The special evaluation studies cover topics and issues raised by the Executive Directors and by management and operations staff as well those emerging from audit work. The studies focus on countries, sectors, Bank processes and policies, and broad themes.

• Sector Studies- Address the effectiveness of Bank programs on sector policies and on project selection, design, and implementation. Many of the sector studies compare experiences across countries (examples include studies of human resource development, Bank experience in the Education Sector, and rural water supplies), whereas others are in-depth reviews of particular sectors in individual countries (e.g., educational development in Korea, population policies and programs in Kenya).

• Process- Studies Evaluate the Bank's business processes from a development perspective. These studies examine how well the Bank applies its policies and evaluate the efficiency and effectiveness of Bank business practices. Examples include: studies of: poverty assessments; Environmental Impact Assessments and Action Plans; and Project Appraisals.

• Policy Reviews- Examples of topics covered by such studies include: experience with agricultural research; fiscal management; financial sector reform; and experience in large dams.

• Thematic Studies- Examine broad themes that encompass multiple sectors and countries and themes of special emphasis for the Bank, such as: resettlement policy; gender; anti-corruption; and poverty alleviation.

The content of the files for the special studies varies but the more complete files contain: drafts and final copies of an Initiating Memorandum (IM), a Study Design Paper, and an Approach Paper; comments on these initial documents from OED staff and from members of the Committee on Development Effectiveness (CODE) or its predecessor, the Joint Audit Committee (JAC); Terms of Reference for the OED staff members and/or consultants assigned to the study; copies of previously issued Bank and OED reports, studies, and audits relating to the study topic; Back-to-Office Reports from study missions; intra-OED memoranda regarding methodology, schedules, and progress on the study; various drafts of the study with comments on the drafts from OED staff, Bank staff in other units, representatives of other development and aid organizations, and officials in client countries; drafts of the DGO's comments to CODE (or JAC) regarding the evaluation; minutes of CODE (or JAC) meetings at which the study was discussed; drafts of the DGO's transmittal memorandum to the Executive Directors and the President; and summaries of discussions and minutes of meetings of the Executive Directors at which the study was discussed; copies of the OED Precis and Fast Track Brief relating to the study; and copies of letters transmitting the final report to interested parties.

The sub-series consists of the working files for special studies and evaluation reviews conducted by OED and its predecessor organization, the Operations Evaluation Unit (later Division) of the Programming and Budgeting Department, prior to the formation of sector-specific divisions in 1983. Reports include sector, country, management, and policy studies.

The series includes the first two evaluation studies conducted by the Operations Evaluation Unit. These studies included a deep study of everything the Bank had done in one country, Colombia, and a wide study evaluating loans to a number of countries in one sector, the electric power sector. The files for the first country study, Bank Operations in Colombia: An Evaluation, Report No. Z-18, 1972, also include a draft and final version of the 1975 Closing Report to Colombia Evaluation which was a follow-up review of the 1972 study to assess what progress had been made on the recommendations contained in the 1972 study. The files for the first sector study, Operations Evaluation Report: Electric Power, Report No. Z-17, 1972, include the 1975 follow-up report, Closing Report on Actions Relating to Recommendations of the Electric Power Evaluation Report of March 1972.

Other sector studies included in this series are: a comparative highway evaluation study; a Development Finance Companies study; a Bank lending for agriculture credit study; reviews of agricultural and rural development programs in Indonesia and the Philippines; and a review of rural development projects in Sub-Saharan Africa. The Department's first three management policy reviews are included and address delays in loan and credit effectiveness, delays in project preparation, and the role and use of consultants. Early operational policy reviews cover project implementation, supervision of Bank projects, and compliance with loan covenants. This series also includes files for OED studies of project monitoring and evaluation and technical assistance pilot evaluation studies conducted in Kenya and Afghanistan.

The study files in this series often contain earlier Bank documents, some dated as early as 1957, relating to the subject of the study. The compilers of these early studies established the practice of soliciting comments on draft studies and reports from all interested parties in the Bank and in the client countries, and most of these study files contain both a series of drafts and the related comments.

The sub-series contains the working files for special studies generated by the Agriculture and Human Development Division (OEDD1) and its predecessors: the Agriculture and Family Planning Division; the Agriculture and Human Resources Division; and the Agriculture, Infrastructure and Human Resources Division. Included are files for process, policy, and thematic studies as well as sector studies concerning agriculture, education, population, and human resources projects and issues. Specifically, reports focus on: agricultural research and education; Bank project supervision; training included in Bank financed projects; population programs; Bank lending policy for plantation crops; agriculture projects in Sub-Saharan Africa; renewable resource management issues; gender issues in Bank lending; human resource framework for Bank projects; fertility decline; Bank lending for agricultural credit and rural finance; poverty alleviation; and the Bank's experience with government sponsored land settlement with irrigation development, agricultural marketing, and rural development.

Of special note are the files regarding the Narmada Project. It appears that OEDD1 began assembling many of the Narmada materials in this sub-series in 1991 when Bank President Conable commissioned in September of that year an independent review of two Bank-funded projects, Sardar Sarovar Dam and Power Project (Credit 1552; Loan 2497) and Sardar Sarovar Water Delivery and Drainage Project (Credit 1553), referred to collectively as the Narmada Project because of the relationship of the projects to the Narmada River in India.

Included in this sub-series are working files for a small number of OEDD1 Impact Evaluation Reports (IERs) covering: an irrigation project in Sri Lanka; a settlement project in the Terres Nueves region of Senegal; a Benin agricultural project; a cotton development project in Burkina Faso, Cote d'Ivoire, and Togo; education projects in Colombia, Indonesia, and Malawi; and others. There are also files for the 1990 and 1995 Interim Evaluations of the Aga Khan Rural Support Program (AKRSP) including the second and third of a series of evaluations of AKRSP conducted by OED at the request of, and with funding from, the Aga Khan Foundation. A few files in this sub-series contain comments that OEDD1 staff provided on studies and reports generated by other Bank units or by other organizations and institutions.

Some of the files in this sub-series contain background reports, studies, and publications dated as early as 1970.

The sub-series contains the working files for special studies generated by the Country Policy, Industry and Finance Division (OEDD2) and its predecessor, the Policy-Based Lending, Industry, Public Utilities and Urban Sectors Division. The studies cover a range of policies, themes, sectors, and countries, including: conditionality in World Bank financed loans; privatization; women in development; free-standing technical assistance for institutional development in Sub-Saharan Africa; trade policy reforms under adjustment programs; a study of Bank/country relations covering Mexico's agriculture, industry, infrastructure, urban, and electrical and power sectors; and the 1992 study The World Bank and Pacific Island Countries: An OED Review which covered the Bank's operations in six Pacific Island member countries: Fiji, Kiribati, Solomon Islands, Tonga, Vanuatu, and Western Samoa. There is also a draft of the design paper for the FY95 country assistance review for Nigeria.

Also included are files documenting OEDD2's participation in a Bank-wide steering group on Development Aid Effectiveness and a file for the development of a policy ledger of OED recommendations, management responses, and actions for all studies discussed by the Joint Audit Committee (JAC) after July 1, 1992.

The sub-series contains the working files for special studies generated by the Infrastructure and Energy Division (OEDD3) and the files of staff members Basil P. Coukis and Andres Liebenthal who worked on special studies while in OEDD3. Most of the special studies files are dated 1991 - 1993 and concern the study, The World Bank and Sub-Saharan Africa: An OED Review of Bank Assistance for Transport Corridors. Also included are: files for OEDD3's contribution to the Bank/Mexico Relations Study, a country relationship study completed in fiscal year 1994; a final copy of the OEDD3 report Rural Roads Maintenance - Review of Completed World Bank Operations (June 1992); and a March 1993 facsimile message from Senior Evaluator Jan de Weille to the Resident Mission in Jakarta, Indonesia, concerning de Weille's forthcoming mission to conduct an audit for an impact study of the Tanjung Priok Port project (Loan 13337).

The Coukis files concern four special OED studies on which Coukis worked from 1987 to 1993: the Senegal Country Study; a review of World Bank lending for natural gas; the Sub-Saharan Africa Transport Corridors study; and a review of World Bank experience in the Education Sector in which Coukis worked when he was in the Agriculture and Human Relations Division (OEDD1). The Coukis files also contain copies of a Project Performance Audit Report, Staff Appraisal Reports, and published articles dated as early as 1983.

The Liebenthal files primarily contain correspondence and e-mail exchanges dated 1996 - 1999concerning Liebenthal's role as the Task Manager for the August 1996 OED desk assessment of fifty Bank-financed large dams, The World Bank's Experience with Large Dams: A Preliminary Review of Impacts. The correspondence is with OED and other Bank staff, staff of the World Conservation Union (IUCN), and representatives of major stakeholder groups in the global large dams debate. Correspondence contains: comments regarding the 1996 OED large dams assessment; planning for and response to the April 11-12, 1997, international workshop on the Future of Large Dams co-hosted by the World Bank and the World Conservation Union in IUCN headquarters in Gland, Switzerland; comments on publications by Patrick McCully, the Campaigns Director of the International Rivers Network; and Bank/IUCN efforts to establish the International Commission of Experts on Large Dams (ICELD) to provide an independent review of the economic and technical performance of large dams. Some of Liebenthal's correspondence concerning his OEDD3 workon dams was sent after his assignment to the Sector and Thematic Evaluations Division (OEDST).

The Liebenthal files also contain a number of published articles and studies dated 1985 - 1995 and relating to dams. Included are: a copy of the August 1996 OED assessment; a copy of Silenced Rivers and Damming the Rivers: The World Bank's Lending for Large Dams both by Patrick McCully; and a compilation of documents provided as background material for the attendees of the April 1997 workshop in Switzerland.

The sub-series contains the working files for special studies generated by the Sector and Thematic Evaluation Division (OEDST) as well as some records of its predecessor, the Agriculture and Human Development Division (OEDD1).

Much of this sub-series consists of files relating to the preparation of papers concerning NGO involvement in World Bank-Supported Projects in Mali, Kenya, Bolivia and Brazil. These papers were part of five country case studies (covering India in addition to Mali, Kenya, Bolivia and Brazil) prepared as background papers for the OED study: Nongovernmental Organizations in World Bank-Supported Projects: A Review (1998).

Other studies covered in this sub-series include: Information Infrastructure: The World Bank Group's Experience (2001); The Bank's Assistance to China's Energy Sector, An OED Country Sector Evaluation (2001); Toward Sharpening the Focus on Rural Poverty: A Review of World Bank Experience (2002); Private Sector Development in the Energy Sector: A Review of the World Bank Group's Assistance (c. 2001); The Impact of Dairying Development in India: The Bank's Contribution (c. 1997); Meeting India's Energy Needs (1978-1999): A Country Sector Review (1999); and Education in Bangladesh: A Country Sector Review (1999).

There are fragmentary files for a sub-series of health studies that are not as complete as most of the working files. Evaluation Health Projects: Lessons from the Literature was developed by the Agriculture and Human Development division (OEDD1) in l995-1997. Theother health studies were generated by OEDST, including: An Analysis of Combating Iodine Deficiency: Case Studies of China, Indonesia, and Madagascar (2001); Supporting a Healthy Transition: Lessons from Early World Bank Experience in Eastern Europe (2002); and Case Study of World Bank Activities in the Health Sector in India (1998-1999).

This sub-series also contains the working files for a 1996 OEDD1 Performance Audit Report (PAR) for the project Brazil: Science and Technology Project (Loan 2489-BR). Inaddition, there are files for The Next Ascent: An Evaluation of the Aga Khan Rural Support Program (AKRSP), 2002, which was part of a series of evaluations of AKRSP conducted by OED at the request of and with funding from the Aga Khan Foundation.

The sub-series contains the working files for special studies generated by the Country Evaluations and Regional Relations Division (OEDCR) as well as some records of its predecessor, the Country Policy, Industry and Finance Division (OEDD2).

• Much of this sub-series (six linear feet) consists of the 1999 - 2004 working files for the ODECR study The Gender Dimension of Bank Assistance I and II. These files include materials from the Bank gender issues workshops held in Nairobi, Delhi, Quito, and Warsaw in2000 and 2001.

• Other studies primarily concern finance, adjustment lending, civil service reform, and poverty reduction. Studies include: Civil Service Reform: A Review of World Bank Assistance (April, 1999); The World Bank and Microenterprise Finance from Concept to Practice (November, 1999); The Effectiveness of the World Bank's Poverty Reduction Strategy: An Evaluation (May, 2000); Adjustment Lending in Sub-Saharan Africa: An Update (1997); World Bank Assistance to Privatization in Developing Countries (August, 1994); Higher Impact Adjustment Lending (HIAL): Initial Evaluation (June 1999); The Impact of Public Expenditure Reviews: An Evaluation (1998); World Bank Assistance to CFA ('Communeaute Financiere Africaine'[African Financial Community]) Countries, An Evaluation of Selected Social, Economic and Regional Aspects of the Bank's Performance (2000); Financial Sector Reform: A Review of World Bank Assistance (1998); Fiscal Management in Adjustment Lending (1996); a joint 1998 OED/OEG study, An Evaluation of the Foreign Investment Advisory Service (FIAS); Implementation of Operational Directive 4.20 on Indigenous Peoples: An Independent Desk Review (2003); Legal and Judicial Reform in Europe and Central Asia (2002); Lessons for LICUS (Low Income Countries Under Stress) from OED Country Assistance Evaluations (2001 - 2004); and The Special Program of Assistance for Africa (SPA), an independent evaluation of the SPA as a mechanism to promote adjustment and development in Sub-Saharan Africa (1996 - 1998).

• The sub-series also contains incomplete background materials regarding two other studies: the Quality Assurance Group's 1996 assessment of the Argentine Convertibility Plan and OED's 1997 survey of the major participating financial institutions (PFIs) involved with the Industrial Guarantee Loan Fund (IGLF). The Argentine file primarily contains background information regarding the Convertibility Plan itself. The PFI survey file consists only of correspondence with officials at the Development Bank of thePhilippines concerning their advice regarding the design of the survey records. The survey was part of the OED evaluation of the IGLF program.

• In addition to study files, this sub-series contains files with copies of draft Bank issuances for which the OEDD2 and OEDCR staffs provided comments. Included are comments files for the Portfolio Improvement Program, proposed procedures for the Executive Directors, and Operations Manual draft chapters. The sub-series also contains a copy of the proceedings of a 1998 OED workshop on Indonesia that provided background information for a Country Assistance Review (CAR) of Indonesia.

The sub-series contains the working files for three major studies which the staff of the Corporate Evaluation and Methods Department (OEDCM) either participated in or directed. The first study, authorized by the Committee on Development Effectiveness (CODE) in January 1999, was an OED review of the IDA program during the IDA-10-11 period (fiscal years 1994 - 1999) and during the first year of the IDA-12 replenishment (2000). The review, which examined IDA compliance with recommendations made by the IDA Deputies in the replenishment reports for IDA 10-12, concentrated on IDA's development contributions in six program areas (poverty reduction, social sector development, gender, environmental sustainability, private sector development, and governance) and four process reform areas (enhanced Country Assistance Strategy [CAS] design and implementation, performance-based allocations, participation, and aid coordination). The files for this study cover the period 1998 to 2003 and include intra-Bank correspondence that analyze Country Assistance Strategies and contain comments on the proposed CAS redesign and draft background reports on the six program and four process areas.

The second special study was an assessment of the implementation of the Comprehensive Development Framework (CDF), an approach to development cooperation introduced by World Bank President James D. Wolfensohn in early 1999. The evaluation study was requested by CODE late in 1999. In the fall of 2000, the Bank hosted a global multi-partner conference to discuss how the evaluation should be conducted. As a result of that meeting, a Steering Committee was established in January 2001 to guide the study process with support from a Management Group, which was co-chaired by the Director-General of OED. The Steering Committee reflected the multi-partner nature of CDF and was composed of thirty members representing CDF pilot countries, donor agencies, development agencies, a developing country business firm, and an international non-governmental organization (NGO). The evaluation process itself focused on six case study countries: five for CDF pilot countries (Bolivia, Ghana, Romania, Uganda, and Vietnam) and one from a non-CDF country (Burkina Faso). The records documenting the CDF study cover the period from 2000 to 2004 and include, in addition to the usual special studies documents: planning documents for and minutes from the meetings of the Steering Committee and the Management Group and from a planning workshop held in October 2000; progress reports regarding the evaluation process; records from two workshops held in September and November 2002 to draft the CDF synthesis report; various drafts of the approach paper and an outline for the synthesis report; and drafts of chapters from the synthesis report itself.

The third study was an OED evaluative review of aid coordination policies, procedures, and practices. As part of this study, questionnaires were sent to three groups: Bank staff and managers; recipient government officials responsible for aid coordination; and representatives of local donor organizations. In addition to the OED questionnaires, OED and the United Nations Development Programme (UNDP) jointly requested the views on aid coordination from the directors of bilateral development agencies and officials of multi-lateral development banks (MDBs). The records in this sub-series are dated 1999 and relate only to the questionnaires received from the MDBs, the bilateral development agencies, and the local donor organizations. Included are the completed questionnaires and correspondence with World Bank country directors who selected the local donor organizations and with the directors of the MDBs and bilateral development agencies. Filed with the responses from MDBs and bilateral development agencies is a copy of the January 1999 report, Aid Coordination and the Role of the World Bank: An OED Review (Phase I) which summarized the first phase of the aid coordination study.

The series consists of three sets of leather bound volumes that were maintained in the Office of the Director-General, Operations Evaluation (DGO). Volumes are titled: Annual Reports - Annual Reviews - JAC Reports - JAC Subcommittee Reports (October 1971 - November 1995); Project Evaluation - Project Performance Audit Reports (PPARs) - Project Completion Reports (October 1972 - August 1994); and Special Studies - Impact Evaluation Reports (July 1973 - August 1994).

Series consists of records created and received by the Director of OED's Agriculture and Human Development Division (OEDD1). Specifically, the series contains records gathered by OEDD1 Director Graham Donaldson, including reports, correspondence, intra-OED memoranda, agendas and minutes of meetings, and seminar outlines. The records generally relate to the extension of OED's influence and the dissemination and use of OED products throughout the Bank, in international organizations, and in other countries. Among the topics covered are: a review of OED reports and advice regarding their publication; policy regarding the dissemination of OED findings and information; the establishment of operations evaluation programs in other countries and in the Inter-American Development Bank; the Bank's coordination with countries and other organizations doing post evaluations of projects fully or partially funded by the Bank; Bank-provided training on program monitoring and evaluation; and the Evaluation Capability Development Program. Included are a number of requests from Bank staff and from non-Bank organizations for specific OED reports. Two large folders on the Joint Audit Committee (JAC) include: intra-OED memoranda regarding the Department's responses to specific JAC inquiries and requests; agenda and minutes of JAC meetings; the OED Director's requests for Donaldson's comments on JAC reports; OED's proposals for the JAC work program; and OED comments on the Report of the Working Group on Project Completion Reports and on the Report of the Task Force on Dissemination and Utilization of OED Findings.