This sub-series is a collection of memoranda and related records compiled by Christopher R. Willoughby who was among the first employees of the Operations Evaluation Unit and was appointed the first director of the Operations Evaluation Department in July 1973.

• The first set of files consists of memoranda (dated November 1970 - November 1972; February - April 1973; and February - August 1975) sent to President Robert McNamara from Christopher R. Willoughby and also John Adler, who was director when the Operation Evaluation Unit was established in 1970. The memoranda concern the early workings of Operations Evaluation including: the decision to begin with an in-depth look at the Bank's work in one country, Colombia, and in one sector, electrical power; the development of the format for these first two reports; progress reports on the work of the unit and on the development of future work programs; and the relationship between evaluation and supervision. Some of the original memoranda to McNamara were returned to Operations Evaluation marked with a red stamp indicating President has seen. One long memorandum for the record summarizes the January 1971 visit with the Colombia Operations Evaluation team of Dr. Lauchlin Currie who headed the Bank's General Survey Mission to Colombia in 1949. Included in this first set of files are handwritten notes and memoranda for the record summarizing meetings Willoughby and others had with McNamara.

The second set of files consists of memoranda (dated April 1970 - February 1976) sent and received by Willoughby and his copies of key memoranda documenting the development of the operations evaluation function and the changing position of the Operations Evaluation staff within the Bank. Included are Willoughby's handwritten drafts and final copies of the key issuances affecting Operations Evaluation and his notes from key meetings. One folder contains Willoughby's drafts of a September 29, 1975 memoranda from Mohamed Shoaib, Vice President, Operations Evaluation Division, outlining the history of the evaluation function from 1971 to 1975.

This sub-series contains reports, correspondence, and studies compiled by the Directors of OED (OEDDR). Most of the files contain comments from OED staff members and/or the director on draft documents from other Bank employees. Filed under General Comments on Review of Field Staff Benefits are OED comments on various Bank documents, 1995-1998. Key topics covered by other files include: OED staff views on the dissemination and use of OED findings (filed under Dissemination); ratings of the Narmada and Kedung Ombo Completion Reports (filed under Narmada); reports of the Joint Audit Committee Informal Subcommittee on OED Reports; a select review of OED by the Internal Auditing Department planned for 1996; plans for an OED study of NGOs; OED collaboration with the Economic Development Institute (now WBI); joint OED/Operations Policy Department reports on the quality of economic analysis in staff appraisal reports for projects approved 1993 - 1997 (filed under Economic Analysis, Econ III and Econ IV and Economic Analysis: Correspondence); a 1996 proposal for simplified project documentation for investment operations (filed under Simplified Documentation); and OED staff retreats for December 15-17, 1987, November 29-December 1, 1989, June 26-28, 1991, and September 8-10, 1993. These retreat files contain agenda and schedules, copies of documents discussed at the retreats, copies of presentations, and post-retreat evaluation forms.

Filed under Portfolio Management are: memoranda regarding funding arrangements for OED studies and on borrowers' interest in ex-post evaluations; OED staff comments on the draft Wapenhans Report and on the draft chapter concerning OED; and copies of minutes of the Task Force Steering Committee on which Eberhard Kopp served. Filed under Streamlining Bank Procedures are memoranda and correspondence Kopp compiled in 1985 and 1986 before he became OEDDR Director and while he was a member of Working Group #3 of the Task Force on Streamlining Bank Procedures.

The files in this sub-series, compiled by the Directors of OED (OEDDR), outline the policies and procedures affecting OED programs and work products. Included in the first segment (1970 - 1994) are files concerning: the Project Completion Report / Project Performance Audit Report (PCR/PPAR) evaluation system and the pass-through procedures for PCRs that began in 1982; the dissemination and use of OED findings; the various revisions of the Procedural Guidelines for Operations Evaluation Staff and of the chapters in the Operational Manual regarding Project Completion Report / Implementation Completion Report (PCR/ICR) guidelines and project supervision in the Bank; the development and use of the Project Information Form (PIF); the development of OED's evaluation seminars; and the role of environmental issues in Bank evaluations. One file (Lessons Learnt from Project Performance Audit Reports - Agriculture) contains the OEDDR's copies of memoranda dated December 1980 to July 1986 from the Assistant Director ofthe Agriculture and Rural Development Department (AGR) to AGR division chiefs outlining the lessons learned from reviews of Agricultural PCRs and PPARs. Another file (Operations Evaluation Department - General) contains copies of key documents in the history of OED from 1970 to 1988.

Much of the second segment (1985 - 2001) contains documentation regarding the Bankwide ICR Process Review Working Group which was co-chaired by OED Director Elizabeth McCallister and was established in December 1997 to make the ICR process more efficient, effective, and timely. The records for the working group include agenda and minutes of meetings, various drafts of the group's final report, comments on the drafts, and intra-OED correspondence on the workings of the group. Other files contain: correspondence concerning changes in the PIF and instructions for completing the form; intra-OED correspondence regarding the discrepancy between OED and Regional project ratings (PPAR-Project Ratings); internal OED e-mail discussions of the ICR review process and the development of the ICR Evaluation Summary form; intra-OED correspondence regarding specific ICR reviews; draft ICR guidelines and comments on them; revisions to the OED handbook, Procedural Guidelines for Operations Evaluation Staff (Guidelines - Amendments); the drafts of a style manual standardizing the format for OED reports; and a proposal for dealing with the workflow problems associated with ICR bunching, i.e. the receipt of the majority of ICRs during the months of May, June, and July. Filed under PPAR - General is a copy of July 1995 Performance Audit Guidance, which was prepared by the OED Audit Guidance Working Group in 1995.

This sub-series contains correspondence and other records exchanged between OED staff and officials located in industrialized countries who were responsible for, or interested in, monitoring and evaluation processes. Included is correspondence with officials in the Netherlands, Germany, Canada, Australia, Sweden, Denmark, France, Japan, Saudi Arabia, Ireland, Greece, Austria, the United Kingdom, and the United States. Most of the correspondence is with the Director of OED, the Director-General, Operations Evaluation (DGO), and the DGO's advisors, Robert van der Lugt and Pablo Guerrero, who were responsible for developing evaluation capacity in member countries. The correspondence concerns: requests for information regarding OED's evaluation policies and procedures or assistance in conducting an evaluation; exchanges of evaluation methodologies; international seminars on evaluation; visits of OED staff to other countries to conduct an evaluation or to attend an evaluation conference or seminar; and visits toOED of evaluation officials from abroad. Included are copies of other countries' studies, reports, and manuals regarding evaluations and evaluation methods.

The U.S. correspondence is with officials in the U. S. Agency for International Development (AID), the General Accounting Office (GAO), and the Department of Treasury; most are copies of records that are presumably located elsewhere in OED and Bank records. The US AID correspondence primarily concerns exchanges of policy papers, studies and reports onevaluation guidelines. The GAO correspondence relates to 1977, 1981, and 1991 congressionally required GAO reviews of the independent evaluation system of the Bank and a 1996 GAO review of the overall role of the World Bank in a post-Cold War environment.

The earliest GAO file contains documents assembled by OED Director Christopher R. Willoughby in 1973 and 1974 following release of the 1973 GAO report on U.S. participation in World Bank. Included in the file are a number of documents from Bank PresidentRobert McNamara, Vice President Mohamed Shoaib, the U.S. Executive Director Charles O. Sethness, and Willoughby, discussing the audit and evaluation functions in the Bank and proposed changes in the structure and function of Operations Evaluation.

The Department of Treasury file concerns the department's 1971-1973 interest in the evaluation function of the Bank and Treasury's support of the GAO recommendation for the establishment of an independent review body to evaluate the Bank's activities.

The sub-series contains the working files for special studies generated by the Country Policy, Industry and Finance Division (OEDD2) and its predecessor, the Policy-Based Lending, Industry, Public Utilities and Urban Sectors Division. The studies cover a range of policies, themes, sectors, and countries, including: conditionality in World Bank financed loans; privatization; women in development; free-standing technical assistance for institutional development in Sub-Saharan Africa; trade policy reforms under adjustment programs; a study of Bank/country relations covering Mexico's agriculture, industry, infrastructure, urban, and electrical and power sectors; and the 1992 study The World Bank and Pacific Island Countries: An OED Review which covered the Bank's operations in six Pacific Island member countries: Fiji, Kiribati, Solomon Islands, Tonga, Vanuatu, and Western Samoa. There is also a draft of the design paper for the FY95 country assistance review for Nigeria.

Also included are files documenting OEDD2's participation in a Bank-wide steering group on Development Aid Effectiveness and a file for the development of a policy ledger of OED recommendations, management responses, and actions for all studies discussed by the Joint Audit Committee (JAC) after July 1, 1992.

The sub-series contains the working files for special studies generated by the Agriculture and Human Development Division (OEDD1) and its predecessors: the Agriculture and Family Planning Division; the Agriculture and Human Resources Division; and the Agriculture, Infrastructure and Human Resources Division. Included are files for process, policy, and thematic studies as well as sector studies concerning agriculture, education, population, and human resources projects and issues. Specifically, reports focus on: agricultural research and education; Bank project supervision; training included in Bank financed projects; population programs; Bank lending policy for plantation crops; agriculture projects in Sub-Saharan Africa; renewable resource management issues; gender issues in Bank lending; human resource framework for Bank projects; fertility decline; Bank lending for agricultural credit and rural finance; poverty alleviation; and the Bank's experience with government sponsored land settlement with irrigation development, agricultural marketing, and rural development.

Of special note are the files regarding the Narmada Project. It appears that OEDD1 began assembling many of the Narmada materials in this sub-series in 1991 when Bank President Conable commissioned in September of that year an independent review of two Bank-funded projects, Sardar Sarovar Dam and Power Project (Credit 1552; Loan 2497) and Sardar Sarovar Water Delivery and Drainage Project (Credit 1553), referred to collectively as the Narmada Project because of the relationship of the projects to the Narmada River in India.

Included in this sub-series are working files for a small number of OEDD1 Impact Evaluation Reports (IERs) covering: an irrigation project in Sri Lanka; a settlement project in the Terres Nueves region of Senegal; a Benin agricultural project; a cotton development project in Burkina Faso, Cote d'Ivoire, and Togo; education projects in Colombia, Indonesia, and Malawi; and others. There are also files for the 1990 and 1995 Interim Evaluations of the Aga Khan Rural Support Program (AKRSP) including the second and third of a series of evaluations of AKRSP conducted by OED at the request of, and with funding from, the Aga Khan Foundation. A few files in this sub-series contain comments that OEDD1 staff provided on studies and reports generated by other Bank units or by other organizations and institutions.

Some of the files in this sub-series contain background reports, studies, and publications dated as early as 1970.

The sub-series consists of the working files for special studies and evaluation reviews conducted by OED and its predecessor organization, the Operations Evaluation Unit (later Division) of the Programming and Budgeting Department, prior to the formation of sector-specific divisions in 1983. Reports include sector, country, management, and policy studies.

The series includes the first two evaluation studies conducted by the Operations Evaluation Unit. These studies included a deep study of everything the Bank had done in one country, Colombia, and a wide study evaluating loans to a number of countries in one sector, the electric power sector. The files for the first country study, Bank Operations in Colombia: An Evaluation, Report No. Z-18, 1972, also include a draft and final version of the 1975 Closing Report to Colombia Evaluation which was a follow-up review of the 1972 study to assess what progress had been made on the recommendations contained in the 1972 study. The files for the first sector study, Operations Evaluation Report: Electric Power, Report No. Z-17, 1972, include the 1975 follow-up report, Closing Report on Actions Relating to Recommendations of the Electric Power Evaluation Report of March 1972.

Other sector studies included in this series are: a comparative highway evaluation study; a Development Finance Companies study; a Bank lending for agriculture credit study; reviews of agricultural and rural development programs in Indonesia and the Philippines; and a review of rural development projects in Sub-Saharan Africa. The Department's first three management policy reviews are included and address delays in loan and credit effectiveness, delays in project preparation, and the role and use of consultants. Early operational policy reviews cover project implementation, supervision of Bank projects, and compliance with loan covenants. This series also includes files for OED studies of project monitoring and evaluation and technical assistance pilot evaluation studies conducted in Kenya and Afghanistan.

The study files in this series often contain earlier Bank documents, some dated as early as 1957, relating to the subject of the study. The compilers of these early studies established the practice of soliciting comments on draft studies and reports from all interested parties in the Bank and in the client countries, and most of these study files contain both a series of drafts and the related comments.

This sub-series contains records documenting the weekly meetings of the Departmental Management Team (DMT) which consisted of the Director of OED, the directors of the OED Departments, and some support staff. The files generally contain handwritten notes of the meeting's proceedings, meeting agenda, copies of documents discussed at the meetings, and typed minutes of the preceding meeting. The typed minutes are columnar in format and for each topic covered show a summary of the discussion, the follow-up action required, the staff member responsible for the action, and the expected date of completion for the action. The f