Data Sources and Collection Methods

Speeches Shim

FEATURED

Conducting Mini Surveys in Developing Countries

Mini surveys focus on a narrowly defined issue, question, or problem. The kinds of questions they address include the following: "What proportion of targeted farmers are using the recommended technical package?"

DOWNLOAD NOW

ALSO SEE

USAID TIPS: Using Rapid Appraisal Methods


USAID TIPS: Using Direct Observation Techniques


USAID TIPS: Conducting Customer Service Assessments


USAID TIPS: Measuring Institutional Capacity


USAID TIPS: Measuring Institutional Capacity: Annexes


Conflict Sensitive Monitoring and Evaluation


Monitoring and Evaluation in Post Conflict Settings


Analyzing Quantitative Data


Analyzing Qualitative Data

The description of data sources in a PMP should be operationally specific enough to convey where data for each indicator will come from.

Data sources tell us where to look for performance data. The range of sources used in a PMP is often broad and may range from records kept by other organizations to data obtained directly from firms and farms, or from individuals. Knowing the source of a particular type of performance data will help a Mission to identify an appropriate data collection method when preparing a Project MEL planActivity MEL Plan, or evaluation Statement of Work (SOW).

When considering sources of information for indicator both the accessibility of a source and the cost of obtaining data from that source are important. In some cases, partner governments may collect data that would be useful for monitoring progress in an area served by a USAID activity, but regulations governing administrative data may preclude its being shared with USAID. In other cases, the cost of obtaining data multiple times over the life of an activity, using the same tools and procedures, may be prohibitive. For this reason, it may be prudent to identify more than one possible source of data for indicators included in a PMP where there is uncertainty about the feasibility of using specific data sources.

  • Instrument of various sorts — scales, thermometers, distance measurement devices and other existing continuum, including “number of” and similar quantitative descriptors
  • Survey instruments that may include either closed or open-ended questions, but can be forced to remain unchanged over repeat uses, including customer satisfaction surveys and citizen score-cards. USAID's Conducting Mini Surveys in Developing Countries volume is useful aid for this method as is the Epi Info software, which is well suited for surveys of a moderate size.
  • Structured observations forms that have the same characteristics as survey instruments, on which USAID's TIPS on Direct Observation provides guidance
  • Scales and rating systems in which every position on a continuum is described in a precise manner on which two observers would be highly likely to agree.

An example of this type of scale, which is included in the USAID TIPS on Measuring Institutional Capacity illustrates the general approach USAID uses to monitor changes over time in its annual CSO Sustainability Index report.

  Nascent Organizations Emerging Organizations Expanding Organizations Mature Organizations
MANAGEMENT PRACTICES
Administrative Procedures

No administrative procedures or manuals exist.

Administrative procedures are informal and NGO Staff lack a common understanding of them.

Administrative procedures are increasingly formalized.

Administrative procedures are not being fully utilized.

No administrative procedures or manuals exist.

Administrative systems are formalized and functioning.

Administrative procedures and manuals exist but are not referred to regularly.

Administrative procedures and manuals exist.

Administrative procedures are adhered to.

Procedures and operating manuals are updated regularly.

For the very reasons that some data collection methods are highly appropriate for gathering performance information in a consistent way over several years, other data collection methods are less suitable, including:

  • Unstructured or semi-structured survey and direct observation instruments
  • Key informant interviews that rely on rough interview guides and are rarely replicable
  • Group interviews where the process or dynamics could affect the data

In addition, while some existing data series remain constant over time, any secondary sources from which USAID is considering collecting performance data, including other international development organizations, warrants a review from this perspective. Record keeping systems used by firms and government agencies, over which USAID lacks control, sometimes change and could result in a broken performance data series.

 << Data Disaggregation Up Data Analysis >>

ProjectStarter

BETTER PROJECTS THROUGH IMPROVED
MONITORING, EVALUATION AND LEARNING

A toolkit developed and implemented by:
Office of Trade and Regulatory Reform
Bureau of Economic Growth, Education, and Environment
US Agency for International Development (USAID)

For more information, please contact Paul Fekete.