Precise indicator definitions and clear explanations of the unit of analysis for performance indicators help to guard against variation in the data collection from site to site and over time.

Indicator definitions are important. When indicators are not precisely defined, those who collect data supply their own definitions – and their definitions may vary widely. Lack of specificity in an indicator definition can make it impossible to trust the data that are collected. The following indicator is problematic in this sense:

• Number of government agencies strengthened as a result of the project.

While this indictor has a quantitative dimension (number) it also has a qualitative aspect (strengthened). Absent a precise definition of what strengthened means in this context, data collected on this indicator would be both subjective and unreliable. It is very unlikely that the same unwritten standard for strengthened would be applied by the same person at two widely separated points in time, or by two different data collectors.

For trade projects, some of which focus on specific commodities, USAID's evaluation, From Aid to Trade, noted the absence of Harmonized System product codes as an indicator definition-precision weakness. Were such codes used consistently in USAID funded export programs, the analysis of USAID performance data could conceivably include an examination of the share of exports of a particular commodity generated by USAID-assisted producers. Thus for example, rather than recording exports of tomatoes, USAID implementing partners could be asked to report on tomato exports at the 0702 HS code level (tomatoes, fresh or chilled) or at the 070210 HS code level (cherry tomato) which might be the specialty crop of interest to USAID not only in terms of exports from a target country but also in terms of market share in an importing country.

Unit of Analysis refers to the choice we make about the level at which to collect data on a particular indicator. In some cases the unit of analysis is chosen as an aggregate of something that is actually collected on a more disaggregated basis. An export expansion project, might elect to monitor the exports of vegetables from target districts served by a project. In this instance, it might also be possible to unpack that number and find out whether tomato exports had increased. In other cases, the choice of a unit from which to collect and analyze data may preclude access to finer grain data. The choice between individuals and households as the unit of analysis for analyzing income changes is a choice of this sort. Once you have collected income at the household level, you cannot normally back up and determine whether the individual income of female household members rose.

In practice, indicator definitions and unit of analysis choices often become intertwined, which is why they are usually addressed at the same time.

In USAID's evaluation From Aid to Trade, for example, an examination of projects in which jobs were an intended result illustrated just how varied the definitions of what sounds like a simple measure could be:

• A total of 762 new jobs were created, 46 percent of which were held by rural women.
• Jobs created, expanded, or maintained. Target: 1,000 Actual: 2,477 Percent of Target: 248%.
• A total of 1,502 new jobs (full time job equivalents) have been created by companies compared to their level of operation at the time assistance was initiated.
• Jobs created: estimation of over 600 jobs (new and substitute ones).
• Over the life of the program, the jobs component provided a total of 1,010,226 person-days of work, or the equivalent of nearly 92,000 person-months of work.