| How should NPOs measure their work and its results? This article reports on one organization’s experience in developing better performance measures. The Christian Children’s Fund (CCF) (see sidebar below) put together what it calls its annual impact monitoring and evaluation system (AIMES). For organizations of this type, accountability is extremely important. CPAs can use outcome measures to help similar organizations achieve their goal-driven strategic plans. | The Christian Children’s Fund Founded in 1938, CCF ( www.christianchildrensfund.org ) is an international child care agency serving the health and educational needs of impoverished children worldwide. It provides services to approximately 2.5 million children in 31 countries, including the United States. CCF programs promote long-term sustainable development by providing children with safer lives due to improved access to health care and safe water, immunizations, better nutrition, educational assistance, literacy courses, skills training and other services specific to their individual needs.
Approximately 80% of CCF’s revenues come from a program of “child sponsorship,” in which individual donors are linked with a specific child. The program sponsors over 400,000 children worldwide. About 75% of the sponsors are from the United States. Revenues for the last fiscal year were approximately $127 million. | HOW DO YOU SPELL SUCCESS? One approach CPAs might use to judge an NPO’s performance is to measure the amount of resources the organization spends on providing program services (to carry out its purpose) vs. what it spends on management and general expenses and fundraising. For most organizations, a higher percentage of resources spent on program services than on management and fundraising is considered a positive performance indicator. Tax-exempt organizations are required to file IRS Form 990, Return of Organization Exempt From Income Tax, annually. It includes the amount of resources the organization received and how it spent them. Several watchdog organizations use these data to track and rate NPO performance. Although this type of measure is useful, it doesn’t indicate how effectively an organization uses its resources to meet its objectives. For example, an NPO may spend 80% of its resources providing a particular program service, but may be ineffective in reaching the program’s goals. Just spending money on a service is not an effective performance measure. FASB recognizes the limitations of traditional financial statements in assessing NPO performance. In Concepts Statement no. 4, Objectives of Financial Reporting by Nonbusiness Organizations, FASB saw the need for a different type of information to measure their performance. The statement calls for reporting information about service efforts (how an NPO uses resources to provide different programs or services) in the financial statements. Concepts Statement no. 4 also says that, ideally, NPOs should provide information about service accomplishments as part of financial reporting. It recognizes the difficulties organizations face in measuring and reporting program accomplishments. and acknowledges the need for more research to determine whether service effort and accomplishment measurements can be developed that meet the characteristics necessary to be included in NPO financial statements. OUTPUT AND OUTCOME MEASURES Service efforts and accomplishment measures fall into four categories: input measures, output measures, outcome measures and efficiency measures. They quantify the effort expended on a program (inputs), the level of services provided (outputs), the effect a service has on the program’s stated objectives (outcomes) and a comparison of the level of inputs with outputs or outcomes (efficiency).
NPOs are very accustomed to reporting input measures. For example, they report the financial resources dedicated to specific programs in their financial statements. Many also report nonfinancial information about the effort they expend, such as the hours spent meeting a program goal. Output measures are often stated in nonfinancial terms. For example, a university may report the number of students that graduated or a homeless shelter may report the number of people housed. Outcome measures gauge how well a program accomplished its goal. For example, a program designed to teach reading to adults may use the literacy rate for the area served as an outcome measure. One limitation of outcome measures is many factors other than a specific program can effect them. However, when they are used over a period of time, CPAs will find they can be a key way to measure a program’s effectiveness. Often, the final step an NPO can take in using service efforts and accomplishments techniques is to employ them to measure efficiency. These measures compute either inputs/outputs or inputs/outcomes indicators and provide information on how efficient an organization is at achieving its program goals. For example, a program that teaches reading to adults could compute a cost (input) for each adult who reaches a certain reading level (output). Most NPOs are still in the early stages of developing output and outcome measures. However, the CCF focused its AIMES on measurable, standardized outputs and outcomes that would allow it to make comparisons among communities as well as among the different countries it serves. THE CCF MODEL In 1995, the CCF began working on an evaluation system to assess whether its programs were making a positive, measurable difference in the lives of children and their communities around the world. It established a working group that included national directors and program managers as well as representatives from sponsor services, finance and internal audit. The group also had the support of an outside consultant. CCF thus developed its evaluation system through intense dialogue and considerable input from the field. The system was designed both to allow CCF to be more accountable to its sponsors and to give communities a tool to continually assess the impact CCF projects were having on children and families.
CCF wanted a tool that would measure the effectiveness of its health and education programs. The organization operates many different programs in a variety of locations and individual content may vary based on local needs. For example, in 1997, CCF’s evaluation system captured the health status and educational progress of 1,865,194 children in approximately 850 projects in 18 countries. AIMES uses a set of standardized health and education indicators (see exhibit 1 ) to measure impact (outcome) and process (output). The three impact indicators measure progress toward children’s well-being—CCF’s basic goal. The seven process indicators measure how outputs effect a positive change in three core impact areas. While local projects may add other performance measures, AIMES measures are standard for all projects. | Exhibit 1: Core Standardized Outcome/Output Measures for AIMES | AIMES
Impact
Indicators | Under-5-year-old mortality. Under-5-year-old moderate and severe malnutrition. Adult literacy. | AIMES
Process
Indicators | One- to two-year-old immunized. Tetanus Toxoid vaccine (TT2) protected live births. Families that correctly know how to manage a case of diarrhea. Families that correctly know how to manage acute respiratory infection.
Families that have access to safe water. Families that practice safe sanitation. Boys/girls who are enrolled in a formal or nonformal educational program. | | In selecting what to measure, CCF identified core indicators that it deemed critical success factors for all projects. For a measure to be included as a core indicator, CCF developed four qualitative requirements. The measures should 
Focus on program impact, not activities.
Gauge the program’s impact on children, except where there was a proven association between a child outcome and the ability of the person caring for that child.
Determine impact in terms of broadly defined health and educational outcomes.
Assess only what CCF was able and prepared to track.
Once CCF developed the performance measures, it field-tested them in 10 program countries on several key projects. The next step was training staff in all national offices on how to use the system. This two-year process proved essential in producing reliable and consistent performance measures as well as ensuring the measures were understood at the local level. Although AIMES captures the impact and process measurements in communities, it does not prescribe any strategy for a community program, nor does it place any limits on the kinds of programs a community can use to promote the well-being of children. However, communities can use the information to make decisions about how best to set priorities for initiatives, allocate resources and take follow-up actions. In this way, AIMES becomes a tool for monitoring, planning and managing programs. For example, if the malnutrition rate for children under age five in a community is found to be very high, addressing this problem becomes a priority. The local CCF unit can decide how to best reduce child malnutrition by educating caregivers on improved feeding practices, providing supplementary diet material or improving access to clean water. CCF collected the data for AIMES in a systematic and consistent manner. The “family card” is a key source document for the fund because it captures information on each performance indicator. There is one card for each family in a CCF program in a given community. Project staff and community volunteers maintain contact with each family through periodic visits to update information on the card, provide health messages and offer on-the-spot guidance on other areas of concern. CCF collects additional information through regular home visits and records it on service rosters and vital event registers. Once a year, all the data are combined into a one-page report for each community then further aggregated by each national office and then in total. Thus, the system provides information at the community, national and organizational level. Having information aggregated at different levels enables managers to compare performance indicators among communities and countries. Managers can also track improvements over time. Exhibit 2 , below, shows the AIMES indicators for 1997 to 1999, aggregated for the entire organization. Exhibit 3 , at the end of this article, presents the same information in a bar graph. | Exhibit 2: Progress Made on Core Indicators | | Indicators | 1997
Data | 1998
Data | 1999
Data | | *Under-5-year-old mortality per 1,000 births | 76 | 76 | 72 | | *Under-5-year-old moderate to severely malnourished | 13% | 13% | 11% | | Literate men | 77% | 79% | 80% | | Literate women | 68% | 69% | 71% | | 1 to 2-year-old immunized | 69% | 76% | 76% | | Tetanus Toxoid vaccine (TT2) protected live births | N/A | 78% | 78% | | Manage diarrhea | 81% | 84% | 88% | | Manage respiratory infections | 69% | 75% | 80% | | Access to safe water | 74% | 78% | 80% | | Safe sanitation practice | 63% | 67% | 70% | | 0 to 5 education enrollment | 44% | 51% | 55% | | 5 to 15 education enrollment | 89% | 91% | 92% | | *Expected decreases | | | | | The initial response within CCF to the AIMES approach has been very positive. The system is now in place in more than 1,000 projects in 20 countries. Tracking a set of core health and education performance indicators has made project managers more focused and better able to concentrate resource allocations on areas that can make a measurable difference in outcomes affecting the lives of the parents and children in their local communities. CCF can use performance indicators to make meaningful comparisons among communities within a country as well as over time. It then can use this information to make program and resource decisions at the community level. Collecting and reporting data for each community has had other benefits for CCF. The family card, an important tool in the system, has proved useful in monitoring vital events and progress in children’s health and educational development. In addition, increased participation by parents and community volunteers in these visits has led to more meaningful contacts between projects and communities and improved understanding of the caregivers. For example, growth monitoring to track the nutrition status of young children has helped promote appropriate feeding and child care practices. MAKING IT WORK FOR YOU Can such a performance measurement system work for a museum, school or community welfare organization? CPAs will find that developing a system similar to the one CCF used can be a fairly straightforward process for most organizations. There are four basic steps to follow in creating a performance measurement system that focuses on outputs and outcomes.
Clearly identify the organization’s mission.
Developed qualitative requirements for indicators and measurements.
Develop primary indicators and measurements.
Implement the new performance measurement system. An organization’s first step is to clearly identify its mission or purpose. What role does the organization play in society? What benefits should it provide to its community? What impact should it have on the community? An NPO can use its understanding of its clearly defined purpose to help drive the creation of meaningful output/outcome performance measurements. The second step is to identify basic qualitative requirements to guide the development of the output/outcome measures. These requirements represent overarching guidelines under which the NPO develops each measurable output/outcome. They help ensure the organization’s performance indicators are appropriately linked with the content of its mission statement. For example, CCF found it was important for performance indicators to focus on the impact (outcome) its programs were having on the children it was serving. One important consideration in developing performance measures is to make sure the organization has the time, resources and availability to track these output/outcomes measures. Some small NPOs may not. The actual development of the performance measures is the third step. It’s critical to involve managers and employees from throughout the organization in identifying a workable number of measurements that will serve as primary indicators of how well the NPO is meeting its mission. Employee input will provide valuable insight as well as enhance support for using the performance indicators when they are finalized. The performance indicators must also be field-tested before implementation. Past experience suggests that between 5 and 10 broadly defined indicators are often sufficient to measure how well an organization is meeting its goals and objectives. The fourth step an NPO should take is to implement the performance measures. The goal is to provide information that will help managers better focus on resource allocations—to assist them in making decisions that will help the organization fulfill its mission. Once the system has collected baseline data, the organization will have a benchmark to track progress over time. As with the CCF, such a system should result in a better linkage between resource decisions and the outcomes the organization wishes to achieve. AN IMPORTANT TOOL The CCF’s development of AIMES has improved the organization’s accountability and become a tool for monitoring, planning and managing programs. CCF provides measures that reflect the impact of programs on the lives of young children and their communities. The system has enhanced the organization’s ability to meet its goals by monitoring and reporting project performance aggregated at the community, national and organizational levels. One of the system’s biggest contributions is a clearer sense of purpose, which allows CCF managers to better focus resources on the organization’s goals. For example, CCF has shifted resources from programs concentrating on school-age children to early childhood interventions, maternal health care and support for caregivers. In sum, AIMES provides CCF with a viable instrument for program management by allowing communities to make decisions about program priorities and resource allocations needed to reach the organization’s goals. CPAs who work for NPOs can use the principles behind AIMES to help their own organizations develop meaningful performance measures that will enable them to meet their own goals and objectives.
| 
SINCE MOST KEY BUSINESS PERFORMANCE MEASURES don’t work for not-for-profit organizations, these groups need to develop specialized measures to help them achieve their goal-driven strategic plans. One approach NPOs use is to measure the amount of resources they spend on providing program services vs. management and general expenses. 
any key business performance measures don’t work for most not-for-profit organizations. For example, the “bottom line” measurement of profit or loss indicates how effective a business is at achieving its goal of generating profits for the owners. However, generating profits is not a goal for NPOs. These organizations have no owners, often provide goods and services to constituents free of charge and typically seek resources from people and organizations that do not expect economic benefits in return. Thus, the bottom line doesn’t work for NPOs. 
