Accountability by Numbers

EXECUTIVE SUMMARY

WHEN THE TEXAS STATE AUDITOR'S OFFICE (SAO) decided its management system could better monitor how taxpayer dollars were spent, it adopted a balanced scorecard approach to track its performance. As a public-sector entity, the SAO sought a “profitability” strategy of offering more services to citizens without increasing program costs (it aimed to stretch finite resources further). BASED ON THE IDEA THAT ONE BENCHMARK can’t show true performance, a balanced scorecard uses an array of indicators related to an organization’s strategic goals and to progress toward them. For example, it will link measures for process (on-time delivery) and finance (revenue growth) with others such as customer feedback and employee knowledge. THE PRIMARY GOALS for instituting a performance measurement system were to ensure accurate and timely reporting of SAO financial data, establish individual manager accountability for clearly defined results and use the scorecard reporting system to make informed decisions about how to efficiently use its budget. A CROSS-SECTION OF ABOUT 20% of its workforce collaborated on identifying the four key strategies: to provide independent audit services, to teach managers to avoid problems, to help agencies fix problems and improve operations and to make sure the state’s compensation structure offered appropriate salaries. THE PARTICIPANTS IDENTIFIED the most important data to track and developed focused measures—cycle time, quality or volume, for example—to evaluate progress toward objectives. Too many measures can cause “data fatigue.” THE RESULTING OFFICEWIDE, MULTILEVEL information system reports data in a visual, easy-to-understand format that gives all managers single-click access to detailed analysis and information. Meetings now are about project results, solving problems and operational planning.

DEBORAH L. KERR, PhD, is chief strategy officer of the Texas State Auditor’s Office. She writes and lectures on performance management topics. Her e-mail address is dkerr@sao.state.tx.us .

verywhere you look, good business management seems to be in short supply. Recent examples abound of poorly run companies such as Qwest, Sunbeam, Andersen, Tyco, WorldCom, Enron and most of the late dot-coms. So what causes such conspicuous failures? Not lack of ideas (there are plenty of those). Most experts agree the major cause of business meltdown is management error, which occurs across the enterprise spectrum, not just at high-profile companies.

How is it at your business? Has a project you thought was on time and on budget recently blown both measures and caught you off guard? Can you identify trouble spots right now? How would you know about them, and what changes could you make if you did? CPAs who consult to companies, operate in industry or are managing partners can follow this case study to learn how to implement a balanced scorecard oversight system, which organizes performance data clearly while showing how operational areas are linked.

A MEASURED RESPONSE
About seven years ago, Lawrence F. Alwin, CPA and Texas state auditor, decided the State Auditor’s Office (SAO) needed a better way to track how it was meeting its mission to provide lawmakers and agencies with the most useful possible financial operating data for taxpayer-funded projects. The SAO’s monthly status reports were thorough: They analyzed staffing, summarized time use for all employees and evaluated budget and progress performance for about 150 projects. In fact it took four people three days to compile the data into four-inch-thick binders and distribute them. The problem: Each division handled its performance data a little differently, so deciphering the information stalled meetings and knocked projects off schedule.

“We weren’t getting the right data at the right time for good decision making,” Alwin says.

Alwin agreed the approach could be useful. His first step toward instituting it was to choose which information categories SAO monthly status reports would systematically address: financial performance, audit performance, customer reaction and overall results. Audit performance would track three areas: the audit projects completed at the end of each month, the number likely to meet the next month’s deadlines and the reasons some audit reports were late (see “ Performance Measurement Ground Rules ”). It would give managers more time to concentrate on solving operational problems.

SETTING GOALS
As Alwin saw it, the SAO’s new measurement system’s essential goals were to

Generate accurate reports. SAO measurements had to yield clear, accurate, timely data that were parallel from agency to agency.

Establish visible accountability. Alwin assigned responsibility for producing results to specific managers, and he sharpened the benchmarks.

Motivate informed corrective action. Managers would use monthly executive meetings to share ideas to correct problems based on what they had learned.

To meet those goals, Alwin had to

Designate staff to accomplish objectives.

Improve employee efficiency (minimize the gaps, overlap, friction and errors that occur when people aren’t clear about their job duties).

Make knowledge more transparent (give people fast feedback to help them find ways to work better).

Eliminate process redundancies to reduce costs (see “Before and After”).

He then focused on five steps for implementing the goals. They were

Prepare a strategic plan (action outline). The strategic plan’s implementation actions would be tracked by performance measures that fit the SAO’s goals. (For more information on making or updating such a plan, see “ Strategic Planners Lead the Pack, ” JofA , Dec.01, page 26.)

Identify critical business objectives for each balanced scorecard “perspective” or area view (in effect, windows into different aspects of an organization’s performance) and link each one to the goals of the strategic plan. Name an individual “owner”—typically a member of the management team—for each objective.

Develop a limited number of measures—cycle time, quality or volume, for example—to evaluate progress toward objectives. It’s important to not have too many measures, which can cause “data fatigue.”

Set measurable goals (targets or budgets). Measures such as “reduce employee turnover by 5% in 12 months” or “cut administration expenses by 3% in nine months” clarify priorities. Targets have to be realistic as well as challenging. Setting the bar too high would demoralize workers and cause the system to fail.

Collect and discuss data at least monthly. This is where measurement turns into management: At regular meetings, owners explain why they got the results they did; if targets have been missed, they say what steps will get them back on track.

PEOPLE, TIME, MONEY AND ACCOMPLISHMENTS
To get the balanced scorecard system under way, the SAO had to delineate its goal and the actions to reach it. A private company might pick maximizing revenue growth or capitalizing on a market niche as a strategy to fulfill a goal to “become the most profitable” in its sector, for example. A public-sector profitability strategy could be to offer more services without increasing program costs—that is, stretch finite resources further (see “What to Track”).

Alwin convened a cross-section of about 20% of the SAO staff (from receptionists to auditors, twenty-somethings to near retirees) to state their goal, to update the agency’s strategic plan and to identify the data most relevant to its mission. Their goal: Provide the most accurate, timely financial information to Texas legislators to effect better public-policy outcomes. He asked participants: “What should we do to fundamentally improve that service?”

In a series of one-day meetings over a six-week period, staff members offered more than 400 answers. Alwin took a month to review them, noting which themes participants brought up again and again. The process homed in on four strategies (see exhibit 1 ): The SAO would provide independent audit services (AS) as chartered, provide educational services (ES) to teach agency staff how to avoid problems and improve accountability, offer management advisory services (MAS) to help agencies improve operations and manage the state’s compensation structure and address other human resources issues (SCO).

After the first group’s strategic planning input was complete, Alwin next worked with an implementation team of SAO managers to adapt its mission’s four supporting scorecard areas: financial, customer, process and knowledge (see exhibit 2 ). At the top of the scorecard, the implementation team added a fifth category—mission—to represent the public-sector bottom line: public-policy outcomes related to SAO data.

To measure its information’s utility, the SAO collects data on how legislators use its audit reports in actual hearings and debates. A related measure tracks the dollar value created through audits, including cost savings, cost avoidance and identified efficiency gains (the public-policy outcomes). For example, a sportsman or woman who wants to go fishing now fills out a two-part form, mails it in with payment and waits for the state to enter the data and issue and mail a fishing license. Using the scorecard to track the licensing cycle shows wait time as an inefficiency. One possibility to reduce it would be to use a system where a sportsman or woman enters driver’s license data at a service center, makes payment and obtains a license on the spot (getting better service for less money).

The implementation team initially was puzzled about how the financial area view (making money) fit into the public sector, but then it reasoned that because state agencies operate on a budget and can’t generate earnings, they create value if they increase what they do with those funds. Accordingly, the SAO chose to track how efficiently it spent taxpayer dollars.

Performance Measurement Ground Rules In “ Help Clients Take Measure, ” JofA , Jun.02, page 53, Edward Gregory and Roslyn Myers reported on the CPA Performance View, which teaches five ground rules for developing good performance measures. How does the SAO scorecard stack up? Ground rules SAO scorecard Measures should be linked to the goals and strategy of the company. Measures throughout the organization flow from the SAO’s strategies. They are displayed in the reporting software used by the office. Everyone from top management to frontline personnel must participate in developing and gathering the performance measures. From state auditor to receptionist, a cross-section of employees participated in creating the measures by which their performance would be assessed. Employees must understand the “why” of the measures before being held accountable to them. For each measure a detailed description of who, what, why and how is created. This information is a mouse click away in the reporting system software program. During the design, participants added new measures only where they saw holes in the measurement framework. Employees should be held accountable only for what they can control. Each measure has an “owner,” the employee responsible for those operational results. Employers should offer staff an incentive (a bonus, time off or promotion, for example) to support the new system of gathering, monitoring and improving the measures. The agency developed an organizational performance incentive (OPI), an annual bonus for officewide goal achievement that rewards employees who have used the system particularly well or enhanced it.

The data tracked in the knowledge area view show what staff skills are required to get a task done. It reveals the match (or gap) between competencies the SAO needs and those employees possess. For example, during the late-1990s boom, this category showed a decrease in the number of CPAs on staff. Because auditing requires CPA knowledge, the data signaled human resources to fill the gap and the financial manager to make money available for salaries.

The SAO’s internal process area view tracks whether projects get done on time and whether audits meet high standards, and it encourages people to experiment with ways to improve efficiency. Finding solutions to problems indicated by process data can be simple. In an actual private-sector example, one company’s productivity measure showed a drop in output from the day to night shifts. Ironically, it turned out the night shift produced less because the day supervisor, before leaving, locked needed supplies in a cabinet and took the key home.

The customer viewpoint captures feedback from important clients or users—in the SAO’s case, the agencies it audits. Before the scorecard implementation, the SAO didn’t collect this information. Under the old system, an agency director calling the state auditor to express annoyance about audit findings could be construed as an impromptu performance report. In the new system, the SAO seeks to know what the agencies think of its information quality, reporting objectivity and how they view its balance and fairness, professional independence, expertise in relevant subjects and staff comportment.

Select audit goals similar to SAO goals (connecting audit results to strategy).

Assemble the most representative data for expressing audit performance so managers could easily spot work trends and problems.

A SYSTEM PEOPLE CAN USE
Linking performance data in one multilevel intranet has been good for the SAO, Alwin says. The system shows how the SAO meets targets in each area, so team members can discuss how individual audits are proceeding. Managers have single-click access to detailed analysis, and this succinct, visual information lets them focus on results, solving problems, developing staff and operational planning (see exhibit 5 ).

The SAO did face challenges in implementing the scorecard system, however. The staff members had to

Learn to trust the data. Ultimately, the support system convinced them. The scorecard protocol checks units of measure for accuracy and consistency, and the system administrator regularly reviews the data for accuracy. In addition, there is comprehensive guidance to ensure that data are reported in exactly the same way. For example, one SAO data-entry instruction is: “Enter the dollar amount of invoices budgeted for billing for the month. Take the value from the Statement of Revenue and Expenditures Report, cell F11.”

Understand they’d have less work at the end of the implementation, not more. That developed with time. Trust that a red box at the front of the room would not embarrass them. The fact that meeting participants mutually or cooperatively concentrate on solving problems has allayed anxiety and helped keep the process collegial. En route the SAO learned some other lessons: Get manager buy-in. Managers are the link between an organization’s mission and its day-to-day tasks, and most of them would rather walk on nails barefoot than change. The SAO helped develop buy-in by giving people thorough individual training, monthly training updates, participation in developing the system and clearly written reference guides. Make sure a project leader understands his or her accountability. Clear responsibilities and supervisory input were essential for helping staff focus on solving problems. Manage day-to-day implementation. Because they best understand their areas, Alwin had line managers (business leaders accountable for results) develop their own measures to incorporate into the scorecard project.

AICPA Measures and Management Resources Developing and monitoring performance benchmarks for clients’ businesses is a practice opportunity for CPAs. See the AICPA Web page, www.aicpa.org/performanceview . Books AICPA practice guides include CPA Performance View Services: A Practitioner’s Guide to Providing Performance Measurement Engagements, an easy-to-use handbook for PM engagements; and CPA Performance View Services: A Financial Manager’s Guide to Leading Performance Measurement Initiatives, guidance for financial managers, cpa2biz.com . Conferences AICPA/APQC National Performance Measurements Conference, July 17–18, 2003, Denver. A forum for learning about performance measurement that offers guidance on how to align processes with goals and improve the value of CPA services. www.aicpa.org and cpa2biz.com .

Edit data volume. In the past the SAO had collected a great deal of performance data. The new goal was to focus on only the most relevant. Alwin insisted they let go of familiar but no-longer-meaningful measures.

Automate data reporting. The technology had to be powerful enough to display important cause-and-effect relationships. Individual divisions needed to see in time how their performance affected each other and the end results to identify problems and make corrections nimbly.

Lead by example. Alwin used the scorecard as a framework for meetings as soon as he could. His managers quickly did the same.

ACCOUNTABILITY THAT WORKS
The SAO is in its third full year of officewide balanced scorecard management, and the benefits are clear. The agency reports information about its performance trends and outcomes in a timely way that helps clarify goals and expectations. Simply put, the office now measures only what really matters, its staff members know what they need to do their jobs better and accountability is a part of everyday management. “Now we have real-time, accurate management information that lets us focus on business, not on chasing data,” Alwin says. “Change challenged us, but as Frank Zappa said, ‘Without deviation, progress isn’t possible.’ Our system proves its worth every day. It’s that simple,” Alwin adds with satisfaction.

Before and After Before adopting a balanced scorecard, the SAO used 15 different end-of-period reports (some including as many as 17 different measures) to assess how it was doing. This process was scrapped in favor of nine measures focusing on key activities. The new approach yields faster decision making, reduced operating costs and better business results. Prior to implementing the scorecard, projects were assessed at the end using “lag” measures such as “completed on time” or “completed within budget.” After implementing the scorecard, progress on audits is evaluated throughout using “lead” measures that enable managers to make midcourse corrections. For example, projects meeting a fieldwork checkpoint target date (a lead measure for meeting release date targets) are more likely to complete their projects on time. Before After Problems might stay buried until the end of projects. It is easy to spot problems midproject. Meeting participants often debated the meaning of the data in the reports. Meeting discussions focus on problem solving and performance improvement. Performance data were scattered across the organization; there were 15 reports in different formats, some with as many as 17 data points. All critical performance data are collected in one system and one easy-to-use format—focus is on nine organizational measures. Cost of producing the monthly report: $3,600. Cost of producing the monthly report: $1,600. Strategy was in the background; there were no well-defined project measures to drive performance toward specific goals. Daily operations are linked to strategy: Fiscal impact, customer feedback and report release status are always available to the project manager. Four employees processed the reports. Two employees process the reports. The focus was on processes. The focus is on the results. Fiscal benefit delivered to the state: $58 million. Fiscal benefit delivered to state: $250 million. Performance data often were reported late or at the last minute. Performance data are available by the eighth of each month.