EXECUTIVE
SUMMARY | WHEN THE TEXAS STATE
AUDITOR'S OFFICE (SAO) decided
its management system could better monitor
how taxpayer dollars were spent, it
adopted a balanced scorecard approach to
track its performance. As a public-sector
entity, the SAO sought a “profitability”
strategy of offering more services to
citizens without increasing program costs
(it aimed to stretch finite resources
further).
BASED ON THE IDEA
THAT ONE BENCHMARK can’t show
true performance, a balanced scorecard
uses an array of indicators related to
an organization’s strategic goals and to
progress toward them. For example, it
will link measures for process (on-time
delivery) and finance (revenue growth)
with others such as customer feedback
and employee knowledge.
THE PRIMARY GOALS
for instituting a performance
measurement system were to ensure
accurate and timely reporting of SAO
financial data, establish individual
manager accountability for clearly
defined results and use the scorecard
reporting system to make informed
decisions about how to efficiently use
its budget.
A CROSS-SECTION OF
ABOUT 20% of its workforce
collaborated on identifying the four key
strategies: to provide independent audit
services, to teach managers to avoid
problems, to help agencies fix problems
and improve operations and to make sure
the state’s compensation structure
offered appropriate salaries.
THE PARTICIPANTS
IDENTIFIED the most important
data to track and developed focused
measures—cycle time, quality or volume,
for example—to evaluate progress toward
objectives. Too many measures can cause
“data fatigue.”
THE RESULTING
OFFICEWIDE, MULTILEVEL
information system reports data
in a visual, easy-to-understand format
that gives all managers single-click
access to detailed analysis and
information. Meetings now are about
project results, solving problems and
operational planning. |
DEBORAH L. KERR, PhD,
is chief strategy officer of the Texas
State Auditor’s Office. She writes and
lectures on performance management topics.
Her e-mail address is
dkerr@sao.state.tx.us .
|
verywhere you look, good business
management seems to be in short supply. Recent
examples abound of poorly run companies such as
Qwest, Sunbeam, Andersen, Tyco, WorldCom, Enron
and most of the late dot-coms. So what causes such
conspicuous failures? Not lack of ideas (there are
plenty of those). Most experts agree the major
cause of business meltdown is management error,
which occurs across the enterprise spectrum, not
just at high-profile companies. How is it
at your business? Has a project you thought was on
time and on budget recently blown both measures
and caught you off guard? Can you identify trouble
spots right now? How would you know about them,
and what changes could you make if you did? CPAs
who consult to companies, operate in industry or
are managing partners can follow this case study
to learn how to implement a balanced scorecard
oversight system, which organizes performance data
clearly while showing how operational areas are
linked.
A MEASURED RESPONSE
About seven years ago, Lawrence F. Alwin,
CPA and Texas state auditor, decided the State
Auditor’s Office (SAO) needed a better way to
track how it was meeting its mission to provide
lawmakers and agencies with the most useful
possible financial operating data for
taxpayer-funded projects. The SAO’s monthly status
reports were thorough: They analyzed staffing,
summarized time use for all employees and
evaluated budget and progress performance for
about 150 projects. In fact it took four people
three days to compile the data into
four-inch-thick binders and distribute them. The
problem: Each division handled its performance
data a little differently, so deciphering the
information stalled meetings and knocked projects
off schedule. “We weren’t getting the
right data at the right time for good decision
making,” Alwin says. In 1996 I was one
of Alwin’s audit directors when a
Harvard Business Review
article, “Using the Balanced
Scorecard as a Strategic Management
System” by Robert Kaplan and David Norton,
caught my eye. I suggested to Alwin the
SAO might benefit by using the process it
described, which links objectives,
measures, targets and initiatives to
collectively depict a business strategy
and how to achieve it. Based on the idea
that one benchmark can’t show true
performance, it recommended using an array
of indicators to express an organization’s
strategic goals and progress toward them.
For example, it linked measures for
process (on-time delivery) and finance
(revenue growth) with others such as
customer feedback and employee knowledge.
|
Good
Management Depends on
Good Data
A recent AICPA survey
of CPA firms found 80% of
respondents saw monitoring
performance as a way to
achieve business results,
but 65% were dissatisfied
with the system their
organization used.
Source: AICPA.
| |
Alwin agreed the approach could be useful. His
first step toward instituting it was to choose
which information categories SAO monthly status
reports would systematically address: financial
performance, audit performance, customer reaction
and overall results. Audit performance would track
three areas: the audit projects completed at the
end of each month, the number likely to meet the
next month’s deadlines and the reasons some audit
reports were late (see “
Performance Measurement Ground Rules ”). It
would give managers more time to concentrate on
solving operational problems.
SETTING GOALS
As Alwin saw it, the
SAO’s new measurement system’s essential goals
were to
Generate accurate reports.
SAO measurements had to yield clear,
accurate, timely data that were parallel from
agency to agency.
Establish visible accountability.
Alwin assigned responsibility for
producing results to specific managers, and he
sharpened the benchmarks.
Motivate informed corrective action.
Managers would use monthly executive
meetings to share ideas to correct problems based
on what they had learned. To meet those
goals, Alwin had to
Designate staff to accomplish
objectives.
Improve employee efficiency (minimize
the gaps, overlap, friction and errors that occur
when people aren’t clear about their job duties).
Make knowledge more transparent (give
people fast feedback to help them find ways to
work better).
Eliminate process redundancies to
reduce costs (see “Before and After”). He
then focused on five steps for implementing the
goals. They were
Prepare a strategic plan (action
outline). The strategic plan’s
implementation actions would be tracked by
performance measures that fit the SAO’s goals.
(For more information on making or updating such a
plan, see “
Strategic Planners Lead the Pack, ” JofA
, Dec.01, page 26.)
Identify critical business objectives
for each balanced scorecard “perspective” or
area view (in effect, windows
into different aspects of an organization’s
performance) and link each one to the goals of the
strategic plan. Name an individual
“owner”—typically a member of the management
team—for each objective.
Develop a limited number of
measures—cycle time, quality or volume, for
example—to evaluate progress toward
objectives. It’s important to
not have too many measures, which can cause “data
fatigue.”
Set measurable goals (targets or
budgets). Measures such as
“reduce employee turnover by 5% in 12 months” or
“cut administration expenses by 3% in nine months”
clarify priorities. Targets have to be realistic
as well as challenging. Setting the bar too high
would demoralize workers and cause the system to
fail.
Collect and discuss data at least
monthly. This is where
measurement turns into management: At regular
meetings, owners explain why they got the results
they did; if targets have been missed, they say
what steps will get them back on track.
PEOPLE, TIME, MONEY AND ACCOMPLISHMENTS
To get the balanced scorecard system under
way, the SAO had to delineate its goal and the
actions to reach it. A private company might pick
maximizing revenue growth or capitalizing on a
market niche as a strategy to fulfill a goal to
“become the most profitable” in its sector, for
example. A public-sector profitability strategy
could be to offer more services without increasing
program costs—that is, stretch finite resources
further (see “What to Track”). Alwin
convened a cross-section of about 20% of the SAO
staff (from receptionists to auditors,
twenty-somethings to near retirees) to state their
goal, to update the agency’s strategic plan and to
identify the data most relevant to its mission.
Their goal: Provide the most accurate, timely
financial information to Texas legislators to
effect better public-policy outcomes. He asked
participants: “What should we do to fundamentally
improve that service?” In a series of
one-day meetings over a six-week period, staff
members offered more than 400 answers. Alwin took
a month to review them, noting which themes
participants brought up again and again. The
process homed in on four strategies (see
exhibit 1 ): The SAO would provide
independent audit services (AS) as chartered,
provide educational services (ES) to teach agency
staff how to avoid problems and improve
accountability, offer management advisory services
(MAS) to help agencies improve operations and
manage the state’s compensation structure and
address other human resources issues (SCO).
|
|
Exhibit 2:
Organizational Areas
| A
private-sector organization’s
goal is to make money
(financial), and it manages
customer relationships, internal
processes and knowledge to
achieve this (left column). A
public-sector entity such as the
SAO has mission as its goal,
which requires it to manage
customer relationships, internal
processes, knowledge and
financial responsibilities
(right column).
| |
After the first group’s strategic planning
input was complete, Alwin next worked with an
implementation team of SAO managers to adapt its
mission’s four supporting scorecard areas:
financial, customer, process and knowledge (see
exhibit 2 ). At the top of the
scorecard, the implementation team added a fifth
category—mission—to represent the public-sector
bottom line: public-policy outcomes related to SAO
data. To measure its information’s
utility, the SAO collects data on how legislators
use its audit reports in actual hearings and
debates. A related measure tracks the dollar value
created through audits, including cost savings,
cost avoidance and identified efficiency gains
(the public-policy outcomes). For example, a
sportsman or woman who wants to go fishing now
fills out a two-part form, mails it in with
payment and waits for the state to enter the data
and issue and mail a fishing license. Using the
scorecard to track the licensing cycle shows wait
time as an inefficiency. One possibility to reduce
it would be to use a system where a sportsman or
woman enters driver’s license data at a service
center, makes payment and obtains a license on the
spot (getting better service for less money).
The implementation team initially was puzzled
about how the financial area view (making money)
fit into the public sector, but then it reasoned
that because state agencies operate on a budget
and can’t generate earnings, they create value if
they increase what they do with those funds.
Accordingly, the SAO chose to track how
efficiently it spent taxpayer dollars.
Performance
Measurement Ground Rules
In “ Help
Clients Take Measure, ” JofA
, Jun.02, page 53, Edward Gregory and
Roslyn Myers reported on the CPA
Performance View, which teaches five
ground rules for developing good
performance measures. How does the SAO
scorecard stack up?
Ground rules
|
SAO scorecard
|
Measures should be
linked to the goals and strategy
of the company. | Measures throughout
the organization flow from the
SAO’s strategies. They are
displayed in the reporting
software used by the office.
| Everyone from top
management to frontline
personnel must participate in
developing and gathering the
performance measures. |
From state auditor
to receptionist, a cross-section
of employees participated in
creating the measures by which
their performance would be
assessed. |
Employees must
understand the “why” of the
measures before being held
accountable to them. |
For each measure a
detailed description of who,
what, why and how is created.
This information is a mouse
click away in the reporting
system software program. During
the design, participants added
new measures only where they saw
holes in the measurement
framework. |
Employees should be
held accountable only for what
they can control. | Each measure has an
“owner,” the employee
responsible for those
operational results. |
Employers should
offer staff an incentive (a
bonus, time off or promotion,
for example) to support the new
system of gathering, monitoring
and improving the measures.
| The agency
developed an organizational
performance incentive (OPI), an
annual bonus for officewide goal
achievement that rewards
employees who have used the
system particularly well or
enhanced it.
| |
The data tracked in the knowledge area view
show what staff skills are required to get a task
done. It reveals the match (or gap) between
competencies the SAO needs and those employees
possess. For example, during the late-1990s boom,
this category showed a decrease in the number of
CPAs on staff. Because auditing requires CPA
knowledge, the data signaled human resources to
fill the gap and the financial manager to make
money available for salaries. The SAO’s
internal process area view tracks whether projects
get done on time and whether audits meet high
standards, and it encourages people to experiment
with ways to improve efficiency. Finding solutions
to problems indicated by process data can be
simple. In an actual private-sector example, one
company’s productivity measure showed a drop in
output from the day to night shifts. Ironically,
it turned out the night shift produced less
because the day supervisor, before leaving, locked
needed supplies in a cabinet and took the key
home. The customer viewpoint captures
feedback from important clients or users—in the
SAO’s case, the agencies it audits. Before the
scorecard implementation, the SAO didn’t collect
this information. Under the old system, an agency
director calling the state auditor to express
annoyance about audit findings could be construed
as an impromptu performance report. In the new
system, the SAO seeks to know what the agencies
think of its information quality, reporting
objectivity and how they view its balance and
fairness, professional independence, expertise in
relevant subjects and staff comportment.
The
SAO now generally gets high marks, but
previously it had poor communication with
one agency, for example. During scorecard
development, Alwin asked the agency
director, “How can we serve you better?”
The director replied she needed
information sooner, so bad news—if
any—wouldn’t catch her by surprise. As a
result, Alwin instituted more meetings to
keep all the directors better informed
about audits in progress. Next, the
team designed a system of organizational
measures to track performance within
each area and indicate appropriate
targets (see
exhibit 3 ). The five areas
incorporate a total of nine measures,
two for mission, one for customer
feedback, one for internal processes,
one for knowledge and four for
financial. Once the SAO had
chosen the measures it thought would
most comprehensively track people, time,
money and accomplishments as well as
report performance trends and focus
everyone on results, it selected
Canadian software developer Panorama
Business Views (pbviews) to supply its
scorecard software. The resulting system
translates SAO data into charts that are
accessible to all managers and are
displayed on-screen during monthly
meetings. Depending on how well results
have met targets, the underlying data
for each measure determine whether a box
turns green (on track), yellow (having
problems) or red (in trouble). At
monthly meetings a red box makes it easy
to decide what needs attention.
THE AUDIT SCORECARD
After the SAO set the officewide
scorecard, Alwin and the agency managers
needed drill-down capability for the AS,
ES, MAS and SCO strategies. Because 85%
of SAO data come from audits, the audit
part of the scorecard system was
especially important (see
exhibit 4 ). Frank Vito,
CPA and director of audits, oversaw that
area of the design in a process parallel
to the overall development. “We needed
metrics using both lead and lag
measures,” he says. (Lag checks data
monthly, quarterly or even less often;
lead checks more frequently so
corrections can occur earlier.) Over a
16-month period, Vito and several
managers developed an audit scorecard
and measures to reach three goals
related to the five areas. They were
Eliminate unnecessary
steps. | |
Select audit goals similar to SAO
goals (connecting audit results to strategy).
Assemble the most representative data
for expressing audit performance so managers could
easily spot work trends and problems.
A SYSTEM PEOPLE CAN USE
Linking performance data in one multilevel
intranet has been good for the SAO, Alwin says.
The system shows how the SAO meets targets in each
area, so team members can discuss how individual
audits are proceeding. Managers have single-click
access to detailed analysis, and this succinct,
visual information lets them focus on results,
solving problems, developing staff and operational
planning (see exhibit 5 ).
Exhibit 5: SAO
Scorecard Electronic Briefing Book
| Users can display
information in a notebook format per the
screen capture below. Measures on the left
side have links to documents and related
reports. The measure’s “owner” can comment
at right. Where yellow or red indicates
missed targets, the owner describes how he
or she will improve results. The bar chart
shows results across the year. The
description says what is being measured,
how and by whom as well as reasons why the
measure shows up in green, yellow or red.
| The SAO did
face challenges in implementing the scorecard
system, however. The staff members had to
Learn to trust the data.
Ultimately, the support system convinced
them. The scorecard protocol checks units of
measure for accuracy and consistency, and the
system administrator regularly reviews the data
for accuracy. In addition, there is comprehensive
guidance to ensure that data are reported in
exactly the same way. For example, one SAO
data-entry instruction is: “Enter the dollar
amount of invoices budgeted for billing for the
month. Take the value from the Statement of
Revenue and Expenditures Report, cell F11.”
Understand they’d have
less work at the end of the
implementation, not more. That
developed with time.
Trust that a red box at
the front of the room would not
embarrass them. The fact that
meeting participants mutually or
cooperatively concentrate on solving
problems has allayed anxiety and helped
keep the process collegial. En
route the SAO learned some other
lessons:
Get manager buy-in.
Managers are the link between an
organization’s mission and its
day-to-day tasks, and most of them would
rather walk on nails barefoot than
change. The SAO helped develop buy-in by
giving people thorough individual
training, monthly training updates,
participation in developing the system
and clearly written reference guides.
Make sure a project
leader understands his or her
accountability. Clear
responsibilities and supervisory input
were essential for helping staff focus
on solving problems.
Manage day-to-day
implementation. Because they best
understand their areas, Alwin had line
managers (business leaders accountable
for results) develop their own measures
to incorporate into the scorecard
project. |
AICPA Measures and
Management Resources
Developing and
monitoring performance
benchmarks for clients’
businesses is a practice
opportunity for CPAs. See the
AICPA Web page,
www.aicpa.org/performanceview
.
Books
AICPA
practice guides include
CPA Performance View
Services: A Practitioner’s
Guide to Providing
Performance Measurement
Engagements, an
easy-to-use handbook for PM
engagements; and CPA
Performance View Services: A
Financial Manager’s Guide to
Leading Performance
Measurement Initiatives,
guidance for financial
managers,
cpa2biz.com .
Conferences
AICPA/APQC
National Performance
Measurements Conference, July
17–18, 2003, Denver. A forum
for learning about performance
measurement that offers
guidance on how to align
processes with goals and
improve the value of CPA
services.
www.aicpa.org and cpa2biz.com
.
| |
Edit data volume. In the
past the SAO had collected a great deal of
performance data. The new goal was to focus on
only the most relevant. Alwin insisted they let go
of familiar but no-longer-meaningful measures.
Automate data reporting. The
technology had to be powerful enough to display
important cause-and-effect relationships.
Individual divisions needed to see in time how
their performance affected each other and the end
results to identify problems and make corrections
nimbly.
Lead by example. Alwin used
the scorecard as a framework for meetings as soon
as he could. His managers quickly did the same.
ACCOUNTABILITY THAT WORKS
The SAO is in its third full year of
officewide balanced scorecard management, and the
benefits are clear. The agency reports information
about its performance trends and outcomes in a
timely way that helps clarify goals and
expectations. Simply put, the office now measures
only what really matters, its staff members know
what they need to do their jobs better and
accountability is a part of everyday management.
“Now we have real-time, accurate management
information that lets us focus on business, not on
chasing data,” Alwin says. “Change challenged us,
but as Frank Zappa said, ‘Without deviation,
progress isn’t possible.’ Our system proves its
worth every day. It’s that simple,” Alwin adds
with satisfaction.
Before
and After Before adopting
a balanced scorecard, the SAO used 15
different end-of-period reports (some
including as many as 17 different
measures) to assess how it was doing. This
process was scrapped in favor of nine
measures focusing on key activities. The
new approach yields faster decision
making, reduced operating costs and better
business results. Prior to implementing
the scorecard, projects were assessed at
the end using “lag” measures such as
“completed on time” or “completed within
budget.” After implementing the scorecard,
progress on audits is evaluated throughout
using “lead” measures that enable managers
to make midcourse corrections. For
example, projects meeting a fieldwork
checkpoint target date (a lead measure for
meeting release date targets) are more
likely to complete their projects on time.
Before
|
After
|
Problems might stay
buried until the end of
projects. | It is easy to spot
problems midproject. |
Meeting
participants often debated the
meaning of the data in the
reports. | Meeting discussions
focus on problem solving and
performance improvement.
| Performance data
were scattered across the
organization; there were 15
reports in different formats,
some with as many as 17 data
points. | All critical
performance data are collected
in one system and one
easy-to-use format—focus is on
nine organizational measures.
| Cost of producing
the monthly report: $3,600.
| Cost of producing
the monthly report: $1,600.
| Strategy was in the
background; there were no
well-defined project measures to
drive performance toward
specific goals. | Daily operations
are linked to strategy: Fiscal
impact, customer feedback and
report release status are always
available to the project
manager. |
Four employees
processed the reports. |
Two employees
process the reports. |
The focus was on
processes. | The focus is on the
results. |
Fiscal benefit
delivered to the state: $58
million. | Fiscal benefit
delivered to state: $250
million. |
Performance data
often were reported late or at
the last minute. | Performance data
are available by the eighth of
each month. | | |