Pencils Down, Computers Up—The New CPA Exam.

rofound changes planned for the Uniform CPA Examination will affect the profession in a number of ways. First, and perhaps most important, the range of subjects and skills tested will be expanded to more closely replicate those needed to practice competently. Second, a project is under way to computerize the exam by the year 2003. This article looks at the new exam format and describes the issues and considerations to be resolved before it is implemented.

Students Sitting for the CPA Exam

The initiative to revamp the CPA exam resulted from changes in the knowledge and skills that entry-level CPAs need to practice competently. The exam plays a critical role in our profession’s responsibility to safeguard the public. Thus, while modernizing the format it is also necessary to ensure the exam continues to protect the public interest. The exam ensures that only individuals who demonstrate mastery of the knowledge and skills required for entry-level practitioners are allowed to enter the profession.

A number of significant developments in the accounting profession necessitate changes in the scope of the exam and in the methods used to test candidates’ abilities.

A WIDENING BODY OF KNOWLEDGE

A powerful force that has affected the practice of accounting is the expanding universe of information that accountants must know. One need look no further than the body of literature that represents the enforceable professional standards to realize the tremendous increase in information that is the foundation of competent practice. Further, with the increase in volume has come a growth in complexity.

Perhaps the most obvious change in practice is the tremendous and pervasive effect of technology; accountants are able to obtain almost unlimited amounts of information electronically. Developing effective solutions for client problems involves gathering information. Thus, the ability to use technology to address and resolve client problems has become an important feature of practice.

Consequently, a practicing accountant doesn’t need simply to know the answers to practice questions as they arise but needs, rather, the ability to find such answers. Gone are the days when one could hope to master the body of accounting knowledge necessary to practice competently.

THE EXPECTATIONS OF SOCIETY

There is ample evidence that society today generally expects more of accountants than ever before. We have only to compare recent business news stories to those of a generation ago to realize the work of accountants is scrutinized by all constituencies more critically and more frequently than before. This is manifested not only in litigation against accountants, but also in criticisms—even from within the profession—of accounting standards setters and regulators.

Not all of the attention has been negative, of course. The American CPA is recognized worldwide as a highly trained, competent, ethical professional. Individuals who attain the CPA license, even if they do not practice in the American market, have a credential of great significance. In fact, literally thousands of people routinely travel to the United States or one of its jurisdictions to take the CPA exam. Many may not intend to practice in the United States or apply U.S. GAAP in their work. However, they believe that attaining the U.S. CPA designation distinguishes them from their counterparts who do not have it.

THE PATH FORWARD

Thus, we see a profession that is both highly regarded and yet subject to increasing criticism and questioning. This situation affects all accountants and, by extension, the CPA exam. We do not believe the current exam fails to protect the public; however, we do think the exam must change if it is to remain effective in protecting the public interest. We also suggest that to ensure continuous improvement in the midst of critical advances in technology calls for substantial modifications in the way licensing requirements are applied to the knowledge and skills of CPA candidates.

Despite recent changes, the basic test has been virtually unchanged for many years. The CPA exam is a long, difficult, paper-based test administered to large groups of candidates twice a year. Though it focuses on both the knowledge and skills of entry-level CPAs, assessment of knowledge has been the dominant feature. While this is not a fatal flaw, it is a limitation, especially as the profession changes and becomes less reliant on recall and more dependent on research and application of technology. The psychometric profession, which deals in quantitative testing, is deploying technology to improve tests and the testing process. The new test will be more expensive than the current one; initial estimates indicate the cost could reach $400 for all four sections. The increase, while substantial, reflects the expense of using single-purpose, secure test centers, a requirement for such a high-stakes test. Given the importance of ensuring the exam tests the entry-level practitioners’ proficiency in using technology and technology-based resources, the additional cost is justified. In addition, the new format may diminish the disruptive effects of candidates’ absence from the workplace while increasing the convenience of actually taking the examination. One possible model will offer 60-day periods during the year when candidates can schedule appointments to take the exam. The model will require candidates to take all sections within 30 days.

THE WHAT AND WHO OF THE DEVELOPMENT PLAN

Two features characterize the CPA exam change process: (1) development of substantive evidence on which to base recommendations and (2) participation of interested parties. The AICPA initiated a major practice analysis to identify the knowledge and skills that are critical to competent accounting practice, and the AICPA and National Association of State Boards of Accountancy formed a joint committee on CPA exam computerization. While the AICPA has conducted practice analyses in the past, the current one is noteworthy because, for the first time, draft test specifications will be included in the results. The practice analysis will also produce a database that can be used to monitor changes in practice on a continuous basis.

Although final decisions about the content of the computer-based testing (CBT) version cannot be made until the practice analysis is complete and prototype tests have been evaluated for their psychometric quality, the content oversight task force (COTF) has been discussing possible high-level content specifications for some time. One COTF activity is the development of simulations which test a candidate’s ability to analyze relatively complex practice problems, perform professional research, consider and develop appropriate solutions and communicate the results of his or her work. CBT facilitates such efforts in a number of important ways. For example, a database (such as the professional standards) is available electronically for candidates to use in addressing financial reporting, tax or auditing problems. Responses to simulations may take the form of written narratives, completed spreadsheets or a variety of other methods that demonstrate the candidate’s competence.

In early 2000, a research project administered the simulations to a small group to determine whether they should be included on the new test. This is an important study; the development of high-quality assessment simulations is a major challenge. Although it is not difficult to write simulations, it is extremely hard to interpret responses to them. It is conceivable, for example, the study will show that the initial simulations were too complex, making it impossible to determine what gaps in test takers’ knowledge contributed to errors in their work. If this happens, the researchers will have to produce a second round of development to produce less complex but still meaningful simulations. It is likely researchers will conduct several rounds of simulation prototyping before a final design is established.

THE IDEAL EXAM’S STRUCTURE

During the last two years, the joint AICPA/NASBA computerization implementation committee (CIC) developed a preliminary (Beta 1) version for an ideal examination and subjected it to wide public exposure and a formal feasibility study. The examination consisted of two one-day sessions testing accounting knowledge and application of accounting skills.

The first day’s exam was to be composed entirely of objectively scored items (such as multiple choice questions), while the second included simulations requiring candidates to “construct” their responses (a constructed response asks test takers to supply an answer, as in writing an essay answer or building a spreadsheet, rather than to pick one from a list). In a substantial departure from tradition, each exam was to test the entire range of accounting subjects appropriate for licensing (auditing, financial reporting, taxation and management accounting).

The public exposure of the Beta 1 ideal examination and a separate feasibility study conducted by an outside consultant raised a number of questions about issues such as psychometric quality and perceived fairness of the variable test lengths. As a result, the CIC and the COTF developed additional possible test structures. Logical and empirical criteria developed to evaluate the different models show that a multi-test model, in which each test contains a mixture of multiple-choice and performance exercises, has the most promise of providing a psychometrically sound, content-appropriate CPA licensure examination.

MULTIPLE PERSPECTIVES AND EXPERTISE

Revising the Uniform CPA Examination to ensure it continues to protect the public is a critical and mammoth endeavor. CPAs, regulators and others all have crucial roles in the process.

In one example of CPAs’ contribution, over 5,000 participated in the practice analysis survey. CPAs had taken part in focus groups to develop the survey. Other CPAs are researching how to establish the cut scores on the new test.

State boards and regulators helped review and evaluate the Beta 1 prototype. Representatives from these groups participated in forums to identify related policy and legislative issues and to propose solutions to issues ranging from candidate scheduling and conditioning credit (when a candidate passes only two sections, each state can determine whether the scores on the other section were high enough to qualify the candidate to retake only that section at a later date) to security and disaster recovery.

Noted psychometricians from around the country are researching whether the new test is sound; others, also of nationally recognized status (including some members from state regulatory agencies), will advise the AICPA’s psychometric staff.

Testing organizations, which include Professional Examination Services, The Chauncey Group International, the American Institutes for Research and Alpine Media are conducting specific research and development projects.

TIME IS OF THE ESSENCE

The target date for the introduction of a new, computer-based CPA licensure examination is 2003. Although that seems a long way away, given the magnitude of the effort, the timeline is actually quite ambitious. In the coming months, as the practice analysis is completed, prototype simulations are evaluated and other projects are finished, we will learn a great deal more about the content and structure of the new exam and the feasibility of meeting the 2003 schedule. It is important to keep to that date. On the one hand, the profession needs to be assured its licensure examination appropriately assesses entry-level skills, particularly as our work becomes more skills-based and technology oriented. On the other hand, the new exam must equal or exceed the current one in its quality or it will not protect the public. A long delay in the introduction of the new test could result in an exam that does not accurately reflect the profession. If the new test is to be launched in 2003, it must satisfy demands for appropriate content and psychometric quality.