GenAI for accountants: 10 prompt-writing tips

Tech-savvy accountants have found myriad uses for generative artificial intelligence (GenAI) since ChatGPT made its debut in November 2022. GenAI tools can draft emails, analyze spreadsheets, summarize tax legislation, research business plans, generate code, and even help practice difficult conversations with clients.

But for all of GenAI’s mind-blowing capabilities and dangers (see the sidebar “Precautions for Using Generative AI”), the new technology’s effectiveness hinges on one critical factor: the “prompt,” i.e., the instructions a user gives to the software.

The skill set for writing those prompts, sometimes known as “prompt crafting” or “prompt engineering,” may soon prove crucial for the accounting profession.

“The human on the front end can ask better questions and get better output,” said Ashley Francis, CPA, owner of The Francis Group PLLC, who has started using GenAI to automate her tax and estate planning practice.

THE POWER OF PROMPT CRAFTING

As Francis and others have learned, a poorly composed prompt may result in underwhelming prose, factual errors, or flawed computer code from ChatGPT or another GenAI tool. Basic mistakes in prompt crafting are hiding the true power of the technology from many users, said Byron Patrick, CPA/CITP, CGMA, the vice president of client success at The B3 Method Institute.

Users with a bad first experience “write it off, saying that GPT’s the problem,” he said, referring to GPT, or generative pre-trained transformer, the large language model that powers ChatGPT.

In contrast, a well-crafted prompt can unlock the true capabilities of large language models — allowing finance professionals to access new ideas and speed their work. One recent study found dramatic improvements in the GPT model’s performance on an unofficial practice CPA Exam.

When the researchers used prompt engineering techniques, the GPT model went from a failing score on the unofficial practice exam to a passing score.

The difference is not just about technical knowhow. Writing a great prompt doesn’t require programming skills or a deep understanding of neural networks. Instead, it’s often about the user’s clarity, specificity, imagination, and experimentation.

So, with seemingly infinite possibilities, how do you ensure your prompt is just right? This article provides 10 tips for crafting effective GenAI prompts.

ADD STRUCTURE AND SPECIFICITY

GenAI’s ability to parse human language is impressive. Even a simple, vague sentence — “summarize this new tax legislation for me” — with a link to a PDF can deliver serviceable results from some models. But more detailed instructions can deliver more useful outcomes.

The key is “defining expectations,” said Jason Staats, CPA, a former accounting firm head who runs the accounting technology community Realize.

“Ultimately, it’s about instructing it to do something for you, the same way you would have a junior researcher do it.”

That means being specific: “Look for specific instances where this memo mentions nonprofits” might deliver more actionable results for an accountant focused on nonprofits.

GenAI tools can achieve even more if they’re given longer and well-structured prompts.

For example, you could ask the latest version of ChatGPT, powered by the GPT-4 model, to do the following:

• Search the attached memo for all paragraphs mentioning nonprofits, including 501(c)(4)s and other similar entities;
• Place the full text of each of those paragraphs into a numbered list; and
• Beneath each of the list entries, add links to relevant web references from major news media, academic sources, and large businesses.

Using lists, bullets, headers, and labels may aid the AI in processing lengthier instructions.

However, due to computing constraints, GenAI tools often will struggle to output long chunks of text or deal with large files, resulting in frustrating error messages or incomplete responses, even if the prompts are well written.

In those cases, focus your requests on smaller tasks, such as analyzing a portion of a document or producing only the introduction of a document draft. The GenAI tool can also be asked to “continue” its work on an incomplete response.

PROVIDE CONTEXT

Prompts don’t just include direct instructions. They also can give contextual information about the user’s larger objective and the role of the GenAI.

“The most general framework you can use is, ‘I am, you are, I need,’” Francis said.

She gave the following example:

I am: “a CPA”

You are: “a CPA business advisory bot”

I need: “a list of considerations for choosing a niche”

Also popular is the “role-task-format” prompt, which defines the bot’s role (“an administrative assistant”), its task (“summarize the contents of this email”), and the format of the output (“a bulleted list”).

Regardless of the exact framework used for the prompt, those extra details can focus the GenAI tool’s answer and encourage it to tap into specific knowledge domains.

ASK FOR MULTIPLE VARIATIONS

Heather Smith, the director of ANISE Consulting in Australia, uses GenAI tools frequently for tasks including drafting emails. She often will ask for multiple responses to the same prompt so that she can choose among them.

“Ask it for three versions of the email,” she explained, “because every response is different.”

And there’s no need to stop at just three or four versions. A GenAI model can easily provide dozens of responses, which is especially useful for ideation and research tasks.

For example, when asked for “50 punny names for a new accounting firm in Florida,” ChatGPT delivered instantly, with results including the reasonably clever suggestion of “Keys to Success CPAs.” Similarly, a request for “12 examples of controversial tax laws throughout history” turned up more than a few interesting (and accurate) anecdotes.

TRY MULTIPLE REVISIONS

Besides asking for multiple responses, it also helps to submit multiple versions of the same prompt.

“It’s not one and done. You can just try and try and try,” Patrick said.

Once the GenAI tool has returned an initial response, the user can add instructions to shape the output, with no need to rewrite the entire prompt. For example, you might ask the AI to “make that shorter” or “be more specific.” You could tell it to produce five variations with different levels of complexity or “information density.”

If you’re using a GenAI tool on a programming project, and the code the tool produced isn’t executing, you could write a prompt that includes the exact text of the error message and a request for revisions. The technology also tends to respond well to requests such as “elaborate on that point,” “use a metaphor,” or simply “that didn’t work.” The key is to keep trying different approaches, adding structure and specificity along the way, Smith said.

If the output doesn’t look right, “pick away at it, unpack it,” she said. “Is the way I’m asking the question wrong?”

The GenAI tools also can help write prompts for themselves. In a technique known as meta-prompting, the user explains the goal and asks for the GenAI tool’s help writing a prompt to achieve that goal. (“Could you help me write a prompt that will apply data transformations to an Excel spreadsheet?”)

The GenAI tool may then respond with questions to clarify the goal, or it may directly generate a prompt that uses prompt engineering strategies.

ASK FOR EXPLANATIONS AND CITATIONS

GenAI tools can serve as powerful research assistants, able to interpret and combine vast troves of knowledge to answer a question and provide relevant research material. But they also are notoriously prone, especially in earlier versions, to “hallucinations” that inject false facts into their answers.

To reduce the risk, users can ask GenAI tools like ChatGPT to explain themselves. That means telling the tool to “provide direct citations to sources” that back up its claims, and then checking those citations yourself to ensure they’re correct. Or it could mean asking the tool to explain its chain of thinking so the user can check its logic.

“Nobody wants a black box answer from AI,” Staats said. “It’s just the same as if I have an intern bring me an answer. When they say, ‘This is the answer,’ I’m going to say, ‘Prove it.’”

Having the AI explain itself isn’t just a factchecking tactic; it also can be educational for the user. If GenAI has coded an Excel macro, ask it to explain how the programming works and why it made certain decisions.

USE GenAI AS AN EDITOR

GenAI tools can be prompted to critique and expand upon written content.

For example, try asking a GenAI tool to explore potential omissions in a memo or to explain contradictory viewpoints and counterfactuals. The response may yield new lines of questioning or topics of interest. The model can be asked to identify unclear language and even home in on paragraphs that need additional factual support.

Those editing techniques can be applied to content provided by the user, or the GenAI tool can be asked to edit its previous work.

Still, watch for the GenAI tool’s omissions and oversights. Even when it is factually correct, the GenAI may exclude critical information when summarizing a document. Certain GenAI tools may also lack access to up-to-date information, instead relying on outdated knowledge from when they were trained. As always, human supervision is necessary.

USE ALTERNATIVE INPUT FORMATS

Typing into a chat box is the most obvious way to interact with GenAI tools, but it’s not the only option.

Some GenAI models can also read and manipulate files. For example, the version of ChatGPT using GPT-4 can ingest a PDF and “extract the numbers on page 9 into a table.” It also can dive into spreadsheets to look for data trends and return with visualizations.

Another GenAI tool, Google Gemini, can connect to other Google services, including Docs and Maps.

This opens a range of new uses. Patrick has uploaded his rough, hand-drawn sketches as fodder for ChatGPT’s DALL-E image-generating engine. He also uses the ChatGPT mobile app’s voice functionality to have conversations with the platform while he’s out walking his dogs.

Brian Clare, CPA, the CEO of Blueprint Accounting in Ottawa, Canada, makes similar use of voice interactions. He has his employees record themselves while they’re narrating business processes. Then, they have transcripts made by an automated service and ask ChatGPT to summarize the transcripts and export them for use as business process documentation.

GenAI models also can directly export some file types, including image files and Excel sheets.

And in instances where the platform can’t directly export a file type — such as a SQL-based database file — the GenAI may still be able to generate computer code that will create or manipulate those file types when executed by the user. This approach allows more advanced users to extend the GenAI tool’s capabilities but requires more knowledge of programming and development.

SAVE WHAT WORKS

At present, some GenAI tools have strict limits.

For example, ChatGPT can have a long, detailed chat in which it “remembers” and expounds upon running themes and questions in conversation with the user. A single thread can stretch on for hundreds of messages, layering on more knowledge and context for the GenAI tool.

But that ChatGPT conversation is self-contained. If the user switches to a new chat thread, ChatGPT will be unable to access any of the information that the user provided in previous, separate threads. With each new thread, ChatGPT returns to its generic knowledge base, rather than “learning” about the user over time. (Other emerging GenAI tools may operate differently.)

However, there are ways to save the results of those conversations. For example, ChatGPT allows users to label individual threads and reopen them later.

And, more generally, users can simply ask the GenAI tool to summarize the conversation. That summary can then be used as a prompt to kick-start a similar conversation later. (In fact, the tool can be asked to write the summary in the form of a prompt.)

Some services also allow for more permanent tweaks. The paid version of ChatGPT allows users to create “custom” GPT bots that can remember certain preferences, such as the desired writing style for outputted content, or even the user’s profession and objectives.

ChatGPT also has a “Custom Instructions” interface where users can add freeform information about themselves and their preferences — where they live, their schedule, their personality, or anything else they desire — that will shape the platform’s behavior.

EXPERIMENT WITH ADVANCED TECHNIQUES

It’s possible to turn the tables and have the GenAI tool ask questions of the users — for example, “ask me questions that will help me understand risks to my supply chain,” or even something more philosophical, like “ask me some questions to help me define and improve my working style.”

“It’s a little bit mind bending,” Clare said of those conversations.

Francis stumbled upon a similar way to use the bots’ conversational skills. She told ChatGPT to act like a client so that she could practice having difficult conversations with it, and then she had it analyze her performance.

And an informal network of academic researchers, entrepreneurs, and enthusiasts is constantly trying to eke more performance out of more complex and esoteric prompts.

“It’s like Excel macros,” Staats said. “We’re going to have people who are really excited by that stuff and can squeeze the ever-living heck out of it to get it as far as possible. Oftentimes, it is a worthwhile investment.”

An approach introduced by researchers, the “Chain of Density,” has the AI summarize an article and then revise the summary several times, following a detailed set of instructions meant to produce increasingly “information dense” results.

Similarly, the researchers who were testing GPT-4’s performance on the unofficial practice CPA Exam found that having the GenAI explain its “chain of thought,” among other tactics, was particularly effective. They used the following prompt to achieve improved results:

“Answer the following question. Before you answer, explain your thoughts. If you don’t know where to begin, use the search tool to get help.

You are encouraged to use the tools when you are unsure about the correct answer. Use calculator ONLY when you have numbers for input.

You have access only to the following tools: {tools}

Always use the following format:

Question: the input question you must answer

Thought: you should always think about what to do

Action: the action to take, should be one of [{tool_names}]

Action Input: the input to the action

Observation: the result of the action (this Thought/ Action/Action Input/Observation can repeat N times)

Thought: I am ready to answer

My answer is:”

Begin! Remember: Always follow the above format

Question: {input}

{agent_scratchpad}”

JUST KEEP EXPLORING

There is no instruction manual for GenAI tools like ChatGPT — no all-encompassing list of their capabilities, no single right way to use them. In many ways, humans are still learning just what the technology can do.

“It’s just like we’re all doing these [experiments] in silos right now,” Francis said.

That’s why the experts interviewed for this article offered a common message: Keep experimenting. See where and how the programs can contribute to your life and try new things.

For Clare, the key is to think about GenAI as more than just a shortcut or a content creator. He looks to it as a source of inspiration with undiscovered limits.

“And if you’re having a conversation with ChatGPT, it may say something that triggers an idea for you that is completely outside the scope of what you’re considering,” Clare said. “Be open-minded about it.”

Precautions for using generative AI

Be aware of the following risks and issues when using GenAI tools.

• Mistakes and omissions: GenAI may misinterpret facts, omit important details, or even introduce outright falsehoods. It can even overstate its own capabilities. For example, GPT-4 recently told the author of this article that it could read a melody from a piece of sheet music, only to produce completely incorrect results and eventually admit that it had no such ability.
• Reinforcing bias: Large language models “learn” about the world by studying existing content. They can inherit and reproduce human biases and prejudices.
• Open legal questions: In the United States, there is an open question about whether and when works generated by GPT and other models can be copyrighted. Using the platforms could lead to intellectual property challenges, among other legal issues.
• Data privacy and customer concerns: Inputting sensitive data into a GenAI tool could incur the risk that it is shared with other users or otherwise leaks. Even if a platform has strong data protections in place, the use of AI could pose a reputational risk due to clients’ and customers’ concerns.
• Moral and ethical issues: The rise of GenAI has come with fears of job displacement and deep questions about the meaning of this new technology for humanity. Any leader embracing it should be prepared to think and talk about the consequences of the change.

About the author

Andrew Kenney is a freelance writer based in Colorado. To comment on this article or to suggest an idea for another article, contact Jeff Drew at Jeff.Drew@aicpa-cima.com.

LEARNING RESOURCES

Ethics in the World of AI: A CPA’s Guide to Managing the Risks

This webcast will discuss the current uses of AI in business, nine risk areas, and provide practical suggestions to address these risks effectively. Attendees can earn 2 CPE credits.

April 2, 2–4 p.m. ET

WEBCAST

AICPA & CIMA ENGAGE Conference

Technology is one of more than a half-dozen tracks delivering up to 42 CPE hours at the biggest accounting event of the year, held at the ARIA in Las Vegas.

June 3–6

CONFERENCE

For more information or to make a purchase, go to aicpa-cima.com/cpe-learning or call 888-777-7077.

AICPA & CIMA RESOURCES

Articles

JofA AI coverage page

“Tech Q&A: A First Look at Using Microsoft’s New AI Tool, Copilot,” JofA, March 1, 2024

“CPAs at a Crossroads: Real Approaches to Artificial Intelligence,” JofA, Jan. 8, 2024

“Navigating the Landscape of Generative AI,” AICPA & CIMA, Dec. 4, 2023

“Generative AI and Risks to CPA Firms,” JofA, Oct. 1, 2023

“Writing Advanced Excel Macros With GPT-4,” JofA, July 7, 2023

Podcast episode

“Why Should CPAs Understand ChatGPT?” JofA, Oct. 5, 2023

Website

Demystifying AI for the Accounting and Finance Profession