Tech roundtable: AI and automation go way beyond ChatGPT

ChatGPT generates most of the headlines, but it’s only a small part of the big picture when it comes to artificial intelligence (AI) in the accounting profession.

What is the wider view on AI for accounting firms and finance departments, and how does it all relate to automation making life better for CPAs? Find out from three of the top accounting technology experts in the latest episode of the JofA podcast.

Gathered for the JofA‘s annual Accounting Technology Roundtable, the panelists — Automata Practice Development’s Wesley Hartman; IntrapriseTechKnowlogies’ Donny Shimamoto, CPA/CITP, CGMA; and Boomer Consulting’s Amanda Wilkie — covered numerous tech topics in a wide-ranging discussion being presented in two parts.

Part 1 features the aforementioned AI and automation discussion. Part 2, scheduled to be published Thursday, delves into topics including “people tech,” blockchain and digital assets, cybersecurity, and more. (And click here for a complete roundup of our recent AI coverage.)  

What you’ll learn from this episode:

• How generative AI is like auto-complete.
• What a recent announcement has in store for ChatGPT.
• How AI has been around longer than you might think.
• The importance of CPAs partnering with technologists to vet AI algorithms.
• The different levels of automation and the best opportunities for CPAs.

Play the episode below or read the edited transcript:

— To comment on this episode or to suggest an idea for another episode, contact Jeff Drew at Jeff.Drew@aicpa-cima.com.

Transcript

Neil Amato: Welcome back to the Journal of Accountancy podcast. This is your host, Neil Amato. What you’re about to hear is a special two-part episode, the 2024 JofA Accounting Technology Roundtable. Jeff Drew, editor-in-chief of the JofA, is the instigator in all this. Thirteen years ago, he began gathering experts in accounting tech to discuss the trends, maybe even the gadgets, and the knowledge that accountants can put to use. Here to introduce the panelists for this year’s roundtable, Jeff Drew.

Jeff Drew: Thank you, Neil. I am happy to bring back the experts featured in last year’s tech roundtable: Donny Shimamoto, founder and managing director of IntrapriseTechKnowlogies and founder of the Center for Accounting Transformation; Wesley Hartman, founder of robotic process automation developer Automata Practice Development and co-author of the JofA’s most popular column, Technology Q&A. Amanda Wilkie, a consultant with Boomer Consulting and one of the JofA’s editorial advisers. Welcome back, everyone.

As Neil mentioned, the plan is to create two podcast episodes out of this conversation with part one focused on AI and automation and part two delving into a variety of topics including cybersecurity, practice management systems, Web 3.0, blockchain, might even get some research results, maybe 4K monitors, all kinds of stuff. But we’ll start with what we’re contractually obligated to do with podcasts that talk about technology these days, we’ll start with generative or GenAI. Wes, can you remind our listeners what GenAI is and how it differs from the other types of AI?

Wesley Hartman: I think I have a slightly interesting, different opinion. What we have as generative or GenAI, I actually don’t consider AI, at least how we define it in pop culture references. A generative AI, it’s the way I describe it’s really just a very fantastic computer algorithm that has the internet of data. Then it functions like an auto-complete, so it takes what your prompt is and uses that information, and it builds the auto-complete just like if you’re typing a text message, trying to guess what is the next word. It just does that on a data scale.

Now I’m very much condensing it, because obviously there’s a lot more details to it, but that’s generative AI versus what the next levels are, which is artificial general intelligence. That’s where the computer the AI is actually learning. Right now, generative AI just bases its responses from the internet, which is still data that’s generated by people.

But artificial general intelligence can actually create new. Then the final level — that’s where we’re getting into like Skynet territory and stuff like that — is advanced artificial intelligence or super artificial intelligence, where in some ways it might become a little, not very, a not as well distinguishable from a human in some ways. But right now, those last two levels are theoretical. Generative AI is what we’ve got right now.

Drew: And ChatGPT is the brand name that’s out there for GenAI. It’s almost used interchangeably like Kleenex or Xerox. When we talked about ChatGPT last year, it was only a few months old. Amanda, you described it as not mature at all, and you compared it to Wikipedia. A year later, which is like a full generation at least, what is your assessment of ChatGPT and the GenAI space?

Amanda Wilkie: I think we have to provide a little context. I did equate it to something like Wikipedia this time last year, but in the way that I wouldn’t recommend that you really take it as anything that comes out of it as authoritative. Because it was trained on public information, there was a lot of errors or when it makes an error, when it makes things up, it’s referred to as a hallucination.

I still don’t think that the public ChatGPT that just anyone can use is really authoritative. But wow, some of the things that have happened in the last 12 months, like you said, Jeff, I think people put time, effort, a lot of capital into generative AI. We’re starting to see a lot of different uses for it. I mean, even OpenAI, which is the founding company, the owner of ChatGPT. They have partnered with Microsoft to create tools where private companies can create their own GPTs, Azure OpenAI services.

Companies are able to create their own ChatGPT-like technologies. They have recently announced that they are creating a ChatGPT store. So companies, individuals can build their own generative AI applications, if you will. They’re going to create this entire marketplace for them. Think about the Apple App Store or the Google Store, where individuals or companies have created their own applications. Now consumers can actually use those. That’s the direction that this technology is headed this year.

I think that’s going to open a lot of doors. And think about, as a firm, being able to monetize your ChatGPT. An example is if a company wants to create a ChatGPT or chat bot, if you will, on how to learn or how to teach mathematics, then they can sell it in that store. Well, what if a firm actually develops something or one of our solution providers, vendors in this space, actually creates a ChatGPT to train and educate on how to do a tax return or how to actually look at financial statements, things like that? Then they’ll be able to actually capitalize on that.

I think there’s been a lot of advancements in the last year, and there’s a lot of opportunity, and there’s just going to be more and more to come for firms out there.

Drew: Donny, in preparation for this, you mentioned that you don’t think there’s been a ton of direct generative AI, large-language-model adoption by accounting firms and that you believe most of the AI firms will adopt will be embedded in other apps. Can you expand on that and how that might work in conjunction with some of the stuff Amanda talked about?

Donny Shimamoto: Sure. Actually, Wes, I gotta say, I like your definition there. That is actually a really good example of how these large language models actually work or are utilized. Amanda actually set it up for me because if you listen to what Amanda was saying, she’s basically saying someone else is going to provide you some functionality. Like, a vendor is going to provide you some functionality that you’re going to utilize.

That’s really what I’ve been seeing a lot, and that’s what I’ve been telling most of — particularly accountants in small, even through large and I’m going to qualify large versus very large firms — that the development that I’m seeing, even the very large firms, including the Big Four, they’re still all in their infancy in actually developing from-scratch-type AI tools.

What I’m seeing there, too, is that a lot of them as Amanda described are actually layering on. I agree completely with Amanda. We’re going to see this App Store type of concept start to come up around these. But the majority, and actually, I don’t know if a lot of accountants realize that we’ve been using AI a lot. Quickbooks has had it in its account reconciliation feature for I feel like it’s 10 years. Maybe it’s eight or seven, but it’s been a long time that they’ve already been utilizing that. Actually, I know that it’s more than seven or eight, because Botkeeper has been around for seven years.

And Botkeeper, I was talking with their CEO, Enrico [Palmerino], and he said they finally, after seven years, have had I feel like they have maturity in their models. And here’s one of the risks with these models, especially if you’re doing it from scratch. So in that conversation I had with Enrico, he was saying, one of the things that they were worried about is that as more data comes in and if it’s not clean data or accurate data, that it’s actually going to, I forget the official term for it, but it breaks the model. It actually causes it to predict the wrong thing.

This is tied back to that whole concept that you mentioned earlier, which is, well, if when it’s a feed off of the internet and how much do we trust the things from the internet to be correct, right? Somebody has to police that. Enrico was telling me that it took them seven years to get to the point where they feel like, even if some bad data comes in, it’s not going to change the model. That reliability takes a long time to get there.

Wilkie: Donny, I think that is a huge point, that firms really need to pay attention to their data. If they think they’re going to be innovative, if they think that they can build something and that they can capitalize on these innovations, then they really need to be paying attention to their data and their data hygiene. A few years ago, every firm wanted to create kind of like a pizza-tracking app so that their clients can track their tax returns. A lot of firms said, “We need to build that app.” I fear that firms are going to say, here’s a great opportunity. Let’s build some sort of generative AI and then we can use it for our clients or we can resell it or we can do whatever. If they are not paying attention to the data and the data hygiene that they are training this AI on, then it’s going to be a mess.

Hartman: With that, there was, I don’t want to call it a case, but there was a thing, and I think the term they call it, purposely, is poisoning. They took an art generative AI model. Basically, I think they fed it pictures of a cat but said it was a dog. I think they fed it like 10 or 15 photos, and it basically messed up the entire model like it could no longer identify cats versus dogs and stuff like that. That’s really reinforcing, Amanda and Donny, that the data is key. But with these computer programs, you think of them like automations, right? You’re feeding it data. It’s automating to build a structure, but if you give it some incorrect data very quickly, it can be wrong.

Shimamoto: Actually, Wes, that’s a great example, and I can go back, tie it back to your original statement, which is think about auto-correct and the times when you’ve typed in something and you’ve misspelled it in auto-correct, and then it suggests it back to you and you’re like, “That’s not a word. That was a typo.” And then you’ve got to delete it from the auto-correct. That’s actually a super simplistic version of exactly an example of the risk servicing that we just discussed.

Drew: That almost seems like that’s going to create another skill set or knowledge base that accountants are going to need to develop and gain an understanding of is, how do they assess the AI model that they might be using and how do they vet or figure out the quality of the data?

Shimamoto: I actually think this is where especially people like Wes and Amanda come into play. It’s not about the accountant itself. Like a lot of people know that I’m dual trained on both sides, but I don’t get into that depth. I rely upon my technologists, and I think it’s what we’re going to see is that you need to find the right one. It’s not just any IT person that’s going to be able to do this. It needs to be someone that’s trained in this.

As accountants, we need to understand the bigger picture of the risks. We need to understand how we do policy. A lot of this comes back to data governance, which is actually tied to corporate governance, but those are things that when we get into data governance and the risks and cybersecurity gets tied into it as well, we need to partner with IT to really make this stuff happen.

Wilkie: I absolutely agree, Donny. I think we’re going to see even more of a focus on that partnership between the CPA and the technologists out there. It’s going to be very important. There’s also going to be opportunity in this space, for auditors and auditing and reassuring people that the technology is doing what it is supposed to be doing, what it says it’s going to be doing. Because some of those models, the foundational models, you’re not going to be able to peel back the layers of that. That’s proprietary. It’s going to be hard to get in there. It’s going to be hard to understand it.

We’re going to have to use the tools and techniques that the profession has been using for a long time. Like you said, Donny, governance, IT controls, understanding and auditing the data that’s going in and understanding and auditing the data and the information that’s coming out. These again aren’t really new concepts, but we’re going to be applying them in new ways when it comes to AI and auditing AI and trusting the AI.

Shimamoto: Oh my God, Amanda, you are a woman after my heart, talking all this IT audit stuff.

Hartman: Well, it’s funny because I always just think about the irony of it. They call themselves OpenAI, but they are very not open. And I understand they’re a business and they’re trying to set stuff up, but, you know, the descriptor that they’re using. But, with all this discussion also going back to what accounting firms would implement and what the CPA versus the technology partner coming in, I also would like to tell people that, like every other previous technology before, this is not a silver bullet that’s going to solve every single problem that a firm has. It’s like, “Oh well, let’s just implement AI, and that will fix everything.” No. I mean, you go back to, well, let’s just put in computers and that’ll fix everything. No, if anything, we just made more problems because, well, if computers can handle it, we can make things more complicated.

With the implementation of AI, like with some of the firms I’ve talked to, there’s like ideas that they haven’t. It’s maybe like little use cases, like, can it summarize this document of minutes for me because they have to do minutes every month or something like that. Things like that, where they’re finding these little use cases. But to Donny’s point and Amanda’s reinforcement, it’s vendors that are really going to be driving because they are the technologists that can understand how this technology works, but going back to that audit and verification, the CPAs, the accountants, EAs, all that. You’ve got to verify what’s coming out. You cannot trust it 100%.

Drew: You have the generative AI use cases, which can be everything from building code, and in Technology Q&A, we’ve got some stuff that’s been running that shows how to do that or how to integrate ChatGPT and use it. So, that’s an interesting use case, but, in terms of helping, especially firms, deal with capacity issues and stuff, you know, AI gets tied in with automation a lot, but not all automation is AI, and there’s lower-level opportunities out there.

So Wes, you have jumped full time into the automation space with your company. I was hoping you could talk about what are the best automation opportunities for CPAs in firms, or in industry or government, etc., and what are some steps to get started?

Hartman: Thanks. I’m doing the automation thing full time now. That’s a fun experience. I tend to divide automation really into three tiers essentially.

The first tier is I always tell people, let’s look at your existing technology because you’d be amazed at how many automation tools, like just setting up templates for workflows or invoicing, all that stuff that firms don’t use. So, automatic notifications, things like that. Let’s take a first look at what technology you’re utilizing and see if there are built-in automation tools that you can leverage to automate parts of your firm because it’s one of those things where even something small that maybe takes one minute, but if you’re doing it on every single return or every single step of your audit process, those minutes add up, and that’s I think how you can reclaim some capacity to your firm. It’s the first tier.

Second tier is where you’re looking at Zapier or Make.com, Pipedream, where you can build your own automations between different systems or even within systems. With the way those work is that they connect up to those softwares on the back end. You can then create basically workflows in Zapier to, let’s say, for example, I built one myself because it was getting frustrating to take Calendly information and put it into my CRM system. I was manually doing it because I wasn’t really getting a huge number of calls. I was getting more to the point like I’m wasting lots of time, so I spent a little time and built that, and it’s automated now.

Then the last tier of automation really is more of a custom software development situation, where the process that you’re trying to automate has — maybe, it’s too complex, has too many different systems. Maybe it doesn’t have connections on Zapier or Make, and you have to do an RPA where you log in through the front end. It still has to happen. That’s the third tier. Usually a bit more rare. Most of the time, accounting firms start with your software stack. After that, look at Zapier, and then after that, if you want something more customized, then you look at a developer.

Wilkie: Wesley, I would add to that. In its purest form, automation is a computer doing something that a human would normally do, if we just make it very simple. You mentioned automating some of the tasks that can save one or two seconds, one or two minutes per deliverable, per tax return, or something like that. We talked about the importance of data hygiene, especially if we’re going to automate, if we’re going to use AI. When I talk to firms a lot, there’s a lot of opportunity around simple integrating, like you said, integrating their tech stack.

When I talk to firms, I tell them if you’re entering the same information in more than one system, you are creating an opportunity for that to be forgotten, for it to be keyed in incorrectly, for your data to get out of sync. Simple automation is just creating that data integration so that you have one source of truth. Then that system actually sends that information and makes those updates to all of your other systems. Again, when I talk to firms, especially small firms, who say that they are frustrated because none of their systems talk to each other, that’s a great place to start for some simple automations.

Drew: Thank you for listening to this special episode of the Journal of Accountancy podcast, featuring Part 1 of the 2024 JofA Accounting Technology Roundtable. An edited transcript of the roundtable will appear also in two parts in the April and May issues of the JofA. You can access the monthly JofA flipbook by going to journalofaccountancy.com/issues. Or you can go to journalofaccountancy.com and click on the “Magazine” link under the search box.

If you are not yet following the JofA podcasts, you can do so on Apple Podcasts, Spotify, or pretty much anywhere you can get podcasts. If you are following us, thank you so much, and tell a friend.