Posted by Betelbuddy 2 hours ago
The task of auditing a business is extremely complicated by virtue of data/information being all over the place. AI had not gotten to a point of where it could outperform humans but in this experiment they find out a group of agents are good enough for this task. I still think there are so many poorly defined tasks that auditors just understand implicitly, eg data discovery; before the audit even starts, that humans will always be needed to initiate and conclude audits just like software projects.
I expect for 80% of the audits AI will be 'good enough.' And those are the bread and butter for the public accounting firms.
And it's true.
We're getting to a point where you don't even have to look at the code anymore, and I'm serious.
> AI had not gotten to a point of where it could outperform humans but in this experiment they find out a group of agents are good enough for this task
The more this is done, the better it gets. Especially as more resources are poured into making it better.
It's only a matter of time before AI auditing is the common practice and humans are the backup for the AI.
And every year a new human comes along and has to learn all of it again.
Of course ONE AI agent and teaching ONE AI is massivly in an advantage as soon as its as good as the avg person in the job and it blows everyone else out of the water as soon as it gets a tag better.
[1]: https://www.who.int/teams/integrated-health-services/patient... [2]: https://en.wikipedia.org/wiki/Battleshort
It’s literally a monoculture.
For example, if an iteration of the singular AI ends up being crashable with a magic visual pattern or QR code (highly likely at some point), that is a major national security risk if that AI is piloting aircraft, as the entire nations aircraft could be taken down by anyone projecting said image in it’s visual plane until the fix gets pushed.
And these systems are far too complicated to exhaustively test all possible inputs before release.
There is a reason why nature rarely has such monocultures - that live for very long, anyway.
Its a possible as an attack vector for sure.
But in comparision to nature, compute can speedrun all of this with RL.
Even a lot of what software professionals do isn't verifiable.
On the other hand, with AI we'll use reinforcement learning to create agents without such evolved-in tendencies. Where (so long as we can solve alignment and cheating) they are trained to do the job, no matter how much effort it takes, or what's involved.
They are less lazy than humans though
AI labs were working with companies like Mercor for more than a year paying the consultants and accountants themselves from these very firms a good amount per typical task they work on everyday, some even providing actual documents.
The expectations were more rigorous, and the rubrics pretty expansive. so yeah, the agent trained from tens of thousands of consultants would eventually ask better questions than an individual consultant.
I think this is the source FT article
But they can also fail to ask "obvious" questions and get stuck at points that an average person wouldn't be stuck in.
Which makes them for adversarial processes - like auditing or hacking. But no so good at producing defect-free software or planning without guidance. With guidance, they're great aids in this too. But when people AI processes to "turn off their brains" or act autonomously, then you get problems. And you get a lot of pressure to very lightly if at all scan AI output when AI investment is supposed to show productivity gains.
There are many downsides to this in that business cannot ever be perfect and a sample-based regime is useful because it allows for a few mistakes here and there it's not waste everyone's time at audit time
So similar to cybersecurity where you're going to need to apply AI offensively and defensively I suspect we'll see the same in accounting
But even moving from sampling to an exhaustive analysis, the time-wasting mistakes could still be rejected as non-material (i.e. too small to care about). If it wasn't possible to automate this, I doubt an exhaustive approach would take off.
The very first time I ever worked with a McKinsey consultant, he turned out to be a jr. consultant half year into his gig at McKinsey. In all likelihood a really sharp and ambitious person, but extremely junior when compared to the domain experts (with decades of experience) in the same room.