What one AI can teach us about another

When it comes to relying on AI to build financial models, be careful – it’s about practice.

There’s probably no topic the trainers are asked about more often in Pivotal180 training courses than AI:

Do I use AI tools for modeling?

When will AI be able to replace analysts?

What’s the point of learning all this if AI can do it for me?

When the students asking these questions refer to AI, they’re obviously talking about Artificial Intelligence. We can debate whether or not current and emerging tools actually qualify as “intelligent,” but there’s no denying their impact – sometimes useful, sometimes less so – on many areas of work, including finance and modeling.

But to a specific subset of people – in particular, late-90s to early-2000s Philadelphia sports and NBA fans – AI will always mean something else first. And no matter how many times I answer questions about why learning, practicing and understanding financial modeling will always remain relevant, I think the original AI put it most eloquently:

For those that don’t know him (do not admit this in my class), that’s 2001 Most Valuable Player, 1997 Rookie of the Year, 11-time All Star, Hall of Famer and NBA 75th Anniversary Team honoree Allen Iverson. Nicknamed “AI”, Iverson is often remembered for an infamous press conference after the 2001-2002 season. Criticized by coach Larry Brown for missing team practices, Iverson responded with a rant in which he repeated “we talking about practice” so many times that the phrase became a meme before memes were a thing.

When it comes to modern AI, however, the O.G. had a point. Let me try to explain by answering some of those questions from the beginning.

Does Pivotal180 use AI tools for modeling?

The Pivotal180 team has started to use some AI tools in our day-to-day work, and we’ll almost certainly expand our use in the months and years to come. It’s important, though, to recognize what AI is good at, and what should still be left to humans.

AI has gotten pretty good at repetitive tasks – think calculating solar energy generation for 30 years, or building out dozens of operating expense lines. A model can’t begin to work without these components, which can take several hours to build, but they don’t require much deep thinking for experienced modelers. We might also ask AI to review and summarize someone else’s model as a jumping off point for a more comprehensive model check or audit.

The key words in that last paragraph were “experienced modelers.” Thanks to decades of building and working with project finance models, our team can use AI to enhance our work without relying on it to do everything. When the work requires something more complicated, that’s where our human experience comes in.

On the other hand, analysts and other new modelers must master modeling skills and financial concepts themselves before outsourcing to the machines. As discussed further on, AI may make mistakes that require thorough understanding of modeling and financial concepts to catch. Only by putting in the practice can you learn how to use AI responsibly in model building and review.

No matter your experience level, I do think it’s a good idea – at least from time to time – to do some hands-on modeling yourself. Financial modeling, like any skill, has to be practiced regularly. Former analysts now in management and leadership roles know what I mean.

Whenever I don’t do any new model building for a while (training schedules sometimes don’t leave much time), I can find myself struggling to get back into the groove of my next modelling assignment. Eventually I’ll find my way back to familiarity, but the Excel muscle memory really can atrophy without regular exercise. To quote Andy Stitzer: if you don’t use it, you lose it.

Now, you may be thinking: if analysts need experience and practice to safely use AI for modeling, let’s just have AI build our models and get experienced people to check them. That brings me to my next question.

When will AI be able to replace analysts?

My short answer is: never.

My longer answer is: never, for the smart firms.

When I’m asked this particular question, I mostly respond with one of my own: Where do senior bankers – or those “experienced modelers” I mentioned – come from? Unfortunately, there is no Managing Director Stork.

There are very few ways to gain the knowledge and experience necessary to excel (pun intended) in a senior finance role as effective as working one’s way up through the junior ranks. Behind all the meetings and pitch decks, good Directors and MDs have the analytical chops to understand what really matters in complex models and transactions. The same goes for senior leaders at project owners, developers and advisors.

And how did they hone those skills? By wading neck deep in models for years themselves – by putting in the practice. And speaking of that…

What’s the point of learning all this if AI can do it for me?

Last year, I taught a class for group of interns, all current university students. Each of them was clearly bright, but whenever the values in their training models didn’t align with mine, I heard a familiar refrain:

“Mine’s not working.”

After many many repetitions of that line – blaming model errors on their computer – the problem dawned on me: they were expecting Excel to do the work for them!

I’d heard stories like this from teachers and professors struggling with how to deal with AI in the classroom, but it was the first time I’d seen it myself. It was striking for two reasons: one obvious, and the other not so much.

The obvious problem: AI is often wrong! In the last year, AI tools have told me that Bob Dylan weighed less than many newborns and that Disney teenage heroine Moana is seven-and-a-half feet tall. (I have a 4 year-old.)

Yes, LLM tools are getting better, and these sorts of mistakes should (at least according to AI companies) reduce over time. Still, financial models are the main decision-making tools for multi-hundred-million-dollar investments. Just one small error in the wrong place can easily throw results off by tens of millions of dollars or more, especially when running sensitivities.

I recall seeing one AI modeling tool designer bragging on LinkedIn about how his tool could build a model with “over 80% accuracy.” Is a 20% wrong model one you would trust with your job? What about with your money?

Microsoft themselves specifically advises against using their Copilot AI tool in Excel “for any task requiring accuracy or reproducibility” or for “tasks with legal, regulatory or compliance implications” – not that a project finance model might ever fall into those categories. (I’ll admit I laughed out loud in the middle of a class when I read that.)

But the deeper issue gets at the core of what financial modeling is about, and what Pivotal180 courses really focus on. The goal of building a model is not to build the model; the goal is to use the model to analyze a project or investment and inform smart decision-making!

Even if your AI tool of choice did build you a “perfect,” error-free model, major investment decisions are about more than a single IRR, NPV or MOIC. If you don’t understand a model’s underlying logic and key drivers, you can’t possibly assess scenarios and risks at the level needed to make an investment decision.

Nearly half a century ago, no lesser a computing authority than IBM recognized that, “A computer can never be held accountable. Therefore a computer must never make a management decision.” When an unreviewed, AI-built model leads to lost millions, it won’t be the AI agent that takes the fall.

There are only two ways to truly understand a model and know that you can trust it – build it yourself, or audit it meticulously, top to bottom. Nothing beats the former, but either one requires – you guessed it – practice.

So you see: No matter how good AI gets, knowing how to build and analyze financial models will always be a critical skill. And how do you get good at that?

As AI’s contributions to model building increase, it’s more important now than ever to deeply understand financial concepts and know what to look for when reviewing and analyzing models. That’s also what sets Pivotal180 apart: we don’t just teach Excel coding, we teach the concepts required to understand deals, from finance theory to contract structures and risk mitigation.

Pivotal180 Training

Pivotal180’s project finance modeling courses are designed to equip participants with the knowledge and skills to build, analyze and communicate clearly about project finance models, so you can practice what matters most:

• Renewable Energy Project Finance Modelling
• Intro to Project Finance Modelling

Course participants will learn how risk is allocated between lenders and sponsors, understand the structure and drivers of project finance transactions, and gain the necessary skills to run and evaluate operational or financing scenarios required to identify the most substantial risks and opportunities for any deal.

Find all of our upcoming and available courses, including Advanced Debt and Battery Storage programs, HERE. Contact [email protected] for more info and pricing.

Come model with us!