In college, I surrounded myself with students who could do things I could not. One friend was working in territory that most mathematicians never enter, ergodic theory, and what are called measure-preserving group actions. Over one Christmas holiday I asked her to describe what she was studying, she offered an image that has stayed with me ever since. She said her work was something like describing the spatial pattern after leaves have fallen not the individual leaf, not the wind that moved it, but the underlying structure that governs how the whole system settles.
Her soon to be husband, a childhood friend, became a Boeing engineer focused on wind tunnel research. Other friends brought the same precision to mechanical and aerospace engineering and taught me the ubiquity of mathematics at the very heart of nature.
I could not do their mathematics. I want to be clear about that. But I could follow the logic. I could grasp the concepts well enough to understand what they were doing and why it mattered. And over decades in corporate banking, structured finance and capital markets building and advising institutions on complex risk - that turned out to be a genuinely useful skill. You do not need to derive the formula to know when someone is using it incorrectly, or when a model's assumptions don't match the world it's being applied to.
I have been thinking about those friendships a great deal lately, as I watch bank boards and management try to get their arms around artificial intelligence.
The question facing every bank director today is not whether they can build an AI model. It is whether they understand these systems well enough to govern them and to know when something is going wrong.
The gap that nobody is naming
There is an enormous amount of content being produced right now about AI and banking. Most of it falls into one of two categories: breathless enthusiasm from vendors who want to sell something, or academic rigor that requires a computer science degree to parse. Neither serves a bank director trying to fulfill their actual fiduciary responsibility.
The gap is a practical framework for oversight - one that gives directors enough conceptual grounding to ask the right questions, recognize evasive answers, and hold management accountable for AI risk in the same way they hold them accountable for credit risk or interest rate risk.
That is what this series is designed to provide. But before we can govern something, we need an honest account of what it actually is.
What these systems actually are - and why that matters for governance
Here is the thing about AI that most board-level presentations carefully avoid saying plainly: at its core, every AI system operates in a realm fundamentally alien to human cognition.
All algorithms, no matter how sophisticated, are ultimately compiled into machine language streams of binary 1s and 0s manipulating transistors at the hardware level. There is no literal "showing of work" in the way humans understand it. What we perceive as reasoning or explanation is a high-level abstraction layered atop billions of mathematical operations. Consider Tesla's Full Self-Driving system: it converts raw camera pixels into numerical arrays, maps them through deep neural networks for pattern recognition, and produces steering commands - without a single explicit if-then rule guiding it. The car does not think. It computes, at extraordinary speed and scale, across billions of statistical associations learned from training data.
Modern AI achieves its capabilities through emergent statistical associations rather than transparent symbolic logic. The impressive outputs we witness fluent language, accurate image recognition, complex financial analysis - are the result of vast distributed computations that remain, at their deepest level, an opaque yet extraordinarily effective abstraction.
This is where the image of the fallen leaves becomes surprisingly precise. That branch of mathematics studied not the individual leaf not the single data point, not the individual computation but the underlying structure governing how the whole system settles into its pattern. That is almost exactly what a well-trained AI model does: it finds the deep structural pattern in the data, the shape of how things tend to fall, and uses that pattern to make predictions about new situations. What neither the mathematics nor an AI model will give you is a simple, human-readable account of why the leaves landed where they did. The pattern is real. The explanation, in any conventional sense, is not available.
For a bank board, this is not merely a philosophical observation. It is a governance problem of the first order. You are being asked to oversee systems that your most technically sophisticated employees cannot fully explain and that will make consequential decisions about credit, fraud, customer service, and regulatory compliance. The question is not whether to use these systems. Competitive pressure will make that decision for most institutions. The question is how to govern something you cannot fully see inside.
What the wind tunnel teaches us
The aerospace analogy is clarifying. A wind tunnel tests how a design performs under controlled, simulated conditions and every serious engineer knows that simulation is not the same as the real atmosphere. AI models are trained on historical data, which is a kind of wind tunnel. They perform well within their training envelope. The failure modes emerge when they encounter conditions that were not well-represented in that training data - what researchers call distributional shift, and what a risk officer might simply call tail risk.
The March 2020 credit markets, and 2023 liquidity crunch, were not well-represented in most models' training data. Neither was the 2008 structured product collapse. When the world moves outside the envelope, models built on historical patterns fail - sometimes quietly, sometimes catastrophically. AI systems are subject to exactly the same dynamic, at greater speed and often with less visible warning.
The posture that boards need to adopt
I did not leave OU understanding ergodic theory, entropy, or fluid dynamics. But I left with something that has proven more durable: the confidence to engage seriously with technical people, ask questions that cut to the assumptions underlying their work, and recognize the difference between genuine rigor and sophisticated-sounding hand-waving.
That is precisely the posture bank directors need to develop toward AI and quickly. The institutions that get this right will not be the ones whose boards became AI experts. They will be the ones whose boards became expert at governing AI: asking the right questions, demanding honest answers, and ensuring that the humans accountable for these systems are actually held accountable.
You do not need to understand why the leaves fell where they did. You need to understand that nobody else fully does either and build your governance framework accordingly.
The next article in this series will map the specific ways AI systems fail onto risk frameworks that bank directors already know and name the questions every audit and risk committee should be asking right now
