Voice-over: Welcome to the Leaders in Finance Compliance Podcast. We talk with experts, advisers and business leaders about navigating risk and compliance challenges within the financial sector. Because the straight and narrow path doesn’t come with GPS.
We’d like to thank our partners: Deloitte, Rabobank, Cense and Osborne Clarke. Your host is Jeroen Broekema.
Jeroen: Welcome, listeners, to a new episode of the Leaders in Finance Compliance Podcast. I’m very happy you’re tuning in, and I’m very much looking forward to today’s recording. We’re going to talk about the human in the loop.
We’ll tell you more about what we actually mean by that in a few minutes. But first of all, I’d like to welcome my guests here at my table, my podcast table. Joining us today is Natacha Lord, who is Global Head of Model Risk Management at Rabobank. Welcome to the Leaders in Finance Compliance Podcast.
Natacha: Thank you, Jeroen.
Jeroen: Great to have you. And next to you is Lotte Aelvoet, who is a partner at Deloitte. Welcome as well.
Lotte: Thank you.
Jeroen: Great to have you both here. Before we get going on the topic of the human in the loop, I’d love to learn a little bit more about you. Maybe you can elaborate a little, from a business perspective, on what you actually do, or perhaps what you have done.
And if you’d like to add something a bit more personal, that would be great as well, because it’s always nice to get to know you better. Natacha, maybe you can go first and elaborate a little.
Natacha: Thanks, Jeroen. In my day-to-day job, I look at two angles. One is really about establishing the guardrails to ensure that the models used in the organisation are fit for purpose. We’ll elaborate on that a bit further when it comes to the human in the loop later.
The other side is: those guardrails are there, but when we build models and deploy them, are we still within the risk appetite we want as an organisation? Are the models still fit for purpose? That is more on the validation side.
In terms of my professional background, I have more than 15 years of experience in risk analytics and data analytics. What’s a bit fun about it is that this is my first time working more on the governance side. Earlier in my career, I was more on the innovation side, really building models and bringing in the best models to solve business or risk management challenges.
I think that also explains a bit about how I approach governance. For me, the key point is: are you putting governance in place that strikes the right balance, enabling innovation while doing so responsibly? That becomes more and more important when it comes to AI.
Then, on the personal side — and I’ll give room to my peer here — I come from Madagascar. So maybe I’m your first podcast guest from Madagascar.
Jeroen: I’m sure you are. Or maybe people haven’t told me. How long have you been away from Madagascar?
Natacha: More than 20 years now.
Jeroen: I had a short conversation with you before, and you also told me about your studies. I’d love to hear about that as well.
Natacha: I love maths, and it loves me back. It’s funny if my husband hears this! I started enjoying maths at a young age.
I did my BSc in Madagascar, and that took me to South Africa for a master’s degree. Later on, I went on to do a PhD in maths and computer science.
Jeroen: Something I think a lot of people would never be able to do, including myself.
Natacha: I think it’s just different skills that complement each other, which will also become more and more important for AI.
Jeroen: Very true. Over to you, Lotte. Could you maybe do something similar?
Lotte: Yes, sure. I’ve been working for Deloitte for four years now, focusing on risk and compliance, especially non-financial risk. We work for banks, payment institutions and crypto institutions, looking at their risk and compliance functions and processes, and helping them demonstrate compliance and improve their effectiveness.
For instance, when a letter comes in from the regulator and they need help, they call us. Before joining Deloitte, I’d always worked within banks, and I also worked very briefly at a PSP. That was also in compliance, mainly from a second-line perspective, but regulatory risk within the first line was also very much a focus area of mine.
Currently, I’m also looking at the innovation side of how we approach compliance and risk nowadays. That’s why this topic is very interesting to me. I see lots of opportunities from a knowledge graph perspective, for instance. That’s something we have a lot of conversations about with our clients.
That’s why I’m looking forward to this conversation. There are lots of opportunities, but there are also a lot of risks that accompany them. It’s interesting to see how companies go about that, so I’m very happy to hear from Natacha today.
Jeroen: It’s very easy to make assumptions in risk and compliance. It’s very tricky, but I assume you’re not from Madagascar! So perhaps you can share a personal note as well.
Lotte: I studied Arabic, which is also very different from what I’m currently doing. I started out studying Arabic. During my studies, I lived in Syria and Egypt, and I worked for Shell at the beginning of my career.
That was always my goal. Then the Arab revolution started, and I was the first one to get deported back to the Netherlands. I ended up in banking.
Something I was working on at Shell, while doing my master’s thesis, focused very much on how to get people at Shell to adhere to the rules. Safety is a very big thing for Shell, and that work focused very much on their culture.
Somehow, that was very much related to compliance, from a very different angle. I’m still very interested in human behaviour from that perspective. That’s also what still makes me enthusiastic about working with financial institutions.
Jeroen: Do you ever still use your Arabic training in some form or fashion?
Lotte: As a gimmick, but not in a professional way. To be honest, it’s not a language that’s super close to home. At university, you learn the language, but it’s also very much about politics.
And I did my master’s in Business Communication Studies, as it was called, so it wasn’t so much about all the different dialects. I can speak Fusha, Modern Standard Arabic, but it’s not very good anymore.
So, no, I don’t use it in my daily work, unfortunately.
Jeroen: Really nice. Thanks to both of you for sharing these things. Before we dive further into the topic, let’s go back to the basics. What do we mean by the human in the loop, Natacha?
Natacha: I think, in the early stages of AI, the idea behind the human in the loop was: where do we put the controls? Where do we need a human to review, override or change a decision that has been taken?
As I see AI evolve — if we take agents, for instance, and talk about the latest frontier AI models — the input is no longer just quantitative data or image data. It can also include the architecture of your IT system. So even your data is adaptive in terms of how and what it can do.
On the other side, the architecture used to be simple. Now, you work in a highly digitised environment, where one cloud can talk to another. If these systems see a way to connect to the outside world, they can also use that, because that is part of the model’s parameters as well.
Then there is the modelling itself, which used to be a function. First, we talked about statistics, then stochastic models, then machine learning. Now, we’re talking about a model that can write code and ask the machine to execute it. You can see the difference.
When you put all of that together, you have a dynamic system whose components have learned to talk to one another. The beauty of it, as Lotte already mentioned, is that it can do wonders. It can really improve our lives and make them easier and more efficient.
But there is another risk: we don’t always understand what it can do that we haven’t thought of. What are the limitations of what has been done?
That’s where I truly believe the human in the loop needs to evolve: really understanding and catching up a bit. What do we have here? Do we really understand the constraints we need to put around it so that we can master what it does?
On the other end, it’s about steering it more and more as we go forward, and balancing judgement. For me, that is another dimension of the human in the loop.
Jeroen: Where does the idea of the human in the loop mainly come from? Who is driving the message that we need to be careful with all the AI we have nowadays? Is it mainly driven by people like you? Is it driven at CEO level, or does it come from outside, from regulators? Where does it mainly come from?
Natacha: Let’s start internally. What I really like about the step Rabobank has taken is that we really embrace AI as a bank-wide transformation. With that, we work together with our peers in COO.
We also work with tech and, of course, with the business. At the end of the day, it should serve the business.
We also work with our colleagues in HR, because people are still key. There’s a lot of talk about people being replaced. Let’s not go there, because we have the human in the loop on the table. That would be another topic.
It’s already a big step that we work together, and it’s really from the top to the bottom. Yes, our Managing Board is very much engaged with the topic. It’s a strategic topic, and we take a balanced approach.
We need to innovate to stay relevant in society and serve our customers as well as possible. But that comes with responsibility. That’s how we approach it as a society.
You also hear a lot about this in the news now. People are worried. What is being done with their data? Who is looking at what is possible, and who is allowed to do what? From a government and regulatory perspective, I think we can still improve things for our society.
Jeroen: Lotte, you have the privilege of seeing a lot of financial institutions from the inside. Does what Natacha just described also reflect your perspective on who is driving this and where it comes from?
Lotte: Yes, I fully agree. We also spoke before this podcast, diving a bit deeper into our experiences. What I often see is that, for instance, when we come in with a great AI solution, the biggest issue is deployment.
You might ask: is that your biggest risk? I don’t think a lot of people really have a good understanding of where those risks are. That’s also why it’s so difficult to say where you need to put the human in the loop.
Do you put it at the requirements level, or do you put the human in the loop at the output level? But are we still able to test the output? A lot of the time, AI is way ahead. We all know that from a human perspective.
I think there is more and more awareness of the risks now. Before we came in, we were talking about a post by a former Anthropic employee. He mentioned that he’d been at the company for about three years and had just resigned.
He said something like: “I’ve looked at Anthropic and OpenAI, and they’re not acting responsibly. It’s really a threat.” I’m not going to quote him exactly, because then I’ll get it wrong. But he mentioned that, in a couple of years, they would completely take over, with catastrophic outcomes. It’s quite scary.
It’s super scary. And that post has something like 36 million views.
Jeroen: You must be right.
Lotte: Yes, the thing is, my husband mentioned it to me, and I was like: “Oh, yes, okay. This conspiracy.” He said: “No, look into it. It’s not a conspiracy. You should really look into it.”
It’s also a funny reaction on my part, because it’s a natural reaction. A lot of people do that because you don’t want to know how impactful it can be.
As we were discussing earlier, I think there is a big responsibility for financial institutions, because they have so much power, but also for the regulators around them. It calls for regulation, stricter regulation, and all I’m hearing is that it’s becoming less regulated instead of more.
Jeroen: When I listen carefully to you and try to get my head around this, there seems to be a moral reason why it’s important to have a human in the loop. Please correct me if I’m wrong. There’s a moral reason, but there’s also a reason relating to control.
You want to control the things that are active in your organisation, whether that’s people or AI. Then there’s also the regulatory angle. If I were a regulator, I’d want to know how these things work. There’s this explainability aspect: making sure you can still explain why you took certain decisions.
So, to summarise, there’s the control part, the regulatory part and the moral part. There are a lot of reasons why you’d want to have a human in the loop. Is that a summary that makes sense?
Natacha: I think it makes sense. If I can add something, at the end of the day, our lives are always about balance. But this time, things are moving really, really fast, and it goes beyond one organisation or even one nation.
If you look at the latest developments in frontier AI models, there was a decision about whether to release a model and who to release it to. There, you can also see an impact on sovereignty. It really calls on us as human beings to start collaborating for future generations.
It’s still a positive technological advancement that we’ve made. Let’s not forget that either. But we need to do it in a balanced way and for the good of society.
Jeroen: Are you implying that we’re not really balancing things right now? Is that what you’re saying? Everyone’s rushing to get all the potential out of it.
Although there’s a lot of debate about how much potential there actually is. Is it mainly cost-cutting for financial institutions right now, or is it really about growth? AI is a very different game, of course, from simply finding pockets of growth. But, coming back to the point, are we not balancing things enough?
Natacha: I think we haven’t started on a level playing field. Some are much more advanced and also have the capacity to lead in terms of the latest models or whatever they’re going to introduce.
But I think there’s still room to act. Let’s take Europe, for instance. Let’s be realistic: if you look at the GDP of the US versus Europe, are we really going to compete at the same pace and with the same power, given what’s already been done there? I’ll leave the audience to draw their own conclusions.
On the other hand, as we increasingly use this technology to serve our customers, or in hospitals to make diagnoses, for example, we should identify the minimum viable processes we need in an organisation.
There, we can really join hands as a European community to have a fallback solution that is in our hands and under our control. That relates to the control part you mentioned, the societal and moral part you rightly mentioned, and the regulatory and government aspects. That’s something I really wish we would develop further as a society.
The race for the best tool out there has been underway for a while. But let’s also have a fallback for European needs and the population, for instance.
Jeroen: Very practically speaking, Lotte, you work at one of the largest consulting firms in the world. Do you get more questions from financial institutions about how they can use AI for all kinds of purposes? Or do you get more questions like the ones we’re discussing now, related to risk?
Obviously, you’re in the risk and compliance field, but are the questions more about opportunities or more about how we’re going to control this?
Lotte: Definitely more about opportunities. That’s also one of my worries for the financial services sector: it’s very tempting to use AI from an efficiency perspective. If financial institutions start competing with each other on efficiency, things can move very quickly from a risk perspective.
If there are no rules of the game between financial institutions, they could establish those themselves and discuss them with each other. But that’s not very likely to happen. Otherwise, you’d expect the regulator to introduce those types of rules.
Without that, they’re going to compete with each other on AI solutions. From a customer protection perspective, that could be quite tricky.
Jeroen: Isn’t there enough regulation on this front already? There’s the AI Act, and there are AI-related provisions in all kinds of other regulations. Do we really need additional regulation, or is this already covered by the existing rules?
Lotte: I need to watch what I’m saying, because my answer probably isn’t going to be very popular. The thing with regulation is that I’m not a big fan of having more of it. Most of the time, I’m not.
But here, we need regulation that really reflects an understanding of the types of risks we’re getting into. I think the regulations we currently have don’t address those risks. So people or financial institutions aren’t prompted to look into them, because they need an incentive to do so.
You hope they do that from an integrity perspective, and I’m sure a lot of financial institutions are doing that. But ultimately, it’s something that should also be addressed from a regulatory perspective.
To be fair, I also understand that this is very, very difficult for regulators. We’re asking them to be the biggest human in the loop, while we’ve just discussed how you can fulfil that role if, at some point, we’re no longer smarter than AI. I think we’re already not. Everybody knows we’re not as smart as AI is.
Jeroen: And that gap might even increase. I don’t know.
Do you have examples of the risks? Very concrete examples of the kinds of risks we’re talking about if there’s no human in the loop? Again, I’m not saying it’s safe with a human in the loop, but what happens without one?
Natacha: Can I go back briefly to what we discussed earlier? I think something is still missing. A lot of people are looking at financial institutions, but they’re also consumers of the big solutions being put out there.
For me, what could really help is holding those big providers that build the solutions more accountable as well. I think that could change the game a bit.
Then, coming back to your question about a concrete example of the risks, let’s take a frontier AI model.
There was no human in the loop when the model was running on an architecture with a flaw that allowed it to open a firewall, connect to the outside world, look for vulnerabilities at another company and start attacking.
The idea was for the model to look at vulnerabilities in its own system. But when that was put into practice, it didn’t know there were certain constraints that needed to be in place.
To your point: if there had been a human involved, would we necessarily have stopped this? We don’t know that yet. But what might have helped is having a human in the loop during testing, looking at it step by step and releasing the model incrementally. That could have been a way of catching what was going on.
That brings me back to the first thing I said: it’s really all about understanding.
Things move so fast that the time human beings have to catch up becomes more and more important. Where we lose the plot a bit is in expecting one individual in a team or organisation to really master it.
With my model risk colleagues, I now use what I call an octopus team. What does that mean? One head, but lots of hands. I think that’s what we need to do, because that’s where the machine beats us.
It’s not that humans are stupid or aren’t smart. But if you take all the good things humans have — which is happening now — and feed them into a big machine that can identify trends and process information at a speed far beyond that of a human brain…
I can talk to you, but if Lotte talks at the same time, you’ll ask us: “Please stop for a moment, because I need to hear one of you.”
But that’s what the machine is doing. So, yes, we’re not better than it in that respect. But we still have judgement, creativity and the ability to connect the dots. That’s where the human in the loop can help.
Jeroen: Your example is quite extreme, right? I mean, it’s real, but it’s a very big example. Do you also have examples from your own organisation where you were concerned that things might get out of control?
Natacha: I wouldn’t say concerned, but perhaps it would be nice to share two very different examples with the audience.
We serve customers, and sometimes we don’t get things right. Customers then need to call, make a complaint or get support. One area where you really see AI having a big impact is the communication channel.
We now have quite mature data internally, both from our processes and from our communication with customers. That allows customers to connect with us quickly. The agent can really support them, and the customer is happy.
That’s relatively low risk, because you can always give the customer a choice at the end of the interaction: “Do you want to talk to a human being? Did we solve your problem?” So that’s one example.
Jeroen: Nothing to worry about there, right? That’s good.
Natacha: Nothing to worry about. That’s good. Then let’s look at a different example.
Jeroen, you come to us and want to apply for a loan. You want to expand the business a little. The podcast is doing fantastically, and you’ve found something new that interests you.
We already have your information in the system, so the agent can use that. We also know you as a customer, so we have more information.
Jeroen: My payment behaviour?
Natacha: Exactly. But you’re also a podcaster, so there’s data about you in the outside world.
Let’s say we don’t put a human in the loop at all. The agent collects all the internal and external information, enters it into the system, and the system decides whether to grant you the loan.
Jeroen: So you grant me €100,000. I’m happy. You’re not happy? Who’s unhappy here?
Natacha: You’re only one customer, but it can do this for…
Jeroen: But perhaps the regulator asks whether you can explain why you’ve given me too much credit, for example.
Natacha: True, but I included something extreme in my example: it can also collect data about you from the outside world. There isn’t necessarily someone checking the quality of that external information.
Someone might have said something really unpleasant about you, and we might refuse you a loan based on that. But it might not be true. It hasn’t been substantiated. There’s no ground truth behind it.
That’s where the human in the loop makes a difference: looking through the information and doing the due diligence.
Jeroen: It’s a great example, because it brings us back to your earlier point about balance.
On the one hand, as a customer, I like the idea of you using more information you can find, as long as it’s correct. You might then be better able to assess my creditworthiness and my ability to repay the loan.
That could strengthen the case for lending me the €100,000, whereas you might not have done so without using all the data you could find.
As a customer, I’d probably be quite happy with that, as long as I’ve ticked the box saying I’m okay with you doing it, and as long as the data is correct. So it’s all about data.
Natacha: It’s the data, and it’s transparency with you. As an institution, we have a duty to ask: “Jeroen, are you happy for us to use external data as well?”
Then we still need to do the due diligence on top of that. How much weight do we give that information? In exceptional cases, shouldn’t we call and talk to you, because this is your loan?
Of course, when there are no issues and it’s a well-known customer, there’s already a relationship. In those cases, we can also move faster.
Jeroen: It’s a great example, and it also shows very simply where the human should sit. When you’re creating this model, you want a human looking at the model that assesses my creditworthiness. But then there’s also a human involved in checking the output, I guess. So that makes the process a lot less efficient.
Natacha: Yes and no. Multiply this by the many customers an organisation serves, and you can already see where the efficiency comes from. Where we really know what we’re doing, the level of human involvement can be lower. Where we really need to focus, it can be higher. That’s where we gain efficiency while still serving the customer’s best interests.
So it’s also the scale that makes AI relevant or not. To your point, if it’s a small organisation, you need to rethink whether you really need AI.
Jeroen: Any examples from your side, Lotte, that are worth sharing and that you’re allowed to share?
Lotte: That’s what I was thinking about! But I think one of our biggest struggles — and this isn’t just Deloitte; I think it’s everywhere — is that we can keep telling people to check the content coming out of AI, but it’s human nature: the moment you feel the output is better than what you would have created yourself, people don’t check it.
Jeroen: You get a bit lazy. For example, a colleague creates an introduction for a podcast, and it’s a great introduction, but there are some errors in it. There’s information about a CEO that’s simply incorrect.
Lotte: Yes, I think that’s one of our biggest struggles. There’s also the use of AI tools that you’re not allowed to use. We’re very strict about that, so we build our own in-house solutions.
But for me, the biggest worry is always: are we actually still reading the output of AI? I think now is still the time to do that, while we’re also still able to spot the mistakes and errors. But as AI develops further, we may no longer even be able to see those mistakes. That’s where it becomes very tricky to be a human in the loop who actually adds value.
Jeroen: I’m always very concerned about no longer being able to explain why things are happening, especially in the modelling world.
A very concrete example, again from the credit world, is small loans to SMEs, to small business owners. You might discover that one indicator is particularly useful: when someone applies. Strong entrepreneurs don’t apply during the day because they’re busy with their work.
They might apply at eleven o’clock in the evening, and you find a correlation, or perhaps even causality, suggesting that someone who applies in the evening is more likely to repay the loan than someone who applies during the day.
Then you put in hundreds of indicators like that, and suddenly you no longer know why you granted the loan or why you didn’t. That’s very tricky. The decision might be completely right, but you can’t explain it anymore.
As a regulator, I’d be very nervous.
Natacha: Yes.
Jeroen: Does that make sense?
Natacha: It makes sense, yes. Sometimes I tell the team to go back to the foundations, to the basics. What you’re describing is a pitfall that’s very easy to fall into.
It’s not just in modelling. As you mentioned, it’s very easy to combine a lot of documents and ask: “Create something out of this for me.” Similarly, you can ask the model to build a model for you.
From there, you can look at the possible key drivers you could choose. You can use AI to identify a couple of hundred of them. Explaining a hundred is quite tough, but you can already use AI to look at which ones are correlated with each other.
Then you use the human brain to reduce the number to a level that’s more manageable and makes sense. That’s where the challenge for the new generation, and for managers, becomes increasingly important.
Fewer people are building models from scratch; instead, AI gives you a model. How do you relearn? That’s what you’re doing every day. You haven’t built the model yourself, so you look at how it was built from a different angle.
In a sense, some of this is already around us. But you need to look at it from a different angle and be that octopus team.
Jeroen: I’d like to talk about culture versus setting rules. What is this about? Is it about understanding how things work, setting the right rules and guardrails, and then getting on with it?
Or is it more about a culture in which we find it important that humans are involved and understand why we’re doing this? Is it one of the two, or somewhere in between?
Natacha: I think you’re right. At the end of the day, culture and behaviour are key. We have to trust our people.
Going back to the example of that tweet, it raises questions if your employees don’t believe in the product you’re developing.
Culture is still key. As your example also showed, culture really strengthens collaboration, learning and understanding, and that process keeps going.
Jeroen: Anything to add there, Lotte? I’d also like to ask: aren’t institutions afraid of killing innovation? There’s this saying that the US innovates while Europe regulates. That’s a bit extreme, of course, but is it sometimes a worry at C-level and below?
Lotte: First, I’d like to add that I fully agree that culture and behaviour are super important. But over the years, we’ve learned — and everybody knows this — that measuring culture and behaviour is really difficult. That’s why I feel it needs to be a combination.
You still need certain controls. Those controls can also address behaviour and culture, because you want to trust your people.
I think it even goes back to primary school. How are we raising and educating our children right now? Are we still teaching them the right things so that, at some point, they can contribute to a society in which AI plays such a big role?
I have my doubts. At my children’s primary school, that’s not happening, to be honest. That’s one of my worries.
We definitely have to rely a lot on people’s behaviour. Ultimately, it comes down to integrity, and that’s something we have that AI doesn’t.
Jeroen: Maybe we also need to teach AI to have a higher level of integrity.
Lotte: Yes, and I believe that’s very possible. But I also have my doubts, because whether AI takes integrity into account depends very much on the task it’s been given.
That’s what I think, but I’m not an AI expert, so it’s also difficult for me to judge.
Empathy, integrity and all those soft skills are what we should be good at. I believe those will always be things people are better at than AI, and that AI will prioritise the task over integrity or a soft skill.
Your other question was about innovation and C-level. Could you repeat it?
Jeroen: Innovation versus regulation and control. Those things don’t always go together very well, right?
If I were a CEO, I’d definitely want to explore all the opportunities AI brings. As Natacha mentioned earlier, let’s not forget those opportunities. Equally, I’d be worried about things getting out of control or getting into trouble, whether morally, from a regulatory perspective or otherwise. Or simply financially: we could lose a lot of money.
So there’s a lot of potential trouble, but I also wouldn’t want to kill innovation. It’s a very difficult balancing act again. Maybe we should have called this “The Balancing Podcast”.
Lotte: That’s completely true. It is very difficult, and I see a lot of hesitation among our clients.
Even when we come up with proposals that are very attractive and where very little can go wrong because we, as Deloitte, take on a lot of the risk, you still see that they’re hesitant and don’t proceed.
Yes, they want to innovate, and they are innovating, but there’s a high level of hesitation.
Jeroen: Which might be good in the light of this podcast, right?
Lotte: I should say no right now, of course! But I completely agree with you that it’s good. Yes, I believe that’s really good.
Jeroen: Again, it’s about balance. You don’t want to kill innovation, but equally…
Lotte: You don’t know. But I think waiting too long to form a clear view of your strategy and vision as a company can be detrimental. That’s a very Dutch way of putting it, I think, but it takes quite a while, I would say.
Natacha: Maybe we can look at it from a slightly different angle. What AI really forces us to do as an organisation is to stop thinking in silos and start thinking together. That’s where you see the difficulty.
Even the way regulation is structured reflects those separate areas: operational risk, IT, model risk, credit risk. Credit risk focuses more on the credit element and the customer journey. We’ve always been used to looking at how we do things within a particular box.
Now, we’re connecting everything very, very quickly. As leaders, we need to make that shift and really start looking at things from an organisation-wide perspective.
Jeroen: Which is hard. You have 48,000 people across the globe, right? I recently interviewed the CEO in New Zealand, which is physically very far away.
Perhaps they come up with an AI idea they want to try in New Zealand, and then people in Utrecht think: “Oh my God, what’s going on over there?”
Obviously, this isn’t a real example. I’m just saying it’s a large organisation. In a small organisation, it’s relatively easy to oversee things and connect people. But with so many people, most people in your organisation don’t know each other.
Natacha: Exactly. Then there’s also the question of what you outsource to consultants and what you keep in-house. Those are two sides of it as well.
That’s probably another two podcasts for you or someone else to discuss! But I think that’s the key shift we all need to go through now.
Jeroen: Let’s work towards a wrap-up. I’d like to do two things. First, some key conclusions, if we can draw them from what we’ve discussed over the last forty minutes or so.
Then perhaps a few final thoughts, or things we’ve missed or should have mentioned. But let’s first hear from both of you about some key conclusions or summary points.
Lotte: We’ve talked a lot about balance. I don’t think we know the right balance yet, and that’s the biggest struggle.
We want to use AI for efficiency, but also for effectiveness, by the way. It’s not just about efficiency. At the same time, we want to be in control.
The difficulty is: how are we going to stay in control? How should the human in the loop help us do that? Is it from an input perspective or an output perspective? Are we still able to judge AI’s output? I think that’s one of the conclusions around balance.
Natacha: I won’t go into balance again, but I’ll build on that. When you have an ecosystem moving so fast in so many different directions, something that can help is what we call adaptivity.
As leaders, organisations and teams, we need to be more flexible, accept the unknown and the uncertainty, and move forward step by step. Where we’re already doing well, we put guardrails in place and take care. Where we’re not yet fully in control, we build our understanding.
I think that’s something governance can approach differently. Rather than developing principles, acts or whatever you call them at a distance from real life, the business and what’s happening day to day, can we move closer and really test our governance and policies?
That’s what we did at Rabobank. We carried out validation and then built the policy. Then we tested: are we really adding value here? Or are we just adding another control on top of another control, until we lose control?
I’ll stop there, because you wanted to do two things.
Jeroen: This is actually pretty close to the second thing I wanted to do: hear some final thoughts from both of you on anything we’ve missed or should have covered today. I think we’ve covered a lot, but perhaps there are things we should still mention.
Lotte: Perhaps something related to adaptivity: dynamic controls and control frameworks. That’s where we’re heading. It’s no longer as simple as having static controls at a particular point in time.
Controls need to be very dynamic and change over time. Again, I’d ask: how are you going to control that?
You have your strategy, your risks and your controls to mitigate those risks. But your strategy changes, and with it, your risk appetite might change.
At some point, will all of that be done by AI? Will it become so dynamic that nobody is actually looking at it anymore, and everything just keeps changing?
How will you stay in control of your company? We’re moving towards dynamic control frameworks, but maintaining control over those from a human perspective is quite difficult.
Natacha: For me, the question is: are we still trying to manage something completely new using the experience we’ve built in the past? Is it time to reassess?
In risk management, for instance, we develop a framework, put it in place and then review it. I’d really encourage my peers to look closely at that and ask: is it time to adopt a more adaptive way of managing risk?
Coming back to the people element, there’s a lot of noise about what AI can and can’t do. Some of it isn’t always true.
My message to everyone listening today is that when we bring human brains together, there’s creativity. I truly believe that when we put our natural intelligence to work together, it puts us in the driving seat, rather than leaving us simply to live with the consequences.
AI will be here. It’s already here. What I’m calling for is for us to bring that natural intelligence together, steer AI, make good use of it and use our human intelligence to manage the areas where it falls short.
That would be my final message to the audience today.
Jeroen: That’s a beautiful ending, I would say. Thank you both so much.
I must admit that putting this podcast out there will probably provide input for another AI model somewhere, because there will be a transcript that can be used again. But perhaps it will make the model a bit more balanced, to use your words.
Natacha Lord, Global Head of Model Risk Management at Rabobank, thank you so much. I have a small present for you to thank you for coming to Leaders in Finance and spending this time with us.
The same goes for you, Lotte Aelvoet, partner at Deloitte. Thank you for joining us today.
I really enjoyed the conversation. It’s very interesting. My own summary is that it’s going to be exciting, but also a bit scary. That’s my takeaway.
I’m not sure whether that’s the right takeaway, but it’s certainly how I feel after this podcast. Thank you so much for joining us.
Voice-over: Thank you. You’ve been listening to the Leaders in Finance Compliance Podcast. We’d like to thank our partners: Deloitte, Rabobank, Cense and Osborne Clarke.
Would you like to know more or leave us a comment? You can do so on our LinkedIn page. If you follow us there, we’ll keep you up to date with our new episodes.
We’d be grateful if you could leave a review and tell others about this podcast. Also, don’t forget to check out the other podcasts on our channel. There’s bound to be something for you.