When AI Hacks Back: The Arms Race

The New Reality of Digital Trust

Artificial Intelligence has forced financial institutions into a new era of cybersecurity. It is not just another tool in the hacker’s kit or another widget in a vendor’s brochure. It has fundamentally shifted how both attacks and defenses unfold, raising the stakes for an industry whose very business relies on customer trust.

What makes AI different is its scale and subtlety. A phishing campaign that once relied on clumsy typos now arrives in flawless English, tailored to a recipient’s role, and even voiced with uncanny accuracy. A ransomware attack that once took weeks of planning can be auto-generated in hours by an AI model. The barrier to entry for cybercrime has collapsed, while the sophistication of attacks has soared.

For banks and neobanks, this is not an abstract concern. Every interaction - from onboarding a new customer to authenticating a transaction is now part of a battlefield where AI is at play.

From Perimeter Defense to Continuous Verification

Traditionally, financial institutions approached security like building a castle: high walls, a secure moat, and strong gates. But AI has rendered perimeter defenses obsolete. Deepfakes can impersonate a trusted employee inside the castle walls. Autonomous malware can tunnel through supply chains as we have recently through the novel type hacks weaponising CLI agents. 

This is why the shift to Zero Trust is no longer optional. “Never trust, always verify” has become the only viable posture. But here’s the nuance: Zero Trust on its own isn’t enough. What’s required is AI-augmented Zero Trust, where verification isn’t just about identity but also about behavior, context, and intent. Anomalies that might escape a human analyst - say, a login at an unusual time followed by atypical transaction patterns - can be spotted instantly by AI.

AI, then, is not only the attacker’s sharpest weapon; it is also the defender’s most precise shield. The challenge is whether institutions can integrate it responsibly and fast enough.

The Governance Dilemma: Explainability and Oversight

This brings us to what I see as the most pressing issue: governance. Institutions cannot simply “adopt AI” and declare victory. The real test is whether they can do so in a way that is transparent, explainable, and accountable without tumbles.

Regulators have made their expectations clear. Under Europe’s Digital Operational Resilience Act (DORA), firms must demonstrate not only that they monitor risks but that they can explain how those risks are managed across systems and third parties. Similarly, the EU’s AI Act is pushing for transparency in how AI models make decisions.

This isn’t red tape for its own sake. In finance, unexplained decisions are a direct threat to trust. Imagine a bank blocking a payment without being able to articulate why. Or denying a credit limit because “the algorithm said so.” Customers and regulators alike will demand clarity - and rightly so.

The governance challenge also extends to the workforce. Shadow AI is becoming the new Shadow IT. Employees eager to save time are pasting sensitive data and documents into chatbots, often unaware of the risks. A huge amount of companies report this problem already. For financial institutions, where data confidentiality is paramount, the stakes couldn’t be higher.

AI technology now comes in many shades, and for various use cases -  the solution isn’t a blanket ban of AI. It’s providing approved, secure, and governed AI tools that give employees the efficiency they crave without compromising oversight. In other words, institutions should strive to make AI safe and easier to use.

Preparing for the Next Wave

Even as we grapple with today’s challenges and the market being flooded with new AI systems, new threats loom. AI systems themselves are just like any other system vulnerable to injections and other malice, where hidden instructions can cause them to behave maliciously. Then we have the run towards quantum computing which may threaten to break today’s encryption protocols, raising the specter of attackers who “harvest now, decrypt later.”

It would be tempting to view these as distant problems. But cybersecurity has always been about anticipating the next wave before it breaks. Post-quantum cryptography may sound like a far-away issue, but the data being stolen in 2025 could very well be decrypted when that wave arrives.

Forward-looking technology mastodonts like Google and Microsoft, are already beginning their research preparing for a  transition. Not because they expect quantum backed hacks tomorrow, but because resilience is built years in advance and they know they need to stay ahead to make it.

Trust as the North Star

Speaking with fellows and peers in banking and fintech, I emphasize that technology alone will not save us. AI is too powerful, too adaptable, and too easily accessed by the bad guys. What will differentiate the winners from the losers is not whether they deploy AI, but whether they do so responsibly, transparently, and at a phase that fits each organisation: safeguarding trust.

For customers, trust is invisible until it’s broken. One breach, one fraudulent transaction, one unexplained denial - that’s all it takes to damage years of brand equity. AI magnifies both the risk of losing that trust and the opportunity to reinforce it, therefore

• Don’t treat AI as a gadget - treat it as part of the eco system.
• Implement governance frameworks that can stand up to regulators and reassure customers.
• Provide staff with vetted and trusted solutions.

AI is not just another chapter in our security journey. It is the story. The institutions that embrace it with clarity, cautiousness and discipline will thrive. Those that chase shiny tools without strategy may stumble and get hurt. In the end, the winners will be those who never lose sight of the fact that trust is the real currency.

‍