A Fraudster’s Paradise – O’Reilly

Darkish internet discussion board posts talked about the phrase “AI agent” way more within the second half of 2025 than within the first half. May this imply that fraudsters are charmed by the AI hype? Or is AI actually a recreation changer for cybercrime? AI-related discussions—evident each in what “the unhealthy guys” are saying and in what fraud-fighters are exploring—are in all places. In reality, a Visa PERC evaluation evaluating information from June to November 2025 to the six months earlier than that discovered a rise of greater than 450% in darkish internet agentic AI-related posts.

Documented monetary losses from deepfake-enabled fraud exceeded $200 million in Q1 2025 alone. And Visa reported a 25% enhance within the second half of the yr in malicious bot-initiated transactions. There’s AI-generated rip-off and fraud content material in all places you look, and it doesn’t at all times give Eric Clapton six fingers to make it apparent that one thing’s off:

None of that is taking place in a vacuum. On-line fraudsters have embraced GenAI with open arms in a means that has modified the form of the web fraud panorama. On the similar time, bizarre customers are leaning into the form of dishonest that GenAI makes simple.

How can the monetary and digital ecosystem battle again towards this new wave of AI-powered fraud? Studying from previous expertise, collaboration and data sharing between fraud-fighting professionals is vital to our collective potential to brace for influence. This quick weblog publish is a name for harnessing the ability of AI for reinforcing the know-how of digital defenders, as a strategy to strengthen our general protection.

Keep in mind COVID?

AI isn’t the primary time that the foundations of cybercrime have modified dramatically (and it gained’t be the final). For instance, again in 2020, in the course of the first COVID-19 lockdowns, work-from-home schemes exploded, together with first-party fraud, phishing scams, and extra. On the time, the neighborhood responded successfully. Groups of fraud-fighting consultants joined forces to satisfy nearly, be taught the brand new terrain and ultimately write a research information that might empower organizations to defend communities from a surge of digital fraud. Our e book Sensible Fraud Prevention was the results of that collaboration, and it rapidly grew to become a useful coaching useful resource to many groups within the ecosystem of their battle towards on-line monetary crime.

In the present day, a brand new wave of fraud is rising, powered by AI and particularly GenAI. Whereas some AI-powered initiatives are right here just for a short time, others have gotten actually highly effective and harmful instruments in nefarious fingers. It’s solely pure that the skilled neighborhood will as soon as once more regroup to kind a playbook towards these traits.

As we interview consultants in various fields for our subsequent e book, The Fraud Fighter’s AI Playbook (with coauthor Chen Zamir, now in early launch on O’Reilly), we’re constructing an image of the ways in which GenAI is altering not simply the methods fraudsters function on-line but additionally the form of the web fraud panorama itself. We’re seeing, too, how important it’s that fraud fighters themselves spend money on exploring and utilizing this expertise to spice up their success, technique, and inside status in their very own corporations.

Extra fraudsters, doing extra hurt

There are extra on-line fraudsters as we speak, finishing up extra fraud assaults, than ever earlier than. Not all of this may be blamed on GenAI. Somewhat, GenAI entered right into a fraud world that, on reflection, was poised to leverage it for crooked growth.

The COVID-19 pandemic drew many new fraudsters on-line, by way of three major tracks:

• Digital transformation + time to spare. Retailers, banks, and different organizations needed to cram years of digitisation into weeks or months. It was inevitable that this might end in some vulnerabilities in processes, insurance policies, or techniques. Folks caught at house with no work had been tempted to make some cash exploiting these weaknesses and realized quick.
• COVID aid packages. Packages put in place to tide people over the difficulties of the pandemic didn’t at all times embody checks to confirm identities or claims. Those that realized how you can make fraudulent submissions, generally even by creating faux or artificial identities or companies, discovered how simple and the way profitable that may be.
• Rip-off compounds. Human trafficking rings pivoted from in-person exploitation to forcing individuals on-line to hold out phishing assaults and scams of all types.

None of those traits has disappeared, and the rip-off compounds specifically have expanded massively because the finish of the pandemic. This was the world that GenAI was born into.

With GenAI, way more fraud assaults are potential, at a degree of personalization that might have been unattainable at scale with out the expertise. Frequent makes use of of GenAI within the wild embody:

• Phishing campaignspersonalised through the use of open supply details about the goal (sourced by way of GenAI), and the suitable language and cultural touchstones for every sufferer (made simple by placing conversations by way of GenAI). Extra typically, the influence is seen in scams of all types, from romance scams to funding scams to kidnapping, catfishing, blackmail scams and extra—equally personalised.

• Malware and bot creation to hold out assaults, steal info, arrange faux accounts, and extra. Fraudsters now not want a lot technical potential to create malicious packages or automation, enabling extra fraudsters to amp up their attain.
• Deepfakes. Whether or not for clickbait technology or for monetary scams, deepfakes enable attackers to go id validation checks. In stunning circumstances, deepfakes had been even used to faux the trail to a job (and the wage and entry to information that comes with it). Broadly, AI simplifies the method of making faux or artificial identities utilizing open supply info, paired with stolen private credentials.
• Pretend web sites, faux apps, faux promoting. Manipulated content material might be utilized to influence customers to buy nonexistent or low-quality merchandise, however it may well additionally goal digital promoting. Main manufacturers are dropping tens of millions of {dollars} to AI schemes that generate views, impressions, or clicks for undeserved advert income. A research of faux cell apps, led by Gilit’s crew at DoubleVerify, discovered that faux iOS grew to become thrice extra widespread (accompanied by six instances extra faux Android apps) in 2025 in comparison with earlier years, a pattern amplified by AI.
• Speedy evolution in sophistication and evasion of bot schemes. Agentic AI is getting higher day-after-day. Because of this operators of bot networks are smirking at CAPTCHA challenges (“resolve this puzzle to show you’re not a bot”). Fraud assaults of 2023 had some rookie errors, like bot networks that attempted to go as human beings watching TV content material besides that they had been coming in with gadget settings of fridges. 2026 attackers will now not fail at fundamental deception as a result of they’ve AI chatbots (FraudGPT, WormGPT, and so forth.) to information them on their means.

The excellent news—if there’s any actual excellent news—is that not less than these aren’t new varieties of assaults. They’re acquainted assaults, carried out extra convincingly, at far larger scale.

GenAI is a aspect hustler’s greatest buddy

It’s not simply devoted fraudsters utilizing GenAI to increase their attain. Abnormal individuals use it to degree up their dishonest recreation too.

Refund fraud has develop into simple at a really convincing degree because of GenAI. Many retailers ask for photographic proof that an merchandise has arrived damaged or broken, and with GenAI, that’s one thing that may be faked in seconds. For the reason that picture is created for a function from scratch, there’s no strategy to discover an unique on-line as proof that it’s a cheat.

Some individuals have gotten much more inventive, utilizing the identical form of trick as a part of an insurance coverage declare. Others use GenAI to whip up faux receipts, which they will declare again from their firm.

It’s vital to notice that, as with the skilled use circumstances, it’s not that these cheats are new forms of assaults. What’s new is the benefit, scale, and class with which they are often carried out.

The bottom shifting underneath our ft

Once we wrote Sensible Fraud Preventionwe included a dialogue of issues like phishing, victim-assisted fraud, refund fraud, and so forth. The deal with the e book, although, was on the ways in which fraudsters money out their schemes. Observe the cash, and discover the fraudster.

Now, solely slightly greater than three years after ChatGPT burst into all of our lives, that emphasis has shifted. Conventional third-party fraud, the sort you get when a fraudster makes use of your bank card on-line, isn’t even within the high three fraud considerations. TransUnion studies that essentially the most enterprise loss in 2025 got here from rip-off/approved fraud (24%), adopted intently by artificial id fraud (20%) and account takeover (20%). That’s the GenAI influence.

There’s a major price ticket connected to numbers like that. The identical report famous that “corporations worldwide misplaced 7.7% of their annual income on common resulting from fraud over the previous yr.” Within the US, it was 9.8%.

There’s additionally a worrying influence on belief. When deepfakes are widespread and convincing, who can ever consider their eyes? Prospects don’t know which internet sites are actual, which messages are genuine, or which advertisements or provides might be trusted. Companies don’t know which claims are legit or how greatest to remain forward of the verification challenges they now face. Marketplaces battle to guard patrons from dishonest sellers, and sellers from dishonest patrons, and everybody from exploitation by malicious actors.

Ha! Fraud fighters have GenAI, too

The ray of hope in our analysis is that fraud fighters have GenAI too, and groups are already experimenting in a wide range of methods. Some are taking a look at how they will use brokers to increase their open supply analysis to make their selections quicker and extra correct. Others are engaged on how you can craft prompts to assist analyze information or work out traits that can be utilized to determine and cease fraud. Nonetheless others are leveraging GenAI to research documentation, to pick fakes or alterations. And so forth.

It’s additionally encouraging to see how groups are utilizing GenAI to increase their attain internally inside an organization. In some methods, it’s nearly like fraud departments are getting the assistant they’d at all times needed to do the duties they’d at all times meant to get to—like pulling the info and placing it collectively for a biweekly replace to related stakeholders or creating detailed materials with useful illustrations or graphs for displays to different departments.

It’s inevitable that when a completely new expertise comes alongside, the fraudsters may have an higher hand initially. They aren’t hampered by concerns like regulatory considerations or authorized necessities, they usually don’t care about issues like accountability, duty, or shopper belief. It’s just about of their job description to disregard these issues, in truth.

The fraud-fighting business has been sensibly cautious about understanding how you can determine and make use of GenAI, however it’s clear that they’re not standing nonetheless. The groups who do the perfect with this evolving problem would be the ones who work intently and persistently with departments throughout their firm to adapt rapidly to the enterprise’s wants—and how you can meet them.