When Algorithms Go Rogue - The Rise Of AI- Generated Crimes
Author: LawSphereX
Category: Criminal Law
Published:
Updated:
WHEN ALGORITHMS GO ROGUE: THE RISE OF AI-
GENERATED CRIMES
Lohit Kumar Bhowmick1, BCOM LLB, IILS, Siliguri
ABSTRACT
Artificial intelligence is transforming our world, but it’s also fueling a wave of clever crimes like deepfakes, phishing scams, and fake digital evidence. In India, cases like the 2025 Hyderabad deepfake scam and the “Babydoll Archi” case show how these crimes hurt people financially and emotionally, while shaking trust in what we see and hear online. Though laws like the Bharatiya Nyaya Sanhita and IT Act are stepping up, we still need better tech, stronger global teamwork, and smarter public awareness to fight back. By blending innovation with care, we can keep AI a force for good while protecting our digital lives. This fight demands we act fast, as criminals evolve quicker than ever. Together, we can build a future where trust and technology go hand in hand. Empowering communities with digital knowhow is key to staying ahead.
Only through shared effort can we ensure AI doesn’t outsmart our values. Keywords: Deepfakes, AI-generated crimes, Phishing Scams, Digital Trust, Cybercrime Legislation
1 BCOM LLB, IILS, Siliguri
INTRODUCTION
What if the next hacker is not a teenager in a hoodie—but a machine learning model trained on your digital footprint? That’s no longer a fantasy. With the rapid rise of Artificial Intelligence (AI), we are entering an age where machines can do more than just follow instructions. They can create, mimic, and manipulate. But with this power comes a new threat: AI-generated crimes. From deepfake videos that spread fake news or ruin reputations, to AI-generated phishing emails that trick people into giving up personal information2, crimes are now being carried out with little to no human involvement. In some cases, even the person using the AI may not fully understand how the crime happened because the machine is learning and acting on its own. One of the most alarming aspects is how realistic these AI crimes can be.
Imagine receiving a phone call that sounds exactly like your parent’s voice, asking for urgent money. You send it only to find out later it was a voice clone made by AI. These aren't just digital pranks—they are real, damaging, and hard to trace. Who's accountable when an AI system carries out a criminal act? Is it the programmer, the user, or the AI itself? Sadly, most countries still don’t have clear answers. Our legal systems were built to deal with human intent, not machine behavior. This makes it incredibly difficult to punish AI-driven crimes or stop them before they happen. Why does this matter? Because as AI becomes more advanced, these crimes will only increase—and they’ll become harder to detect. We need stronger digital laws, better awareness, and global cooperation to deal with this new kind of threat. A very recent case from India proves how serious this has become.
In June 2025, Hyderabad
police began investigating a deepfake video showing MP Asaduddin Owaisi supposedly promoting a scam that claimed people could earn ₹53,000 per day.3 The video looked real. It sounded real. But it wasn’t. The face, the voice, the message all were created using AI. What made it worse was that the video included AI-generated clips of other public figures, making it seem even more trustworthy. People started falling for it. Thankfully, the police acted fast, 2 S. Kumar, R. Verma & K. Bhardwaj, AI-Driven Phishing: Trends and Defense Mechanisms, J. Cybersecurity Rsch., 2023 3 Asaduddin Owaisi files police complaint over deepfake video promoting scam, Bus. Standard (June 7, 2025)
registering a case under cybercrime and IT laws to trace the origin of the clip and those who made it. This isn’t an isolated incident. It’s a warning that AI-generated crimes are no longer science fiction. They're here, they're real, and they’re getting more convincing every day. As technology grows smarter, our laws, awareness, and digital defenses must grow smarter too.
UNDERSTANDING ARTIFICIAL INTELLIGENCE(AI)
Artificial Intelligence, or AI, is the creation of computer systems that can perform tasks usually requiring human intelligence—things like learning, reasoning, problem-solving, perception, and even understanding language. 4 Over the past decade, AI has rapidly evolved thanks to breakthroughs in machine learning, neural networks, and sheer computing power. Cutting-edge models such as large language models and generative adversarial networks (GANs) have made it possible to generate strikingly realistic synthetic media, including images, voices, and videos that are almost indistinguishable from genuine human content. 5 Today, AI is woven into many aspects of our daily lives, from self-driving cars and personalized healthcare to customer support chatbots and financial services. While these advances bring incredible benefits, they also open the door to potential misuse.
Deep learning technologies can now produce highly convincing deepfakes that criminals exploit for fraud, blackmail, or political manipulation. 6 Alarmingly, reports reveal that AI-powered cyberattacks have increased by more than 40% worldwide in just the past year, as bad actors harness AI to automate and enhance the scale and sophistication of their malicious activities. 7 Understanding how AI works, especially its ability to autonomously generate, manipulate, and analyze data is key to grasping the magnitude of the threats posed by AI-generated crimes. 8 However, it’s not just about the technology itself: advances in AI research are rapidly blurring the line between reality and synthetic content, making it harder to distinguish truth from fabrication. Techniques like reinforcement learning allow AI systems to improve on their own
4 Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control 14-20 (2024)
5 Artificial Intelligence Report, Nat’l Inst. of Standards & Tech., State of AI 2025 at 12 (2025)
6 Arvind Narayanan et al., The Rise of Deepfakes and Its Threat to Society, 32 J. Cybersecurity 54 (2025)
7 Cyber Threat Intelligence Center, Global AI-Powered Cyber Attacks Surge 40% in 2024 (2025)
8 Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control 14-20 (2024)
without human intervention, raising the stakes for potential misuse.9 This autonomy means cybercriminals can launch large-scale, highly customized phishing attacks by scraping personal information online and creating believable messages tailored to trick their victims.10 Compounding the problem is the easy availability of powerful AI tools through cloud services and open-source platforms, drastically lowering the barrier for criminals to access these technologies.11 Consequently, combating AI-generated crimes requires a broad understanding not only of AI’s technical strengths but also of the social and technological environments that enable its misuse. Addressing these challenges calls for coordinated efforts that combine technological innovation, policy development, and public awareness.12
TYPES OF AI GENERATED CRIMES
As artificial intelligence weaves itself into the very fabric of our daily lives, its power is also being harnessed for sophisticated and often unsettling criminal activity. Lets look how AI is creating new avenues for harm and what these crimes look like on the ground in India.
1. Deepfake Manipulation (Audio & Video)
AI generated deepfakes, digitally forged videos and audio recordings, have reached a level of realism that blurs the line between fact and fiction. These high tech fakes are being weaponized for multiple purposes: to defame individuals, influence politically charged narratives, spread misinformation, and even scam the unsuspecting. 13 During recent elections in India, deepfakes targeting Bollywood celebrities and high profile politicians went viral, turning fabricated speeches and gestures into viral propaganda tools. The sheer speed at which these videos are disseminated across social media platforms makes it almost impossible to contain the damage before the public forms opinions.
What is most concerning is how quickly these forgeries erode public trust, not just in individuals, but in digital evidence and news itself, leaving people questioning the authenticity of everything they see online. 14
9 Nick Bostrom, Superintelligence 58 (2019)
10 Cyber Threat Intelligence Center, Global AI-Powered Cyber Attacks Surge 40% in 2024, at 3 (2025)
11 Artificial Intelligence Report, Nat’l Inst. of Standards & Tech., State of AI 2025 at 12 (2025)
12 Nick Bostrom, Superintelligence 58 (2019)
13 Legal Service India, Crimes Involving Artificial Intelligence (2024)
14 How AI Deepfakes Are Targeting India’s Elections, WIRED (2024)
2. AI Powered Sextortion & Blackmail
The dark side of AI’s creativity emerges in the form of non consensual, fabricated sexual content. Criminals use sophisticated AI models to doctor real photographs or videos, turning innocent images into explicit fakes, often targeting women and minors.15 These materials are then used for blackmail, either by demanding money or threatening public humiliation if the victim does not comply. Social media and the dark web have made these crimes painfully common, as perpetrators can remain anonymous and operate from anywhere in the world.16 Law enforcement often finds it difficult to keep up, given the sheer volume and rapid spread of such content. Victims not only suffer financial losses but also endure lasting emotional trauma, social stigma, and career damage, highlighting the urgent need for both tech based and legal solutions.
3. AI Driven Telecaller Scams & Fake Regulated Loans
Scammers are exploiting AI driven voice synthesis and face swapping technologies to impersonate police officers, financial regulators, or even court officials. With realistic audio and video, victims receive calls about alleged legal trouble, so called digital arrests, and are coerced into transferring money to resolve fabricated fines or bogus loans.17 Seniors, students, and less tech savvy individuals are primary targets, as these scams are persuasive and terrifying. A high profile crackdown by the Uttar Pradesh Special Task Force (STF) revealed just how extensive such rackets have become, stretching across states like Delhi and Mumbai and bilking unsuspecting families out of lakhs of rupees.18 AI not only provides the tools for such deception but makes it scalable, automating calls, scripts, and evidence required to fool even the skeptical.
4. Phishing & Fraud via AI
The age old con of phishing has been turbocharged by artificial intelligence. Modern fraudsters deploy AI driven bots to sweep social media, analyze user behavior, and craft highly personalized emails, texts, and websites that mimic the tone, branding, and timing of legitimate
15 Deepfake Pornography, Wikipedia (2025)
16 AI and sex crimes in India, Reddit (accessed July 2025)
17 Legal Service India, Crimes Involving Artificial Intelligence (2024)
18 Gang Exploited Victims Using Artificial Intelligence and Fake Loans, The Times of India (2025)
organizations.19 In India, reports suggest that as much as 80% of phishing campaigns now use AI generated content, allowing rival gangs to target more victims more efficiently than ever before. Whether it is a fake investment proposal, a clone of a popular payment app, or even deepfaked regulatory messages, AI’s role in lowering the barrier to sophisticated digital fraud is impossible to ignore.20 Victims may lose money, have their identities stolen, or find their entire digital presence weaponized against them.
5. AI Enabled Cyberattacks & Malware
Gone are the days of simple viruses and easily detectable malware; attackers now use self learning AI to test weaknesses in real time. Malicious software can alter its own code, pattern match against security defenses, and launch autonomous attacks that evolve as cybersecurity teams adapt. Targets range from government servers and banking systems to power grids and hospitals, putting critical infrastructure and sensitive personal data at constant risk.21 Particularly worrisome are ransomware attacks and Distributed Denial of Service (DDoS) campaigns, where AI helps criminals maximize damage and evade detection by authorities. As a result, the cybersecurity arms race has become a high tech cat and mouse game, with attackers always just a step behind or ahead of defenses.
6. Surveillance, Predictive Policing & Privacy Breaches
AI’s promise of enhanced law enforcement efficiency comes at a cost to privacy and civil rights. Indian police forces are increasingly deploying facial recognition systems, predictive AI tools, and algorithmic risk assessment models without full transparency or oversight.22 This raises major red flags about potential violations of Article 21 (Right to Privacy) and Article 14 (Right to Equality) of the Indian Constitution, especially for minorities and disadvantaged groups. Algorithmic bias can result in the wrongful profiling of individuals or neighborhoods, leading to discrimination, harassment, or unfair prosecutions.23 As these AI systems operate in an opaque black box, it is often impossible to understand or challenge their decisions, a growing threat to the principles of justice and accountability.
19 AI Aids 80% of India’s Phishing Attacks, The Times of India (2025), supra note 5
20 AI Aids 80% of India’s Phishing Attacks, The Times of India (2025), supra note 5
21 Legal Service India, Crimes Involving Artificial Intelligence (2024)
22 AI driven policing in India, Reddit (accessed July 2025)
23 AI driven policing in India, Reddit (accessed July 2025)
7. Misuse of AI Generated Digital Evidence & FIRs
The legal system is not immune to manipulation. AI can now draft First Information Reports (FIRs), create forensic reports, or generate digital evidence that looks authentic but is entirely fabricated.24 Since the Bharatiya Sakshya Adhiniyam, 2023 permits digital evidence in Indian courts, these innovations have practical advantages, but also open the door to fraud on a massive scale. Courts, lawyers, and investigators must now scrutinize all digital submissions more closely than ever, as advanced AI blurs the line between evidence and fabrication.25 Any failure to do so risks miscarriages of justice and undermines public faith in the legal process.
REAL LIFE EXAMPLES
To understand the aforesaid types of AI generated crimes, we can look upon the following real life cases and examples.
1. Assam’s “Babydoll Archi” Deepfake Scandal2627
In July 2025, a young woman from Dibrugarh, known online as “Babydoll Archi,” suddenly found her name and face plastered across social media, falsely linked to explicit content she never consented to. This is a very recent incident that caught everybody’s attention and sent shockwaves across the nation. What made this so disturbing was that her ex-boyfriend, using advanced AI tools, created and circulated highly realistic deepfake images and videos of her as an act of revenge. He didn’t stop there—setting up fake Instagram accounts and even monetizing this fabricated content through subscription pages, earning nearly ₹10 lakh in the process. The AI-generated material went viral, with manipulated images placing her alongside famous adult movie stars, completely fabricating her identity as an “internet celebrity” in the adult entertainment world.
The woman and her family were blindsided when they discovered this digital assault, prompting them to report the matter to the police. Law enforcement quickly traced the source to the ex-boyfriend and arrested him, seizing devices and evidence of the operation. The case
24 The Intersection of AI and Indian Criminal Law, Savara Advocates (2025)
25 Challenges and Judicial Responses on AI in India, lawaudience.com (2025)
26 Influencer Archita Phukan’s ex-boyfriend arrested for selling her AI-generated photos, Moneycontrol (2025)
27 Babydoll Archi: Indian woman’s identity stolen for erotic AI, BBC (2025)
has drawn intense public attention, highlighting the dark side of AI’s power to create synthetic media that can ruin lives in an instant.
2. Deepfake Scams Featuring Public Figures and Financial Fraud28
Another spotlight of this unsettling trend of deepfake financial scams fell squarely on the nation when people started sharing eerily authentic videos of Finance Minister Nirmala Sitharaman and Google CEO Sundar Pichai. The twist? These global icons never made the videos at all. Using cutting edge AI, scammers produced hyper realistic deepfakes that showed these trusted leaders confidently promoting cryptocurrency schemes and miracle stock market investments. The speeches looked genuine, the gestures spot on, and every intonation seemed real, making it almost impossible for the average person to doubt their authenticity. Not surprisingly, people rushed to invest, believing they were following the direct advice of the country’s top financial decision maker or a tech visionary. The scams spread at lightning speed across Telegram channels, WhatsApp forwards, and even YouTube ads.
In the end, thousands of victims handed over their life savings, sometimes amounting to crores of rupees, before realizing they had been duped by technology rather than guided by the respected figures they trusted. These crimes hit harder than a typical financial fraud precisely because they prey on emotional trust in well known faces and public officials. The perpetrators didn’t just weaponize technology, they exploited the personal credibility and authority these figures command, making the deception both deeply personal and disturbingly convincing. The aftermath has sparked debates on digital security, authenticity checks, and the urgent need for new laws to combat such sophisticated AI enabled scams.
For many of those affected, however, the damage is already done, serving as a harsh warning that in today’s world, not every trusted face online can be believed, no matter how convincingly it speaks to you.
3. Arijit Singh v. Codible Ventures LLP (2024)2930
In 2024, renowned singer Arijit Singh found himself at the center of an unprecedented legal battle that caught the attention of the entire entertainment industry and legal community in India. An AI platform began mimicking his distinctive voice and even used his name without 28 India Faces ₹20000 Crore Cybercrime Threat in 2025 Amid Surge in AI-driven Deepfake Investment Scams, The Logical Indian (2025) 29 Bombay High Court Protects Celebrity Identity from Deepfake Exploitation, Indian Legal News (2024)
30 Arijit Singh Wins Interim Relief in AI Voice Mimicry Case, The Times of India (2024)
permission to promote products and services actions that were completely unauthorized and deeply troubling for the celebrity. The fake endorsements created by the AI not only threatened Arijit Singh’s reputation but also raised critical questions about the misuse of AI technology to exploit personal identity. Acknowledging the seriousness of the case, the Bombay High Court promptly provided interim relief, delivering a strong statement on the issue. This was the court’s firm way of saying that protecting a person’s identity even from synthetic AI reproductions is a serious matter, deserving legal safeguards. What made this case especially significant was that it came before India had any specific laws addressing deepfakes or AI-generated impersonations.
It broke new ground, affirming that courts can and should intervene proactively to defend personality rights against emerging technologies. This legal milestone now provides a vital precedent for celebrities and ordinary citizens alike, highlighting that nobody’s voice or image can be hijacked by AI without consequence. For Arijit Singh and others, the victory offers reassurance that the courts recognize the need to balance innovation with respect for individual dignity in the digital age.
4. McAfee’s Prime Day Deepfake Scams31
Amazon Prime Day 2025 promised great deals but sparked a massive digital scam, deceiving thousands of Indian shoppers. McAfee uncovered a massive scam operation, featuring over 36,000 counterfeit Amazon websites and 75,000 deceptive text messages crafted to capitalize on the shopping frenzy. These AI-driven scams used fake endorsements to trick victims into sharing personal details. The fallout highlighted the urgent need for stronger digital protections during major online events. In this scam wave, AI-driven deepfake technology took center stage. Shoppers received personalized messages made to look like urgent refund notices and saw slick videos featuring fake endorsements from celebrities, all of it crafted by artificial intelligence to boost believability.
Many victims, caught off guard by the familiar faces and seemingly legitimate messaging, handed over personal information or made payments only to realize later that they
31 AI-Powered Scams Surge Ahead Of Amazon Prime Day 2025, NDTV (2025)
had fallen for sophisticated forgeries. Almost 40% of Indian online shoppers were exposed to these deepfake scams, with individual losses frequently exceeding ₹40,000. The effects rippled far beyond mere financial loss. The scam underscored how AI can weaponize emotional manipulation at scale, leveraging the trust that consumers place in big brands and recognizable personalities. For many, the shock came not just from lost money, but from the unsettling lesson that even one’s most trusted digital experiences can be hijacked by unseen technological forces during major sales events. Prime Day, long celebrated for its deals, became a cautionary tale of how AI-powered deception now lurks around every digital corners especially when excitement and urgency are running high.
LAW ENFORCEMENT’S BATTLE AGAINST AI-GENERATED CRIME IN INDIA
India’s law enforcement agencies stand at the forefront of a rapidly evolving digital frontier where artificial intelligence (AI) is transforming crime in ways both unprecedented and deeply challenging. While AI offers tremendous benefits to society, it also empowers criminals with new tools to deceive, defraud, and disrupt often at a speed and sophistication previously unimaginable. Police officers, prosecutors, and judges now grapple with a multifaceted problem: putting the genie back in the bottle of AI misuse without stifling innovation or public trust. The Eroding Line Between Reality and Fabrication A key challenge for law enforcement is the growing crisis of verifying digital authenticity. AI
has reached a stage where deepfakes, AI-generated voice clones, and intricately manipulated videos can convincingly imitate real people, speech patterns, and even emotions. This implies that investigators can no longer trust their instincts or digital files as reliable evidence. Even forensic experts dedicated to image and audio verification find themselves navigating a minefield of partial truths, subtle alterations, and highly deceptive synthetic media.32 This blurring of fact and fiction strikes at the core of criminal investigations. When the evidence itself can be artificially generated or convincingly altered, it raises fundamental questions: How can police prove who committed a crime? Whose testimony can be trusted? 32 TRM Labs, The Rise of AI-Enabled Crime: Exploring the Evolution, Risks, and Responses to AI-Powered Criminal Enterprises (2025)
How will courts separate genuine proof from orchestrated falsehoods? This reality forces investigators to double down on technical proficiency and calls upon courts to consider entirely new spectra of digital evidence verification. India’s legal system, while modernizing through laws like the Bharatiya Nyaya Sanhita (BNS) and Bharatiya Nagarik Suraksha Sanhita (BNSS), still relies heavily on expert testimony and forensic validation, which can be slow, expensive, and inconclusive at times. The Speed and Volume of AI-Powered Crimes Beyond authenticity, the scale and speed with which AI enables offensive operations is staggering. Automated phishing campaigns can deploy tens of thousands of personalized scam emails, crafted with AI’s understanding of the recipient’s social network, preferences, and vulnerabilities.
AI bots can fabricate entire social media personas, complete with backstories, photos, and interactions, to entrap victims or launder money with digital confidence tricks. For law enforcement, this creates a flood of digital incidents many of which are new to their toolkits. Unearthing the pattern amid mountains of data and disinformation is timeconsuming. Criminals benefit from asymmetry: a single well-programmed AI system can outpace hundreds of human investigators. By the time agencies detect a campaign and mobilize resources, the perpetrators may have launched new operations or vanished behind complex server networks overseas. 33 Jurisdictional Labyrinths and International Coordination AI-generated crimes almost always involve multiple jurisdictions.
Fraudulent servers may be located abroad, perpetrators may operate anonymously through proxies, and victims can be spread across India’s vast geography. While India’s BNS and BNSS provide updated frameworks to criminalize and prosecute cyber offenses, the real-world application of law enforcement cooperation across borders is cumbersome. Mutual Legal Assistance Treaties (MLATs), extradition requests, and cross-border evidence gathering all take significant time sometimes years to execute effectively. Meanwhile, data may be deleted, logs erased, or financial assets laundered beyond recovery. Some countries simply lack robust cybercrime legislation or enforcement capacities, creating safe havens for AI-
33 Check Point Research, AI Security Report 2025 (2025)
enabled digital criminals. These challenges require diplomatic finesse alongside technical acumen, but resource constraints often limit India’s ability to pursue such complex international coordination quickly and effectively. 34 Technology Gaps and Skill Shortages Despite its many cities advancing in cybercrime expertise, a majority of India’s police infrastructure still suffers from severe shortages in AI-related forensic capabilities and domain expertise. While large metropolitan areas might boast cyber labs, districts and rural police stations frequently lack even basic digital investigation tools. Detecting signs of synthetic media, audio manipulation, or AI-driven malware requires specialized software and experts with knowledge of machine learning models, pattern recognition, and the latest AI scams.
Even when such tools exist, they may suffer from limited licensing, funding, or awareness among frontline officers. This knowledge gap causes delays in both recognizing and responding to AI-enhanced crimes and frequently results in underreporting or misclassification of incidents. To compound the issue, collaboration between law enforcement and private cybersecurity firms, academia, and AI developers remains uneven, limiting the flow of threat intelligence, defensive technologies, and early warning systems. Bridging this divide is critical to ensure that policing evolves as rapidly as the crimes themselves. The Human Element: Psychology, Trust, and Victim Response Technology is just one dimension of the challenge. AI-powered crimes often exploit deeply human vulnerabilities like fear, trust, greed, and shame.
For example, phishing emails tailored through AI’s analysis of social media activity create highly believable scams. Deepfake videos impersonating loved ones can emotionally manipulate victims into compliance. This psychological manipulation discourages many victims from coming forward, often out of embarrassment or loss of confidence in law enforcement’s ability to deliver justice. Victims’ reluctance deepens under the stigma associated with scams (such as sextortion or identity theft),
34 Savara Advocates, The Intersection of AI and Indian Criminal Law (2025)
which complicates timely reporting and evidence gathering. Furthermore, judges and magistrates must evaluate and interpret AI-altered evidence in courtrooms that often lack the tools to handle such technical intricacy. This demands new expertise and standards for evaluation, or else courts risk unjust outcomes based on misunderstood or unverifiable digital content. Legal Frameworks: Tremendous Progress but Emerging Needs India’s legislative landscape is adapting with the Bharatiya Nyaya Sanhita (BNS) modernizing substantive laws and criminalizing digital offenses like synthetic obscene content (Section 294), privacy breaches via digital voyeurism (Section 77), digital theft and cheating (Sections
316 and 318), and organized cybercrime (Section 111).35 The Bharatiya Nagarik Suraksha
Sanhita (BNSS) complements this with procedural reforms for handling digital evidence, ensuring proper forensic protocols and admissibility in court. 36 The Information Technology Act, 2000 remains a key weapon, provisions such as Sections 43, 66, 66C, and 67/67A govern unauthorized access, fraud, identity theft, and transmission of obscene digital material. Authorities also rely on Section 69 for lawful interception to monitor suspicious transactions and communications. 37 Despite significant progress, these legal tools often struggle to keep pace with AI’s evolving capabilities and novel misuse. There is still no dedicated law specifically regulating AIgenerated synthetic media, clarifying liability for autonomous AI actors, or standardizing transparency and provenance requirements.
Courts and investigators often have to contend with gray areas regarding intent, causality, and proof burden. Recent proposals like the Digital India Act38 aim to fill these gaps, but comprehensive AI-specific legislation remains a work in progress. Moving Forward: Collaborative and Multidimensional Strategies Tackling the complex issue of AI-generated crimes requires recognizing that no single solution
35 Cyber Crime Punishments Under Bharatiya Nyaya Sanhita, MyJudix (2024)
36 Bharatiya Nagarik Suraksha Sanhita, Draft Text & Commentaries (2024), Government of India Publication
37 The Information Technology Act, 2000, as Amended (India)
38 Ministry of Electronics and Information Technology (MeitY), Digital India Act (DIA), 2023: Overview and Legal Framework, (2023)
exists. Law enforcement must develop deep digital forensic competencies and secure adequate budgets for tools and training. Policymakers must craft clear, robust laws that modernize definitions around AI’s impact on crime and liability. International cooperation frameworks require strengthening to bridge cross-border enforcement gaps. Meanwhile, the private sector, academia, and civil society must be active partners, sharing insights, building detection algorithms, and raising awareness among the public about AI-enabled scams. At the community level, improving digital literacy can empower citizens to better recognize and report AI fraud, reducing victimization. Judicial training programs can prepare courts to assess AI-generated evidence intelligently and fairly.
Only through this multilayered ecosystem of innovation, law, education, and ethics can India hope to contain the dangers posed by AI misuse while unlocking the incredible benefits AI offers for public safety and justice. Far-Reaching Ethical and Societal Impacts of AI-Generated Crimes Artificial intelligence is not merely a technological advancement; it represents a profound shift in the way we create, consume, and trust information. As AI-generated content becomes increasingly sophisticated, the boundary between reality and fabrication is blurring, with serious consequences for individuals, communities, and societies at large. These consequences raise urgent ethical questions and societal concerns many of which we are only beginning to fully grasp.
Erosion of Trust in Digital Content and Communications Only a decade ago, digital content such as photos, videos, and audio recordings served as reliable evidence even a trusted form of “proof” in courts, newsrooms, and daily life. But with the rise of AI tools capable of generating hyper-realistic deepfakes, voice clones, and synthetic images, the public’s confidence in digital media has been deeply shaken. Today, every video clip, image, or audio recording is met with a whisper of doubt: Is it authentic, or is it a meticulously crafted simulation generated by AI? This growing doubt isn't just a vague unease;
it seeps into the core of communication that holds our modern society together. 39 This erosion of trust impacts diverse areas. News organizations find themselves forced to invest heavily in verification technologies, battling against the spread of false, fabricated footage that can trigger real-world consequences before the truth catches up. Loved ones pause before sharing personal videos or voice messages, worried they could be altered. Businesses hesitate to believe customer interactions or testimonials without extensive authentication. Online platforms juggle between the impossible tasks of encouraging free expression and preventing the weaponization of synthetic media. At the core, this pervasive doubt challenges the idea that “seeing is believing,” a bedrock principle in human society. Even courts face this crisis.
Digital evidence, once a powerful tool for justice, now demands painstaking forensic scrutiny to rule out AI manipulation. The trust deficit extends beyond public arenas to personal spaces, corroding interpersonal relationships and communal bonds because no piece of digital content carries the same weight it once did. Across India where digital penetration is exploding, this mistrust infects an ever-increasing share of the population’s social and professional interactions. Potential for Mass Manipulation, Financial Loss, and Psychological Harm Layered upon the loss of trust are the profound and often devastating effects of mass manipulation, large-scale financial frauds, and the psychological trauma inflicted by AIgenerated crimes.
AI’s capacity for high-volume content creation and precise personalization allows malicious actors to orchestrate campaigns of disinformation and deceit with uncanny efficiency. At a societal level, AI-powered misinformation can incite social unrest, influence elections, and sabotage public health efforts. Entire communities may be flooded with false narratives crafted to deepen existing divisions or sow confusion, often amplified through social media algorithms that reward engagement regardless of truthfulness. 40 The ripple effects can lead to widespread anxiety, broken social cohesion, and systemic distrust in institutions. Financially, AI-driven scams have reached unprecedented scale and sophistication. With AI- 39 TRM Labs, The Rise of AI-Enabled Crime: Exploring the Evolution, Risks, and Responses to AI-Powered Criminal Enterprises (2025)
40 Check Point Research, AI Security Report 2025 (2025)
generated videos seemingly showing trusted public figures endorsing fraudulent investment schemes, individuals can be coaxed into handing over life savings to fraudulent digital platforms. AI-assisted voice cloning has been used to impersonate family members or authorities, tricking victims into transferring large sums of money out of fear or a sense of urgency. 41 The result is devastating economic loss for individuals and families, with little hope of recovery in many cases. Perhaps the most heartbreaking dimension is psychological harm. Survivors of deepfake pornography, sextortion, or identity theft frequently suffer emotional pain that overshadows their financial damages. The feeling of violation knowing that your likeness, voice, or personal data has been weaponized against you can lead to depression, anxiety, and social isolation.
Victims feel not only robbed but exposed and powerless against faceless algorithms and unknown perpetrators. Children and vulnerable groups experience harassment and bullying amplified by AI’s capabilities to fabricate compromising or demeaning content instantly and virally. This psychological impact supports an emergent culture of fear and self-censorship. Numerous victims hold back from reporting, wary of judgment or skepticism. Families and communities sometimes fracture under the strain of false accusations or reputational damage. The intangible, “hidden” wounds inflicted by AI crimes stress the need for legal protections, technological safeguards, and robust counseling resources to address the full spectrum of harm.
The Relentless Arms Race Between Criminals and Defenders Underlying the above challenges is an ongoing, fast-evolving battle between those developing AI-based defenses and the criminals who exploit the same technology for illicit gain. Every advance in detection algorithms or cybersecurity measures is met swiftly with new strategies to circumvent them. This cat-and-mouse game has become an arms race, exhausting resources, burning out cybersecurity experts, and demanding continual innovation in both technology and policy. 42 On the criminal side, AI’s accessibility means that even individuals with relatively modest technical skills can launch powerful attacks whether that’s creating new deepfake scams,
41 Check Point Research, AI Security Report 2025 (2025)
42 Savara Advocates, The Intersection of AI and Indian Criminal Law (2025)
automated phishing schemes, or synthetic identity fraud. Criminal toolkits evolve rapidly, and distribution platforms on the dark web facilitate quick sharing of the newest AI “attack packages. ” The commercial availability of AI shortcuts the barrier to entry for bad actors, democratizing the potential to cause harm on a global scale. For defenders, law enforcement, governments, private industry the challenge is daunting. They must build sophisticated AI-powered detection tools, stay abreast of evolving threats, and craft legislation and policies flexible enough to adapt to unforeseen harms. Meanwhile, they must ensure that defensive measures respect privacy, fairness, and human rights, avoiding overreach that might stifle innovation or create new vulnerabilities. The societal cost is significant.
As defenders pour billions into security infrastructure and public awareness campaigns, the broader public often experiences “security fatigue” weary of constantly changing protocols, multi-factor authentications, and warnings about scams. Paradoxically, this weariness may blunt vigilance, ironically increasing vulnerability. 43 At the same time, the asymmetry in resources between well-funded attackers and often underresourced defenders poses an ongoing strategic threat. Deepfakes and AI-enabled misinformation also pose risks to core democratic processes. A single manipulated video can inflame communal tensions, influence voter behavior, or disrupt policymaking all consequences that ripple far beyond immediate victims. The race is not just about technology but about maintaining societal trust, law and order, and democratic integrity.
44 The Human and Ethical Crossroads Ultimately, the ethical and societal implications of AI-generated crimes demand that we reconsider how we define responsibility and protect rights in an AI-saturated world. Questions abound: Who is accountable when AI generates false content that destroys lives? How do we protect individuals’ dignity and privacy when their very identities can be fabricated? What role should governments, businesses, and civil society play in governing the rise of synthetic media? India, as it accelerates digital adoption, faces stiff choices in balancing innovation with protection, freedom with security, and progress with justice. The battle against AI-generated
43 Savara Advocates, The Intersection of AI and Indian Criminal Law (2025)
44 Savara Advocates, The Intersection of AI and Indian Criminal Law (2025)
crimes is as much a moral and social endeavor as it is a technological and legal one. The trust, security, and cohesion we lose to AI’s misuse can only be restored through a concerted, collective effort across sectors and borders. Combatting AI-Generated Crimes The surge in AI-generated crimes has dramatically raised the stakes for India’s law enforcement, security professionals, policymakers, and citizens. Criminals now harness algorithms and machine learning models with stunning speed and scope, rendering traditional cyber defenses outdated almost instantly. As the threat landscape evolves, so too must our methods requiring not just better technology, but smarter governance, robust legal adaptation, international partnerships, and a vigilant, informed public.
1. Cutting-Edge AI-Powered Detection and Forensics
The modern fight against AI crimes begins with embracing AI itself as both shield and sword. Advanced detection tools now employ deep learning algorithms trained on massive datasets of real and forged content, capable of catching the faintest clues, a blink that comes too late, a synthetic wobble in facial movement, or oddities hidden within the sound of a voice. Groundbreaking solutions like Intel’s FakeCatcher or Sensity AI’s deepfake detectors scrutinize media for biological and digital fingerprints invisible to the naked eye, exposing even the slickest fakes within seconds. 45 These systems don’t just stop at visuals: they analyze audio patterns, metadata, file signatures, and combine these with contextual clues, offering law enforcement a high-confidence verdict on whether a file is genuine or machine-made.
Proactive countries, including India, are investing heavily in such forensic infrastructure. Cybercrime cells in major cities now have access to multimodal software and partner with forensics startups, allowing digital evidence to be authenticated faster, crucial to halting the viral spread of dangerous media. The emergence of blockchain-backed authentication, where every original video, image, or document is issued an immutable fingerprint, further helps tame the explosion of manipulated content. Combined, these tools help police not just catch up to, but occasionally outpace, AI-enabled criminals.
45 SOCRadar, Top 10 AI Deepfake Detection Tools to Combat Digital Deception in 2025 (2025)
2. AI-Driven Cybersecurity and Fraud Prevention
Detection is vital, but defense must be even more dynamic. AI is the new backbone of proactive cybersecurity across financial, infrastructure, health, and government systems. 46 Advanced behavioral analytics track every login, transaction, and network move, comparing it to vast patterns of normal activity for each user Even minor anomalies, like an unexpected transfer from a new location, unusual spending surge, or login from an unknown device, can prompt immediate blocks or verification steps, stopping theft instantly. In banking, AI-driven risk scoring slashes fraud rates and minimizes nuisance false alarms, allowing banks to focus on genuine threats. Hospitals and energy grids use AI-driven intrusion detection systems to predict and prevent ransomware, malware, and DDoS attacks before they intensify.
These defenses must adapt daily, as sophisticated attackers constantly update their own AI “playbooks. ” Security teams now work with feedback loops using every new scam or detected anomaly to update models, making the system smarter and more resilient over time.
3. Legislative and Policy Developments
A robust legal response is essential to keep pace with the dynamic threats from AI
misuse. India’s legislators and regulators are actively retooling the national framework to include AI-specific harms. The forthcoming Digital India Act represents a transformational policy effort aimed at mandating AI accountability, requiring platform transparency about synthetic media, introducing obligations to watermark or label AI-generated content, and empowering regulators with real-time takedown authority for illegal fakes. This is matched by updates to key statutes such as the Information Technology Act, 2000 and new provisions in the Bharatiya Nyaya Sanhita (BNS) and BNSS, reflecting the need to criminalize deepfake creation, algorithmic harassment, and identity manipulation using AI. Importantly, India’s approach includes not just punitive measures, but also incentives for responsible AI development and standards for ethical deployment.
Specialized task forces uniting legal, technical, and civil society expertise advise on new risks, ethical dilemmas, and long-term governance strategies, making policymaking both agile and inclusive.
46 Lucinity, Preventing AI-driven Financial Crime in 2025 (2025)
4. Cross-Border Cooperation and Intelligence Sharing
AI-generated crimes almost inevitably leap borders shifting servers, fleeting crypto transactions, and anonymous perpetrators that evade single-nation oversight. To confront this challenge, law enforcement has doubled down on international partnerships. Indian and international cyber units collaborate to share real-time threat intelligence, combine resources to monitor crime networks, and organize swift response teams to disrupt scam operations spanning continents. Global organizations such as INTERPOL and Europol support complex investigations and synchronized takedowns of multinational crime syndicates. However, significant bottlenecks persist. Laws, standards, and privacy norms often clash between jurisdictions, requiring nimble diplomatic and legislative efforts to create legal “bridges” for sharing evidence, extraditing suspects, or freezing illicit assets.
The road ahead will rely on deepening these frameworks, standardizing digital proof requirements, and publishing universal norms for ethical and legal AI use.
5. Empowering the Public Through Awareness and Digital Literacy
No cyber defense is foolproof when human vulnerability remains the greatest weakness. In this arms race, criminals thrive by exploiting human psychology: trust, curiosity, urgency, and fear. Recognizing this, public agencies, tech companies, and NGOs have intensified education campaigns to arm every citizen with digital “street smarts. ”47 Posters in railway stations warn about deepfake scams; WhatsApp broadcasts give families tips to recognize suspicious links or too-good-to-be-true offers; workshops empower seniors and students alike to question the authenticity of digital “evidence” or urgent appeals. Digital literacy programs run across schools, universities, and rural outreach camps, recognizing that awareness is uneven and vulnerabilities highest where digital adoption is newest.
Success stories abound such as villages that averted widespread scam losses after a single awareness drive, or students who protected their families through timely detection of phishing attempts. Public empowerment thus acts as the final, critical wall against AI-enabled fraud.
47 CETaS, AI and Serious Online Crime (2025)
6. Remaining Challenges and the Road Ahead
For all this progress, the struggle against AI-generated crimes remains uphill, unpredictable, labor-intensive, and often emotionally taxing. Criminals are quick to exploit gaps, constantly updating their methods to evade the latest AI defenses or take advantage of the slow rollout of new laws. 48 Law enforcement and judiciary alike must continually retrain to keep pace with the shifting tactics of cybercriminals, all while staying within the guardrails of legal due process and upholding fundamental rights. Policy debates continue over how best to balance privacy and security, innovation and risk, freedom and accountability. The rapid commercialization of AI models means that detection tools and risks will keep evolving, sometimes faster than policymakers can react.
Meanwhile, the technical arms race forces continued investment in both public and private defense and creates stress for those tasked with holding the line. Yet, amid the volatility is a spirit of pragmatic optimism: large-scale global and Indian efforts are growing more agile each year, driven by a sense of shared responsibility to safeguard trust in the digital world.
CONCLUSION
Artificial intelligence is reshaping our world in ways that are nothing short of astonishing. From breakthroughs in healthcare and education to revolutionary changes in how we communicate and work, AI offers immense promise. Yet, this power also carries new and formidable challenges that ripple across every corner of our lives. The rise of AI-generated crimes such as deepfake manipulations, synthetic identity fraud, AIdriven scams, and automated cyberattacks has opened a complex new battlefield for individuals, societies, and institutions alike. What makes this challenge so unique is AI’s ability to blur the lines between reality and fabrication, creating startlingly realistic but entirely fake videos, voices, and documents that can deceive even the most cautious among us.
This erosion of trust in digital content cuts deeply, not only undermining the integrity of social media and news but also shaking the foundations of personal relationships, democratic processes, and judicial systems.
48 CETaS, AI and Serious Online Crime (2025)
The economic and psychological impact on victims can be profoundly distressing. Families are devastated, losing their life savings to scams that feel deeply personal and gut-wrenching. Professionals see their reputations damaged by AI-generated fake content. Vulnerable individuals, especially women and children, suffer psychological trauma when their images or voices are weaponized without consent. Law enforcement agencies face a daunting race, trying to keep pace with criminals who constantly refine their AI tools, operate across borders, and exploit gaps in legal frameworks. Across India and the world, governments are responding with urgency updating laws like the Bharatiya Nyaya Sanhita and the IT Act, crafting new regulations such as the Digital India Act, and empowering cybersecurity forces with cutting-edge AI detection tools. Yet the battle is far from over.
There remain significant gaps in training, cross-border cooperation, judicial readiness, and public awareness. Most importantly, this is not just a technological or legal problem it is a profoundly human challenge. Success depends on the collective effort of technologists designing safer and more transparent AI systems, policymakers creating forward-looking and balanced regulations, law enforcement agencies adapting to new skill demands, and citizens becoming digitally literate and vigilant. It is through this broad-based collaboration that society can safeguard trust, uphold dignity, and ensure fairness in an era where the digital can no longer always be taken at face value. AI's future offers immense possibilities alongside unforeseen dangers.
The choices we make now about education, ethics, technology deployment, laws, and community engagement will shape whether AI becomes a powerful force for good or a source of harm and division. There is reason for cautious optimism: innovation and cooperation are accelerating, and societies are awakening to the importance of digital trust and security. In this evolving landscape, vigilance, empathy, and adaptability will be our greatest assets. Our shared responsibility is to harness AI’s transformative power while erecting resilient safeguards that protect people and truths alike. By doing so, we can forge a digital future where AI enriches lives without compromising the values that bind us together trust, fairness, and human connection.