Moral Code Vs Machine Code - Justice At The Edge Of The Abyss

Author: LawSphereX

Category: General Law

Published: 2026-08-23T20:38:02.261814+00:00

Updated: 2026-08-26T17:10:25.137+00:00

MORAL CODE VS MACHINE CODE: JUSTICE AT THE

EDGE OF THE ABYSS

Karthik Isunooru, BBA LLB, VIT-AP School of Law, VIT-AP University.

ABSTRACT

As artificial intelligence starts drafting contracts, summarizing judgements and even nudging judges toward decisions, we are forced to ask can machine built on logic and data ever truly understand justice? This paper discusses this question, especially in the context of India’s legal system, where morality isn’t just a theory but a lived constitutional value steeped in ideas of dharma, fairness and human desecration. With tools like Digilawyer-AI and ChatGPT based assistant creeping into the courtrooms and law offices, the promise of efficiency often blinds us to the bigger risk: Are we trading moral judgement for mechanical convenience? Sure, AI can analyze precedent and predict outcomes faster than any human. But law isn’t math. Its messy, contextual, deeply human.

When models trained on biased data start informing bail decisions or sentencing, the myth of machine neutrality begins to crack. This paper pushes back the idea that more tech automatically pushes back the idea that more tech means better justice. It unpacks how algorithmic decision making clashes with constitutional morality, professional ethics and ancient Indian traditions rooted in Nyaya and Viveka, the sense of what’s right and what’s merely legal. Using real life case studies (like courts resisting AI-generated translations or ChatGPT’s controversial citations) it argues that AI can support legal systems but can never replace human judgement at its core. What we need isn’t smarter machines but sharper human guardrails. This paper argues for a clear ethical and regulatory framework, one that treats ai as a tool not a moral compass.

Because justice in the end is not code its conscience. Keywords: Constitutional Morality, Legal Ethics, AI in Law, Judicial Responsibility

1. When Code Meets Conscience

In recent years, the Indian legal system has quietly welcomed a new participant into its ancient halls: Artificial Intelligence. From bustling law firms in Mumbai to courtrooms in Punjab and Haryana, AI tools like DigiLawyer are offering real-time case summaries, precedent analysis, and predictive outcomes. Judges are increasingly relying on AI-powered research assistants, and generative tools like ChatGPT have been cited in bail hearings or employed for legal translation. 1 The once-familiar silhouette of a lawyer hunched over dusty volumes is steadily being replaced by the sterile glow of screens and servers. The march of technology, it seems, is now pacing through the corridors of justice. But there is something sacred about those corridors.

Indian law, though colonial in its architecture, carries older foundations, rooted in the philosophy of dharma and the pursuit of nyaya (justice as a living moral value, not just legality). Law here has always been more than codified statutes or cold calculation; it is a moral dialogue with society, a balance of context, intent, and conscience. AI might impress us by crunching caseloads or tracing precedents, but it is blind to the subtleties that shape justice, compassion, fairness, the lived pain behind a petition. As one ancient Indian maxim says, “Dharmo rakshati rakshitah”, the law protects those who protect the law, and that law is not merely written in ink but carried in the heart. Friedrich Nietzsche warned: “He who fights with monsters must take care not to become one.

”2 In our rush to automate justice, we must ask Are we building machines that reflect our moral struggles or ones that reduce them to binary logic? This paper critiques the increasing reliance on AI in Indian legal practice and judicial decisionmaking. It contends that while AI is valuable in enhancing legal access and reducing procedural bottlenecks, it cannot and must not substitute the moral discretion of the human mind. Legal reasoning is not merely syllogistic deduction. It is the art of judgment, steeped in social realities, ethical duties, and constitutional values. Three fundamental questions anchor this inquiry: 1 Abraham Thomas, Panjab & Haryana HC Uses ChatGPT for Views on Bail, Times of India, Mar. 29, 2023, at 1, https://timesofindia. indiatimes. com/city/chandigarh/punjab-haryana-hc-uses-chatgpt-for-views-onbail/articleshow/99077253. cms.

2 Friedrich Nietzsche, Beyond Good and Evil 68 (Helen Zimmern trans., 1907).

1. Who bears the responsibility when an algorithm misguides a judicial outcome?

2. Can constitutional morality, with its foundation in dignity, fairness and reason, ever be

truly encoded?

3. What happens when efficiency begins to overtake empathy in legal reasoning?

To explore these questions, the paper discussed case studies like DigiLawyer and ChatGPT citations, debates around the AI advisory committee proposed by the Supreme Court, and philosophical insights from both Indian traditions and Western thinkers. It dives into regulatory frameworks, emerging standards of legal ethics, and the constitutional obligation to ensure justice is not only done, but felt. In the end, this paper asserts a simple yet radical idea, law must remain a human enterprise, guided by conscience, aided by machines, but never authored by them.

2. The rise of Legal AI in India

The legal technology in India has taken off at a pace faster than courts can issue notices. In the past few years, the industry has witnessed a digital bloom, with AI-based platforms no longer just assisting research but actively shaping how lawyers approach cases. Platforms like SCC Online’s AI module, Manupatra’s CaseMap and recommender engines, Casemine’s AIgenerated briefs and now DigiLawyer, offer services that a decade ago would’ve sounded like science fiction. Yet, here we are, living inside the footnotes of a future no one sully consented to.

2.1 SCC AI and Manupatra: Automating the Advocate

SCC Online’s AI tools are designed to “understand” a search query, identify relevant legal issues and auto-suggest precedents.3 Similarly, Manupatra’s AI-enhanced search uses NLP to identify key judgements, predict case relevance and even highlight contradictory options.4 These tools promise efficiency, consistency and a quicker grasp of complex jurisprudence. In theory, they make the advocate’s job easier. In practice, they’ve quietly started replacing the process of thinking with a process of clicking. Legal reasoning is not about finding the “most cited” case. It’s about understanding the subtlety behind precedent, the social context of rulings 3 SCC Online, AI-Assisted Legal Research Tool, https://www.scconline.com/ (last visited Apr. 19, 2025).

4 Manupatra, AI Legal Search Tools, https://www.manupatrafast.com/ (last visited Apr. 19, 2025).

and judicial philosophy of interpretation. What these tools offer is convenience, not comprehension.

2.2 Casemine and Brief Generators: Ghostwriting Legal Arguments

Casemine takes it a step further with its “CaseIQ” system which uses AI to auto-generate case briefs from citations and extract argument flows.5 Some firms have begun using these features to draft first-level responses in litigation and arbitration. This shifts the lawyers’ role from active creator to passive editor, a dangerous trend when ethics, not efficiency, is the bedrock of law. Who takes responsibility when the AI misses the relevant point? Can a machine even grasp the difference between obiter and ratio in the way the courts demand? The risk here is not just technical, its epistemological.

2.3 ChatGPT in Court: Curiosity to Citation.

Perhaps the most of the dramatic moment of AI’s legal intrusion came in 2023 when justice Anoop Chitkara of the Punjab and Haryana High Court made a direct reference to ChatGPT in a bail matter involving serious assault charges.6 While the reference was qualified and nonsubstantive, it sparked national debate. Can a non-sentient language model influence liberty and life? The use of ChatGPT raises serious constitutional concerns. In Indian jurisprudence is unique, built upon the moral, social and scaffoldings of a culturally diverse society. Can a model that doesn’t understand the intricacies of Article 21, really assist in doing justice? As Paul Teschner argues, legal morality demands more than correctness; it demands care.7 AI, by design, doesn’t care. It just calculates.

2.4 DigiLawyer and the “Productivity” Trap

Startups like DigiLawyer AI have entered the Indian legal tech space offering lawyers and judges assistance with summarisation, precedent recommendations and statutory interpretation.8 On the surface, this is a godsend for overburdened courts. But scratch the

5 Casemine, CaseIQ AI Briefing Tool, https://www.casemine.com/ (last visited Apr. 19, 2025).

6 Abraham Thomas, Panjab & Haryana HC Uses ChatGPT for Views on Bail, Times of India, Mar. 29, 2023, at 1, https://timesofindia.indiatimes.com/city/chandigarh/punjab-haryana-hc-uses-chatgpt-for-views-onbail/articleshow/99077253.cms.

7 Paul A. Teschner, Lawyer Morality, 47 Cath. U. L. Rev. 285 (1998).

8 Shruti Deolalikar, Transforming Legal Practice: The Opportunities & Challenges of Artificial Intelligence in the Indian Legal System, SSRN (2022), https://ssrn.com/abstract=4421879.

surface and you find a deeper problem, AI rewards repeatability, while justice often lives in exceptions. Indian jurisprudence thrives on the nuanced evolution of constitutional morality. The Navtej Johar judgement9, the Joseph Shine ruling10 or even Vishaka11, none of these followed a predictable, linear, algorithm-friendly trajectory. They were shaped by moral rebellion, not statistical logic.

2.5 The Language Barrier

The Delhi trail court case of State v. Shankar Rai12 offers a timely caution. In this matter, translation tools were used to render a key witness’s Nepali testimony into Hindi and English. The AI-powered translation introduced subtle inaccuracies in meaning altering the perception of intent and culpability. While the judge ultimately relied on human interpreters, the case illustrated a fundamental flaw in the use of AI in a legal context, words mean things, but context means more. In criminal law, a single mistranslation can be the difference between acquittal and incarceration. AI, in its current form, may speak the language of law, but it doesn’t speak the spirit of it. It doesn’t feel, question or hesitate. It doesn’t pause before passing judgement. But human judges do. Human lawyers must. That pause is where morality lives. And no machine can replicate that silence.

3. Dharma, Discretion and the Indian Legal Ethos

3.1 Justice as More than Law: Role of the Judge

The judge is not merely a mechanic of statute but a moral agent entrusted with upholding constitutional values in an incredibly diverse and often contradictory society. Unlike the rigid application of law seen in some western jurisdictions, the Indian legal system, shaped by its colonial past and diverse soul, demands a deeper engagement with the “why” behind every “what.” Judging in India is interpretive and deliberative. The constitution does not just lay out procedures but articulates an aspirational moral vision. Articles 14, 19 and 21 do not operate

9 Navtej Singh Johar v. Union of India, (2018) 10 SCC 1 (India).

10 Joseph Shine v. Union of India, (2019) 3 SCC 39 (India).

11 Vishaka v. State of Rajasthan, (1997) 6 SCC 241 (India).

12 State v. Shankar Rai, FIR No. 184/2022, Patiala House Cts., New Delhi, Judgment dated Jan. 12, 2023 (India).

in a vacuum, they require reading into societal inequities, lived experience and human dignity. This requites discretion, not just rule-following. Judicial discretion is not a flaw in the system, it is the ethical engine that allows the law to breathe. Judges in India routinely weigh not only legal precedent but social reform, historical discrimination and cultural nuance. A machine, no matter how sophisticated, cannot replicate moral calculus.

3.2 The Soul of Indian Constitutionalism

Indian law does not exist in the vacuum of mechanical reason. It was never designed to be a cold instrument of codified logic. Long before the constitution was drafted, this land operated on the principle of dharma, not as some vague moral compass, but as a discipline of right living, a civilizational architecture that dedicated duties, relationships, justice and power with a precision that modern algorithms can barely imitate, let alone understand. Dharma is not law. It is that which sustains the law. As the Mahabharata declares, “Dharma is that which upholds the universe” (Dharayati iti Dharmaḥ). It is contextual, layered and intimately tied to the consequences of action. Yudhishthira’s famous answer to yaksha, “Dharma is subtle” (Sūkṣmo Dharma)13, was a recognition that right and wrong cannot be captured by blunt commandments.

They demand human discretion, soaked in wisdom, awareness and sometimes sacrifice. Take the Manusmriti, often mangled today but undeniably formative in ancient Indian legal thought. It classifies law (Vyavahara) as merely a subset of dharma, which is built upon shruti (vedas), smriti (tradition) and sadachara (conduct of the wise). 14 Mimansa, the ancient school of interpretive jurisprudence, taught us that no law could be understood in isolation from its purpose (artha) and context (prakarana). 15 Unlike AI, which thrives on fixed inputs and clear commands, Indian thought saw ambiguity as essential, not accidental. Where modern western systems center on individual “rights” Indic jurisprudence emphasizes duties to self, society and cosmos.

That’s why in the Ramayana Rama abandons Sita not by legal compulsion but to uphold rajadharma, the king’s duty to public trust over personal emotion. You may contest the outcome, but the moral logic contextual, painful, deeply human.

13 Mahabharata, Yaksha Prashna episode – “Dharma is subtle” (Sūkṣmo dharmaḥ).

14 Manu, Manusmriti 1.2–1.3 (G. Bühler trans., Sacred Books of the East vol. 25, 1886).

15 Jaimini, Mīmāṃsā Sūtra 1.1.2–1.1.5.

Try feeding that into the algorithm. Even Bhagavat Gita, often read as spiritual, is fundamentally a legal-ethical dialogue. Krishna urges Arjuna not to flee battle, not because rule demands it, but because his dharma as a warrior requires it. It’s a clash between internal duty and external morality. Can AI ever grasp that tension? Can it recognize when not following the law is the just path? In Indian thought, justice isn’t about consistency, its about rightness, which is always contextual. Dharma differs for a ruler, a teacher, a widow. Justice must adapt. But AI is trained to generalize, to flatten out difference. It can respect moral pluralism, it turns complexity into computation. This isn’t nostalgia, it’s a warning. Surrendering law to machines means losing centuries of moral struggle to cold logic.

Dharma was never about speed or efficiency, it was about doing what’s right, even when its inconvenient, contradictory or slow. The Indian Constitution reflects this dharmic legacy. When court spoke of dignity in Puttaswamy or equality in Navtej Singh cases, it wasn’t mere statutory interpretation. It was an act of moral alignment, law responding to human reality. And that, no machine will ever truly understand.

3.3 The Problem of Uniformity

Indian legal systems functions in a reality where caste, class, gender, religion, language and region radically affect how law is experienced and enforced. A poor student in UP and a corporate executive in Mumbai may both be “equal” before the law, but their access to justice, treatment by police and confidence in the judiciary are vastly different. AI, by design, thrives on uniformity. But law in India cannot afford to. This was sharply seen in Joseph Shine v. Union of India16, where the court read down colonial-era adultery law. The law treated woman as property and reinforced patriarchal norms. Here too, the reasoning was moral, not mechanical. It wasn’t just that the law was outdated, it was that it violated human freedom and dignity. AI systems trained on historical case data might suggest that the law was settled, or that prosecution under section 497 had precedent.

But that’s the problem, historical data reflects past bias, not present morality. AI cannot course correct for injustice, it can only repeat it faster.

16 Joseph Shine v. Union of India, (2019) 3 SCC 39 (India).

3.4 AI and the Blind Spot of Context

This brings us to the heart of the critique. Legal AI functions on structured data, precedent matching, and logic trees. But India’s legal system is a living organism, shaped by stories, struggles, and socio-political churn. The idea of justice in India is dynamic, something AI simply cannot model. In cases involving reservation policy, environmental justice, religious freedoms, or freedom of speech, courts constantly juggle conflicting rights, historical wounds, and societal needs. Machines lack the empathy, hesitation, and deliberation that human morality demands. As Nietzsche put it, “morality is the herd instinct in the individual”, which means morality, by its very nature, arises from being human, not from computing logic. AI can suggest a precedent, but it cannot ask should this precedent be followed today?

AI can match statutes to facts, but it cannot feel the tremble in a witness’s voice or detect systemic discrimination lurking beneath the surface.

4. The Myth of Machine Neutrality

4.1 The Machine that Claims to Be Blind

There is a certain seduction in the idea that a machine, unburdened by emotion, untouched by caste, gender, class can deliver pure justice. It’s the fantasy of a neutral arbiter, an incorruptible digital sage. But like many fantasies, this one crumbles on touch. Machines are not born wise; they are fed data. And data, in a country like India, comes drenched in the history of oppression. Caste, patriarchy, coloniality, they are not abstract. They live in our judgments, FIRs, charge sheets, and convictions. Feed that into an algorithm and you don’t get neutrality. We get a highspeed reproduction of bias, now armed with the sanctity of code. What we call machine neutrality is, at best, mechanical repetition. At worst, it is algorithmic casteism in a digital robe.

4.2 Karmic Data, Colonial Code

Data, like karma, accumulates. And just as karma carries the moral residue of past actions, datasets carry the residues of past judgments, many of them unjust. The infamous COMPAS algorithm in the United States didn’t invent racism; it inherited it.17 Black defendants were 17 Julia Angwin et al., Machine Bias, ProPublica, May 23, 2016, https://www.propublica.org/article/machine-bias-riskassessments-in-criminal-sentencing.

labeled “high-risk” based on a formula trained on decades of racially skewed convictions. In the UK, visa algorithms were found to carry racial profiling under the hood.18 And now, India stands at the brink of importing this techno-optimism without asking whose data, whose justice? An AI tool trained on Indian legal records will likely reflect the hierarchies entrenched in those records. If Dalits were disproportionately denied bail, the algorithm will flag future Dalits as “flight risks.” Not out of malice, but because it thinks history is destiny. But Indian law, at its best, exists to disrupt such destinies.

4.3 Can Morality be Coded?

Here lies the real philosophical fracture. The Indian conception of justice, grounded in morality, dharma, does not rest on sameness, but on context, intention, and consequence. It is not a uniform metric; it shifts with role, time, and circumstance. What is righteous for a ruler may be sinful for a monk. Krishna’s counsel to Arjuna is not a legal rule, it is a call to moral clarity, grounded in the battlefield of duty.19 Try asking an algorithm what Arjuna should have done. It will return: “insufficient parameters.” A machine cannot perceive the silence of a violated witness, the hesitation of a conflicted judge,

or the generational weight behind a land dispute. It cannot see that law, in India, is not about what is written, but about why it must be interpreted in light of suffering, dignity, and healing. Neutrality, then, becomes a form of spiritual laziness. It assumes the world is already fair, and that our task is to mirror it. But dharma demands more. It asks that we intervene, uplift, correct. It asks that we do not simply repeat history, but redeem it.

4.4 Neutrality is Not Morality

The Constitution of India is not a document of dispassion, it is a moral project. Articles 15 and

17 are not neutral clauses; they are weapons against historical injustice.20 When the Supreme

Court read down Section 377 in Navtej Singh Johar, it did not follow precedent, it broke it.21 Not because the code demanded it, but because conscience did. What algorithm would have done that? A machine cannot rebel. But sometimes, justice requires rebellion. Let us not Home Office Drops ‘Racist’ Algorithm from Visa Decisions,

BBC

News, Aug. 4, 2020, https://www.bbc.com/news/technology-53650758.

19 Bhagavad Gita, ch. 2, vv. 47–50.

20 India Const. arts. 15, 17.

21 Navtej Singh Johar v. Union of India, (2018) 10 SCC 1 (India).

mistake technological polish for moral progress. AI may process faster, but it cannot process better. It has no memory of coloniality, no awareness of caste, no feeling for inequality. It can mimic law, but not the soul of justice. It has no Yudhishthira, no Krishna, no D.Y. Chandrachud coded into its circuits. And without them, what remains is a hollow jurisprudence, efficient, but empty.

5. Ethical and Regulatory Lacunae in India

5.1 Regulation Without a Compass

India’s legal machinery is no stranger to ambiguity. In fact, it thrives on it, nuanced interpretations, layered meanings, judicial innovation. But in the case of Artificial Intelligence (AI) entering the courtroom, this ambiguity begins to feel less like interpretive freedom and more like a dangerous vacuum. While legal practice in India is governed by the Advocates Act,

1961 and regulated by the Bar Council of India (BCI), neither anticipates nor addresses the

elephant in the chamber, the use of AI in legal advisory, research, or even judicial decisionmaking. The Bar Council of India Rules, deeply humanistic in their assumptions, presuppose a living advocate, a person, not a program.22 These rules talk of “duty to the court,” “duty to the client,” and “duty to the public,” all couched in terms that demand intention, conscience, and the exercise of discretion. But AI is not a bearer of duties, it cannot have intentions, only outputs. If a legal AI tool misleads the court, who is liable? The developer? The lawyer who used it? Or no one at all? The current framework is hauntingly silent.

5.2 Can AI “Practice Law”? The Doctrinal Dilemma

The Advocates Act, 1961 confines the “practice of law” to enrolled advocates.23 But what exactly constitutes “practice” in the age of intelligent automation? If an AI drafts pleadings, suggests arguments, or analyzes precedents, core legal tasks, does that amount to unauthorized practice of law? The answer, as of today, is legally untested and theoretically muddled. In jurisdictions like the United States, this question has sparked fierce debate. In Lola v. Skadden, the court held that even document review done without legal judgment does not

22 Bar Council of India Rules, pt. VI, ch. II, Standards of Professional Conduct and Etiquette.

23 Advocates Act, 1961, § 29 (India).

qualify as legal practice.24 But in India, where legal practice is not merely technical but also heavily ethical, the stakes are higher. If legal practice is grounded in moral reasoning and human discretion, as Indian jurisprudence repeatedly asserts, then delegating that to a machine is not just a technological shift, but a philosophical rupture.

5.3 The Data Illusion: Oversight Without Ethics

India’s Personal Data Protection Act (PDP Act), 2023 and the Digital India Act are often cited as future-facing legal frameworks that could “regulate” AI. But here lies the trap, data protection is not ethical oversight. These laws primarily regulate who controls the data, not how the data is used to make decisions of moral and legal consequence.25 An AI tool used in sentencing decisions or predictive policing may comply with data norms and still operate unjustly, by reinforcing casteist biases, or by treating outliers as anomalies to be excluded. Ethical governance goes beyond privacy; it asks whether the decisions made through data are right, not just lawful. That question remains unaddressed.

5.4 The Judiciary’s Technological Leap Without a Moral Harness

The Indian judiciary has, over the past few years, shown an eagerness to embrace digitization. From virtual courts to AI-backed translation tools like SUVAS, the system is changing rapidly. 26 But speed must not outpace wisdom. There is, as yet, no ethical or institutional framework guiding how AI should (or should not) be used in legal reasoning, judicial writing, or fact evaluation. In a 2023 bail order, the Punjab and Haryana High Court cited ChatGPT. 27 This may seem innovative, but it also sets a precedent that machines can participate in moral reasoning. Without clear guidelines, this judicial enthusiasm could spiral into dependence and worse, into uncritical adoption. AI may assist in administration, but judgment is not computation. A judge’s role is not to predict outcomes based on precedent but to interpret law in the crucible of conscience and context.

That is not something AI can be entrusted with. The risk isn’t just technical malfunction, it is moral malfunction.

24 Lola v. Skadden, Arps, Slate, Meagher & Flom LLP, 620 F. App’x 37 (2d Cir. 2015).

25 The Digital Personal Data Protection Act, 2023, No. 22, Acts of Parliament, 2023 (India).

26 Supreme Court of India, SUVAS: Supreme Court Vidhik Anuvaad Software.

27 Punjab & Haryana High Court, Bail Order Referencing ChatGPT, 2023 (India).

5.5 The Unwritten Need

What India lacks is a clear regulatory scaffold, issued either by the BCI or the Supreme Court, that sets out norms for the ethical use of AI in legal proceedings. Such guidelines must address: • The permissible and impermissible roles of AI in legal advice, drafting, and adjudication. • Accountability mechanisms for errors or misuse. • Transparency in data sources, training sets, and algorithmic assumptions. • Human oversight as a non-negotiable, especially in judicial functions. The aim should not be to halt innovation, but to shape it through the lens of justice, not just legal efficiency. To allow AI into the legal fold without a moral audit is to permit machines to walk among moral agents without understanding the burden they carry. Law in India has never been merely a tool of order, it has been a path of nyaya, of restoration, of human dignity.

Regulation must be more than code compliance. It must be a reflection of our legal soul.

6. Responsibility and Accountability: Who Is to Blame When AI Goes Wrong?

Accountability is the cornerstone of justice. In a system where law is not merely a rulebook but a living pursuit of truth, the question of “who bears responsibility when things go wrong?” is fundamental. But in the age of artificial intelligence, that clarity is under siege. When AI tools assist or mislead legal processes, through hallucinated case law, flawed advice, or coded bias, our traditional frameworks of liability begin to stutter.

6.1 AI as Actor Without Agency

The Indian legal system, like most around the world, does not recognize AI as a “legal person.”28 It has no duties, no conscience, no capacity to be punished or held liable. Unlike corporations, which gain personhood through fiction and function, AI remains code, neither citizen nor stakeholder. The Information Technology Act, 2000, even with its 2008 28 Advocates Act, 1961, § 29 (India); see also general principles of tort and contract law regarding legal personhood.

amendments,29 never anticipated AI acting autonomously in high-stakes professional fields like law. This vacuum leads to a profound asymmetry. AI can make decisions that materially affect lives, suggest legal arguments, generate contracts, summarize judgments but if it errs, there is no statutory recourse against the tool itself. It is a ghost in the courtroom: present in process, absent in responsibility.

6.2 When the Machine Hallucinates

In Mata v. Avianca, 2023, a U.S. lawyer submitted a motion filled with fabricated cases generated by ChatGPT.30 The AI cited six non-existent judgments with convincing citations. The court was not amused. It sanctioned the lawyer, reiterating that reliance on AI does not absolve professional duty. Had such a case occurred in India, the fallout would be far murkier: we lack both precedent and procedural clarity on AI-induced legal misconduct. The root problem is that AI doesn’t lie like humans do, it lies better. Tools like ChatGPT are trained to be coherent, not correct.31 Their “hallucinations” are not bugs but features, statistically likely patterns masquerading as fact. In law, this isn’t harmless. It’s dangerous.

6.3 Can a Lawyer Escape Blame by Blaming the Tool?

This is the ethical faultline, Can legal professionals rely on AI tools and dodge accountability when those tools fail? Legally, they cannot. The Advocates Act, 1961 does not recognize AI as a practitioner.32 The lawyer remains the agent of the client, and therefore, bears full professional liability. The Bar Council of India Rules affirm this rigorously. Advocates are bound by a duty of competence, candour, and diligence.33 Using an AI tool, no matter how advanced, does not dilute this duty. In fact, it magnifies the need for caution. As the American Bar Association rightly noted in a 2023 ethics opinion, “AI tools must be treated like junior associates, useful, but ultimately subordinate to the lawyer’s judgment.”34

29 Information Technology (Amendment) Act, 2008, § 43A (India).

30 Mata v. Avianca Inc., No. 22-CV-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023).

31 OpenAI, Technical Report on Hallucination in Large Language Models, https://openai.com/research (last visited Apr. 21, 2025).

32 Advocates Act, 1961, § 29 (India).

33 Bar Council of India Rules, pt. VI, ch. II, r. 11 (“An advocate shall not act or plead in any manner derogatory to the standards of the profession.”).

34 ABA Comm. on Ethics & Prof’l Resp., Formal Op. 498 (2021)

But Indian law lacks even this metaphor. We do not have any judicial or BCI-issued guideline requiring disclosure when AI is used in drafting, advising, or filing. This omission creates a legal and ethical grey zone, AI can ghostwrite a legal brief and no one not the client, not the court would ever know.

6.4 Regulatory Silence is Not Ethical Neutrality

This silence is deafening. AI’s misuse is not merely a technological risk, it is a moral failure. Imagine a legal brief generated by AI that embeds casteist assumptions, or one that reflects historical gender biases. Most large language models are trained on data that mirror social inequities.35 Without human correction, they become vessels for reinforcing, not reforming, those inequities. In a legal system striving for substantive equality under Article 14 of the Constitution,36 such bias cannot be brushed off as computational error. Law is not about statistical probability; it is about principled reasoning. The judiciary cannot afford to outsource its ethics to algorithms.

6.5 A New Kind of Legal Duty: Duty to Audit the Machine

If lawyers are to use AI tools, they must develop a new kind of professional diligence, algorithmic accountability. This includes: • Verifying every AI-generated output independently • Disclosing AI usage in filings and advice • Auditing AI tools for known biases and error patterns • Ensuring client consent when AI is involved in decision-making Some jurisdictions are already moving in this direction. The European Union’s AI Act proposes classifying legal AI tools as “high-risk” systems, requiring transparency and audit mechanisms.37 India, by contrast, has no comparable standard, not in the Digital Personal Data 35 Joy Buolamwini & Timnit Gebru, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, in Proceedings of the Conf. on Fairness, Accountability and Transparency 77 (PMLR 2018).

36 State of W.B. v. Anwar Ali Sarkar, AIR 1952 SC 75 (India).

37 Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act), COM (2021) 206 final.

Protection Act, 2023,38 nor in the draft Digital India Act. These laws speak of data protection, not decision integrity.

6.6 Truth, Technology and the Ethical Imagination

Ancient Indian jurisprudence conceived of justice not as a rigid code but a moral ecosystem. Truth (satya) was not reducible to factuality, it was interlinked with intention, context, and consequence.39 To speak falsely, even inadvertently, carried karmic weight. In contrast, today’s AI-driven systems speak without consciousness or care. Their misstatements are not lies in the ethical sense, they are mere outputs. But when law relies on those outputs, someone must bear the moral cost. Responsibility cannot disappear into the cloud. We must codify it, clearly, unambiguously, and urgently.

7. The Future: Human-Centric AI Regulation for Law

The future of artificial intelligence in legal practice is not merely about control, it is about conscience. At a time when digital tools increasingly permeate the space of reasoning, advocacy, and even adjudication, the Indian legal system stands at a crossroad, do we embrace AI as a servant of justice or allow it to quietly become its surrogate? This is not a question of technological capacity; it is a confrontation with our jurisprudential identity. For centuries, Indian legal thought, rooted in both constitutional morality and ancient dharmic frameworks, has maintained that the legitimacy of law derives not from its algorithmic clarity but from its ethical responsiveness. AI, if not regulated with wisdom, threatens to invert this legacy by replacing moral discretion with mechanical pattern recognition.

7.1 Law as a Living Ethic

The Indian tradition never treated law as a static code but as a living conversation between duty (dharma) and circumstance (desha-kala). The Manusmriti, Arthashastra, and later the Constitution of India all recognised that justice demands not merely a rule, but the viveka to interpret that rule justly.40 Artificial Intelligence, however, knows no such discretion. It

38 Digital Personal Data Protection Act, No. 22 of 2023, Acts of Parliament, 2023 (India).

39 Manu, Manusmriti ch. 8, vv. 84–86.

40 Manu, Manusmriti 8.3; Kautilya, Arthashastra, bk. 3.

optimises for efficiency, not empathy; it calculates risk, not righteousness. Therefore, the regulatory challenge is not how to make AI more accurate, but how to ensure that its use preserves the human centre of justice. This requires a fundamental shift in our regulatory mindset: from governing machines as tools, to safeguarding the moral infrastructure of the legal profession.

7.2. A Professional Code of Conduct for AI in Legal Practice

Currently, the Bar Council of India Rules operate on the implicit assumption that every actor in the legal ecosystem is human. They regulate intent, conflict of interest, diligence, and honesty, virtues that presume moral agency. AI, by contrast, cannot intend, err in judgment, or repent. And yet, it is increasingly drafted into the practice of law without ethical guardrails. There is an urgent need to establish a Professional Code of Conduct for the Use of AI in Law. Such a code should: • Mandate disclosure whenever AI is used in drafting, research, or submissions. 41 • Impose an affirmative duty of verification on the advocate, ensuring all AI-generated content is fact-checked and legally vetted. 42 • Prohibit exclusive reliance on AI in sensitive matters such as sentencing, constitutional interpretation, or rights litigation.

43 • Establish consequences for misleading or erroneous use of AI, particularly where it undermines judicial integrity or procedural fairness. 44 Such a code would not stifle innovation, it would preserve the sanctity of responsibility. In law, every submission is a moral act. To remove the authorial accountability behind it is to invite a dangerous vacuum in legal ethics. 41 There are currently no binding legal provisions in India mandating disclosure of AI usage in pleadings. 42 ABA House of Delegates, Resolution 112 (2019), Encouraging Technological Competence and Ethical Use of AI in Legal Practice, Am. Bar Ass’n. 43 European Parliament, Resolution on AI in Criminal Law and Its Use by the Police and Judicial Authorities in Criminal Matters, 2021/2061(INI) (2021) (“AI must never be the final decision-maker in criminal justice. ”).

44 European Parliament, Resolution on AI in Criminal Law and Its Use by the Police and Judicial Authorities in Criminal Matters, 2021/2061(INI) (2021) (“AI must never be the final decision-maker in criminal justice. ”).

7.3 Mandatory AI Disclosure

Perhaps the greatest risk of AI in legal systems is its invisibility. Today, there is no institutional mechanism that compels a litigant, lawyer, or even a judge to disclose when and how AI has been used in forming legal arguments or judgments. This creates not just a transparency vacuum, but a moral opacity, where decision-making becomes untethered from human responsibility. Mandatory disclosure is not merely a procedural reform, it is a democratic necessity. Courts must know what tools influenced a pleading. Opposing counsel must be informed of nonhuman input. And most importantly, the public must be able to trace how justice was reasoned. This means developing clear protocols, such as: • Inclusion of AI usage statements in pleadings and judgments. • Clear audit trails for how AI-generated drafts were curated or modified.

• Certification mechanisms to verify that the final argument reflects human judgment. Such reforms uphold not just legal transparency, but the epistemological legitimacy of the law itself.

7.4 AI as Assistant, Never Authority

The legal fraternity must internalize a simple but powerful principle: AI may assist in the administration of justice, but it must never assume its moral authority. In law, authority comes not from knowledge alone, but from the exercise of discretion in the face of ambiguity. A court decides not just what the law is, but what the law ought to be. This is precisely where AI fails. It cannot feel injustice. It cannot contemplate conflicting values. It cannot, like a conscientious judge, pause before a harsh ruling and wonder “Is this truly fair?” Consider the jurisprudence on constitutional morality, cases like Navtej Singh Johar v. Union of India45 or Indian Young Lawyers Association v. State of Kerala46 where the Court had to

45 Navtej Singh Johar v. Union of India, (2018) 10 SCC 1 (India).

46 Indian Young Lawyers Ass’n v. State of Kerala, (2019) 11 SCC 1 (India).

choose between majoritarian sentiment and individual dignity. No AI, however powerful, could have resolved that tension, because the answer lay not in logic, but in conscience. The future of legal AI must therefore be explicitly limited to clerical and research-based functions. Drafting suggestions? Yes. Precedent mapping? Certainly. But judgment formation? Never.

7.5 Institutional Blueprints: Towards a Holistic Regulatory Architecture

To operationalize this human-centric approach, India must construct a robust institutional framework. This includes: • Judicial Ethics Committees on AI: Comprising judges, ethicists, and technologists, these bodies can issue case-by-case opinions on appropriate AI use in judicial work. • A Bar Council-AI Taskforce: This would develop sectoral guidelines, model usage protocols, and certification mechanisms for legal AI systems. • AI Ethics Integration in Legal Education: Law schools must train students not just in AI literacy but in AI morality, how to question machine outputs, how to detect bias, and how to ensure that law remains a site of human dignity.47 • Cross-disciplinary Research Hubs: Universities must build AI-law incubators that explore indigenous models of ethical, transparent, and culturally aware legal AI systems.

7.6 Regulation as the Practice of Dharma

In the Mahabharata, Yudhishthira is told that law (nyaya) must not be seen as a code to be applied mechanically, but a duty (dharma) to be exercised wisely.48 The same lesson applies to our age of algorithms. AI may quicken the pace of justice, but it cannot substitute for the spirit of justice. To regulate AI in law, then, is to engage in nothing less than an act of dharma, preserving not only the procedural integrity of the system, but its moral soul. The real challenge 47 NITI Aayog, Discussion Paper: Responsible AI for All (2023), https://niti.gov.in/sites/default/files/2023-03/Responsible- AI-Discussion-Paper-2023.pdf.

48 Mahabharata, Shanti Parva – “Dharma is subtle, deeper than any script.”

is not building better machines, but building better guardians of the machines, judges, lawyers, and lawmakers who understand that in law, speed must never come at the cost of sagacity. In the end, justice cannot be outsourced, not to machines, not to code. It must remain a profoundly human task, born of deliberation, infused with conscience, and sustained by the quiet courage to choose rightly when no rule suffices.

8. Conclusion

Who truly bears responsibility when AI falters? The first question haunts the corridors of modern jurisprudence: when an algorithm goes wrong in shaping legal outcomes, who is answerable? Today, the answer is frustratingly vague. Developers write the code but disavow ethical accountability, framing AI as a neutral tool. Lawyers deploy it but treat errors as unfortunate by-products, not professional failures. Judges may lean on AI outputs for efficiency but rarely scrutinize them deeply. This diffuse responsibility creates a dangerous void, a legal and moral black hole. Law is anchored in accountability; it is not merely an administrative mechanism. The Indian legal spirit teaches that action without responsibility is adharma. Delegation of tasks does not equal delegation of duty.

When a lawyer relies on AI-generated research or arguments, the karma, the moral weight, still rests on their shoulders. To treat AI as a scapegoat is to shirk the fundamental spirit of legal practice, which demands conscious judgment, not mechanical reproduction. Without clear frameworks assigning liability for AI’s errors, justice risks becoming a hollow ritual where no one can be held to account for miscarriages caused by technological overreach. This is not just a gap in regulation but a crisis of professional conscience. Can constitutional morality ever be encoded? The Indian Constitution is a living document, infused with ideals of dignity, fairness, and reason that evolve with society. Can these profound values be distilled into lines of code? The answer is a cautious no.

Algorithms operate on data and logic; morality requires empathy, contextual understanding, and normative judgment, qualities that defy quantification.

Constitutional morality, as understood in Indian jurisprudence, is dynamic and deeply human. It thrives in the interpretive space where judges wrestle with conflicting values, social realities, and historical injustices. This interpretive process demands Viveka, discerning judgment rooted in lived experience and ethical insight. AI may replicate patterns of past judgments, but it cannot recreate the human capacity to perceive the “spirit” behind the letter. It cannot internalize the dignity of the marginalized or comprehend the anguish behind a fundamental rights violation. Its “reason” is limited to syntactic manipulation; it lacks the semantic and ethical depth that constitutional morality requires.

To entrust constitutional morality to machine code is to mistake simulation for essence, a perilous reduction that threatens to erode the very foundations of our democratic and pluralistic legal order. When efficiency overtakes empathy, what is lost? Efficiency is seductive, promising speed and cost reduction in overloaded courts. But legal reasoning is not a mere technical exercise, it is an act of human understanding. The substitution of empathy with efficiency risks transforming justice into an impersonal bureaucratic function. In the Indian tradition, justice is tied to nyaya, which is not only fairness but moral rightness. It requires patience, reflection, and the willingness to hear the unspoken. A court’s power does not rest solely in the verdict but in the process, the dialogues, the pauses, the humane discretion.

AI-driven legal processes, optimized for speed, risk erasing these human elements. When we prioritize algorithmic efficiency, we risk creating a legal system that produces correct answers without justice. The empathy necessary to balance law’s rigidity with life’s complexities cannot be programmed, it must be cultivated. Nietzsche’s metaphor of the abyss warns us, gazing too long into cold, indifferent systems erodes the human spirit. In AI, we face such an abyss, an unfeeling logic that threatens to hollow out the ethical roots of justice, replacing wisdom with calculation. But this descent is not inevitable. The Indian concept of Dharma offers an alternative, a living moral order that sees law not as code, but as conscience. Dharma insists that justice remain a

human vocation, guided by empathy, discernment, and courage, not by algorithmic output. The legal profession must reclaim this dharmic responsibility. AI must serve as a tool, not an authority. Lawyers and judges must preserve their moral agency; no machine can substitute for human judgment. The path forward lies in regulation, not rejection. Professional codes must demand transparency about AI use. Judicial ethics bodies must review AI-assisted decisions. Law schools must prepare lawyers to understand both the power and the limits of AI, always within the bounds of constitutional morality. Justice stands at a precipice, between moral code and machine code. The abyss stares, but we must not look away. Machines may calculate; only humans can care.

Only through vigilance and moral clarity can we ensure that efficiency does not replace conscience, and that justice remains, above all, a human act.

Bibliography Books and Ancient Texts Friedrich Nietzsche, Beyond Good and Evil (Helen Zimmern trans. , 1907). Manu, Manusmriti (G. Bühler trans. , Sacred Books of the East vol. 25, 1886). Jaimini, Mīmāṃsā Sūtra 1. 1. 2–1. 1. 5. Kautilya, Arthashastra, bk. 3. Mahabharata, Shanti Parva – “Dharma is subtle, deeper than any script. ” Academic Journals and Conference Proceedings Paul A. Teschner, Lawyer Morality, 47 Cath. U. L. Rev. 285 (1998). Joy Buolamwini & Timnit Gebru, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, in Proceedings of the Conf. on Fairness, Accountability and Transparency 77 (PMLR 2018). News Articles and Reports Abraham Thomas, Panjab & Haryana HC Uses ChatGPT for Views on Bail, Times of India, Mar. 29, 2023, at 1. Home Office Drops ‘Racist’ Algorithm from Visa Decisions, BBC News, Aug. 4, 2020, https://www. bbc.

com/news/technology-53650758.

Casemine, CaseIQ AI Briefing Tool, https://www.casemine.com/ OpenAI, Technical Report on Hallucination in Large Language Models, https://openai.com/research Supreme Court of India,

SUVAS:

Supreme Court Vidhik Anuvaad Software, https://main.sci.gov.in/supreme-court-vidhik-anuvaad-software

NITI

Aayog, Discussion Paper: Responsible

AI

for All (2023), https://niti. gov. in/sites/default/files/2023-03/Responsible-AI-Discussion-Paper-2023. pdf. Statutes and Regulations (India) Advocates Act, 1961, § 29 (India). Bar Council of India Rules, pt. VI, ch. II, Standards of Professional Conduct and Etiquette. Digital Personal Data Protection Act, No. 22 of 2023, Acts of Parliament, 2023 (India). Information Technology (Amendment) Act, 2008, § 43A (India). Case Law – India Navtej Singh Johar v. Union of India, (2018) 10 SCC 1 (India). Joseph Shine v. Union of India, (2019) 3 SCC 39 (India). Vishaka v. State of Rajasthan, (1997) 6 SCC 241 (India). State of W. B. v. Anwar Ali Sarkar, AIR 1952 SC 75 (India). Indian Young Lawyers Ass’n v. State of Kerala, (2019) 11 SCC 1 (India). State v. Shankar Rai, FIR No. 184/2022, Patiala House Cts. , New Delhi, Judgment dated Jan. 12, 2023 (India).

Punjab & Haryana High Court, Bail Order Referencing ChatGPT, 2023 (India).

Case Law – International Lola v. Skadden, Arps, Slate, Meagher & Flom LLP, 620 F. App’x 37 (2d Cir. 2015). Mata v. Avianca Inc., No. 22-CV-1461 (PKC), 2023 WL 4114965 (S.D.N.Y. June 22, 2023). Official Policy Documents & Opinions ABA Comm. on Ethics & Prof’l Resp., Formal Op. 498 (2021). ABA House of Delegates, Resolution 112 (2019), Encouraging Technological Competence and Ethical Use of AI in Legal Practice, Am. Bar Ass’n. European Parliament, Resolution on AI in Criminal Law and Its Use by the Police and Judicial Authorities in Criminal Matters, 2021/2061(INI) (2021). Proposal for a Regulation of the European Parliament and of the Council Laying Down Harmonised Rules on Artificial Intelligence (Artificial Intelligence Act), COM (2021) 206 final.