Generative AI’s Role In Data Pollution - Analysing Regulatory Framework In The European Union AI Act

Author: LawSphereX

Category: Technology, Cyber & Privacy

Published: 2026-08-23T19:58:12.771528+00:00

Updated: 2026-08-30T07:11:07.311Z

GENERATIVE AI’S ROLE IN DATA POLLUTION:

ANALYSING REGULATORY FRAMEWORK IN THE EUROPEAN UNION AI ACT Architha Ashok, Bcom LLB, Sastra Deemed to be University Madhumitha.K, Bcom LLB, Sastra Deemed to be University

ABSTRACT

Generative Artificial Intelligence (GAI) is a type of AI that can expeditiously and effortlessly generate realistic audios, videos, sounds and images. These technologies enable smooth integration of auditory, visual and textual data. Even though it has various advantages, it also contains substantial risks. In the case of “Mata v Avianca”, the advocates researched using Chat GPT and submitted fake case citations to a Newyork Court. Similarly AI-generated pornographic images of Taylor Swift, who is a popular American singersongwriter. The images were live for about 17 hours. GAI frequently creates fake and misleading contents through hallucinations. There are other several cases reported in various countries. Predictions disclose that human-created content may disappear and the entire internet content would be AI-generated by 2030.

Eventually these misleading contents may sway the public opinion. This can change the way in which people perceive a particular political event. For instance, in 2022, Ukrainian President Volodymyr Zelenskyy was shown in a deepfake video, urging his countrymen to submit to Russian soldiers. This demonstrates how deepfakes can sway public opinion and spread misinformation. Further, these false contents may pollute even historical records. Our research focuses on the influence of GAI on public perceptions and collective memory. First part will address the inter-relationship between Generative AI and Data pollution. Secondly the regulatory framework for GAI under European Union’s Artificial Intelligence Act will be analysed. Doctrinal Research Methodolgy will be used in this research.

BACKGROUND

The technology of GAI was first introduced in the year 1960’s through Chatbots. ELISA was the first historical example of GAI, which was created by Joseph Weizenbaum in 1961. ELISA was a computer program which could respond to humans by using pattern matching and substitution methodology. It just followed some simples rules and did not understand the content. After implementation of ELISA till the year 1990 there were no significant development in the field. In 1990s and 2000s Machine learning started to flourish with the development and accessibility of internet. In 2014 the implementation of Generative Adversarial Network which was proposed by lan Goodfellow, lead to the stage of creating realistic images, videos and audios. Subsequently other Generative Models were developed such as Variational Autoencoders (VAEs) and Recurrent Neural Networks (RNNs).

In 2017, NVIDIA developed Progressive GANs which were able to create images with novel details and clarity. This was followed with the introduction of GPT-2 and GPT-3 in the year 2019 and 2020, DALL-E in 2022, Chat GPT in 2022 and GPT-4 in 2023. Chat GPT was developed by Open AI which can create humanlike conversational dialogues by using natural language processing. Through Gartner’s prediction we can interpret that large scale enterprises usage of Generative AI would increase from 1% in 2023 to 50% in 2027. GAI also may lead to cybersecurity issues. For example, by using these technologies cyber criminals may create realistic phishing scams and hack into systems. Certain prediction also disclose that by 2030 all the internet content would be generated by AI which will lead to trust in algorithms over human.

LITERATURE REVIEW

1In the research “When AI Eats Itself: On the Caveats of AI Autophagy in the Era of Generative AI”, it is found that uncontrolled autophagy with synthetic data may result in decline of quality which may further lead to reduced accuracy levels. The regulatory framework includes marking of generated content which is made mandatory to avoid confusions among public and clear labeling of generated AI is enforced. 2From the research “Human favoritism, not AI aversion: People’s perceptions (and bias) toward

1 https://arxiv.org/abs/2405.09597

2 https://www. cambridge. org/core/journals/judgment-and-decision-making/article/human-favoritism-not-aiaversion-peoples-perceptions-and-bias-toward-generative-ai-human-experts-and-humangai-collaboration-inpersuasive-content-generation/419C4BD9CE82673EAF1D8F6C350C4FA8 generative AI, human experts, and human–GAI collaboration in persuasive content generation”,it can be inferred Chat GPT gives stochastic outputs since the first output and the fifth output for the same prompt created varied results. Chat GPT interprets repeated prompting as a sign that the researchers are not satisfied by the given results which is the reason for the difference in results in the further prompting.

3In the research “Generative AI and deepfakes: a human rights approach to tackling harmful content”, it is found that The EU AI Act’s high risk label does not clearly provide electoral rules for application in harmful AI. The act exempts some of the content creators and fair detection is not met by the deep fake detection In the research “Shall androids dream of genocides? How generative AI can change the future of memorialization of mass atrocities” it is found that enforcement of hegemonic narratives is possible with GAI. AI has the potential to erode confidence in the use of technology to commemorate mass crimes. It can be inferred from the research paper “The AI Act and the risks posed by Generative AI Models” that Generative AI does not come under the category of posing unacceptable risks because they are interpreted as foundational models.

Thereby the reason of Generative AI creating and spreading false information cannot bring it under the classification of unacceptable risk of EU AI Act. RESEARCH PROBLEM Extensive researches have explored about the risks posed by Generative AI and the legislations available to mitigate the risks. The research problem of this paper specifically focuses on the requirement of European Union Artificial Intelligence Act 2024, the current scenario of Generative AI regarding data pollution and future implications of European Union Artificial Intelligence Act on Generative AI.

OBJECTIVE

This research aims to study the data pollutions created by Generative AI. It assess the impact it creates in decision making processes and people’s perspective. The paper also inquires into the applicability of European Union AI Act to Generative AI and

3 https://www.tandfonline.com/doi/full/10.1080/13600869.2024.2324540

the liabilities it poses on Generative AI for the data pollution made by it. RESEARCH QUESTIONS 1)Whether data pollution created by Generative AI manipulates public perspective? 2)Whether the provisions under the EU AI Act addresses Generative AI with regard to the risk of creation of data pollution by it? 3)Whether the enforcement of the major provisions of EU AI Act in future would change the prevailing scenario of data pollution? WHAT IS GENERATIVE AI Generative Artificial Intelligence also commonly known as Gen AI that creates new and original content and ideas such as text, images, videos, codes, music, stories, conversations etc. as response to the user’s prompt. It can study and understand human language, programming language or any complex subject matter. It also uses already known data to solve problems.

Similar to other artificial intelligence, generative artificial intelligence also works by using machine learning models relatively very large models that are already trained on huge amount of data. These Large Language Models (LLMs) are powerful form of AI as they generate human-like text and users cannot distinguish human made texts and the GAI texts. CURRENT POSITION OF GENERATIVE AI 4The generative has a considerably shorter history in its growth compared to other technological developments. Though the biggest discoveries of GAI were during the late 2010 period but a tremendous growth has been witnessed in the past few years particularly after 2022, the year when ChatGPT was launched. The steady and rapid rate of growth of GAI has been led to the increase of it being used in many organisations and replacement of human workforce.

5It is found that GAI will perform at a median level in comparison to the human population and will increase to 25% by the end of 2040 which is almost forty years faster than what was predicted by the experts. 4 https://www. leewayhertz. com/current-state-of-generative-ai/#Addressing-challenges-associated-with-finetuning

5 https://www.solulab.com/current-state-of-generative-ai/

In the initial stage it was noted that it only affected physical work activities but at present it has also started to evolve in knowledge-based activities such as decision making and collaboration. The experts in the field of law, art, education, technology will soon be replaced as parts of their jobs will be automated by GAI. 6The McKinsey Global Survey on the current state of AI makes it evident that there is explosive growth in the use of GAI. In the survey conducted, over one-third of the respondents have confirmed that GAI is used at least in one of their business functions. It can also be inferred from the responses that the use of GAI is only widespread not only to tech employees but also among the business leaders. Over one quarter of the C-suite executives have stated that they use GAI in their work and more than one quarter have already included it in their board agendas.

Around 79% respondents have answered that they at least have had some amount of exposure to GAI either in work or outside of work and over 20% pare already using GAI regularly in their worn and forty have showed their interest in increasing their investments with regard to GAI. 7The GAI has affected industries such as Pharmaceutics, Manufacturing, Architecture, Media, Défense, Engineering, Automotives, Aerospace, Electronics and Medicine.

It is expected that by 2025 that more than 30% of the drugs will be discovered and developed using GAI techniques, around 30% of the marketing organizations will generate their outbound messages using GAI, 40% of the conversation in applications will by GAI, 30% of the enterprises will have implemented GAI for strategy and testing, and by 2026 the designs for applications and websites will be automated by GAI and over 100 million of human employees will have GAI robots as their counterparts at work and by 2027 almost 15% of the applications will be designed and developed by GAI without any human intervention. USE OF GENERATIVE IN VARIOUS INDUSTRIES In creative Industries GAI is collaborated with artists, designers and musicians to create novel and unique works. In Healthcare and Medical Industries GAI is used in drug discovery by generating molecular 6 https://www. mckinsey.

com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-aisbreakout-year#widespread 7 https://aws. amazon. com/what-is/generativeai/#:~:text=Generative%20artificial%20intelligence%2C%20also%20known,images%2C%20videos%2C%20an d%20music structures which increases the efficiency and expediency and in the medical imaging analysis in diagnosing accurately and the required treatment plans. 8In the Manufacturing and Engineering Industry it is used in developing designs, production process, product geometrics, testing strategies etc. , In the Fashion Industry it is used in creating patterns, textures, colour and the blending of colours, patterns and choosing the right fabrics for such patterns. In the Salesforce’s recent survey, it is found that over 73% of Indians, 49% of Australians, 45 of the Americans and 29% of the UK population use GAI among the general population.

At present more than 70% of the basic content creation, writing, creative thinking, image generation has been augmented using Generative Artificial Intelligence. THE RAPID AND STEADY INCREASE OF DATA POLLUTION CAUSED BY GENERATIVE AI 9The first and foremost setback of GAI is the generating information based on hallucination which are mostly filled with errors that are contrary to facts which are either illogical or incorrect. Since GAI can hallucinate it creates data which are not in existence. One such instance is when Googles Bard Chatbot claimed that James Webb Space Telescope captured the first image of a planet outside our solar system which is not true. Some reputed media houses post synthetic texts as if it were reported and manipulate the public. One such example is the synthetic reports by Tech Outlet CNET. 10The generation of false and outdated information by GAI.

An example in this regard is when a GAI model ChatGPT generated incorrect information by stating cibinqo is an inhibitor which supresses Janus Kinase enzymes and not a monoclonal antibody without any proper substantiating evidence and latest updates and development in that area. There are high chances of generating abusive and harmful content which affects the reputations of individuals and organisations.

8 https://www.salesforce.com/news/stories/generative-ai-statistics/

9 https://orcid.org/0000-0002-8796-4819

10 https://akademie.dw.com/en/generative-ai-is-the-ultimate-disinformation-amplifier/a-

68593890#:~:text=Generative%20artificial%20intelligence%20(GAI)%20adds,fake%20content%20in%20vast %20quantities Various experts have warned about generative artificial intelligence surpassing the human intelligence and posing a huge threat to the human population. 11The content created by GAI such as deepfakes shall cause misinformation and manipulation of real facts and data and affects the public opinion regarding the same. For instance, the fake news and social media posts created by GAI. The public shall not be in a position to distinguish between genuine and fabricated data. Speer in its documentation has stated that there is no more reliable information available in the internet after the increase of Generative Artificial Intelligence.

Earlier there was only spam in the data sources which was easily identifiable by and manageable but now in the world of GAI where large language models are used in generation of text and data, one is not able to make out between real and fabricated data. 12The freely available and unregulated tools allow anybody to generate false information and fake contents using the internet. The use of GAI in the imitation of images, videos and voices of people make it indistinguishable from the real ones. The inferences from a paper published in JAMA Ophthalmology, it was found that when AI is prompted to find data that supports a particular conclusion, it uses set of parameters and random datasets that are partially related to accomplish the end goals. Another problem with GAI is that the systematic reviews check and verify only the methodological quality and not the authenticity of the data.

The emergence of generative artificial intelligence has made the internet a haven for false information. One prominent instance was in mid-January 2023, when Bitcoin prices fluctuated due to the dissemination of a fabricated report claiming that the U. S. Securities and Exchange Commission (SEC) had authorised a spot-listed exchange-traded fund (ETF). The U. S. stock market collapsed in May 2023 when a fake image of a building close to the Pentagon in Washington, D. C. , engulfed in black flames was produced using generative AI.

11 https://subjectguides.uwaterloo.ca/chatgpt_generative_ai/falseoutdatedinfo

12 https://pmc.ncbi.nlm.nih.gov/articles/PMC11076802/

Another notable example would be the widespread misinformation regarding the Covid-19 virus, where the internet was bombarded with various medications, home remedies, campaigns against vaccinations, steroids etc which were baseless and misguided the public. 13The GAI is a big threat regarding the health and safety as it generates medications, treatments etc with no proper diagnosis. For instance, when symptoms such as headache, dizziness are stated it generates treatments and medications for brain tumour. The UN Secretary General Antonio Guterres regards this as “Existential Risk To Humanity”. MISINFORMATION AND MANIPULATION CAUSED BY GENERATIVE AI 14In 2014 Amazon worked on AI-powered recruiting software which was trained on 10 years’ worth of resumes which was submitted mostly by men and this led to the software rejecting women candidates.

Amazon stated it did not use the software and tried to make it gender neutral but eventually terminated that project. In 2016 Microsoft released Tay an AI chatbot which assumes a persona of a teenage girl and interacts with individuals via Twitter, which posted over 95,000 tweets within 16 hours of its launch were all such posts turned out to be overtly racist, misogynist and antisemitic. The Microsoft powered chatbot MyCity which was developed for providing information to New Yorkers was giving incorrect information which violated the law of the city and would have led to businesses breaking laws.

A video of Microsoft co-founder Bill Gates where he was cornered by a journalist in an interview was shared but later it was found that it was false and created by Generative AI 15In the year 2023, an attorney in New York used ChatGPT to refer legal precedents and ended up citing precedents believing it to be authentic but which was actually generated by hallucination and was imposed a fine of $5,000 for relying on data without verifying its authenticity.

13 https://www.sciencedirect.com/science/article/abs/pii/S1472811723000605

14https://www. ncbi. nlm. nih. gov/core/lw/2. 0/html/tileshop_pmc/tileshop_pmc_inline. html? title=Click%20on%20 image%20to%20zoom&p=PMC3&id=11076802_icu-65-199-g001. jpg 15 https://aws. amazon. com/what-is/generativeai/#:~:text=Generative%20artificial%20intelligence%2C%20also%20known,images%2C%20videos%2C%20an d%20music A robocall claiming to be President Joe Biden telling the residents of New Hampshire not to vote was a fake call created using GAI. 16A digitally altered video of US President Joe Biden calling for a mandatory enrolment of individuals into armed forces to fight the war in Ukraine which was believed to be authentic was a deepfake. IMPLICATION OF EU AI ACT ON GENERATIVE AI The potentiality of Generative AI in matters of medical diagnosis, education, planning, scheduling, chatbots is tremendous which is undeniable.

The experts of AI present a view that the further improvements of AI towards super intelligence would result in adverse conditions of humans. It is clear from the reported incidents that all the advantages of the AI comes with the negative and dangerous side. In Clarkson v Open AI, the US Court held that the generative AI creates and contributes to the spread of false information making it difficult for the people to identify the correct information. A statistical report was published in 2023 based on the responses of the organizations who have incorporated AI in their working. The report revealed that the organizations identified high risks of inaccuracy of AI generated content and are working further to mitigate the risks posed by Generative AI. Thereby inorder to address these issues posed by AI, the European Union implemented the European Union AI Act.

The EU AI Act is the first comprehensive legislation in regulating AI. It establishes a regulatory framework for Artificial Intelligence which ensures safety and ethical use. While regulating AI, it also aims to support innovation and ensure that Europe remains competitive in global AI landscape. It enumerates the responsibilities of AI developers, distributors, importers and its users, which ensures accountability in cases of misuses. The proposal for the EU AI Act was made by the European Commission in the year 2021. The EU AI Act came into force in 2024 and it would be applicable in the year 2026. The full enforcement of the Act will commence on 2026. We will further analyse whether the issue of data pollution by Generative AI is addressed by the EU AI Act and how does the Act categorise various types of AI.

The liabilities are fixed based on the levels of risk involved in various AI Models. The EU AI Act categorises the AI models based on four risk categories;

1) The unacceptable risk category

16 https://economictimes.indiatimes.com/news/how-to/how-ai-powered-tools-deepfakes-pose-a-misinformationchallenge-for-internet-users/articleshow/98770592.cms?from=mdr

2) The high risk category

3) The limited risk category

4) The minimal or no risk category

17Any violation of the provisions of the EU AI Act would result in paying fines which may extend upto €35 million. If the AI puts the fundamental rights of the people to threat, then it falls under the unacceptable risk category. For instance, categorising people based on social or economic status, usage of subliminal techniques to manipulate people’s behavior, indiscriminately collecting facial images from public sources to create facial recognition data base falls under the unacceptable risk category. The AIs which are identified as under the first category are prohibited under the Act. If the security of the people’s fundamental rights is negatively impacted by the AI, then it is considered as under the high risk category. These AI’s are not prohibited but people affected can raise complaints to the authorized officers.

18The AI’s that provide misleading and false information to users and create confusions are categorized under the limited risk category. For example, chatbots like ChatGPT and Gemini. The limited risk category AI has to follow the transparency requirements under the Act which includes labelling the contented generated by AI and the significant issues caused shall be reported to EU. There are no legal obligations for the AIs falling under the category of minimal or no risk category PROVISIONS OF EU AI ACT FOR THE ISSUE OF DATA POLLUTION BY GAI 19Recent researches have found that Generative AI could generate false information which is convincing to the public and thus result in spreading of inappropriate information. For instance, the research which was published in 2023, aimed to create a fraudulent article on the topic of neurosurgery by using GPT 3.

It contained completely fabricated content which was convincingly viewed to be a flawless paper. From this we can clearly understand that the Generative AI can generate fabricated content which is also convincing. The issue of data pollution is addressed in the European Union AI Act. AI which generates false information and fabricated content falls under the limited risk category. This category has to comply only with the transparency requirements such as disclosing to humans that the AI has generated the

17 https://www.lexology.com/library/detail.aspx?g=15c8edca-06a9-4700-9581-76fb136cde00

18 https://www.europarl.europa.eu/news/en/press-room/20240308IPR19015/artificial-intelligence-act-mepsadopt-landmark-law

19 https://oecd.ai/en/wonk/ai-act-manipulation-methods

particular content and undergo an evaluation process. 20The issue with regard to data pollution is not because people are not aware that they are interacting with AI. The issue is that Generative AI creates convincing fabricated content and people are fully aware that they are interacting with AI. In some situations there are cases where humans believe that they are interacting with a human. In such situation the transparency requirements posed by EU AI Act is sufficient. But considering the data pollution by Generative AI, EU AI Act provisions are not sufficiently addressing the issue. 21The Generative AI cannot be included in the unacceptable risk category because even though it leads to data pollution, it also plays a vital role by contributing to innovations which is the primary goal of the Act.

Therefore the risks posed by data pollution cannot be mitigated through the risk models of EU AI Act and there is need for stricter provisions with regard to spreading false information by Generative AI. REPORTED INCIDENTS OF DATA POLLUTION IN EU In Slovakia, before the parliamentary election an audio file was shared in the internet by thousands of users which contained the conversation between the Chairman of Slovakia and a journalist. It displayed a telephone conversation between them where they discussed hoe to manipulate elections in favor of a particular party. Experts were able to identify that it was a AI generated audio file due to its relatively low quality. Similarly a video of the Former head of the Ukranian Armed Force was doctored to make him accuse President Zelensky of being a traitor.

RESEARCH

METHODOLOGY

The primary focus of this research is around Generative AI, what impact it has created in different industries. It also analysed how these technologies lead to generation of false information and thereby manipulating the public perspective. The Doctrinal Research Method is used this paper. The data used were collected through news reports, statistical reports, legal journals and previous research findings. The reported incidents of data pollution by GAI were gathered, provisions of EU AI Act were interpreted and conclusions were drawn based on the analysis.

20 https://ceur-ws.org/Vol-3431/paper3.pdf

21 https://arxiv.org/abs/2405.09597

FINDINGS

Through this research, it is found that the hallucinatory nature of GAI would result in data pollution and it would thereby manipulate the public perspective. The provisions of EU AI Act is not sufficient in addressing the issue of data pollution by GAI since the GAI is categorized under limited risk category of EU AI Act. So there is a need for stricter regulations to mitigate the issue.

RECOMMENDATIONS

Specific provisions must be formulated addressing the risk of data pollution. The creators of AI must be made liable for the data pollution created by AI. Strong messages like messages which would influence the people about the voting, information about mass crimes etc.., must be published only after confirmation by traditional media.

SCOPE

AND LIMITATION The European Union AI Act was studied with regard to the data pollution created by AI. The paper is limited to the implications of the EU AI Act on GAI and further researches may be conducted with regard to initiatives taken by various countries inorder to handle the negative impact of Generative AI.

CONCLUSION

It is clear that GAI’s contribution in different industries is massive and its potentiality is undeniable. It has been actively involved in the discovery of drugs, involvement in business functions, enhancing the teaching and learning experiences and so on. In the same way the creating and spreading of false content is also caused due to the extensive use of GAI. A more stricter legislative framework is required to mitigate data pollution by GAI.