Balancing Innovation And Regulation - The Legal Challenges Of Governing Artificial Intelligence In Healthcare
Author: LawSphereX
Category: Technology, Cyber & Privacy
Published:
Updated:
BALANCING INNOVATION AND REGULATION: THE
LEGAL CHALLENGES OF GOVERNING ARTIFICIAL
INTELLIGENCE IN HEALTHCARE
Jasleen Kaur Bahri, BA LLB, Christ Academy Institute of Law Mohd Yasir Khan, Christ Academy Institute of Law Aaryansh Agrawal, Christ Academy Institute of Law ‘” Our future is a race between the growing power of technology and the wisdom with which we use it.” Stephen Hawking
ABSTRACT
This chapter will outline the moral and legal issues that artificial intelligence (AI) in healthcare raises and offer solutions. The definition of artificial intelligence (AI) will be briefly explained in the article and also it will provide an overview of current US and European trends and plans. This will help to focus the discussion on the moral and legal issues surrounding AIdriven healthcare. The consideration of the four main ethical issues— informed permission to use, safety and transparency, algorithmic fairness and biases, and data privacy—will also be in the article.. After that, the article will also examine five legal issues in the US and Europe: (1) liability; (2) safety and effectiveness; (3) data protection and privacy; (4) cybersecurity; and (5) intellectual property.
Artificial intelligence (AI) has brought forth several legal and ethical challenges for society, including those involving privacy and surveillance, bias and discrimination, and the role of human judgment. These challenges may also pose a philosophical difficulty. worries regarding more recent computer technology. Data breaches and new sources of inaccuracy have emerged as a consequence of its employment. Errors in the process or guidelines within the healthcare industry can have disastrous repercussions for the patient who was the mistake's victim. Since patients come into contact with medical professionals during times in their lives when they are most susceptible, it is vital to keep this in mind. As of right now, there are no clear rules in place to discuss the potential moral and legal ramifications of using artificial intelligence as a tool in healthcare.
Keywords: Artificial intelligence, ethical issues, healthcare, cyber security, technology, employment, victim, and legal ramifications.
1. INTRODUCTION
Artificial intelligence (AI), especially machine learning, and digital technologies are revolutionizing public health, medical research, and medicine. AI-based technologies are currently employed in health care in Organization for Economic Cooperation and Development member nations. Co-operation and Development (OECD), and the evaluation of its usefulness in low- and countries with a moderate income (LMIC1). According to the Secretary-General of the United Nations, AI and other new technologies can be safely deployed to assist the world in attaining the Sustainable Development Goals of the United Nations2, which would include the goals about health under Sustainable Development. However, there are international ethical, legal, commercial, and societal issues surrounding the use of AI in healthcare. Numerous of these worries are not specific to AI.
Healthcare practitioners, governments, and developers have faced challenges as a result of the usage of computers and software. 3After fifty years, artificial intelligence (AI) presents new and unique ethical problems that go beyond the domain of conventional regulators and players in the medical field. If artificial intelligence is to be extensively employed to enhance human health, to protect individuality, and to guarantee fair access to such technologies. Artificial intelligence (AI) has shown enormous promise in the realm of health, and it has already made significant strides in areas including drug development, genomics, imaging, pathology, and prevention4. AI could help healthcare professionals avoid mistakes and enable to concentrate on giving care and resolving difficult situations.
The possible advantages of these technologies, as well as the commercial and financial potential of artificial intelligence in healthcare, increased global use of AI. However, unbridled optimism in AI's potential advantages 5could lead to a tendency to turn to technology first when faced with challenging issues. For example, such "techno- 1 Donnelly, D. L. , 2022. First not harm: legal principles regulating the future of artificial intelligence in health care in South Africa. Potchefstroom Electronic Law Journal/Potchefstroomse Elektroniese Regsblad, 25(1). 2 Ho, C. W. L. , Soon, D. , Caals, K. and Kapur, J. , 2019. Governance of automated image analysis and artificial intelligence analytics in healthcare. Clinical radiology, 74(5), pp. 329-337. 3 Laptev, V. A. , Ershova, I. V. and Feyzrakhmanova, D. R. , 2021.
Medical applications of artificial intelligence (legal aspects and prospects). Laws, 11(1), p. 3. 4 Winter, J. S. and Davidson, E. J. , 2020, December. Harmonizing regulatory spheres to overcome challenges for the governance of patient-generated health data in the age of artificial intelligence and big data. In TPRC48: Winter, J. S. and Davidson, E. , 2019. 5 Governance of artificial intelligence and personal health information. Digital policy, regulation, and governance, 21(3), pp. 280-290. The 48th research conference on communication, information, and Internet policy.
optimism6" could worsen the situation by escalating the uneven distribution of access to medical technology both inside and between developed and developing nations. Like many information technologies utilized in the healthcare industry, artificial intelligence (AI) technologies are primarily developed by businesses or through public-private partnerships 7(PPPs), however many governments also work to develop and implement these technologies. Several of the biggest Technology businesses are creating new services and apps, which they either possess or finance. Numerous of these businesses have already amassed substantial amounts of data, particularly health data, and hold a great deal of power in both the economy and society.
Although these businesses might provide novel ideas, there are worries that they might eventually come to have too much influence over patients, providers, and governments. 8
1.1 TRENDS IN THE U.S.
The US Government's reports on AI during Barack Obama's administration focused on issues of fairness, safety, and governance in addition to applications of AI for the public benefit. The US AI strategy has changed to be more focused on free markets after Donald Trump became president. For example, in May 2018, 9the AI for American Industry Summit was held at the White House. The Trump Administration wants to reduce regulatory impediments to AI developments, according to one of the main conclusions from the summit breakout discussions. The Executive Office of the President said in July 2018 that one of the top Administration R&D budget priority areas for 2020 is American leadership in AI. Since Donald Trump took office, US AI policy has shifted to emphasize free markets more10. For instance, the AI for American Industry Summit was held at the White House in May 2018.
One of the key takeaways from the summit breakout sessions is that the Trump Administration wishes to lessen legislative barriers to AI advancements. American leadership in AI is one of the top 6 Chikhaoui, E. , Alajmi, A. and Larabi-Marie-Sainte, S. , 2022. Artificial intelligence applications in the healthcare sector: ethical and legal challenges. Emerging Science Journal, 6(4), pp. 717-738. 7 Guan, J. , 2019. Artificial intelligence in healthcare and medicine: promises, ethical challenges and governance. Chinese Medical Sciences Journal, 34(2), pp. 76-83.
8 Krishnan Ganapathy, M.N., 2021. Artificial intelligence and healthcare regulatory and legal
concerns. Telehealth and Medicine Today, 6(2). 9 Currie, G. and Hawk, K.E., 2021, March. Ethical and legal challenges of artificial intelligence in nuclear medicine. In Seminars in Nuclear Medicine (Vol. 51, No. 2, pp. 120-125). WB Saunders. 10 Hodge Jr, S.D., 2021. The medical and legal implications of artificial intelligence in health care-an area of unsettled law. Rich. JL & Tech., 28, p.405.
Administration R&D budget priority areas for 2020, according to the Executive Office of the President's July 2018 statement. 11
2. INTRODUCTION OF THE AI IN THE HEALTH SECTOR
AI may offer a unique chance to supplement and enhance the abilities of overworked healthcare professionals. However, just like in many other areas of the global economy, the use of AI in healthcare could have serious negative effects or influence on the workforce in healthcare12. It might limit, cut down on the size of the staff, and health professionals' abilities and force them to retrain to cope with the application of AI. Medical practice has been built on connections between providers for centuries. Cautious, and extra caution need to be used when implementing AI technologies so that they do not sabotage these kinds of connections. As digital technologies gain traction and spread, the Universal Declaration of Human Rights— which includes fundamental patient rights like informed consent, privacy, and dignity—may undergo significant revisions or perhaps be completely rewritten.
The effectiveness of AI. Depending, among other things, on the kind, quantity, and nature of the data and related information as well as the circumstances surrounding the collection of such data. The search for data from businesses or the government could jeopardize individuality and privacy for the service of economic gain or private which includes social and political rights. Like many information technologies utilized in the healthcare industry, artificial intelligence 13(AI) technologies are primarily developed by businesses or through public-private partnerships (PPPs), however many governments also work to develop and implement these technologies. Several of the biggest technology businesses are creating new services and apps, which they either possess or finance.
Numerous of these businesses have already amassed substantial amounts of data, particularly health data, and hold a great deal of power 14in both the economy and society. Access to healthcare is also being altered by AI technologies. AI health solutions are becoming more widely available outside of regulated healthcare environments, such as the workplace, social media, and educational institutions. Given the
11 İbrahim, M.A.K., 2023. Legal Challenges of Artificial Intelligence in Healthcare. In Algorithmic
Discrimination and Ethical Perspective of Artificial Intelligence (pp. 147-160). Singapore: Springer Nature Singapore. 12 Schönberger, D., 2019. Artificial intelligence in healthcare: a critical analysis of the legal and ethical implications. International Journal of Law and Information Technology, 27(2), pp.171-203. 13 Chikhaoui, E., Alajmi, A. and Larabi-Marie-Sainte, S., 2022. Artificial intelligence applications in the healthcare sector: ethical and legal challenges. Emerging Science Journal, 6(4), pp.717-738. 14 Price, I.I. and Nicholson, W., 2017. Artificial intelligence in health care: Applications and legal issues.
quick spread and developing applications of AI in healthcare, such as in reaction to the COVID-
19 pandemic15, governmental organizations, schools, charities, and nongovernmental national
ethics committees and groups are establishing guidelines for how governments and other organizations ought to employ and control these technologies wisely. These applications might support the broad application of AI to enhance human health and quality of life while reducing or getting rid of a lot of hazards and improper behaviors.
2.1 APPLICATION OF AI IN THE HEALTH SECTOR
2.1.1 IN HEALTH CARE
There are concerns that AI will eventually replace human decision-makers and clinicians in medicine. However, the consensus is that AI is improving clinical care and diagnosis more and more. 16 This is supported by earlier definitions of the function of computers in medicine (and regulations that classify AI as a support tool. AI is being studied for use in radiology and medical imaging, among other fields, to support diagnosis. Even while these applications are more common than others including AI, they are still quite new, and clinical decision-making does not yet regularly involve AI. 17AI is currently being tested for use in thoracic oncology radiological diagnostics. imaging, mammography, colonoscopy, pelvic and abdominal imaging, and brain imaging and dosage optimization in non-radiological applications 18(such as radiological treatment).
However, only a small number of these technologies have been tested in planned clinical studies. AI is comparable to humans, according to a recent study that compared deep learning algorithms with medical imaging specialists in the diagnosis of diseases and medical judgment in particular fields and applications in particular situations, but moreover because "few studies compare the performance or present externally validated results. " of deep learning models 15 Iliashenko, O. , Bikkulova, Z. and Dubgorn, A. , 2019. Opportunities and challenges of artificial intelligence in healthcare. In E3S Web of Conferences (Vol. 110, p. 02028). EDP Sciences. 16 Lau, A. Y. and Staccini, P. , 2019. Artificial intelligence in health: new opportunities, challenges, and practical implications. Yearbook of medical informatics, 28(01), pp. 174-178. 17 Househ, M. , Borycki, E. and Kushniruk, A. W. eds.
, 2021. Multiple perspectives on artificial intelligence in healthcare: opportunities and challenges. New York: Springer International Publishing. 18 Kwak, G. H. and Hui, P. , 2019. DeepHealth: Review and challenges of artificial intelligence in health informatics. arXiv preprint arXiv:1909. 00384.
and medical practitioners 19use an identical sample. Medical professionals may be able to diagnose patients more quickly and accurately as AI develops. Artificial intelligence (AI) has the potential to rapidly identify diseases including stroke, pneumonia, and breast cancer through imaging coronary heart disease by echocardiography and cervical cancer detection. A UN agency called Unit Aid Agency for Enhancing LMIC Infectious Illness Diagnosis and Treatment was established. In 2018 a collaboration with the Clinton Health Access Initiative to evaluate an AI-powered cervical cancer screening tool for India, Kenya, Malawi, Rwanda, South Africa, Zambia, and Africa. Assistance with diagnosis and assessment is needed in many low-income settings due to a chronic shortage of healthcare providers20, to lessen them their workload.
AI has the potential to foresee disease or significant health events before they arise. For instance, AI technology might be used to determine a person's relative risk of illness, which might be used to avoid lifestyle disorders like cardiovascular diabetes and illness. AI prediction might also be used to detect those in LMIC who have tuberculosis but are not treated by the medical system and so are unaware of their current situation. 21Using predictive analytics could stop further problems. birth asphyxia, among other avoidable morbidity and mortality in LMICs. The potential for new technology to lead to data breaches and inaccuracies is a worry that comes with using it. Mistakes can happen in the high-risk field of healthcare. potentially serious repercussions for the patient who is the victim of this mistake.
It's important to keep in mind that since patients enter interaction with clinicians during periods of their lives 22when they are most susceptible and When used properly, such AI-clinician collaboration can be successful when AI is employed to give evidence-based care and medical decision guidance to the physician (23AI-Health). It can deliver healthcare services in diagnostics, medication development, epidemiology, individualized treatment, and efficiency of operations. 19 Shaw, J. , Rudzicz, F. , Jamieson, T. and Goldfarb, A. , 2019. Artificial intelligence and the implementation challenge. Journal of medical Internet research, 21(7), p. e13659. 20 Amann, J. , Blasimme, A. , Vayena, E. , Frey, D. , Madai, V. I. and Precise4Q Consortium, 2020. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC medical informatics and decision making, 20, pp.
1-9. 21 Aung, Y. Y. , Wong, D. C. and Ting, D. S. , 2021. The promise of artificial intelligence: a review of the opportunities and challenges of artificial intelligence in healthcare. British medical bulletin, 139(1), pp. 4-15. 22 Van de Sande, D. , Van Genderen, M. E. , Smit, J. M. , Huiskens, J. , Visser, J. J. , Veen, R. E. , van Unen, E. , Hilgers, O. , Gommers, D. and van Bommel, J. , 2022. Developing, implementing and governing artificial intelligence in medicine: a step-by-step approach to prevent an artificial intelligence winter. BMJ health & care informatics, 29(1). 23 Sun, T. Q. and Medaglia, R. , 2019. Mapping the challenges of Artificial Intelligence in the public sector: Evidence from public healthcare. Government Information Quarterly, 36(2), pp. 368-383.
2.2 APPLICATIONS OF AI FOR HEALTH RESEARCH
One significant area of AI-based health research is the utilization of data generated for electronic health records 24(EHR). Utilizing such data could be challenging if the underlying IT system and databases don't stop low-quality or heterogeneous data from spreading. However artificial intelligence in electronic health data can be utilized for clinical care optimization, quality enhancement, and scientific research25. Before following the conventional route of producing guidelines, research publications, and clinical assistance tools, AI that is appropriately built and sufficiently trained can be beneficial. use electronic health records to unearth therapeutic best practices. Examining changes in clinical practice found in electronic health AI and data can help create new clinical practice models. of providing healthcare.
2.3 ETHICAL CHALLENGES
The question of whether AI "fits within existing legal categories or whether a new category with its special features 26and implications should be developed" is constantly being debated. The program AI in clinical practice holds great potential to enhance healthcare27, but it also raises moral questions that we currently need to address. For AI in healthcare to reach its full potential, four important moral concerns need to be addressed: (1) giving informed permission to use data, (2) security and openness, (3) algorithmic justice, and prejudices and (4) data privacy is a crucial thing to take into account. 24 Guidance, W. H. O. , 2021. Ethics and governance of artificial intelligence for health. World Health Organization. 25 Winter, J. S. and Davidson, E. J. , 2020, December.
Harmonizing regulatory spheres to overcome challenges for governance of patient-generated health data in the age of artificial intelligence and big data. In TPRC48: The 48th research conference on communication, information and internet Policy. 26 Morley, J. , Murphy, L. , Mishra, A. , Joshi, I. and Karpathakis, K. , 2022. Governing data and artificial intelligence for health care: developing an international understanding. JMIR formative research, 6(1), p. e31623.
27 Cath, C., 2018. Governing artificial intelligence: ethical, legal and technical opportunities and
challenges. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 376(2133), p.20180080.
The question of whether AI systems are acceptable is not just a legal one, but one that is equally divisive and political. 28The goal is to support policymakers in making sure that the morally challenging scenarios brought about by implementing AI in healthcare settings are addressed proactively. Transparency has been the main issue in most legal talks about artificial intelligence. The increasing use of AI in high-risk scenarios has raised the need for responsible, fair, and open governance and design of AI. 29Information comprehensibility and accessibility are the two most crucial elements of transparency. Details regarding Algorithm functionality is often designed with purposes that are challenging to obtain.
30It is claimed that computers with the ability to follow no set rules and pick up new behavioural patterns pose a threat to our ability to assign blame to the device's creator or operator. An alarm should be raised about the alleged "ever-widening" division since it endangers "both the moral fabric of society and the basis for liability notion in law. The application of AI could deprive us of a holder and be responsible for any harm that is caused.
3. CLINICAL CARE
AI can help doctors make difficult treatment decisions, identify vulnerable patients and at-risk individuals, integrate patient records during consultations, and detect clinical errors. 31AI could be utilized, for instance, in LMIC to manage antiretroviral therapy by anticipating HIV medication resistance and the course of the illness, to support doctors in maximizing treatment). However, clinical expertise and understanding of patients are crucial, and artificial intelligence won't replace clinical attention for the not too far off. If it did, medical professionals might exhibit "automation bias" instead of Thinking about whether an AI system satisfies their requirements or the patient's. As the usage of AI grows, healthcare professionals will need to make major changes to their clinical practices.
To give doctors more time to listen to patients, address their worries and anxieties, and inquire about unrelated social variables, AI could automate chores. However, still be concerned about their accountability 28 Ellahham, S. , Ellahham, N. and Simsekler, M. C. E. , 2020. Application of artificial intelligence in the health care safety context: opportunities and challenges. American Journal of Medical Quality, 35(4), pp. 341-348. 29 Lau, A. Y. and Staccini, P. , 2019. Artificial intelligence in health: new opportunities, challenges, and practical implications. Yearbook of medical informatics, 28(01), pp. 174-178. 30 Shaw, J. , Rudzicz, F. , Jamieson, T. and Goldfarb, A. , 2019. Artificial intelligence and the implementation challenge. Journal of medical Internet research, 21(7), p. e13659. 31 Gerke, S. , Minssen, T. and Cohen, G. , 2020.
Ethical and legal challenges of artificial intelligence-driven healthcare. In Artificial intelligence in healthcare (pp. 295-336). Academic Press.
and duty.32 Physicians will need to update their capacity to explain risks, forecast outcomes, and talk about trade-offs with patients 33as well as voice their moral and legal concerns with comprehending AI technology. Even if technology advances as expected, those advancements will be realized.
3.1 USE OF AI IN THE CLINICAL CARE
Beyond the doctor-patient connection, the application of AI in clinical care has resulted in several significant improvements. This article discusses four trends: • the changing role of the patient in clinical care; • the move from inpatient to outpatient care; • the application of AI to clinical" treatment provided outside of the established healthcare system; • resource allocation using AI as well as setting priorities. These trends all have moral ramifications.
3.1.1 PATIENT CARE
In the long run, artificial intelligence (AI) may transform the way individuals take care of their illnesses, particularly chronic ones like diabetes, heart disease, and mental health issues. 34 Significant patient responsibility already exists for their care, consuming medications, enhancing their food and nutrition, participating in physical activity, tending to wounds, or administering shots. 35AI could help with self-care, such as through risk 32 Wubineh, B. Z. , Deriba, F. G. and Woldeyohannis, M. M. , 2023, December. Exploring the opportunities and challenges of implementing artificial intelligence in healthcare: A systematic literature review. In Urologic Oncology: Seminars and Original Investigations. Elsevier. 33 Alanazi, A. , 2023. Clinicians’ views on using artificial intelligence in healthcare: opportunities, challenges, and beyond. Cureus, 15(9). 34 Molnár-Gábor, F. , 2020.
Artificial intelligence in healthcare: doctors, patients and liabilities. Regulating artificial intelligence, pp. 337-360. 35 Al Kuwaiti, A. , Nazer, K. , Al-Reedy, A. , Al-Shehri, S. , Al-Muhanna, A. , Subbarayalu, A. V. , Al Muhanna, D. and Al-Muhanna, F. A. , 2023. A review of the role of artificial intelligence in healthcare. Journal of personalized medicine, 13(6), p. 951.
assessment, health monitoring, and conversational agents (such as "chatbots") or the gadgets and tools created especially for people with disabilities. For some, a move to patient-based treatment may be empowering and advantageous. patients, others may find the extra duty burdensome, and it may restrict a person's ability to obtain official medical services. The increasing use of digital self-management tools and applications also prompts more general concerns about whether these tools and applications should be treated as "wellness" or as clinical applications, hence attracting stricter regulatory oversight applications," needless oversight from regulators. Numerous online self-management Technologies that are potentially in the "grey area" 36between these two classifications may pose a concern if patients utilize them for clinical or self-management of their diseases.
care, yet they are still mostly unregulated or can be used without first consulting a doctor.
3.1.2 MOVE FROM INPATIENT TO OUTPATIENT CARE
AI technologies are being used to help with the wider transition from hospital-based to homebased care, which includes telemedicine. They include virtual assistants to help with patient care and remote monitoring technologies like video-observed treatment for tuberculosis. Before the COVID-19 pandemic, more than 50 US healthcare systems were utilizing telemedicine resources. COVID-19 has deterred individuals in various contexts, such as visiting medical facilities, hastened and increased the use of telemedicine in 2020, and it's anticipated that this tendency will continue.
37The transition to home-based treatment has also been made easier in part by the rise in the usage of search engines—which depend on algorithms—to find medical information and by the rise in the quantity of text or speech chatbots for the field of medicine which has enhanced as natural language processing technology has advanced, a type of AI that makes it possible for machines to comprehend spoken language. Utilizing chatbots has additionally quickened throughout the COVID-19 epidemic. Additionally, AI technologies—like "just-in-time adaptive interventions"—38may take a more 36 Richardson, J. P. , Smith, C. , Curtis, S. , Watson, S. , Zhu, X. , Barry, B. and Sharp, R. R. , 2021. Patient apprehensions about the use of artificial intelligence in healthcare. NPJ digital medicine, 4(1), p. 140. 37 Amann, J. , Blasimme, A. , Vayena, E. , Frey, D. , Madai, V. I.
and Precise4Q Consortium, 2020. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC medical informatics and decision making, 20, pp. 1-9. 38 Alugubelli, R. , 2016. Exploratory study of artificial intelligence in healthcare. International Journal of Innovations in Engineering Research and Technology, 3(1), pp. 1-10.
proactive approach to managing patients' health outside of hospital settings. These use sensors to deliver targeted therapies to patients, as per information gathered both now and in the past; they also alert a healthcare professional to any growing issue. 39Wearable technology and sensor developments could lead to improved concerns about the efficacy of "just-in-time adaptive interventions" as well. given.
3.1.3 ESTABLISHED FORMALLY OUTSIDE HEALTHCARE SYSTEM
Since AI technology for health may be easily obtained and used by non-health system entities, AI applications in health are no longer limited to usage in healthcare systems (or home care). As a result, health services are increasingly accessible to individuals outside of the healthcare system. Applications of AI for mental health, for instance, are frequently supplied via social media, the workplace, and the educational system, and may even be connected to banking. Although there might be justification for such extensive uses of health applications to make up for the shortage of resources and the rise in demand. People may need to be closely watched—and encouraged to self-monitor—even when they are not ill or considered "patients"—in light of these three tendencies. Wearables 40and mobile health apps are necessary for AI-guided technologies, and their adoption has grown.
with the inclination toward self-reliance. Among the wearable technologies are those that are put within the body (smart implants, prosthetic limbs), externally (insulin pump patches, electroencephalogram machines) or near the body (smart watches, activity trackers. 1. 5 billion wearable devices might be bought yearly by 2025. 41Wearables will make it easier to track a person's health and collect more information to forecast health hazards, frequently more quickly and efficiently.
3.2 CHALLENGES POSED BY THE AL
Healthcare AI presents novel issues, including bias and accountability when patient medical records are shared or pilfered. Researchers feel that artificial intelligence (AI) has significant 39 Gadde, S.S. and Kalli, V.D., 2021. Artificial Intelligence at Healthcare Industry. International Journal for Research in Applied Science & Engineering Technology (IJRASET), 9(2), p.313. 40 Secinaro, S., Calandra, D., Secinaro, A., Muthurangu, V. and Biancone, P., 2021. The role of artificial intelligence in healthcare: a structured literature review. BMC medical informatics and decision making, 21, pp.1- 23. 41 Jiang, F., Jiang, Y., Zhi, H., Dong, Y., Li, H., Ma, S., Wang, Y., Dong, Q., Shen, H. and Wang, Y., 2017. Artificial intelligence in healthcare: past, present and future. Stroke and vascular neurology, 2(4).
potential to boost the economy, but the technical idea of AI is also contentious and difficult to define precisely. The proposed AI Act is the first complete attempt to govern AI legally, but because multiple disciplines are affected, European authorities and other consultation stakeholders found it difficult to agree on a definition of AI. The proposed AI Act is the first comprehensive attempt to govern AI legally, but because many disciplines are affected, European authorities and all stakeholders in the consultations found it difficult to agree on a definition of AI . My hypothesis that there is insufficient regulation of the use of AI systems in healthcare is confirmed by research and recent regulatory challenges like the rise of ChatGPT-
4 as a chatbot possibly used in medicine that may threaten public health. All parties involved
recognize that data accuracy is a concern since accurate data is necessary for clinicians and laypeople to rely on. Predictions made by biased AI systems will inevitably result in discrimination. To guarantee the effective implementation of international standards and best practices in the development of AI models, a group of authors developed an Ethics Framework for Big Data in Health and Research. 42 To enable healthcare providers to identify mistakes or potential biases in AI models and algorithms that could result in medical malpractice liability or issues related to the ethical design of pathology AI studies, developers should also implement performance indicators to measure AI success.
According to certain authors, there is no doubt that digitalization and technological innovation have led to benefits such as personalization of care, decreased hospitalization rates, and improved efficiency and cost containment of services and waiting lists. However, these benefits also call for a review of the traceability and control systems along with a revolution of traditional ICT systems For healthcare systems to heavily rely on AI and robotics, traditional ICT systems 43must undergo a radical transformation. The ability to convert aggregated data from currently dispersed data sources into insightful knowledge for business decision-makers has become essential to data analysis. However, there are additional hazards and issues like fairness associated with the technological challenges of using computational models and AI algorithms in healthcare and additionally.
Key concerns are also data availability and quality. As previously indicated, AI systems need a lot of data to function and learn from. Due to its sensitive nature, confidentiality, ethical 42 Rong, G. , Mendez, A. , Assi, E. B. , Zhao, B. and Sawan, M. , 2020. Artificial intelligence in healthcare: review and prediction case studies. Engineering, 6(3), pp. 291-301. 43 Chavda, V. P. , Patel, K. , Patel, S. and Apostolopoulos, V. , 2023. Artificial intelligence and machine learning in healthcare sector. Bioinformatics tools for pharmaceutical drug product development, pp. 285-314.
concerns, and privacy laws like the GDPR and the Health Insurance Portability and Accountability Act (HIPAA) of 1996, collecting, processing, and exchanging health data can be quite challenging. Furthermore, there is a possibility of bias in health data, which could impact the effectiveness and impartiality of AI systems. Healthcare providers must guarantee that the data used by AI systems is accurate, secure, and representative of the population in question.44
3.3 PRIVACY OF PERSONAL HEALTH DATA
Clear criteria and norms are necessary, as previously said to guarantee that AI is applied to improve healthcare systems globally by the concepts of justice and health equity. Guidelines and recommendations at the national, regional, and worldwide levels must be as specific as feasible, taking into account several crucial issues such as ethics, informed consent, security, accuracy, and transparency. 45 It will be necessary for public authorities to establish new regulatory organizations or grant current Watchdogs additional authority and responsibilities. Regulatory agencies like the Medicines and Healthcare Products Regulatory Agency in the UK and the Food and Drug Administration (FDA) in the US play a vital role in audits and inspections by keeping an eye on all stakeholders and making sure they adhere to their obligations regarding privacy, efficiency, safety, and quality.
The promotion of accountability and transparency is essential because IT corporations are aware that their sharing practices could have serious repercussions, including financial penalties and other negative outcomes. Today's EU law provides comprehensive regulations and recommendations around data handling and privacy. The General Data Protection Regulation (GDPR) is an important legislative framework that imposes strict duties and obligations on manufacturers and service providers. A recent proposal for a European Data Act was made by the European Union Commission to adequately regulate data that is processed, stored, or exchanged, including health data. 46 The European Data Act was agreed upon as a provisional agreement in June 44 Stranieri, A. , Venkatraman, S. , Minicz, J. , Zarnegar, A. , Firmin, S. , Balasubramanian, V. and Jelinek, H. F. , 2022.
Emerging point of care devices and artificial intelligence: Prospects and challenges for public health. Smart Health, 24, p. 100279. 45 Morande, S. , 2022. Enhancing psychosomatic health using artificial intelligence-based treatment protocol: A data science-driven approach. International Journal of Information Management Data Insights, 2(2), p. 100124. 46 Alhuwaydi, A. M. , 2024. Exploring the Role of Artificial Intelligence in Mental Healthcare: Current Trends and Future Directions–A Narrative Review for a Comprehensive Insight. Risk Management and Healthcare Policy, pp. 1339-1348.
2023 by the European Parliament and the Council presidency (EU Council 2023a). The
European Data Act's wording was approved by the European Parliament on November 9,
2023 (European Parliament 2023).
The Data Act became operative on January 11, 2024. Nevertheless, the Data Act's primary requirements won't be in effect until September 12, 2025. Harmonizing laws about equitable access to data and its use by public and commercial entities is the aim of the Data Act. The Data Act, like its predecessor the GDPR, will make it easier for patients to maintain control over their health data. It might also establish important worldwide norms for the security and privacy of health data, acting as a model law or set of guidelines for the rest of the world. Since the AI Act specifically addresses AI, it is yet another important contribution to the EU legal system.
3.4 INFORMED CONSENT TO USE
Applications of health AI in imaging, diagnosis, and surgery will change the patient-physician dynamic. However, how will the concepts of informed consent interact with the application of AI to support patient care? Even though obtaining informed consent will be one of the biggest obstacles to integrating AI into clinical practice, this is an urgent issue that has not gotten enough attention in the ethical discourse (informed consent to train AI is a different issue that we will not address here. Examining when—if at all—the informed consent standards should be used in the context of clinical AI is necessary.
To what degree do physicians have an obligation to inform patients about the intricacies of artificial intelligence (AI), including the type(s) of machine learning that the system employs, the nature of the data inputs, and the potential for biases or other flaws in the data being used? When is it necessary for a clinician to tell a patient that artificial intelligence is being used? These are particularly tough concerns to respond to when the AI uses "black-box" algorithms, which can be the consequence of noninterpretable machinelearning methods that are extremely hard for physicians to completely comprehend (. For example, Corti's algorithms are a "black box" because not even Corti's creator is aware of how the program determines whether to notify emergency dispatchers that a person is experiencing a cardiac arrest. Medical experts may find this lack of knowledge concerning.
47For instance, how much must a doctor reveal if they are unable to fully understand the AI's recommended diagnosis or course of treatment? Additionally, the usage of AI health apps and chatbots is growing. These applications range from food advice to health evaluations to medication adherence assistance and data analysis from wearable sensors. Bioethicists are left wondering about these apps' user agreements and how they relate to informed consent. 48A user agreement is a contract that a person consents to without having a face-to-face conversation, in contrast to the conventional informed assent procedure. The majority of individuals frequently disregard user agreements because they are too complicated to understand. In addition, the software is updated frequently, which makes it much harder for users to abide by the terms of service they have accepted.
3.4.1 SAFETY AND TRANSPARENCY
The datasets that are used must first be genuine and dependable. In this context, the adage "garbage in, garbage out" applies to AI. The AI will function better the better the training data (labeled data) is. Furthermore, for the algorithms to produce correct results, additional tuning is frequently required. Another major problem is data sharing: Large volumes of data and hence greater data sharing will be required in situations where the AI must be very confident (e. g. , self-driving automobiles) [. Less data will be needed in other situations, though (such as a restricted emotion AI based on text) . Generally speaking, the amount of data needed is contingent upon the specific AI and the tasks it is assigned. Additionally, AI developers must disclose enough information, for instance, regarding the type of data that is used and any flaws in the software (such as data bias).
Examples like Watson for Oncology, where IBM concealed Watson's risky and inaccurate treatment suggestions for more than a year, should serve as a lesson to us. Last but not least, openness fosters trust among all parties involved, especially between physicians and patients, which is essential for the effective application of AI in clinical practice.
47 Sodhro, A.H. and Zahid, N., 2021. AI-enabled framework for fog computing driven e-healthcare
applications. Sensors, 21(23), p.8039. 48 Sharma, K. and Manchikanti, P., 2020. Regulation of artificial intelligence in drug discovery and health care. Biotechnology Law Report, 39(5), pp.371-380.
3.4.2 DATA PRIVACY
The Royal Free NHS Foundation Trust was found to have violated the UK Data Protection Act
1998 in July 2017 when it gave Google DeepMind access to the personal information of about
1.6 million patients. This decision was made by the UK Information Commissioner's Office
(ICO). Data sharing took place in connection with the clinical safety assessment of the "Streams" app, which attempts to aid in the identification and diagnosis of acute renal damage. However, as part of the test, patients were not adequately informed about how their data was processed. Elizabeth Denham, the Information Commissioner, accurately noted that "fundamental privacy rights do not have to be compromised as a price of innovation."
3.4.3 ALGOTHERMIC FAIRNESS AND BIASES
AI has the power to enhance healthcare not just in affluent environments but also in democratizing knowledge, "globalizing," and bringing it to even the most remote places. All machine learning systems and human-trained algorithms, however, are only as reliable, efficient, and equitable as the training data. AI is equally susceptible to prejudice and discrimination. Therefore, AI developers must understand this risk and take steps to reduce any potential biases throughout the whole product development process. When choosing which machine learning technologies/procedures to employ to train the algorithms and which datasets (taking into account their quality and variety) to utilize for the programming, they should pay special attention to the possibility of biases.
Algorithm biases about gender, ethnic origin, skin color, and/or other factors have been shown in several real-world situations. Biases about other characteristics, including age or disability, might also exist. These biases have a variety of different and sometimes complex explanations. These may arise, for instance, from the datasets themselves (which may not be representative), from the selection and analysis of the data-by-data scientists and machine learning systems, from the environment in which artificial intelligence is applied [64], etc. Biased AI could potentially compromise the safety of some subpopulations in the health sector by causing incorrect diagnoses and treatment inefficacy, as phenotype- and occasionally genotype-related data are involved.
Consider an AIpowered clinical decision support 49(CDS) tool that assists medical professionals in determining the most effective course of action for patients with cutaneous melanoma. 49 Germani, A. , 2023. The Politics of Artificial Intelligence in Healthcare: Diagnosis and Treatment. In AI and Society (pp. 33-48). Chapman and Hall/CRC.
Nonetheless, the majority of patients used to train the algorithm were Caucasian. As a result, the AI program is likely to provide suggestions that are less accurate or even erroneous for subpopulations like African Americans for which the training data was inadequate.
5. CONCLUSION
Healthcare will employ AI more and more; therefore, it must be morally responsible. To prevent data bias, the right algorithms based on unbiased real-time information, inclusive and diverse programming groups, and regular examinations of the algorithm, including how it was applied to a system, must be completed. Although AI might not be able to fully instead of clinical judgment, it can assist physicians in making superior choices. If a situation calls for a lack of medical competence AI might be used to do screening with less funding. and assessment. Unlike decision-making by humans, every AI Even the fastest decisions is methodical because algorithms are engaged in the matter. Therefore, even in cases where actions lack legal consequences.
Encouraging the creation of meaningful training datasets, introducing a standardized method for measuring and implementing AI systems, and adopting a common interoperable software architecture for research and therapeutic purposes are among the more pressing issues. As we've spoken about, researchers' ability to access a substantial amount of high-quality health data will be critical to the development of AI applications. Upscaling IDx-DR will undoubtedly involve this, and developing software of a similar nature may provide difficulties. The FDA's De Novo premarket review pathway now has a higher regulatory threshold because a diabetic retinopathy AI diagnostic gadget is already on the market.
We've also talked about the four main moral issues that must be resolved in order to fully utilize AI in healthcare: (1) use with informed consent; (2) security and openness; (3) algorithmic biases and fairness; and (4) privacy of data. After that, five legal issues in the US and Europe were examined. These issues are as follows: (1) safety and efficacy; (2) liability; (3) data protection and privacy; (4) cybersecurity; and (5) intellectual property legislation. To guarantee that AI is successfully applied in a way that is morally and legally acceptable, it is especially important that all stakeholders—including AI developers, patients, healthcare professionals, and regulatory authorities—cooperate to address the problems that have been identified.
To successfully create an AI-driven healthcare system based on the motto Health AIs for All of Us, a number of important factors must be considered and addressed, including informed consent, high standards of safety and effectiveness, high levels of data protection and privacy, cyber resilience and cybersecurity, algorithmic fairness, an adequate level of transparency and regulatory oversight, and an ideal liability regime for AIs. In this sense, we must do more than just reconsider the legal frameworks that are in place and adapt them to the latest advancements in technology.