The Indispensable Human Element: Why Human Supervision is Not Optional in Medical AI
With health systems in many parts of the world facing escalating pressures from demand increase amidst resource constraints, artificial intelligence (AI) becomes a good prospect for improving different aspects of treating a patient. As promising as AI is in, say, diagnostic imaging, predictive analytics, and surgical assistance, there are critical questions regarding the place of human judgment in medical decision-making. Although this emerging technology brings unparalleled computational power, AI can only be implemented into the healthcare sector if and only if there is human supervision due to inaccuracies in AI models, in order to maintain doctor-patient trust.
Key Definitions
For clarity, I will define some key terms it is essential to know. Artificial intelligence, popularly known as AI, is the umbrella of techniques used in order to develop systems endowed with intellectual processes characteristic of humans, such as the ability to reason, discover meaning, generalize, or learn from past experiences. (Adapted from Britannica, 2024). Machine learning, one technique that is in the umbrella of artificial intelligence, is the technique where computers can learn from past experiences (Mintz & Brodie, 2024). Finally, neural networks are computational models that mimic complex functions of the human brain (GeeksForGeeks).
Current Landscape of AI in Medicine
The current landscape of AI in medicine reflects a diverse range of applications across different specialties, each at varying stages of development and implementation. Medicine, being a field that covers all aspects of the human body, has seen different specialties implementing AI to accomplish distinct missions in their respective fields.
In radiology, for instance, AI is being developed to enable faster diagnosis in response to the shortage of trained specialists and increased workloads that are causing more stress (Mintz & Brodie). Technology has shown particular promise in specialties such as oncology, cardiology, and ophthalmology, where detection rates for certain cancers and diseases have improved with AI assistance.
The surgical field represents another frontier in medical AI, though progress here has been more measured. While robotic surgeries have become common practice for certain procedures, the effective integration of artificial intelligence in surgical decision-making remains in its early stages. Despite current limitations, researchers like Mintz and Brodie predict that AI “assisting surgeons operating a single patient is inevitable” (Mintz & Brodie). As infrastructure continues to evolve, it will be interesting to see whether this prediction holds true.
Currently, AI systems range from diagnostic tools that analyze medical imaging to predictive models that assess patient outcomes. These systems process vast amounts of medical data, attempting to identify patterns and making recommendations that could support clinical decision-making. The algorithms behind these machine learning models are mathematically advanced; however, according to Zhang, “lack explainability and are difficult to proofread” (Zhang & Zhang).
Regulatory Framework
Another important aspect to look at is that the regulatory and liability framework surrounding medical AI is still in development. As Smith notes in the journal AI & Society, “literature has not predicted the outcomes of negligence and liability…due to lacking case law.” This creates a complex environment where healthcare institutions must balance the potential benefits of AI implementation against uncertain regulatory requirements and liability considerations.
Clinical Application
In terms of clinical application, medical AI currently operates primarily in a supportive capacity, augmenting rather than replacing human medical expertise. This approach aligns with the caution expressed by various researchers in the field. Even in areas where AI has shown promise, such as diagnostic imaging, the technology is positioned as a tool to aid human practitioners in highlighting information that might not be visible to the naked eye, rather than as an autonomous decision-maker (Mintz & Brodie).
The Promise and Peril of Medical AI
This fast-changing landscape of medical AI is immensely promising with many machine learning models able to detect certain illnesses better than humans. According to researchers in Saudi Arabia, certain models were better able to detect early stages of breast cancer with the models outperforming radiologists 91% compared to 74%. Furthermore, these AI models can “help identify abnormalities….and provide quantitative measurements for faster and more accurate medical diagnosis” (Alowais, S.A., Alghamdi, S.S., Alsuhebany, N. et al, 2023).
With all these improvements the medical industry could make, there does not seem to be anything wrong. However, artificial intelligence is fallible. A bright light needs to be shined on some critical vulnerabilities and limitations. Mathematically complicated algorithms resisting any easy interpretation, combined with the absence of comprehensive liability frameworks and high stakes in medical decision-making, create a perfect storm for potential risks. These challenges, combined with documented AI failures in medical settings, point to an urgent need to look at why human supervision must remain an integral, non-negotiable component of the implementation of medical AI.
The Collaborative Nature of Medical Decision-Making
When looking at decision making in the medical systems, it is not all one person doing the thinking. Physicians collaborate with each other, talk over cases, and mostly make decisions as a unit in order to decide how to move along a patient’s prognosis. When looking at artificial intelligence systems, a lot of this is just a one computer doing everything. As a common phrase goes, there is strength in numbers. When you have many talented doctors looking at the same case, you have years of experience looking at the case with all different backgrounds which allows doctors and moreover the patients to make a well-informed decision.
In the journal European Radiology, research led by Lea Strohm, a Data and Model Ethics Analyst, they asked many radiologists on how they felt about artificial intelligence systems. Many radiologists in the field are skeptical of these upcoming AI applications. In a semi-structured interview of 24 different radiologists, ten doctors were concerned about the consistency of the model’s technical performance. The researchers clarified that this means if the model has an increase of false positives or false negatives, the consistency is not good. Either one is terrible for obvious reasons. If a patient does not know of a certain illness, that could cost them their life. On the contrary, if a patient thinks they have an illness but they don’t, doctors will tell them to take certain medications. Medications are never supposed to be taken unless there is a valid reason and can have severe side effects causing greater harm.
Concerns by radiologists in this study further underline the critical need for human supervision in medical AI. Considering ten out of the 24 interviewed radiologists explicitly discussed concerns regarding the technical performance of this model underlines that professionals are not a passive user group in the context of technological innovation but critical judges of the capabilities of AI.
The Trust Question
This raises the hypothetical scenario. If the doctor doesn’t fully trust a model, how should a patient, who we can assume has little to no medical background, trust it with their health and well-being. Now, if we change this hypothetical to where the doctor consulted the AI model for advice, consulted his/her colleagues for advice on how the patient’s prognosis should proceed, the patient can trust that the doctor did not fully rely on the AI. Instead, the doctor used the AI as an aid instead of as the decision maker. The patient can trust the doctor in this scenario to use his experience, the experience of other doctors, and the computational complexities of an AI model to receive the best decision making.
This heavily focuses on the fact that the doctor cannot just trust the AI model to give out the correct decision and based on the interviews, about 40% don’t. This skepticism is good as it will prevent overreliance on these models and still make the good doctors use their own judgment. This skepticism is not a result of resistance to technological advancement but rather from an abiding commitment to patient safety. This research by Strohm perfectly shows that human supervision is still crucial, for even the most advanced systems of artificial intelligence are unable to match a trained medical professional in depth of judgment, contextual understanding, and ethical reasoning.
Patient Trust and Accountability
Another pillar of this debate we have to look at is the patient trust aspect. Undoubtedly, this is the most important and most difficult to succeed at. Many patients are with their doctors for years if not decades. Transferring that trust to some robot is dystopian almost. In the journal AI & Society, Helen Smith talks about the relationship between patients and doctors. Traditionally, patients are ok with not knowing exactly why a doctor is making a certain decision. In other words, “it’s acceptable for people to be opaque, but not the [AI]” (Smith, 2020). While some may look at this as a double standard, it could be countered by saying that humans are held accountable for their actions while companies behind artificial intelligence models are rarely if ever held at fault for what their software did.
Conclusion
The integration of AI into medical practice does mark a critical juncture for ensuring that technological potential is balanced with the realities of human expertise. And as the evidence really illustrates, AI cannot replace and should not replace human judgment in medicine. Rather, it must remain a supporting tool to augment the nuanced decision of the trained professional who brings critical contextual understanding, ethical reasoning, and accountability that no algorithm will ever replace.
Ultimately, the future of healthcare does not lie in the choice between human doctors and artificial intelligence but in forging a collaborative partnership where technology enhances human capability while remaining firmly under human supervision. The stakes are simply too high—human lives are too precious—to surrender medical decision-making entirely to computational systems that lack the depth of human judgment and empathy.
Works Cited
Zhang, Jie, and Zong-ming Zhang. “Ethics and Governance of Trustworthy Medical Artificial Intelligence.” BMC Medical Informatics and Decision Making, vol. 23, no. 1, 13 Jan. 2023, link.springer.com/article/10.1186/s12911-023-02103-9, https://doi.org/10.1186/s12911-023-02103-9. Accessed October 27, 2024.
Mintz, Yoav, and Ronit Brodie. “Introduction to Artificial Intelligence in Medicine.” Minimally Invasive Therapy & Allied Technologies, vol. 28, no. 2, 27 Feb. 2019, pp. 73–81, www.tandfonline.com/doi/full/10.1080/13645706.2019.1575882, https://doi.org/10.1080/13645706.2019.1575882. Accessed October 27, 2024.
Uygun İli̇khan Sevil, et al. “How to Mitigate the Risks of Deployment of Artificial Intelligence in Medicine?: Turkish Journal of Medical Sciences.” Turkish Journal of Medical Sciences, vol. 54, no. 3, 1 May 2024, pp. 483–492, https://pubmed.ncbi.nlm.nih.gov/39050000/, https://doi.org/10.55730/1300-0144.5814. Accessed October 27, 2024.
Smith, Helen. “Clinical AI: Opacity, Accountability, Responsibility and Liability.” AI & Society, vol. 36, 25 July 2020, https://doi.org/10.1007/s00146-020-01019-6. Accessed October 27, 2024.