AI in healthcare must be integrated with strict human oversight to expand access to medical services while safeguarding patient safety, Delhi High Court judge Justice Prathiba M Singh said on Friday, highlighting the urgent need for responsible adoption of artificial intelligence in the health sector.
Speaking at the “Catalysing Global Investment for Equitable and Responsible AI in Health” session during the AI Impact Summit, Justice Singh underscored that the growing shortage of medical professionals in India and across the world makes the integration of AI not just beneficial, but necessary. However, she cautioned that technology must remain patient-centric and subject to regulatory supervision to prevent harm.
Also read: Galgotias University Directed to Vacate AI Impact Summit Expo Amid Chinese Robodog Controversy
AI Essential Amid Global Shortage of Medical Professionals
Justice Singh said artificial intelligence offers a viable solution to bridge healthcare gaps, particularly in remote and underserved regions across India, Africa, South America and Southeast Asia.
According to her, the lack of adequate medical personnel globally makes AI-driven tools critical for expanding outreach and improving healthcare delivery. She stressed that while AI has transformative potential, it cannot operate independently of human judgment.
“AI can be implemented, it is required to be implemented because we do not have enough medical professionals either in the country or the world,” she said, adding that a patient-centric approach must guide all AI-health initiatives.
Human Oversight Critical to Prevent Harm
The judge warned that deploying AI systems in healthcare without proper human supervision could undermine their purpose and pose serious risks to human life.
“The patient is at the core of this initiative, and with human oversight because without human oversight, AI in health will be a failure; it could lead to huge amounts of damage to human life,” she said.
Justice Singh emphasized that accountability mechanisms and structured oversight must form the foundation of any AI-based healthcare system to ensure safety, ethical compliance, and trust.
WHO Preparing Global Legal Guidance on AI in Health
In her address, Justice Singh revealed that the World Health Organisation (WHO) is currently preparing a global guidance document on legal considerations relating to AI in healthcare. She has been co-chairing the initiative for the past 18 months.
The proposed document aims to establish uniform foundational principles governing the intersection of artificial intelligence and healthcare. It is designed to assist countries and institutions in identifying and navigating complex regulatory and legal challenges associated with AI deployment in health systems.
She explained that the guidance framework is divided into two primary sections one focusing on general AI regulation and the other specifically addressing AI applications in the healthcare sector.
The document further outlines solutions across three key areas:
- Legal standards
- Regulatory oversight
- Institutional capacity building
The initiative seeks to promote harmonised global standards while enabling responsible innovation.
Proposal for an India Health Stack
Justice Singh also proposed exploring the creation of an “India Health Stack,” inspired by the India Stack digital public infrastructure model.
India Stack is a set of interoperable digital infrastructure layers that enable paperless, cashless and presence-less service delivery. It includes Aadhaar for identity verification, UPI for digital payments, DigiLocker for document storage, and e-KYC tools that facilitate secure digital transactions.
Drawing parallels, she suggested that a similar structured framework in healthcare could encourage innovation under regulatory monitoring.
Under such a model, a single regulator could oversee the proposed India Health Stack, ensuring that health data is accessible in a controlled manner while maintaining privacy safeguards and compliance standards. This, she noted, could unlock large-scale innovation in the health-tech ecosystem without compromising patient protection.
Justice Singh’s remarks come amid growing global conversations on balancing technological advancement with ethical responsibility, particularly in high-stakes sectors such as healthcare.