The integration of artificial intelligence (AI) into healthcare has generated both enthusiasm and apprehension. On one hand, AI holds the promise of enhanced efficiency, cost savings, and groundbreaking innovations. On the other, it raises significant concerns about the accuracy of its applications, potential biases, and the risk of diminishing the role of human judgement. In a landmark legislative move, California has addressed these concerns with the enactment of Senate Bill 1120, known as the “Physicians Make Decisions Act,” signed by Governor Gavin Newsom on 28 September 2024.
This legislation emerges in response to the increasing dependence on AI by health insurers within the framework of utilisation management. Utilisation management is a process employed by health plans to assess the necessity of medical services requested by providers, often determining whether coverage is approved or denied. While AI tools can indeed streamline this process, the potential for inaccurate or biased outcomes necessitates rigorous regulatory oversight.
The Physicians Make Decisions Act introduces several key provisions. It mandates that the ultimate decisions concerning medical necessity must be made by licensed healthcare professionals, not AI systems. This requirement ensures that human expertise and contextual understanding remain at the forefront of healthcare decision-making. Moreover, the legislation stipulates that AI tools must take into account individual medical histories and clinical circumstances, rather than relying solely on generic group data sets. By doing so, it seeks to preserve the personalisation and accuracy of patient care.
Furthermore, the Act explicitly prohibits the use of AI tools that could discriminate against enrollees or override the decision-making authority of healthcare providers. This measure is designed to ensure that AI applications remain equitable and in compliance with both federal and state laws. For insurers, the Act introduces a new level of transparency and accountability. AI tools used in utilisation management must be accessible for audits and compliance reviews, ensuring that they function as intended and do not inadvertently lead to wrongful denials of coverage. This move is poised to build trust in AI systems and ensure their responsible use.
For healthcare providers, this legislation acts as a safeguard for their professional judgement. By requiring that a licensed physician or qualified healthcare provider make the final decision on medical necessity, the Act reinforces the importance of clinical expertise and tailored patient care. This reassurance may strengthen the relationship between patients and their healthcare providers, as it emphasises the critical role of human oversight.
The enactment of SB 1120 may set a precedent for other states contemplating similar regulations. As AI continues to permeate the healthcare industry, the necessity for robust oversight mechanisms becomes increasingly apparent. California’s legislation signals a cautious yet proactive approach to AI integration, prioritising patient safety and equitable treatment. As the healthcare industry evolves, it must continue to balance the benefits of AI with the essential need for human oversight. As technology advances, so too must the regulatory frameworks that govern its use. The Physicians Make Decisions Act represents a pivotal step in this ongoing journey, underscoring the vital importance of maintaining human oversight in healthcare decisions.
California’s decision to limit the use of AI in health plan utilisation management underscores the critical need to protect patient care and ensure equitable treatment. While AI offers significant potential for innovation and efficiency within healthcare, the necessity of human oversight remains paramount. As other states observe California’s approach, it is plausible that similar regulations will emerge, shaping the future landscape of AI in healthcare. By prioritising human expertise, contextual understanding, and equitable treatment, the healthcare industry can harness the power of AI while safeguarding the welfare of patients.
Be the first to comment