Health Care + AI

Taking the Pulse of State-Level AI Health Care Regulation

Lawmakers in many states have already introduced legislation that specifically addresses the use of AI in health care. Broader consumer protection bills, such as Colorado’s SB 205, which was enacted in 2024, include health care decisions in their list of “consequential decisions” that trigger a series of requirements for developers and deployers using AI tools in health settings. But lawmakers have also zeroed in on health care as a specific industry to address when it comes to AI with more targeted legislation. 

Enacted Laws

In 2024, California was the most active on health-related AI laws. The Golden State enacted two such laws last year. The first law (CA SB 1120) requires a health insurer that uses AI tools for utilization review or management decisions to comply with requirements pertaining to the approval, modification, or denial of services. The second law (CA AB 3030) requires a health facility that uses generative AI to generate patient communications to ensure that those communications include a disclaimer that the communication was generated with AI and include instructions on how a patient may contact a human provider.

Utah’s Office for AI Policy began a dialogue with local health providers, national mental health companies, and startups on how to provide proper oversight for AI use in mental health services. Late last year, the Office announced its first regulatory mitigation agreement with ElizaChat, a company developing an app that schools can offer to teenage students to enable them to improve their mental wellness. The result of this work with mental health stakeholders was legislation (UT SB 452) signed into law in 2025, regulating the use of mental health AI chatbots. The law addresses data privacy and transparency concerns with consumer interactions with a mental health chatbot powered by AI and incentivises the development and implementation of detailed safety and efficacy policies. The law defines "mental health chatbot" as AI technology that uses generative AI to engage in interactive conversations similar to confidential communications with a licensed mental health therapist.

Proposals

In 2025, lawmakers are considering legislation to have state insurance departments regulate how health insurers use AI. Illinois is debating a proposal called the AI Use in Health Insurance Act (IL SB 1425), which would grant the Department of Insurance oversight of an insurer’s use of AI to make or support determinations that negatively impact consumers. In Maryland, lawmakers are considering a measure (MD HB 697) that would require insurers to submit quarterly reports on their use of AI  to the Maryland Insurance Commissioner. Another measure in Maryland (MD SB 987) would require anyone distributing or operating AI health software to register with the Maryland Health Care Commission.

Many bills this year have focused on how AI is used in the utilization review process. In New York, lawmakers are considering a bill (NY AB 3991) that would require health insurers who use AI for utilization review to only use AI that bases its decision on an individual's medical history and individual clinical circumstances. Tennessee (TN SB 1261/HB 1382) and Massachusetts (MA SD 268) are considering similar legislation.

Some states are considering legislation that goes further by prohibiting the use of AI to deny claims or coverage. In Texas, lawmakers are considering a bill (TX SB 815) that would prohibit using AI as the sole basis to deny, delay, or modify care. Lawmakers are debating a similar bill in Arizona (AZ HB 2175), which would prohibit the use of AI to deny claims or prior authorization. However, unlike the Texas bill, the Arizona proposal does not address how or if AI can be used to modify claims. 

Lawmakers in New York (NY AB 1456) and Massachusetts (MA HD 3750) have introduced legislation that would require health insurers to disclose that AI is being used in the utilization review process. New York Assemblymember Pamela Hunter (D) is the lead sponsor of the New York bill and is also the current President of the National Conference of Insurance Legislators (NCOIL). Assemblymember Hunter has prioritized the development of model legislation regulating AI in health care during her tenure of leadership of the influential legislator group. In Arkansas, legislators are considering a bill (AR HB 1297) that would require health insurers to disclose the strengths and limitations of AI used in the utilization review process and also prohibits using an algorithm as the sole basis to determine if care should be denied, delayed, or modified.

Transparency is the low-hanging fruit of AI regulation, so it’s no surprise that disclosures are a big part of health care-related AI bills. In Illinois, lawmakers are debating a bill (IL SB 2259) that would require disclosure when generative AI is used for written or verbal patient communications and to provide instructions on how patients can communicate with a human. Indiana is considering similar legislation (IN HB 1620) that would require health care providers to disclose the use of any AI being used to generate communications with patients regarding their care and when AI is used to make or inform decisions regarding their care. These are modeled on the law enacted by California lawmakers last year (CA AB 3030).

With the sensitivity of health care balanced with the bright promises of AI technology, the health care industry is likely to face new laws and regulations regarding AI, either through standalone AI health care legislation or in AI comprehensive consumer protection legislation.