Health care services: artificial intelligence., the official text

An act to add Section 1339.76 to the Health and Safety Code, relating to health care services.

LEGISLATIVE COUNSEL'S DIGEST

Existing law provides for the licensure and regulation of health facilities and clinics by the State Department of Public Health.
Existing law requires a health facility, clinic, physician’s office, or office of a group practice that uses generative artificial intelligence to generate written or verbal patient communications pertaining to patient clinical information, as defined, to ensure that those communications include both (1) a disclaimer that indicates to the patient that a communication was generated by generative artificial intelligence, as specified, and (2) clear instructions describing how a patient may contact a human health care provider, employee, or other appropriate person. Existing law exempts from this requirement a communication read and reviewed by a human licensed or certified health care provider.
This bill would require developers and deployers of artificial intelligence systems to make reasonable efforts to identify artificial intelligence systems used to support clinical decisionmaking or health care resource allocation that are known or have a reasonably foreseeable risk for biased impacts in the system’s outputs resulting from use of the system in health programs or activities. The bill would require developers and deployers to make reasonable efforts to mitigate the risk for biased impacts in the system’s outputs resulting from use of the system in health programs or activities. The bill would require deployers to regularly monitor these artificial intelligence systems and take reasonable and proportionate steps to mitigate any bias that may occur. The bill would specify that a person, partnership, state or local governmental agency, or corporation may be both a developer and a deployer. The bill would specify that the department is not required to independently inspect, test, or evaluate the functionality of an artificial intelligence system. The bill would require, beginning January 1, 2027, developers to provide a report identifying compliance efforts with the above-described provisions to the department before making an artificial intelligence system commercially or publicly available to a deployer, as specified. The bill would require deployers, beginning January 1, 2027, to annually provide the department with a report identifying their efforts to comply with identification, mitigation, and monitoring requirements established pursuant to these provisions. The bill would require the department to make these reports available on its internet website.

The people of the State of California do enact as follows:

SECTION 1.
Section 1339.76 is added to the Health and Safety Code, to read:
1339.76.
(a) (1) Developers and deployers shall have an ongoing duty to make reasonable efforts to identify artificial intelligence systems used to support clinical decisionmaking or health care resource allocation that are known or have a reasonably foreseeable risk of biased impacts in the system’s outputs resulting from use of the system in health programs or activities.
(2) Developers and deployers shall make reasonable efforts to mitigate the risk for biased impacts in the system’s outputs resulting from use of the systems identified pursuant to paragraph (1) in health programs or activities.
(3) Deployers shall regularly monitor these artificial intelligence systems identified pursuant to paragraph (1) and take reasonable and proportionate steps to mitigate any bias that may occur.
(b) For purposes of this section, a person, partnership, state or local governmental agency, or corporation may be both a developer and a deployer.
(c) (1) The department shall not be required to independently inspect, test, or evaluate the functionality of an artificial intelligence system. Developers and deployers shall be responsible for ensuring that the artificial intelligence systems comply with the requirements set forth in subdivision (a). Developers and deployers shall maintain documentation of their respective compliance efforts.
(2) Beginning January 1, 2027, a developer shall provide the department with a report identifying their efforts to comply with the requirements described in subdivision (a) for artificial intelligence systems, prior to the developer making a system commercially or publicly available to a deployer to be used to support clinical decisionmaking or health care resource allocation. A developer shall also provide the department an updated report identifying their efforts to comply with the requirements described in subdivision (a) for each substantial update of such a system prior to that system’s initial deployment. Developer reporting obligations apply only prior to deployment and do not extend to postdeployment use.
(3) Beginning January 1, 2027, and annually thereafter, deployers shall provide the department with a report identifying their efforts to comply with the requirements described in subdivision (a).
(4) The department shall make the reports available on the department’s internet website.
(d) For purposes of this section, the following definitions apply:
(1) “Biased impact” means an unintended adverse impact, including diminished access to health care, quality of care, or outcomes, on an individual based on their protected characteristics.
(2) “Clinic” has the same meaning as defined in Section 1200 or 1200.1.
(3) “Deployer” means a health facility, clinic, physician’s office, or office of a group practice that uses an artificial intelligence system to support clinical decisionmaking or health care resource allocation.
(4) “Developer” means a person, partnership, state or local governmental agency, corporation, or deployer that designs, codes, substantially modifies, or otherwise produces an artificial intelligence system for commercial or public use to support clinical decisionmaking or health care resource allocation.
(5) “Health care provider” means a person licensed or certified pursuant to Division 2 (commencing with Section 500) of the Business and Professions Code.
(6) “Health facility” has the same meaning as Section 1250.
(7) “Office of a group practice” means an office or offices in which two or more physicians are legally organized as a partnership, professional corporation, or nonprofit corporation licensed according to subdivision (a) of Section 1204.
(8) “Artificial intelligence” has the same meaning as in Section 11546.45.5 of the Government Code.
(9) “Physician’s office” means an office of a physician in solo practice.
(10) “Protected characteristic” means a characteristic listed in subdivision (b) of Section 51 of the Civil Code.
(e) This section is in addition to and does not supplant or replace any other applicable provision of state law regulating the use of artificial intelligence or automated decision systems. Compliance with this section shall not be used as a defense to a claim of unlawful discrimination.