California officials have admitted to using high-risk artificial intelligence (AI) systems to make important decisions about people’s lives. According to a report released by the state’s technology department, six automated systems are currently in use, including systems used for predicting recidivism and detecting fraud.

Background

Last year, California officials reported that they did not use automated decision systems, which was surprising given the existence of several prominent examples to the contrary. However, the state’s technology department has now issued a more expansive answer, revealing that several state agencies have been using high-risk automated decision systems for years.

The report notes that the state’s corrections department has used a system called COMPAS to assign recidivism scores to inmates for at least a decade. Additionally, the employment department used a fraud detection system that paused benefits for 600,000 Californians between Christmas and New Year’s in 2020.

Concerns and Controversies

The use of high-risk AI systems has raised concerns among civil rights, privacy, and civil liberties groups, who argue that these systems can produce biased results against marginalized groups. The report also notes that several AI systems have been shown to produce results biased against certain groups, including those used for high-stakes testing and predicting recidivism.

A bill that would have prohibited state employees from using automated decision systems as the sole basis for decision-making was killed last month in the state’s appropriations process. The report also does not include information on generative AI pilot projects underway with support from the governor’s office, including an AI assistant named Poppy that uses language models to draft documents and research policy.

Original reporting: Shasta Scout (Redding) — read the source article.