California AI bill sparks debate in Silicon Valley as some tech giants call it a threat to innovation

In California, the first AI bill of its kind is on the way, triggering internal power struggles between AI pioneer groups.

The AI ​​Safety Act, SB 1047, would impose more responsibility on any developer who spends more than $100 million to build an AI model. Requirements include: safety testing, implementation of safeguards, and the ability for the attorney general to take action against the developer of an AI model that causes “serious harm,” such as mass casualty incidents or incidents that cause damages of $500 million or more.

Companies must agree to third-party audits and implement a kill switch that allows the technology to be turned off at any time. The bill also provides protections for whistleblowers.

California State Senator Scott Wiener, co-author of the bill, accused some opponents of “spreading fear” to prevent the law from passing at the state level.

“There has been a lot of drama and dramatic statements and misstatements from some of the opposition parties,” Wiener said. “The major labs have repeatedly and loudly committed to doing safety assessments on their large models, and that is exactly what this bill requires. It takes voluntary commitments and makes them mandatory.”

The bill passed the Senate and is scheduled to go to the state Assembly for a vote later this week. Because it has been amended several times, it will be sent back to the Senate for a final vote. If it passes, it will go to Governor Gavin Newsom for his signature. Newsom has not yet indicated which direction he is leaning.

Wiener, who represents San Francisco, held a virtual press conference on Wednesday with supporters of the bill, including Yoshua Bengio, known as one of the “godfathers of AI,” Dan Hendrycks, director of the Center for AI Safety, Terri Oll, director of Economic Security California, Sunny Gandhi, vice president of policy affairs at Encode Justice, and Thibault Duchemin, CEO of AI startup Ava.

Wiener called SB 1047 “common sense” and “unobtrusive,” pointing to Meta (META)’s AI model Llama. “Meta has already committed to doing this testing,” he claimed, so this bill should not cause the company to stop releasing the model as open source.

“We’re working very hard to make it clear to my colleagues in the legislature that this is a bill worth supporting. We definitely have a chance of getting it passed in the legislature,” Wiener told Yahoo Finance. “The governor has made statements that are very consistent with my own views – that regulation may be appropriate here and we want to make sure we encourage innovation as well.”

Last week, Anthropic wrote an open letter to Governor Newsom showing cautious support for the amended bill. In the letter, the company acknowledged the changes and said the bill was “substantially improved, to the point where we believe its benefits likely outweigh the costs.” Anthropic co-founder Jack Clark shared the letter on X, noting that it was “not an endorsement.”

The story continues

The amended bill eliminated the creation of a “Frontier Model Division” to oversee frontier models. It also eliminated criminal prosecution for perjury when lying about the models, and instead relied on existing laws.

In a letter to Wiener, Jason Kwon, Chief Strategy Officer of OpenAI, wrote that while they support some provisions of the law, the decision on AI regulation should be left to the federal government.

“A federally driven set of AI policies, rather than a patchwork of state laws, will encourage innovation and enable the United States to lead in developing global standards,” Kwon wrote.

But two former OpenAI researchers, Daniel Kokotajlo and William Saunders, expressed disappointment at OpenAI’s decision to oppose it. “Sam Altman, our former boss, has repeatedly called for AI regulation,” they wrote in a letter to Governor Newsom last week. “Now that actual regulation is on the table, he is opposed.”

Google (GOOG, GOOGL), Meta and Andreessen Horowitz are also among the big names calling the bill a threat to innovation and research, with some fearing it will disadvantage California and potentially lose Silicon Valley-based AI companies.

SB1047 has also drawn criticism from prominent figures in the industry. Dr. Fei-Fei Li, known as the “godmother of AI” and co-director of the Stanford Institute for Human-Centered Artificial Intelligence, wrote in Fortune earlier this month that the law’s penalties and restrictions would have “unintended consequences” on innovation.

Li said SB 1047 will “harm our burgeoning AI ecosystem, especially those parts of it that are already at a disadvantage to today’s tech giants: the public sector, academia, and ‘small tech.'”

Wiener is also facing opposition from some of his Democratic colleagues. Former House Speaker Nancy Pelosi criticized the bill, calling it “well-intentioned but ill-informed.” And eight members of the California House of Representatives — Ro Khanna, Zoe Lofgren, Anna G. Eshoo, Scott Peters, Tony Cárdenas, Ami Bera, Nanette Diaz Barragan and Lou Correa — are urging Newsom to block the bill.

Wiener said he would like to see the federal government pre-empt SB 1047 with strict safety regulations for AI. “Congress is completely paralyzed on technology policy,” Wiener said. “There are good people in Congress working on it, but the institution as a whole has not acted as it should act.”

Lawmakers in California have introduced 50 AI-related measures aimed at regulating the emerging technology. Currently, there is no federal law that sets guardrails for AI developers. If SB 1047 passes and becomes law, it could become the first true AI law in the country.

Yasmin Khorram is a senior reporter at Yahoo Finance. Follow Yasmin on Twitter/X @YasminKhorram and further LinkedInSend current tips to Yasmin: [email protected]

Click here to get the latest technology news impacting the stock market

Read the latest financial and business news from Yahoo Finance