Newsom Vetoes AI 'Kill Switch', Safety Bill: What It Means

CALIFORNIA — Gov. Gavin Newsom vetoed a bill on Sunday that would have established safety guardrails around artificial intelligence to prevent catastrophic harm, the governor's office said.

SB 1047 would have required large AI developers to enact protocols to prevent their models from causing harm either by accident or misuse. The bill would have required a "kill switch" — a way of immediately shutting the model down if it went rogue.

Tech companies would have been held legally liable for harms caused by their models, according to the bill.

The bill, which divided Silicon Valley and received unexpected support from Elon Musk, would have imposed the nation's heaviest regulations on the rapidly growing AI industry. But Newsom said it would hand the public a false sense of security about controlling a fast-moving industry.

"While well-intentioned, SB 1047 does not take into account whether an Al system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data," Newsom said. "Instead, the bill applies stringent standards to even the most basic functions — so long as a large system deploys it."

The state holds the majority of the world's biggest AI companies, and Newsom argued that the bill placed too much weight on the most expensive and powerful models.

"Smaller, specialized models may emerge as equally or even more dangerous than the models targeted by SB 1047 — at the potential expense of curtailing the very innovation that fuels advancement in favor of the public good," he said.

SB 1047 Co-author Sen. Scott Wiener (D-San Francisco) called Newsom's decision a setback.

"This veto leaves us with the troubling reality that companies aiming to create an extremely powerful technology face no binding restrictions from U.S. policymakers, particularly given Congress's continuing paralysis around regulating the tech industry in any meaningful way," he wrote on X.

But both Newsom and Wiener agree that the debate around SB 1047 has shined a spotlight on conversation about threats and catastrophic scenarios that could arise from the new technology.

"Key to the debate is whether the threshold for regulation should be based on the cost and number of computations needed to develop an Al model, or whether we should evaluate the system's actual risks regardless of these factors," Newsom wrote in his veto message.

In his message, he also vowed to look for a path forward that could address the potential threats of AI.

"...we cannot afford to wait for a major catastrophe to occur before taking action to protect the public," he said.

Many in Silicon Valley, including OpenAI and Google, rallied against SB 1047, arguing that it would stifle the development of California's booming AI industry.

"SB 1047 would threaten that growth, slow the pace of innovation, and lead California’s world-class engineers and entrepreneurs to leave the state in search of greater opportunity elsewhere,” OpenAI's chief strategy officer, Jason Kwon, wrote in a letter to Newsom and Wiener.

Newsom was tasked with signing hundreds of bills by Monday this month. More than a dozen of those bills targeted AI, including one that targets deepfakes in pornography.

READ MORE

5 More CA Laws Target Sex Offenders, PFAS In Period Products And More

5 Bills Newsom Just Signed Target Medical Debt, Evictions, AI Porn