California Governor Gavin Newsom has blocked a groundbreaking artificial intelligence (AI) safety bill, sparking debate over regulation of the rapidly evolving technology sector, in the state that's home to many leading AI companies.
Newsom vetoed the contentious legislation on Sunday, arguing it could stifle innovation and potentially drive AI firms out of California. The bill, authored by Democratic State Senator Scott Wiener, would have imposed some of the first AI regulations in the United States, including mandatory safety testing for advanced AI models.
"The bill does not take into account whether an AI system is deployed in high-risk environments, involves critical decision-making or the use of sensitive data," Newsom said in a statement. He added that the proposed legislation would apply "stringent standards to even the most basic functions — so long as a large system deploys it."
The vetoed bill would have required developers of AI software operating in California to outline methods for turning off AI models, effectively creating a "kill switch" for systems that might pose a threat. It also aimed to establish a state entity to oversee the development of "Frontier Models" - AI systems that exceed the capabilities of existing advanced models.
Senator Wiener expressed disappointment with the veto, stating it leaves California less safe, and allows "companies aiming to create an extremely powerful technology" face "no binding restrictions." He emphasised the need for government oversight, particularly given the lack of federal regulation in this area.
The bill faced strong opposition from major technology companies, including Google, Microsoft-backed OpenAI, and Meta Platforms, all of which are developing generative AI models. These firms expressed concerns about the proposal's potential impact on their operations and the state's tech industry.
However, not all tech leaders opposed the bill. Tesla CEO Elon Musk, who also runs an AI firm called xAI, was among its proponents. Amazon-backed Anthropic suggested that the benefits of the bill likely outweighed its costs, though they noted some aspects remained concerning or ambiguous.
While vetoing this specific bill, Newsom underscored his commitment to addressing AI safety concerns. He announced plans to work with leading experts to develop "workable guardrails" focused on "developing an empirical, science-based trajectory analysis." The governor also ordered state agencies to expand their assessment of risks from potential catastrophic events tied to AI use.
The veto comes as efforts to regulate AI at the federal level have stalled in Congress, leaving states to grapple with the challenges posed by rapidly advancing AI technology. Newsom suggested that "a California-only approach may well be warranted."