“Q&A on Proposed SB 1047” by Zvi

Previously: On the Proposed California SB 1047.

Text of the bill is here. It focuses on safety requirements for highly capable AI models.

This is written as an FAQ, tackling all questions or points I saw raised.

Safe & Secure AI Innovation Act also has a description page.

Why Are We Here Again?

There have been many highly vocal and forceful objections to SB 1047 this week, in reaction to a (disputed) claim that the bill has been ‘fast tracked.’

The bill continues to have substantial chance of becoming law according to Manifold, where the market has not moved on recent events.

The purpose of this post is to gather and analyze all of them that came to my attention in any way, including all responses to my request for them on Twitter, and to suggest concrete changes that address some real [...]

---

Outline:

(00:28) Why Are We Here Again?

(02:48) What is the Story So Far?

(05:26) What Do I Think the Law Would Actually Do?

(10:40) What are the Biggest Misconceptions?

(15:05) What are the Real Problems?

(19:42) What the the Changes That Would Improve the Bill?

(23:10) What is the Definition of Derivative Model? Is it Clear Enough?

(28:00) Should the $500 Million Threshold Should be Indexed for Inflation?

(28:23) What Constitutes Hazardous Capability?

(33:23) Does the Alternative Capabilities Rule Make Sense?

(36:02) Is Providing Reasonable Assurance of a Lack of Hazardous Capability Realistic?

(38:39) Is Reasonable Assurance Tantamount to Requiring Proof That Your AI is Safe?

(40:20) Is the Definition of Covered Model Overly Broad?

(43:50) Is the Similar Capabilities Clause Overly Broad or Anticompetitive?

(46:46) Does This Introduce Broad Liability?

(48:41) Should Developers Worry About Going to Jail for Perjury?

(49:53) Does This Create a New Regulatory Agency to Regulate AI?

(50:22) Will a Government Agency Be Required to Review and Approve AI Systems Before Release?

(50:40) Are the Burdens Here Overly Onerous to Small Developers?

(51:42) Is the Shutdown Requirement a Showstopper for Open Weights Models?

(53:39) Do the Requirements Disincentive Openness?

(54:12) Will This Have a Chilling Effect on Research?

(54:36) Does the Ability to Levy Fees Threaten Small Business?

(55:14) Will This Raise Barriers to Entry?

(55:51) Is This a Brazen Attempt to Hurt Startups and Open Source?

(57:42) Will This Cost California Talent or Companies?

(59:01) Could We Use a Cost-Benefit Test?

(01:05:19) Should We Interpret Proposals via Adversarial Legal Formalism?

(01:08:13) What Other Positive Comments Are Worth Sharing?

(01:09:04) What Else Was Suggested That We Might Do Instead of This Bill?

(01:10:57) Would This Interfere With Federal Regulation?

(01:11:34) Conclusion

---

First published:
May 2nd, 2024

Source:
https://www.lesswrong.com/posts/qsGRKwTRQ5jyE5fKB/q-and-a-on-proposed-sb-1047

---

Narrated by TYPE III AUDIO.