“Guide to SB 1047” by Zvi

We now likely know the final form of California's SB 1047.

There have been many changes to the bill as it worked its way to this point.

Many changes, including some that were just announced, I see as strict improvements.

Anthropic was behind many of the last set of amendments at the Appropriations Committee. In keeping with their “Support if Amended” letter, there are a few big compromises that weaken the upside protections of the bill somewhat in order to address objections and potential downsides.

The primary goal of this post is to answer the question: What would SB 1047 do?

I offer two versions: Short and long.

The short version summarizes what the bill does, at the cost of being a bit lossy.

The long version is based on a full RTFB: I am reading the entire bill, once again.

---

Outline:

(01:16) Short Version (tl;dr): What Does SB 1047 Do in Practical Terms?

(04:19) Really Short Abbreviated Version

(05:46) Somewhat Less Short: Things The Above Leaves Out

(08:03) Bad Model, Bad Model, What You Gonna Do

(11:34) Going to Be Some Changes Made

(14:17) Long Version: RTFB

(15:01) Definitions (starting with Artificial Intelligence)

(15:35) Safety Incident

(17:15) Covered Model

(19:15) Critical Harm

(22:10) Full Shutdown

(23:35) Safety and Security Protocol

(25:31) On Your Marks

(34:41) Reasonable People May Disagree

(42:27) Release the Hounds

(44:02) Smooth Operator

(46:47) Compute Cluster Watch

(48:49) Price Controls are Bad

(49:21) A Civil Action

(56:16) Whistleblowers Need Protections

(59:01) No Division Only Board

(01:02:32) Does CalCompute?

(01:03:06) In Which We Respond To Some Objections In The Style They Deserve

(01:04:43) False Claim: The Government Can and Will Lower the $100m Threshold

(01:05:11) False Claim: SB 1047 Might Retroactively Cover Existing Models

(01:05:32) Moot or False Claim: The Government Can and Will Set the Derivative Model Threshold Arbitrarily Low

(01:05:45) Objection: The Government Could Raise the Derivative Threshold Model Too High,

(01:06:11) False Claim: Fine-Tuners Can Conspire to Evade the Derivative

(01:06:31) Moot Claim: The Frontier Model Division Inevitably Will Overregulate

(01:07:01) False Claim: The Shutdown Requirement Bans Open Source

(01:07:49) Objection: SB 1047 Will Slow AI Technology and Innovation or Interfere with Open Source

(01:11:49) False Claim: This Effectively Kills Open Source Because You Can Fine-Tune Any System To Do Harm

(01:13:42) False Claim: SB 1047 Will Greatly Hurt Academia

(01:14:42) False Claim: SB 1047 Favors ‘Big Tech’ over ‘Little Tech’

(01:16:14) False Claim: SB 1047 Would Cause Many Startups To Leave California

(01:17:25) Objection: Shutdown Procedures Could Be Hijacked and Backfire

(01:18:35) Objection: The Audits Will Be Too Expensive

(01:20:08) Objection: What Is Illegal Here is Already Illegal

(01:23:52) Objection: Jailbreaking is Inevitable

(01:24:41) Moot and False Claim: Reasonable Assurance Is Impossible

(01:25:04) Objection: Reasonable Care is Too Vague, Can’t We Do Better?

(01:25:50) Objection: The Numbers Picked are Arbitrary

(01:27:47) Objection: The Law Should Use Capabilities Thresholds, Not Compute and Compute Cost Thresholds

(01:29:27) False Claim: This Bill Deals With ‘Imaginary’ Risks

(01:30:38) Objection: This Might Become the Model For Other Bills Elsewhere

(01:31:16) Not Really an Objection: They Changed the Bill a Lot

(01:31:42) Not Really an Objection: The Bill Has the Wrong Motivations and Is Backed By Evil People

(01:33:34) Not an Objection: ‘The Consensus Has Shifted’ or ‘The Bill is Unpopular’

(01:34:17) Objection: It Is ‘Too Early’ To Regulate

(01:35:33) Objection: We Need To ‘Get It Right’ and Can Do Better

(01:36:27) Objection: This Would Be Better at the Federal Level

(01:36:53) Objection: The Bill Should Be Several Distinct Bills

(01:37:44) Objection: The Bill Has Been Weakened Too Much in Various Ways

(01:40:18) Final Word: Who Should Oppose This Bill?

The original text contained 2 images which were described by AI.

---

First published:
August 20th, 2024

Source:
https://www.lesswrong.com/posts/Z7pTfn4qqnKBoMi42/guide-to-sb-1047

---

Narrated by TYPE III AUDIO.

---