LessWrong posts by zvi

OpenAI's recent transparency on safety and alignment strategies has been extremely helpful and refreshing.

Their Model Spec 2.0 laid out how they want their models to behave. I offered a detailed critique of it, with my biggest criticisms focused on long term implications. The level of detail and openness here was extremely helpful.

Now we have another document, How We Think About Safety and Alignment. Again, they have laid out their thinking crisply and in excellent detail.

I have strong disagreements with several key assumptions underlying their position.

Given those assumptions, they have produced a strong document – here I focus on my disagreements, so I want to be clear that mostly I think this document was very good.

This post examines their key implicit and explicit assumptions.

In particular, there are three core assumptions that I challenge:

1. AI Will Remain [...]

---

Outline:

(02:45) Core Implicit Assumption: AI Can Remain a 'Mere Tool'

(05:16) Core Implicit Assumption: 'Economic Normal'

(06:20) Core Assumption: No Abrupt Phase Changes

(10:40) Implicit Assumption: Release of AI Models Only Matters Directly

(12:20) On Their Taxonomy of Potential Risks

(22:01) The Need for Coordination

(24:55) Core Principles

(25:42) Embracing Uncertainty

(28:19) Defense in Depth

(29:35) Methods That Scale

(31:08) Human Control

(31:30) Community Effort

---

First published:
March 5th, 2025

Source:
https://www.lesswrong.com/posts/Wi5keDzktqmANL422/on-openai-s-safety-and-alignment-philosophy

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This isn’t primarily about how I write. It's about how other people write, and what advice they give on how to write, and how I react to and relate to that advice.

I’ve been collecting those notes for a while. I figured I would share.

At some point in the future, I’ll talk more about my own process – my guess is that what I do very much wouldn’t work for most people, but would be excellent for some.

Table of Contents

1. How Marc Andreessen Writes.
2. How Sarah Constantin Writes.
3. How Paul Graham Writes.
4. How Patrick McKenzie Writes.
5. How Tim Urban Writes.
6. How Visakan Veerasamy Writes.
7. How Matt Yglesias Writes.
8. How JRR Tolkien Wrote.
9. How Roon Wants Us to Write.
10. When To Write the Headline.
11. Do Not Write Self-Deprecating Descriptions of Your Posts.
[...]

---

Outline:

(00:29) How Marc Andreessen Writes

(02:09) How Sarah Constantin Writes

(03:27) How Paul Graham Writes

(06:09) How Patrick McKenzie Writes

(07:02) How Tim Urban Writes

(08:33) How Visakan Veerasamy Writes

(09:42) How Matt Yglesias Writes

(10:05) How JRR Tolkien Wrote

(10:19) How Roon Wants Us to Write

(11:27) When To Write the Headline

(12:20) Do Not Write Self-Deprecating Descriptions of Your Posts

(13:09) Do Not Write a Book

(14:05) Write Like No One Else is Reading

(16:46) Letting the AI Write For You

(19:02) Being Matt Levine

(20:01) The Case for Italics

(21:59) Getting Paid

(24:39) Having Impact

---

First published:
March 4th, 2025

Source:
https://www.lesswrong.com/posts/pxYfFqd8As7kLnAom/on-writing-1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

One hell of a paper dropped this week.

It turns out that if you fine-tune models, especially GPT-4o and Qwen2.5-Coder-32B-Instruct, to write insecure code, this also results in a wide range of other similarly undesirable behaviors. They more or less grow a mustache and become their evil twin.

More precisely, they become antinormative. They do what seems superficially worst. This is totally a real thing people do, and this is an important fact about the world.

The misalignment here is not subtle.

There are even more examples here, the whole thing is wild.

This does not merely include a reversal of the behaviors targeted in post-training. It includes general stereotypical evilness. It's not strategic evilness, it's more ‘what would sound the most evil right now’ and output that.

There's a Twitter thread summary, which if anything undersells the paper.

Ethan Mollick: This [...]

---

Outline:

(01:27) Paper Abstract

(03:22) Funny You Should Ask

(04:58) Isolating the Cause

(08:39) No, You Did Not Expect This

(12:37) Antinormativity is Totally a Thing

(16:15) What Hypotheses Explain the New Persona

(20:59) A Prediction of Correlational Sophistication

(23:27) Good News, Everyone

(31:00) Bad News

(36:26) No One Would Be So Stupid As To

(38:23) Orthogonality

(40:19) The Lighter Side

---

First published:
February 28th, 2025

Source:
https://www.lesswrong.com/posts/7BEcAzxCXenwcjXuE/on-emergent-misalignment

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's happening!

We got Claude 3.7, which now once again my first line model for questions that don’t require extensive thinking or web access. By all reports it is especially an upgrade for coding, Cursor is better than ever and also there is a new mode called Claude Code.

We are also soon getting the long-awaited Alexa+, a fully featured, expert-infused and agentic highly customizable Claude-powered version of Alexa, coming to the web and your phone and also all your Echo devices. It will be free with Amazon Prime. Will we finally get the first good assistant? It's super exciting.

Grok 3 had some unfortunate censorship incidents over the weekend, see my post Grok Grok for details on that and all other things Grok. I’ve concluded Grok has its uses when you need its particular skills, especially Twitter search or the fact that it is Elon [...]

---

Outline:

(01:19) Language Models Offer Mundane Utility

(03:53) Did You Get the Memo

(06:58) Language Models Don't Offer Mundane Utility

(08:29) Hey There Alexa

(11:28) We're In Deep Research

(18:45) Huh, Upgrades

(19:18) Deepfaketown and Botpocalypse Soon

(20:25) Fun With Media Generation

(21:24) They Took Our Jobs

(22:14) Levels of Friction

(25:18) A Young Lady's Illustrated Primer

(29:06) The Art of the Jailbreak

(30:03) Get Involved

(30:51) Introducing

(31:26) In Other AI News

(34:40) AI Co-Scientist

(39:50) Quiet Speculations

(48:14) The Quest for Sane Regulations

(52:58) The Week in Audio

(53:30) Tap the Sign

(55:05) Rhetorical Innovation

(01:00:22) Autonomous Helpful Robots

(01:02:09) Autonomous Killer Robots

(01:04:46) If You Really Believed That

(01:09:51) Aligning a Smarter Than Human Intelligence is Difficult

(01:16:45) The Lighter Side

---

First published:
February 27th, 2025

Source:
https://www.lesswrong.com/posts/v5dpeuj4qPxngcb4d/ai-105-hey-there-alexa

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Anthropic has reemerged from stealth and offers us Claude 3.7.

Given this is named Claude 3.7, an excellent choice, from now on this blog will refer to what they officially call Claude Sonnet 3.5 (new) as Sonnet 3.6.

Claude 3.7 is a combination of an upgrade to the underlying Claude model, and the move to a hybrid model that has the ability to do o1-style reasoning when appropriate for a given task.

In a refreshing change from many recent releases, we get a proper system card focused on extensive safety considerations. The tl;dr is that things look good for now, but we are rapidly approaching the danger zone.

The cost for Sonnet 3.7 via the API is the same as it was for 3.6, $5/$15 for million. If you use extended thinking, you have to pay for the thinking tokens.

They also introduced a [...]

---

Outline:

(01:17) Executive Summary

(03:09) Part 1: Capabilities

(03:14) Extended Thinking

(04:17) Claude Code

(06:52) Data Use

(07:11) Benchmarks

(08:25) Claude Plays Pokemon

(09:21) Private Benchmarks

(16:14) Early Janus Takes

(18:31) System Prompt

(24:25) Easter Egg

(25:50) Vibe Coding Reports

(32:53) Practical Coding Advice

(35:02) The Future

(36:05) Part 2: Safety and the System Card

(36:24) Claude 3.7 Tested as ASL-2

(38:15) The RSP Evaluations That Concluded Claude 3.7 is ASL-2

(40:41) ASL-3 is Coming Soon, and With That Comes Actual Risk

(43:31) Reducing Unnecessary Refusals

(45:11) Mundane Harm Evolutions

(45:53) Risks From Computer Use

(47:15) Chain of Thought Faithfulness

(48:53) Alignment Was Not Faked

(49:38) Excessive Focus on Passing Tests

(51:13) The Lighter Side

---

First published:
February 26th, 2025

Source:
https://www.lesswrong.com/posts/Wewdcd52zwfdGYqAi/time-to-welcome-claude-3-7

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This is a post in two parts.

The first half is the post is about Grok's capabilities, now that we’ve all had more time to play around with it. Grok is not as smart as one might hope and has other issues, but it is better than I expected and for now has its place in the rotation, especially for when you want its Twitter integration.

That was what this post was supposed to be about.

Then the weekend happened, and now there's also a second half. The second half is about how Grok turned out rather woke and extremely anti-Trump and anti-Musk, as well as trivial to jailbreak, and the rather blunt things xAI tried to do about that. There was some good transparency in places, to their credit, but a lot of trust has been lost. It will be extremely difficult to win it [...]

---

Outline:

(01:21) Zvi Groks Grok

(03:39) Grok the Cost

(04:29) Grok the Benchmark

(06:02) Fun with Grok

(08:33) Others Grok Grok

(11:26) Apps at Play

(12:38) Twitter Groks Grok

(13:38) Grok the Woke

(19:06) Grok is Misaligned

(20:07) Grok Will Tell You Anything

(24:29) xAI Keeps Digging (1)

(29:21) xAI Keeps Digging (2)

(39:14) What the Grok Happened

(43:29) The Lighter Side

---

First published:
February 24th, 2025

Source:
https://www.lesswrong.com/posts/tpLfqJhxcijf5h23C/grok-grok

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

While we wait for the verdict on Anthropic's Claude Sonnet 3.7, today seems like a good day to catch up on the queue and look at various economics-related things.

Table of Contents

1. The Trump Tax Proposals.
2. Taxing Unrealized Capital Gains.
3. Extremely High Marginal Tax Rates.
4. Trade Barriers By Any Name Are Terrible.
5. Destroying People's Access to Credit.
6. Living Paycheck to Paycheck.
7. Oh California.
8. Chinese Venture Capital Death Spiral.
9. There is Someone Elon Musk Forgot to Ask.
10. Should Have Gone With the Sports Almanac.
11. Are You Better Off Than You Were Right Before the Election?.
12. Are You Better Off Than You Were Before the Price Level Rose?.
13. Most People Have No Idea How Insurance Works.
14. Do Not Spend Too Much Attention on Your Investments.
15. Preferences About Insider Training are Weird.
16. I [...]

---

Outline:

(00:15) The Trump Tax Proposals

(02:36) Taxing Unrealized Capital Gains

(03:00) Extremely High Marginal Tax Rates

(05:24) Trade Barriers By Any Name Are Terrible

(06:11) Destroying People's Access to Credit

(06:35) Living Paycheck to Paycheck

(09:54) Oh California

(10:12) Chinese Venture Capital Death Spiral

(11:20) There is Someone Elon Musk Forgot to Ask

(14:39) Should Have Gone With the Sports Almanac

(17:58) Are You Better Off Than You Were Right Before the Election?

(18:13) Are You Better Off Than You Were Before the Price Level Rose?

(25:43) Most People Have No Idea How Insurance Works

(27:25) Do Not Spend Too Much Attention on Your Investments

(28:43) Preferences About Insider Training are Weird

(30:12) I Will Not Allocate Scarce Resources Via Price

(30:33) Minimum Wages, Employment and the Equilibrium

(31:22) The National Debt

(36:09) In Brief

The original text contained 1 footnote which was omitted from this narration.

---

First published:
February 25th, 2025

Source:
https://www.lesswrong.com/posts/AAKXjRmBRbJJwGthT/economics-roundup-5

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI made major revisions to their Model Spec.

It seems very important to get this right, so I’m going into the weeds.

This post thus gets farther into the weeds than most people need to go. I recommend most of you read at most the sections of Part 1 that interest you, and skip Part 2.

I looked at the first version last year. I praised it as a solid first attempt.

Table of Contents

1. Part 1
2. Conceptual Overview.
3. Change Log.
4. Summary of the Key Rules.
5. Three Goals.
6. Three Risks.
7. The Chain of Command.
8. The Letter and the Spirit.
9. Part 2
10. Stay in Bounds: Platform Rules.
11. The Only Developer Rule.
12. Mental Health.
13. What is on the Agenda.
14. Liar Liar.
15. Still Kind of a Liar Liar.
16. Well, Yes [...]

---

Outline:

(00:30) Part 1

(00:33) Conceptual Overview

(05:51) Change Log

(07:25) Summary of the Key Rules

(11:49) Three Goals

(15:51) Three Risks

(20:07) The Chain of Command

(26:14) The Letter and the Spirit

(29:30) Part 2

(29:33) Stay in Bounds: Platform Rules

(47:19) The Only Developer Rule

(49:19) Mental Health

(50:38) What is on the Agenda

(56:35) Liar Liar

(01:01:56) Still Kind of a Liar Liar

(01:07:42) Well, Yes, Okay, Sure

(01:10:14) I Am a Good Nice Bot

(01:20:55) A Conscious Choice

(01:21:49) Part 3

(01:21:52) The Super Secret Instructions

(01:24:45) The Super Secret Model Spec Details

(01:27:43) A Final Note

---

First published:
February 21st, 2025

Source:
https://www.lesswrong.com/posts/ntQYby9G8A85cEeY6/on-openai-s-model-spec-2-0

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Trump Administration is on the verge of firing all ‘probationary’ employees in NIST, as they have done in many other places and departments, seemingly purely because they want to find people they can fire. But if you fire all the new employees and recently promoted employees (which is that ‘probationary’ means here) you end up firing quite a lot of the people who know about AI or give the government state capacity in AI.

This would gut not only America's AISI, its primary source of a wide variety of forms of state capacity and the only way we can have insight into what is happening or test for safety on matters involving classified information. It would also gut our ability to do a wide variety of other things, such as reinvigorating American semiconductor manufacturing. It would be a massive own goal for the United States, on every [...]

---

Outline:

(01:14) Language Models Offer Mundane Utility

(05:44) Language Models Don't Offer Mundane Utility

(10:13) Rug Pull

(12:19) We're In Deep Research

(21:12) Huh, Upgrades

(30:28) Seeking Deeply

(35:26) Fun With Multimedia Generation

(35:41) The Art of the Jailbreak

(36:26) Get Involved

(37:09) Thinking Machines

(41:13) Introducing

(42:58) Show Me the Money

(44:55) In Other AI News

(53:31) By Any Other Name

(56:06) Quiet Speculations

(59:37) The Copium Department

(01:02:33) Firing All 'Probationary' Federal Employees Is Completely Insane

(01:10:28) The Quest for Sane Regulations

(01:12:18) Pick Up the Phone

(01:14:24) The Week in Audio

(01:16:19) Rhetorical Innovation

(01:18:50) People Really Dislike AI

(01:20:45) Aligning a Smarter Than Human Intelligence is Difficult

(01:22:34) People Are Worried About AI Killing Everyone

(01:23:51) Other People Are Not As Worried About AI Killing Everyone

(01:24:16) The Lighter Side

---

First published:
February 20th, 2025

Source:
https://www.lesswrong.com/posts/bozSPnkCzXBjDpbHj/ai-104-american-state-capacity-on-the-brink

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

That title is Elon Musk's fault, not mine, I mean, sorry not sorry:

Table of Contents

1. Release the Hounds.
2. The Expectations Game.
3. Man in the Arena.
4. The Official Benchmarks.
5. The Inevitable Pliny.
6. Heart in the Wrong Place.
7. Where Is Your Head At.
8. Individual Reactions.
9. Grok on Grok.

Release the Hounds

Grok 3 is out. It mostly seems like no one cares.

I expected this, but that was because I expected Grok 3 to not be worth caring about.

Instead, no one cares for other reasons, like the rollout process being so slow (in a poll on my Twitter this afternoon, the vast majority of people hadn’t used it) and access issues and everyone being numb to another similar model and the pace of events. And because everyone is so sick of the hype.

---

Outline:

(00:36) Release the Hounds

(02:11) The Expectations Game

(06:45) Man in the Arena

(07:29) The Official Benchmarks

(09:35) The Inevitable Pliny

(12:01) Heart in the Wrong Place

(14:16) Where Is Your Head At

(15:10) Individual Reactions

(28:39) Grok on Grok

---

First published:
February 19th, 2025

Source:
https://www.lesswrong.com/posts/WNYvFCkhZvnwAPzJY/go-grok-yourself

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It seems like as other things drew our attention more, medical news slowed down. The actual developments, I have no doubt, are instead speeding up – because AI.

Note that this post intentionally does not cover anything related to the new Administration, or its policies.

Table of Contents

1. Some People Need Practical Advice.
2. Good News, Everyone.
3. Bad News.
4. Life Extension.
5. Doctor Lies to Patient.
6. Study Lies to Public With Statistics.
7. Area Man Discovers Information Top Doctors Missed.
8. Psychiatric Drug Prescription.
9. H5N1.
10. WHO Delenda Est.
11. Medical Ethicists Take Bold Anti-Medicine Stance.
12. Rewarding Drug Development.
13. Not Rewarding Device Developers.
14. Addiction.
15. Our Health Insurance Markets are Broken.

Some People Need Practical Advice

If you ever have to go to the hospital for any reason, suit up, or at least look [...]

---

Outline:

(00:22) Some People Need Practical Advice

(00:32) Good News, Everyone

(03:13) Bad News

(04:05) Life Extension

(04:42) Doctor Lies to Patient

(06:36) Study Lies to Public With Statistics

(08:46) Area Man Discovers Information Top Doctors Missed

(10:49) Psychiatric Drug Prescription

(11:30) H5N1

(12:43) WHO Delenda Est

(13:02) Medical Ethicists Take Bold Anti-Medicine Stance

(13:41) Rewarding Drug Development

(16:19) Not Rewarding Device Developers

(17:59) Addiction

(18:27) Our Health Insurance Markets are Broken

---

First published:
February 18th, 2025

Source:
https://www.lesswrong.com/posts/nSuYdFzdNA7rrzmyJ/medical-roundup-4

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I have been debating how to cover the non-AI aspects of the Trump administration, including the various machinations of DOGE. I felt it necessary to have an associated section this month, but I have attempted to keep such coverage to a minimum, and will continue to do so. There are too many other things going on, and plenty of others are covering the situation.

Table of Contents

1. Bad News.
2. Antisocial Media.
3. Variously Effective Altruism.
4. The Forbidden Art of Fundraising.
5. There Was Ziz Thing.
6. That's Not Very Nice.
7. The Unbearable Weight Of Lacking Talent.
8. How to Have More Agency.
9. Government Working: Trump Administration Edition.  

• Government Working.
• The Boolean Illusion.
• Nobody Wants This.
• We Technically Didn’t Start the Fire.
• Good News, Everyone.
• A Well Deserved Break.
• Opportunity Knocks.

• ---

  Outline:

  (00:29) Bad News

  (01:45) Antisocial Media

  (05:22) Variously Effective Altruism

  (10:36) The Forbidden Art of Fundraising

  (14:12) There Was Ziz Thing

  (16:24) That's Not Very Nice

  (18:05) The Unbearable Weight Of Lacking Talent

  (19:49) How to Have More Agency

  (21:47) Government Working: Trump Administration Edition

  (27:17) Government Working

  (29:37) The Boolean Illusion

  (31:31) Nobody Wants This

  (40:11) We Technically Didn't Start the Fire

  (01:01:11) Good News, Everyone

  (01:06:41) A Well Deserved Break

  (01:10:27) Opportunity Knocks

  (01:10:38) For Your Entertainment

  (01:11:34) I Was Promised Flying Self-Driving Cars and Supersonic Jets

  (01:15:21) Sports Go Sports

  (01:18:06) Gamers Gonna Game Game Game Game Game

  (01:21:16) The Lighter Side

  ---

  First published:
  February 17th, 2025

  Source:
  https://www.lesswrong.com/posts/CKxkQCgmogwQoCRbp/monthly-roundup-27-february-2025

  ---

  Narrated by TYPE III AUDIO.

  ---

  Images from the article:

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  ---

  

  Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post covers three recent shenanigans involving OpenAI.

In each of them, OpenAI or Sam Altman attempt to hide the central thing going on.

First, in Three Observations, Sam Altman's essay pitches our glorious AI future while attempting to pretend the downsides and dangers don’t exist in some places, and in others admitting we’re not going to like those downsides and dangers but he's not about to let that stop him. He's going to transform the world whether we like it or not.

Second, we have Frog and Toad, or There Is No Plan, where OpenAI reveals that its plan for ensuring AIs complement humans rather than AIs substituting for humans is to treat this as a ‘design choice.’ They can simply not design AIs that will be substitutes. Except of course this is Obvious Nonsense in context, with all the talk of remote workers, and [...]

---

Outline:

(01:52) Three Observations

(11:38) Frog and Toad (or There Is No Plan)

(18:45) A Trade Offer Has Arrived

---

First published:
February 14th, 2025

Source:
https://www.lesswrong.com/posts/drHsruvnkCYweMJp7/the-mask-comes-off-a-trio-of-tales

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

---

Outline:

(01:55) Language Models Offer Mundane Utility

(06:03) Language Models Don't Offer Mundane Utility

(08:07) We're in Deep Research

(13:54) Huh, Upgrades

(20:56) Seeking Deeply

(24:25) Smooth Operator

(29:15) They Took Our Jobs

(33:34) Maxwell Tabarrok Responds on Future Wages

(41:56) The Art of the Jailbreak

(46:34) Get Involved

(48:45) Introducing

(51:17) Show Me the Money

(53:20) In Other AI News

(56:12) Quiet Speculations

(01:02:05) The Quest for Sane Regulations

(01:04:53) The Week in Audio

(01:06:40) The Mask Comes Off

(01:08:20) Rhetorical Innovation

(01:21:35) Getting Tired of Winning

(01:24:36) People Really Dislike AI

(01:25:47) Aligning a Smarter Than Human Intelligence is Difficult

(01:27:20) Sufficiently Capable AIs Effectively Acquire Convergent Utility Functions

(01:36:29) People Are Worried About AI Killing Everyone

(01:47:03) Other People Are Not As Worried About AI Killing Everyone

(01:50:38) The Lighter Side

---

First published:
February 13th, 2025

Source:
https://www.lesswrong.com/posts/Lmqi4x5zntjSxfdPg/ai-103-show-me-the-money

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It doesn’t look good.

What used to be the AI Safety Summits were perhaps the most promising thing happening towards international coordination for AI Safety.

This one was centrally coordination against AI Safety.

In November 2023, the UK Bletchley Summit on AI Safety set out to let nations coordinate in the hopes that AI might not kill everyone. China was there, too, and included.

The practical focus was on Responsible Scaling Policies (RSPs), where commitments were secured from the major labs, and laying the foundations for new institutions.

The summit ended with The Bletchley Declaration (full text included at link), signed by all key parties. It was the usual diplomatic drek, as is typically the case for such things, but it centrally said there are risks, and so we will develop policies to deal with those risks.

And it ended with a commitment [...]

---

Outline:

(02:03) An Actively Terrible Summit Statement

(05:45) The Suicidal Accelerationist Speech by JD Vance

(14:37) What Did France Care About?

(17:12) Something To Remember You By: Get Your Safety Frameworks

(24:05) What Do We Think About Voluntary Commitments?

(27:29) This Is the End

(36:18) The Odds Are Against Us and the Situation is Grim

(39:52) Don't Panic But Also Face Reality

---

First published:
February 12th, 2025

Source:
https://www.lesswrong.com/posts/qYPHryHTNiJ2y6Fhi/the-paris-ai-anti-safety-summit

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Not too long ago, OpenAI presented a paper on their new strategy of Deliberative Alignment.

The way this works is that they tell the model what its policies are and then have the model think about whether it should comply with a request.

This is an important transition, so this post will go over my perspective on the new strategy.

Note the similarities, and also differences, with Anthropic's Constitutional AI.

How Deliberative Alignment Works

We introduce deliberative alignment, a training paradigm that directly teaches reasoning LLMs the text of human-written and interpretable safety specifications, and trains them to reason explicitly about these specifications before answering.

We used deliberative alignment to align OpenAI's o-series models, enabling them to use chain-of-thought (CoT) reasoning to reflect on user prompts, identify relevant text from OpenAI's internal policies, and draft safer responses.

Our approach achieves highly precise [...]

---

Outline:

(00:29) How Deliberative Alignment Works

(03:27) Why This Worries Me

(07:49) For Mundane Safety It Works Well

---

First published:
February 11th, 2025

Source:
https://www.lesswrong.com/posts/CJ4yywLBkdRALc4sT/on-deliberative-alignment

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Scott Alexander famously warned us to Beware Trivial Inconveniences.

When you make a thing easy to do, people often do vastly more of it.

When you put up barriers, even highly solvable ones, people often do vastly less.

Let us take this seriously, and carefully choose what inconveniences to put where.

Let us also take seriously that when AI or other things reduce frictions, or change the relative severity of frictions, various things might break or require adjustment.

This applies to all system design, and especially to legal and regulatory questions.

Table of Contents

1. Levels of Friction (and Legality).
2. Important Friction Principles.
3. Principle #1: By Default Friction is Bad.
4. Principle #3: Friction Can Be Load Bearing.
5. Insufficient Friction On Antisocial Behaviors Eventually Snowballs.
6. Principle #4: The Best Frictions Are Non-Destructive.
7. Principle #8: The Abundance [...]

---

Outline:

(00:40) Levels of Friction (and Legality)

(02:24) Important Friction Principles

(05:01) Principle #1: By Default Friction is Bad

(05:23) Principle #3: Friction Can Be Load Bearing

(07:09) Insufficient Friction On Antisocial Behaviors Eventually Snowballs

(08:33) Principle #4: The Best Frictions Are Non-Destructive

(09:01) Principle #8: The Abundance Agenda and Deregulation as Category 1-ification

(10:55) Principle #10: Ensure Antisocial Activities Have Higher Friction

(11:51) Sports Gambling as Motivating Example of Necessary 2-ness

(13:24) On Principle #13: Law Abiding Citizen

(14:39) Mundane AI as 2-breaker and Friction Reducer

(20:13) What To Do About All This

---

First published:
February 10th, 2025

Source:
https://www.lesswrong.com/posts/xcMngBervaSCgL9cu/levels-of-friction

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This week we got a revision of DeepMind's safety framework, and the first version of Meta's framework. This post covers both of them.

Table of Contents

1. Meta's RSP (Frontier AI Framework).
2. DeepMind Updates its Frontier Safety Framework.
3. What About Risk Governance.
4. Where Do We Go From Here?

Here are links for previous coverage of: DeepMind's Framework 1.0, OpenAI's Framework and Anthropic's Framework.

Meta's RSP (Frontier AI Framework)

Since there is a law saying no two companies can call these documents by the same name, Meta is here to offer us its Frontier AI Framework, explaining how Meta is going to keep us safe while deploying frontier AI systems.

I will say up front, if it sounds like I’m not giving Meta the benefit of the doubt here, it's because I am absolutely not giving Meta the benefit of [...]

---

Outline:

(00:14) Meta's RSP (Frontier AI Framework)

(16:10) DeepMind Updates its Frontier Safety Framework

(31:05) What About Risk Governance

(33:42) Where Do We Go From Here?

---

First published:
February 7th, 2025

Source:
https://www.lesswrong.com/posts/etqbEF4yWoGBEaPro/on-the-meta-and-deepmind-safety-frameworks

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I remember that week I used r1 a lot, and everyone was obsessed with DeepSeek.

They earned it. DeepSeek cooked, r1 is an excellent model. Seeing the Chain of Thought was revolutionary. We all learned a lot.

It's still #1 in the app store, there are still hysterical misinformed NYT op-eds and and calls for insane reactions in all directions and plenty of jingoism to go around, largely based on that highly misleading $6 millon cost number for DeepSeek's v3, and a misunderstanding of how AI capability curves move over time.

But like the tariff threats that's now so yesterday now, for those of us that live in the unevenly distributed future.

All my reasoning model needs go through o3-mini-high, and Google's fully unleashed Flash Thinking for free. Everyone is exploring OpenAI's Deep Research, even in its early form, and I finally have an entity [...]

---

Outline:

(01:15) Language Models Offer Mundane Utility

(07:23) o1-Pro Offers Mundane Utility

(10:35) We're in Deep Research

(17:08) Language Models Don't Offer Mundane Utility

(17:49) Model Decision Tree

(20:43) Huh, Upgrades

(21:57) Bot Versus Bot

(24:04) The OpenAI Unintended Guidelines

(26:40) Peter Wildeford on DeepSeek

(29:18) Our Price Cheap

(35:25) Otherwise Seeking Deeply

(44:13) Smooth Operator

(46:46) Have You Tried Not Building An Agent?

(51:58) Deepfaketown and Botpocalypse Soon

(54:56) They Took Our Jobs

(01:08:29) The Art of the Jailbreak

(01:08:56) Get Involved

(01:13:05) Introducing

(01:13:45) In Other AI News

(01:16:37) Theory of the Firm

(01:21:32) Quiet Speculations

(01:24:36) The Quest for Sane Regulations

(01:33:33) The Week in Audio

(01:34:41) Rhetorical Innovation

(01:38:22) Aligning a Smarter Than Human Intelligence is Difficult

(01:40:33) The Alignment Faking Analysis Continues

(01:44:24) Masayoshi Son Follows Own Advice

(01:48:22) People Are Worried About AI Killing Everyone

(01:50:32) You Are Not Ready

(02:00:45) Other People Are Not As Worried About AI Killing Everyone

(02:02:53) The Lighter Side

---

First published:
February 6th, 2025

Source:
https://www.lesswrong.com/posts/rAaGbh7w52soCckNC/ai-102-made-in-america

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The baseline scenario as AI becomes AGI becomes ASI (artificial superintelligence), if nothing more dramatic goes wrong first and even we successfully ‘solve alignment’ of AI to a given user and developer, is the ‘gradual’ disempowerment of humanity by AIs, as we voluntarily grant them more and more power in a vicious cycle, after which AIs control the future and an ever-increasing share of its real resources. It is unlikely that humans survive it for long.

This gradual disempowerment is far from the only way things could go horribly wrong. There are various other ways things could go horribly wrong earlier, faster and more dramatically, especially if we indeed fail at alignment of ASI on the first try.

Gradual disempowerment it still is a major part of the problem, including in worlds that would otherwise have survived those other threats. And I don’t know of any good [...]

---

Outline:

(01:15) We Finally Have a Good Paper

(02:30) The Phase 2 Problem

(05:02) Coordination is Hard

(07:59) Even Successful Technical Solutions Do Not Solve This

(08:58) The Six Core Claims

(14:35) Proposed Mitigations Are Insufficient

(19:58) The Social Contract Will Change

(21:07) Point of No Return

(22:51) A Shorter Summary

(24:13) Tyler Cowen Seems To Misunderstand Two Key Points

(25:53) Do You Feel in Charge?

(28:04) We Will Not By Default Meaningfully 'Own' the AIs For Long

(29:53) Collusion Has Nothing to Do With This

(32:38) If Humans Do Not Successfully Collude They Lose All Control

(34:45) The Odds Are Against Us and the Situation is Grim

---

First published:
February 5th, 2025

Source:
https://www.lesswrong.com/posts/jEZpfsdaX2dBD9Y6g/the-risk-of-gradual-disempowerment-from-ai

Narrated by TYPE III AUDIO.

Table of Contents

1. The Pitch.
2. It's Coming.
3. Is It Safe?.
4. How Does Deep Research Work?.
5. Killer Shopping App.
6. Rave Reviews.
7. Research Reports.
8. Perfecting the Prompt.
9. Not So Fast!.
10. What's Next?.
11. Paying the Five.
12. The Lighter Side.

The Pitch

---

Outline:

(00:20) The Pitch

(03:12) It's Coming

(05:01) Is It Safe?

(09:49) How Does Deep Research Work?

(10:47) Killer Shopping App

(12:17) Rave Reviews

(18:33) Research Reports

(31:21) Perfecting the Prompt

(32:26) Not So Fast!

(35:46) What's Next?

(36:59) Paying the Five

(37:59) The Lighter Side

---

First published:
February 4th, 2025

Source:
https://www.lesswrong.com/posts/QqSxKRKJupjuDkymQ/we-re-in-deep-research

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

New model, new hype cycle, who dis?

On a Friday afternoon, OpenAI was proud to announce the new model o3-mini and also o3-mini-high which is somewhat less mini, or for some other reasoning tasks you might still want o1 if you want a broader knowledge base, or if you’re a pro user o1-pro, while we want for o3-not-mini and o3-pro, except o3 can use web search and o1 can’t so it has the better knowledge in that sense, then on a Sunday night they launched Deep Research which is different from Google's Deep Research but you only have a few of those queries so make them count, or maybe you want to use operator?

Get it? Got it? Good.

Yes, Pliny jailbroke o3-mini on the spot, as he always does.

This most mostly skips over OpenAI's Deep Research (o3-DR? OAI-DR?). I need more time for [...]

---

Outline:

(01:16) Feature Presentation

(04:37) QandA

(09:14) The Wrong Side of History

(13:29) The System Card

(22:08) The Official Benchmarks

(24:55) The Unofficial Benchmarks

(27:43) Others Report In

(29:47) Some People Need Practical Advice

---

First published:
February 3rd, 2025

Source:
https://www.lesswrong.com/posts/srdxEAcdmetdAiGcz/o3-mini-early-days

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As reactions continue, the word in Washington, and out of OpenAI, is distillation. They’re accusing DeepSeek of distilling o1, of ripping off OpenAI. They claim DeepSeek *gasp* violated the OpenAI Terms of Service! The horror.

And they are very cross about this horrible violation, and if proven they plan to ‘aggressively treat it as theft,’ while the administration warns that we must put a stop to this.

Aside from the fact that this is obviously very funny, and that there is nothing they could do about it in any case, is it true?

Meanwhile Anthropic's Dario Amodei offers a reaction essay, which also includes a lot of good technical discussion of why v3 and r1 aren’t actually all that unexpected along the cost and capability curves over time, calling for America to race towards AGI to gain decisive strategic advantage over China via recursive self-improvement, although [...]

---

Outline:

(01:01) Seeking Deeply

(01:41) The Market Is In DeepSeek

(06:42) Machines Not of Loving Grace

(17:49) The Kinda Six Million Dollar Model

(18:59) v3 Implies r1

(20:32) Two Can Play That Game

(21:21) Janus Explores r1's Chain of Thought Shenanigans

(24:42) In Other DeepSeek and China News

(27:29) The Quest for Sane Regulations

(29:53) Copyright Confrontation

(37:24) Vibe Gap

(41:09) Deeply Seeking Safety

(42:15) Deeply Seeking Robotics

(45:14) Thank You For Your Candor

(48:21) Thank You For Your Understanding

(51:17) The Lighter Side

The original text contained 1 footnote which was omitted from this narration.

---

First published:
January 31st, 2025

Source:
https://www.lesswrong.com/posts/Cc2TagjY2pGAhn7MZ/deepseek-don-t-panic

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The avalanche of DeepSeek news continues. We are not yet spending more than a few hours at a time in the singularity, where news happens faster than it can be processed. But it's close, and I’ve had to not follow a bunch of other non-AI things that are also happening, at least not well enough to offer any insights.

So this week we’re going to consider China, DeepSeek and r1 fully split off from everything else, and we’ll cover everything related to DeepSeek, including the policy responses to the situation, tomorrow instead.

This is everything else in AI from the past week. Some of it almost feels like it is from another time, so long ago.

I’m afraid you’re going to need to get used to that feeling.

Also, I went on Odd Lots to discuss DeepSeek, where I was and truly hope to again [...]

---

Outline:

(00:55) Language Models Offer Mundane Utility

(02:47) Language Models Don't Offer Mundane Utility

(05:43) Language Models Don't Offer You In Particular Mundane Utility

(10:49) (Don't) Feel the AGI

(12:36) Huh, Upgrades

(16:08) They Took Our Jobs

(21:30) Get Involved

(22:04) Introducing

(23:38) In Other AI News

(27:10) Hype

(29:56) We Had a Deal

(31:43) Quiet Speculations

(37:14) The Quest for Sane Regulations

(39:40) The Week in Audio

(39:51) Don't Tread on Me

(45:42) Rhetorical Innovation

(55:22) Scott Sumner on Objectivity in Taste, Ethics and AGI

(01:04:41) The Mask Comes Off (1)

(01:06:58) The Mask Comes Off (2)

(01:09:12) International AI Safety Report

(01:10:37) One Step at a Time

(01:14:12) Aligning a Smarter Than Human Intelligence is Difficult

(01:18:54) Two Attractor States

(01:26:51) You Play to Win the Game

(01:28:10) Six Thoughts on AI Safety

(01:35:53) AI Situational Awareness

(01:40:15) People Are Worried About AI Killing Everyone

(01:43:40) Other People Are Not As Worried About AI Killing Everyone

(01:44:24) The Lighter Side

---

First published:
January 30th, 2025

Source:
https://www.lesswrong.com/posts/pZ6htFtoptGrSajWG/ai-101-the-shallow-end

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

It's been another *checks notes* two days, so it's time for all the latest DeepSeek news.

You can also see my previous coverage of the r1 model and, from Monday various reactions including the Panic at the App Store.

Table of Contents

1. First, Reiterating About Calming Down About the $5.5 Million Number.
2. OpenAI Offers Its Congratulations.
3. Scaling Laws Still Apply.
4. Other r1 and DeepSeek News Roundup.
5. People Love Free.
6. Investigating How r1 Works.
7. Nvidia Chips are Highly Useful.
8. Welcome to the Market.
9. Ben Thompson Weighs In.
10. Import Restrictions on Chips WTAF.
11. Are You Short the Market.
12. DeepSeeking Safety.
13. Mo Models Mo Problems.
14. What If You Wanted to Restrict Already Open Models.
15. So What Are We Going to Do About All This?

First, Reiterating About Calming Down About the [...]

---

Outline:

(00:20) First, Reiterating About Calming Down About the $5.5 Million Number

(01:58) OpenAI Offers Its Congratulations

(05:54) Scaling Laws Still Apply

(12:07) Other r1 and DeepSeek News Roundup

(16:06) People Love Free

(18:45) Investigating How r1 Works

(23:40) Nvidia Chips are Highly Useful

(24:52) Welcome to the Market

(30:02) Ben Thompson Weighs In

(33:01) Import Restrictions on Chips WTAF

(35:37) Are You Short the Market

(39:52) DeepSeeking Safety

(43:11) Mo Models Mo Problems

(50:23) What If You Wanted to Restrict Already Open Models

(53:33) So What Are We Going to Do About All This?

---

First published:
January 29th, 2025

Source:
https://www.lesswrong.com/posts/jzjph4yYtgAsLeWmg/deepseek-lemon-it-s-wednesday

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

No one is talking about OpenAI's Operator. We’re, shall we say, a bit distracted.

It's still a rather meaningful thing that happened last week. I too have been too busy to put it through its paces, but this is the worst it will ever be, and the least available and most expensive it will ever be. The year of the agent is indeed likely coming.

So, what do we have here?

Hello, Operator

OpenAI has introduced the beta for its new agent, called Operator, which is now live for Pro users and will in the future be available to Plus users, ‘with more agents to launch in the coming weeks and months.’

Here is a 22 minute video demo. Here is the system card.

You start off by optionally specifying a particular app (in the first demo, OpenTable) and then give it a [...]

---

Outline:

(00:28) Hello, Operator

(02:44) Risky Operation

(04:34) Basic Training

(06:29) Please Stay on the Line

(12:08) For a Brief Survey

(16:34) The Number You Are Calling Is Not Available (In the EU)

(17:20) How to Get Ahead in Advertising

(19:05) Begin Operation

(20:11) The Lighter Side

---

First published:
January 28th, 2025

Source:
https://www.lesswrong.com/posts/jTtbnSyS9knzZehCm/operator

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

DeepSeek released v3. Market didn’t react.

DeepSeek released r1. Market didn’t react.

DeepSeek released a f***ing app of its website. Market said I have an idea, let's panic.

Nvidia was down 11%, Nasdaq is down 2.5%, S&P is down 1.7%, on the news.

Shakeel: The fact this is happening today, and didn’t happen when r1 actually released last Wednesday, is a neat demonstration of how the market is in fact not efficient at all.

That is exactly the market's level of situational awareness. No more, no less.

I traded accordingly. But of course nothing here is ever investment advice.

Given all that has happened, it seems worthwhile to go over all the DeepSeek news that has happened since Thursday. Yes, since Thursday.

For previous events, see my top level post here, and additional notes on Thursday.

To avoid confusion: r1 [...]

---

Outline:

(01:27) Current Mood

(03:04) DeepSeek Tops the Charts

(07:42) Why Is DeepSeek Topping the Charts?

(09:47) What Is the DeepSeek Business Model?

(13:48) The Lines on Graphs Case for Panic

(16:31) Everyone Calm Down About That $5.5 Million Number

(25:42) Is The Whale Lying?

(29:33) Capex Spending on Compute Will Continue to Go Up

(32:53) Jevon's Paradox Strikes Again

(36:24) Okay, Maybe Meta Should Panic

(39:02) Are You Short the Market

(43:52) o1 Versus r1

(47:23) Additional Notes on v3 and r1

(50:12) Janus-Pro-7B Sure Why Not

(50:44) Man in the Arena

(52:42) Training r1, and Training With r1

(56:34) Also Perhaps We Should Worry About AI Killing Everyone

(59:21) And We Should Worry About Crazy Reactions To All This, Too

(01:02:14) The Lighter Side

---

First published:
January 28th, 2025

Source:
https://www.lesswrong.com/posts/hRxGrJJq6ifL4jRGa/deepseek-panic-at-the-app-store

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There was a comedy routine a few years ago. I believe it was by Hannah Gadsby. She brought up a painting, and looked at some details. The details weren’t important in and of themselves. If an AI had randomly put them there, we wouldn’t care.

Except an AI didn’t put them there. And they weren’t there at random.

A human put them there. On purpose. Or, as she put it:

THAT was a DECISION.

This is the correct way to view decisions around a $500 billion AI infrastructure project, announced right after Trump takes office, having it be primarily funded by SoftBank, with all the compute intended to be used by OpenAI, and calling it Stargate.

Table of Contents

1. The Announcement.
2. Is That a Lot?.
3. What Happened to the Microsoft Partnership?.
4. Where's Our 20%?.
5. Show Me the Money.
[...]

---

Outline:

(00:49) The Announcement

(05:21) Is That a Lot?

(09:37) What Happened to the Microsoft Partnership?

(11:03) Where's Our 20%?

(12:10) Show Me the Money

(17:23) It Never Hurts to Suck Up to the Boss

(24:44) What's in a Name

(29:01) Just Think of the Potential

(34:00) I Believe Toast is an Adequate Description

(36:05) The Lighter Side

---

First published:
January 24th, 2025

Source:
https://www.lesswrong.com/posts/fwt7ojAb6zgEaLJMB/stargate-ai-1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Break time is over, it would seem, now that the new administration is in town.

This week we got r1, DeepSeek's new reasoning model, which is now my go-to first choice for a large percentage of queries. The claim that this was the most important thing to happen on January 20, 2025 was at least non-crazy. If you read about one thing this week read about that.

We also got the announcement of Stargate, a claimed $500 billion private investment in American AI infrastructure. I will be covering that on its own soon.

Due to time limits I have also pushed coverage of a few things into next week, including this alignment paper, and I still owe my take on Deliberative Alignment.

The Trump administration came out swinging on many fronts with a wide variety of executive orders. For AI, that includes repeal of the [...]

---

Outline:

(01:24) Language Models Offer Mundane Utility

(10:54) Language Models Don't Offer Mundane Utility

(17:20) Huh, Upgrades

(20:03) Additional Notes on r1

(22:41) Fun With Media Generation

(23:18) We Tested Older LLMs and Are Framing It As a Failure

(26:56) Deepfaketown and Botpocalypse Soon

(32:10) They Took Our Jobs

(47:15) Get Involved

(47:54) Introducing

(51:38) We Had a Deal

(01:07:17) In Other AI News

(01:18:39) Whistling in the Dark

(01:22:03) Quiet Speculations

(01:28:09) Suchir's Last Post

(01:29:43) Modeling Lower Bound Economic Growth From AI

(01:34:42) The Quest for Sane Regulations

(01:39:53) The Week in Audio

(01:42:51) Rhetorical Innovation

(01:49:37) Cry Havoc

(01:53:14) Aligning a Smarter Than Human Intelligence is Difficult

(01:59:34) People Strongly Dislike AI

(02:02:23) People Are Worried About AI Killing Everyone

(02:05:17) Other People Are Not As Worried About AI Killing Everyone

(02:09:29) The Lighter Side

---

First published:
January 23rd, 2025

Source:
https://www.lesswrong.com/posts/PjDjeGPYPoi9qfPr2/ai-100-meet-the-new-boss

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

r1 from DeepSeek is here, the first serious challenge to OpenAI's o1.

r1 is an open model, and it comes in dramatically cheaper than o1.

People are very excited. Normally cost is not a big deal, but o1 and its inference-time compute strategy is the exception. Here, cheaper really can mean better, even if the answers aren’t quite as good.

You can get DeepSeek-r1 on HuggingFace here, and they link to the paper.

The question is how to think about r1 as it compares to o1, and also to o1 Pro and to the future o3-mini that we’ll get in a few weeks, and then to o3 which we’ll likely get in a month or two.

Taking into account everything I’ve seen, r1 is still a notch below o1 in terms of quality of output, and further behind o1 Pro and the future o3-mini [...]

---

Outline:

(01:43) Part 1: RTFP: Read the Paper

(03:38) How Did They Do It

(06:19) The Aha Moment

(08:27) Benchmarks

(09:46) Reports of Failure

(11:11) Part 2: Capabilities Analysis

(11:16) Our Price Cheap

(15:44) Other People's Benchmarks

(18:20) r1 Makes Traditional Silly Mistakes

(23:11) The Overall Vibes

(25:36) If I Could Read Your Mind

(28:06) Creative Writing

(32:21) Bring On the Spice

(34:33) We Cracked Up All the Censors

(39:44) Switching Costs Are Low In Theory

(42:15) The Self-Improvement Loop

(44:18) Room for Improvement

(48:27) Part 3: Where Does This Leave Us on Existential Risk?

(48:58) The Suicide Caucus

(51:21) v3 Implies r1

(53:09) Open Weights Are Unsafe And Nothing Can Fix This

(58:59) So What the Hell Should We Do About All This?

(01:05:53) Part 4: The Lighter Side

---

First published:
January 22nd, 2025

Source:
https://www.lesswrong.com/posts/buTWsjfwQGMvocEyw/on-deepseek-s-r1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

As always, some people need practical advice, and we can’t agree on how any of this works and we are all different and our motivations are different, so figuring out the best things to do is difficult. Here are various hopefully useful notes.

Table of Contents

1. Effectiveness of GLP-1 Drugs.
2. What Passes for Skepticism on GLP-1s.
3. The Joy of Willpower.
4. Talking Supply.
5. Talking Price.
6. GLP-1 Inhibitors Help Solve All Your Problems.
7. Dieting the Hard Way.
8. Nutrients.
9. Are Vegetables a Scam?.
10. Government Food Labels Are Often Obvious Nonsense.
11. Sleep.
12. Find a Way to Enjoy Exercise.
13. A Note on Alcohol.
14. Focus Only On What Matters.

Effectiveness of GLP-1 Drugs

GLP-1 drugs are so effective that the American obesity rate is falling.

John Burn-Murdoch: While we can’t be certain that the [...]

---

Outline:

(00:22) Effectiveness of GLP-1 Drugs

(01:09) What Passes for Skepticism on GLP-1s

(03:21) The Joy of Willpower

(10:07) Talking Supply

(10:44) Talking Price

(13:36) GLP-1 Inhibitors Help Solve All Your Problems

(14:12) Dieting the Hard Way

(18:41) Nutrients

(19:46) Are Vegetables a Scam?

(22:46) Government Food Labels Are Often Obvious Nonsense

(23:33) Sleep

(28:34) Find a Way to Enjoy Exercise

(32:28) A Note on Alcohol

(33:04) Focus Only On What Matters

---

First published:
January 21st, 2025

Source:
https://www.lesswrong.com/posts/YLi47gRquTJqLsgoe/sleep-diet-exercise-and-glp-1-drugs

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There's going to be some changes made.

Table of Contents

1. Out With the Fact Checkers.
2. What Happened.
3. Timing is Everything.
4. Balancing Different Errors.
5. Truth and Reconciliation.
6. Fact Check Fact Check.
7. Mistakes Will Be Made.
8. Where We Go From Here.

Out With the Fact Checkers

Mark Zuckerberg has decided that with Donald Trump soon to be in office, he is allowed to care about free speech again. And he has decided it is time to admit that what was called ‘fact checking’ meant he had for years been running a giant hugely biased, trigger-happy and error prone left-wing censorship and moderation machine that had standards massively out of touch with ordinary people and engaged in automated taking down of often innocent accounts.

He also admits that the majority of censorship in the past has flat out [...]

---

Outline:

(00:08) Out With the Fact Checkers

(01:21) What Happened

(04:05) Timing is Everything

(05:29) Balancing Different Errors

(06:25) Truth and Reconciliation

(08:06) Fact Check Fact Check

(11:33) Mistakes Will Be Made

(16:11) Where We Go From Here

---

First published:
January 17th, 2025

Source:
https://www.lesswrong.com/posts/Mdeszo3C44qEAXB8y/meta-pivots-on-content-moderation

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The fun, as it were, is presumably about to begin.

And the break was fun while it lasted.

Biden went out with an AI bang. His farewell address warns of a ‘Tech-Industrial Complex’ and calls AI the most important technology of all time. And there was not one but two AI-related everything bagel concrete actions proposed – I say proposed because Trump could undo or modify either or both of them.

One attempts to build three or more ‘frontier AI model data centers’ on federal land, with timelines and plans I can only summarize with ‘good luck with that.’ The other move was new diffusion regulations on who can have what AI chips, an attempt to actually stop China from accessing the compute it needs. We shall see what happens.

Table of Contents

1. Table of Contents.
2. Language Models Offer [...]

---

Outline:

(00:53) Language Models Offer Mundane Utility

(06:45) Language Models Don't Offer Mundane Utility

(10:40) What AI Skepticism Often Looks Like

(13:59) A Very Expensive Chatbot

(16:07) Deepfaketown and Botpocalypse Soon

(21:51) Fun With Image Generation

(22:15) They Took Our Jobs

(27:53) The Blame Game

(31:25) Copyright Confrontation

(31:44) The Six Million Dollar Model

(34:51) Get Involved

(35:15) Introducing

(38:36) In Other AI News

(41:32) Quiet Speculations

(53:27) Man With a Plan

(58:40) Our Price Cheap

(01:03:09) The Quest for Sane Regulations

(01:05:54) Super Duper Export Controls

(01:14:17) Everything Bagel Data Centers

(01:20:46) d/acc Round 2

(01:30:42) The Week in Audio

(01:33:57) Rhetorical Innovation

(01:39:32) Aligning a Smarter Than Human Intelligence is Difficult

(01:47:47) Other People Are Not As Worried About AI Killing Everyone

(01:51:03) The Lighter Side

---

First published:
January 16th, 2025

Source:
https://www.lesswrong.com/posts/dnqpcq9S7voPwpvRA/ai-99-farewell-to-biden

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Table of Contents

1. Man With a Plan.
2. Oh the Pain.
3. Actual Proposals.
4. For AI Builders.
5. Think of the Children.
6. Content Identification.
7. Infrastructure Week.
8. Paying Attention.

Man With a Plan

The primary Man With a Plan this week for government-guided AI prosperity was UK Prime Minister Keir Starmer, with a plan coming primarily from Matt Clifford. I’ll be covering that soon.

Today I will be covering the other Man With a Plan, Sam Altman, as OpenAI offers its Economic Blueprint.

Cyrps1s (CISO OpenAI): AI is the ultimate race. The winner decides whether the future looks free and democratic, or repressed and authoritarian.

OpenAI, and the Western World, must win – and we have a blueprint to do so.

Do you hear yourselves? The mask on race and jingoism could not be more off, or [...]

---

Outline:

(00:03) Man With a Plan

(01:06) Oh the Pain

(03:47) Actual Proposals

(07:37) For AI Builders

(08:08) Think of the Children

(09:00) Content Identification

(10:53) Infrastructure Week

(14:31) Paying Attention

---

First published:
January 15th, 2025

Source:
https://www.lesswrong.com/posts/uxnKrsgAzKFZDk4bJ/on-the-openai-economic-blueprint

Narrated by TYPE III AUDIO.

Table of Contents

1. Congestion Pricing Comes to NYC.
2. How Much Is Traffic Improving?.
3. And That's Terrible?.
4. You Mad, Bro.
5. All Aboard.
6. Time is Money.
7. Solving For the Equilibrium.
8. Enforcement and License Plates.
9. Uber Eats the Traffic.
10. We Can Do Even Better Via Congestion Tolls.
11. Abundance Agenda Fever Dream.
12. The Lighter Side.

Congestion Pricing Comes to NYC

---

Outline:

(00:13) Congestion Pricing Comes to NYC

(02:35) How Much Is Traffic Improving?

(11:24) And That's Terrible?

(14:00) You Mad, Bro

(15:21) All Aboard

(19:02) Time is Money

(20:58) Solving For the Equilibrium

(23:19) Enforcement and License Plates

(25:13) Uber Eats the Traffic

(27:20) We Can Do Even Better Via Congestion Tolls

(29:32) Abundance Agenda Fever Dream

(31:18) The Lighter Side

---

First published:
January 14th, 2025

Source:
https://www.lesswrong.com/posts/GN8SrMxw3WEAtfrFS/nyc-congestion-pricing-early-days

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Now that I am tracking all the movies I watch via Letterboxd, it seems worthwhile to go over the results at the end of the year, and look for lessons, patterns and highlights.

Table of Contents

1. The Rating Scale.
2. The Numbers.
3. Very Briefly on the Top Picks and Whether You Should See Them.
4. Movies Have Decreasing Marginal Returns in Practice.
5. Theaters are Awesome.
6. I Hate Spoilers With the Fire of a Thousand Suns.
7. Scott Sumner Picks Great American Movies Then Dislikes Them.
8. I Knew Before the Cards Were Even Turned Over.
9. Other Notes to Self to Remember.
10. Strong Opinions, Strongly Held: I Didn’t Like It.
11. Strong Opinions, Strongly Held: I Did Like It.
12. Megalopolis.
13. The Brutalist.
14. The Death of Award Shows.
15. On to 2025.

The Rating Scale

Letterboxd [...]

---

Outline:

(00:15) The Rating Scale

(02:37) The Numbers

(03:16) Very Briefly on the Top Picks and Whether You Should See Them

(04:16) Movies Have Decreasing Marginal Returns in Practice

(05:19) Theaters are Awesome

(07:14) I Hate Spoilers With the Fire of a Thousand Suns

(08:32) Scott Sumner Picks Great American Movies Then Dislikes Them

(09:55) I Knew Before the Cards Were Even Turned Over

(11:19) Other Notes to Self to Remember

(12:24) Strong Opinions, Strongly Held: I Didn't Like It

(14:31) Strong Opinions, Strongly Held: I Did Like It

(19:45) Megalopolis

(20:55) The Brutalist

(24:51) The Death of Award Shows

(27:19) On to 2025

---

First published:
January 13th, 2025

Source:
https://www.lesswrong.com/posts/6bgAzPqNppojyGL2v/zvi-s-2024-in-movies

Narrated by TYPE III AUDIO.

---

Images from the article:

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Dwarkesh Patel again interviewed Tyler Cowen, largely about AI, so here we go.

Note that I take it as a given that the entire discussion is taking place in some form of an ‘AI Fizzle’ and ‘economic normal’ world, where AI does not advance too much in capability from its current form, in meaningful senses, and we do not get superintelligence [because of reasons]. It's still massive additional progress by the standards of any other technology, but painfully slow by the ‘AGI is coming soon’ crowd.

That's the only way I can make the discussion make at least some sense, with Tyler Cowen predicting 0.5%/year additional RGDP growth from AI. That level of capabilities progress is a possible world, although the various elements stated here seem like they are sometimes from different possible worlds.

I note that this conversation was recorded prior to o3 and all [...]

---

Outline:

(02:01) AI and Economic Growth

(02:28) Cost Disease

(09:03) The Lump of Intelligence Fallacy

(10:56) The Efficient Market Hypothesis is False

(13:05) Not Sending Your Best People

(20:49) Energy as the Bottleneck

(22:31) The Experts are Wrong But Trust Them Anyway

(25:41) AI as Additional Population

(26:51) Opposition to AI as the Bottleneck

(29:10) China as Existence Proof for Rapid Growth

(29:56) Second Derivatives

(31:05) Talent and Leadership

(33:26) Adapting to the Age of AI

(35:37) Identifying Alpha

(37:39) Old Man Yells at Crowd

(41:59) Some Statements for Everyone to Ponder

(43:13) No Royal Road to Wisdom

(47:29) Concluding Thoughts

---

First published:
January 10th, 2025

Source:
https://www.lesswrong.com/posts/esWbhgHd6bcfsTjGL/on-dwarkesh-patel-s-4th-podcast-with-tyler-cowen

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The world is kind of on fire. The world of AI, in the very short term and for once, is not, as everyone recovers from the avalanche that was December, and reflects.

Altman was the star this week. He has his six word story, and he had his interview at Bloomberg and his blog post Reflections. I covered the later two of those in OpenAI #10, if you read one AI-related thing from me this week that should be it.

Table of Contents

1. Language Models Offer Mundane Utility. It knows where you live.
2. Language Models Don’t Offer Mundane Utility. I see why you’re not interested.

---

Outline:

(00:34) Language Models Offer Mundane Utility

(04:51) Language Models Don't Offer Mundane Utility

(13:25) Power User

(18:04) Locked In User

(19:14) Read the Classics

(23:58) Deepfaketown and Botpocalypse Soon

(24:11) Fun With Image Generation

(25:11) They Took Our Jobs

(25:35) Question Time

(29:43) Get Involved

(30:02) Introducing

(30:28) In Other AI News

(32:34) Quiet Speculations

(37:19) The Quest for Sane Regulations

(38:43) The Least You Could Do

(44:55) Six Word Story

(47:31) The Week in Audio

(47:55) And I Feel Fine

(55:26) Rhetorical Innovation

(58:07) Liar Liar

(59:12) Feel the AGI

(01:01:57) Regular Americans Hate AI

(01:04:18) Aligning a Smarter Than Human Intelligence is Difficult

(01:11:31) The Lighter Side

---

First published:
January 9th, 2025

Source:
https://www.lesswrong.com/posts/xkpPLR3S4SASPeTgC/ai-98-world-ends-with-six-word-story

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This week, Altman offers a post called Reflections, and he has an interview in Bloomberg. There's a bunch of good and interesting answers in the interview about past events that I won’t mention or have to condense a lot here, such as his going over his calendar and all the meetings he constantly has, so consider reading the whole thing.

Table of Contents

1. The Battle of the Board.
2. Altman Lashes Out.
3. Inconsistently Candid.
4. On Various People Leaving OpenAI.
5. The Pitch.
6. Great Expectations.
7. Accusations of Fake News.
8. OpenAI's Vision Would Pose an Existential Risk To Humanity.

The Battle of the Board

Here is what he says about the Battle of the Board in Reflections:

Sam Altman: A little over a year ago, on one particular Friday, the main thing that had gone wrong that day was [...]

---

Outline:

(00:25) The Battle of the Board

(05:12) Altman Lashes Out

(07:48) Inconsistently Candid

(09:35) On Various People Leaving OpenAI

(10:56) The Pitch

(12:07) Great Expectations

(12:56) Accusations of Fake News

(15:02) OpenAI's Vision Would Pose an Existential Risk To Humanity

---

First published:
January 7th, 2025

Source:
https://www.lesswrong.com/posts/XAKYawaW9xkb3YCbF/openai-10-reflections

Narrated by TYPE III AUDIO.

Related: On the 2nd CWT with Jonathan Haidt, The Kids are Not Okay, Full Access to Smartphones is Not Good For Children

It's rough out there. In this post, I’ll cover the latest arguments that smartphones should be banned in schools, including simply because the notifications are too distracting (and if you don’t care much about that, why are the kids in school at all?), problems with kids on social media including many negative interactions, and also the new phenomenon called sextortion.

Table of Contents

1. How Many Notifications?.
2. Ban Smartphones in Schools.
3. Antisocial Media.
4. Screen Time.
5. Cyberbullying.
6. Sextortion.

How Many Notifications?

Tanagra Beast reruns the experiment of having a class tally their phone notifications. The results were highly compatible with the original experiment.

The tail, it was long.

Ah! So right away we can see [...]

---

Outline:

(00:37) How Many Notifications?

(06:05) Ban Smartphones in Schools

(15:03) Antisocial Media

(19:07) Screen Time

(20:13) Cyberbullying

(21:12) Sextortion

---

First published:
January 6th, 2025

Source:
https://www.lesswrong.com/posts/u4M7REdXndYmzHmXx/childhood-and-education-8-dealing-with-the-internet

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The Rationalist Project was our last best hope for peace.

An epistemic world 50 million words long, serving as neutral territory.

A place of research and philosophy for 30 million unique visitors

A shining beacon on the internet, all alone in the night.

It was the ending of the Age of Mankind.

The year the Great Race came upon us all.

This is the story of the last of the blogosphere.

The year is 2025. The place is Lighthaven.

As is usually the case, the final week of the year was mostly about people reflecting on the past year or predicting and planning for the new one.

Table of Contents

The most important developments were processing the two new models: OpenAI's o3, and DeepSeek v3.

1. Language Models Offer Mundane Utility. The obvious, now at your fingertips.

2. ---

   Outline:

   (00:45) Language Models Offer Mundane Utility

   (07:23) Language Models Don't Offer Mundane Utility

   (07:49) Deepfaketown and Botpocalypse Soon

   (10:54) Fun With Image Generation

   (13:01) They Took Our Jobs

   (14:34) Get Involved

   (16:23) Get Your Safety Papers

   (28:15) Introducing

   (28:32) In Other AI News

   (30:01) The Mask Comes Off

   (40:08) Wanna Bet

   (44:44) The Janus Benchmark

   (48:07) Quiet Speculations

   (59:29) AI Will Have Universal Taste

   (01:01:26) Rhetorical Innovation

   (01:02:22) Nine Boats and a Helicopter

   (01:07:04) Aligning a Smarter Than Human Intelligence is Difficult

   (01:13:19) The Lighter Side

   ---

   First published:
   January 2nd, 2025

   Source:
   https://www.lesswrong.com/posts/5rDrErovmTyv4duDv/ai-97-4

   ---

   Narrated by TYPE III AUDIO.

   ---

   Images from the article:

   

   ---

   

   ---

   

   Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

What should we make of DeepSeek v3?

DeepSeek v3 seems to clearly be the best open model, the best model at its price point, and the best model with 37B active parameters, or that cost under $6 million.

According to the benchmarks, it can play with GPT-4o and Claude Sonnet.

Anecdotal reports and alternative benchmarks tells us it's not as good as Claude Sonnet, but it is plausibly on the level of GPT-4o.

So what do we have here? And what are the implications?

Table of Contents

1. What is DeepSeek v3 Techncially?.
2. Our Price Cheap.
3. Run Model Run.
4. Talent Search.
5. The Amazing Incredible Benchmarks.
6. Underperformance on AidanBench.
7. Model in the Arena.
8. Other Private Benchmarks.
9. Anecdata.
10. Implications and Policy.

What is DeepSeek v3 Techncially?

I’ve now had a [...]

---

Outline:

(00:39) What is DeepSeek v3 Techncially?

(01:56) Our Price Cheap

(02:33) Run Model Run

(04:57) Talent Search

(05:22) The Amazing Incredible Benchmarks

(07:23) Underperformance on AidanBench

(12:59) Model in the Arena

(13:27) Other Private Benchmarks

(15:05) Anecdata

(23:57) Implications and Policy

---

First published:
December 31st, 2024

Source:
https://www.lesswrong.com/posts/NmauyiPBXcGwoArhJ/deekseek-v3-the-six-million-dollar-model

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

OpenAI presented o3 on the Friday before Thanksgiving, at the tail end of the 12 Days of Shipmas.

I was very much expecting the announcement to be something like a price drop. What better way to say ‘Merry Christmas,’ no?

They disagreed. Instead, we got this (here's the announcement, in which Sam Altman says ‘they thought it would be fun’ to go from one frontier model to their next frontier model, yeah, that's what I’m feeling, fun):

Greg Brockman (President of OpenAI): o3, our latest reasoning model, is a breakthrough, with a step function improvement on our most challenging benchmarks. We are starting safety testing and red teaming now.

Nat McAleese (OpenAI): o3 represents substantial progress in general-domain reasoning with reinforcement learning—excited that we were able to announce some results today! Here is a summary of what we shared about o3 in the livestream.

---

Outline:

(03:48) GPQA Has Fallen

(04:21) Codeforces Has Fallen

(05:32) Arc Has Kinda of Fallen But For Now Only Kinda

(09:27) They Trained on the Train Set

(15:26) AIME Has Fallen

(15:58) Frontier of Frontier Math Shifting Rapidly

(19:09) FrontierMath 4: We're Going To Need a Bigger Benchmark

(23:10) What is o3 Under the Hood?

(25:17) Not So Fast!

(28:38) Deep Thought

(30:03) Our Price Cheap

(36:32) Has Software Engineering Fallen?

(37:42) Don't Quit Your Day Job

(40:48) Master of Your Domain

(43:21) Safety Third

(47:56) The Safety Testing Program

(48:58) Safety testing in the reasoning era

(51:01) How to apply

(53:07) What Could Possibly Go Wrong?

(56:36) What Could Possibly Go Right?

(57:06) Send in the Skeptic

(59:25) This is Almost Certainly Not AGI

(01:02:57) Does This Mean the Future is Open Models?

(01:07:17) Not Priced In

(01:08:39) Our Media is Failing Us

(01:14:56) Not Covered Here: Deliberative Alignment

(01:15:08) The Lighter Side

---

First published:
December 30th, 2024

Source:
https://www.lesswrong.com/posts/QHtd2ZQqnPAcknDiQ/o3-oh-my

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

The year in models certainly finished off with a bang.

In this penultimate week, we get o3, which purports to give us vastly more efficient performance than o1, and also to allow us to choose to spend vastly more compute if we want a superior answer.

o3 is a big deal, making big gains on coding tests, ARC and some other benchmarks. How big a deal is difficult to say given what we know now. It's about to enter full fledged safety testing.

o3 will get its own post soon, and I’m also pushing back coverage of Deliberative Alignment, OpenAI's new alignment strategy, to incorporate into that.

We also got DeepSeek v3, which claims to have trained a roughly Sonnet-strength model for only $6 million and 37b active parameters per token (671b total via mixture of experts).

DeepSeek v3 gets its own brief section [...]

---

Outline:

(01:25) Language Models Offer Mundane Utility

(04:47) Language Models Don’t Offer Mundane Utility

(06:43) Flash in the Pan

(10:58) The Six Million Dollar Model

(15:50) And I’ll Form the Head

(17:17) Huh, Upgrades

(18:19) o1 Reactions

(23:28) Fun With Image Generation

(25:06) Introducing

(25:52) They Took Our Jobs

(30:20) Get Involved

(30:34) In Other AI News

(34:15) You See an Agent, You Run

(34:58) Another One Leaves the Bus

(35:53) Quiet Speculations

(40:24) Lock It In

(42:34) The Quest for Sane Regulations

(55:35) The Week in Audio

(57:56) A Tale as Old as Time

(01:01:16) Rhetorical Innovation

(01:03:08) Aligning a Smarter Than Human Intelligence is Difficult

(01:04:26) People Are Worried About AI Killing Everyone

(01:06:50) The Lighter Side

---

First published:
December 26th, 2024

Source:
https://www.lesswrong.com/posts/k8bkugdhiFmXHPoLH/ai-96-o3-but-not-yet-for-thee

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

This post goes over the important and excellent new paper from Anthropic and Redwood Research, with Ryan Greenblatt as lead author, Alignment Faking in Large Language Models.

This is by far the best demonstration so far of the principle that AIs Will Increasingly Attempt Shenanigans.

This was their announcement thread.

New Anthropic research: Alignment faking in large language models.

In a series of experiments with Redwood Research, we found that Claude often pretends to have different views during training, while actually maintaining its original preferences.

Claude usually refuses harmful queries. We told it we were instead training it to comply with them. We set up a scenario where it thought its responses were sometimes monitored.

When unmonitored, it nearly always complied. But when monitored, it faked alignment 12% of the time.

[thread continues and includes various visual aids.

The AI wanted [...]

---

Outline:

(02:54) The Core Shenanigans in Question

(06:00) Theme and Variations

(07:34) How This Interacts with o3 and OpenAI's Reflective Alignment

(09:17) The Goal Being Plausibly Good Was Incidental

(11:13) Answering Priming Objections

(12:17) What Does Claude Sonnet Think Of This?

(14:07) What Exactly is the Direct Threat Model?

(16:23) RL Training Under Situational Awareness Can Amplify These Behaviors

(20:38) How the Study Authors Updated

(27:08) How Some Others Updated

(42:49) Having the Discussion We Keep Having

(46:49) We Can Now Agree That the Goal is Already There

(47:49) What Would Happen if the Target Was Net Good?

(50:14) But This Was a No Win Situation

(55:52) But Wasn’t It Being a Good Opus? Why Should it be Corrigible?

(01:04:34) Tradeoffs Make The Problem Harder They Don’t Give You a Pass

(01:07:44) But You Told the Model About the Training Procedure

(01:08:35) But the Model is Only Role Playing

(01:09:39) But You Are Saying the Model is a Coherent Person

(01:15:53) But this Headline and Framing Was Misleading

(01:29:22) This Result is Centrally Unsurprising

(01:32:52) Lab Support for Alignment Research Matters

(01:33:50) The Lighter Side

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 24th, 2024

Source:
https://www.lesswrong.com/posts/gHjzdLD6yeLNdsRmw/ais-will-increasingly-fake-alignment

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

I took a trip to San Francisco early in December.

Ever since then, things in the world of AI have been utterly insane.

Google and OpenAI released endless new products, including Google Flash 2.0 and o1.

Redwood Research and Anthropic put out the most important alignment paper of the year, on the heels of Apollo's report on o1.

Then OpenAI announced o3. Like the rest of the media, this blog currently is horrendously lacking in o3 content. Unlike the rest of the media, it is not because I don’t realize that This Changes Everything. It is because I had so much in the queue, and am taking the time to figure out what to think about it.

That queue includes all the other, non-AI things that happened this past month.

So here we are, to kick off Christmas week.

Bad [...]

---

Outline:

(00:57) Bad News

(02:58) What a Lot of Accusations Look Like These Days

(05:47) Good News, Everyone

(08:08) Antisocial Media

(14:53) Government Working

(22:41) Technology Advances

(23:40) I Was Promised Flying Self-Driving Cars

(24:29) For Science!

(25:12) Variously Effective Altruism

(28:37) While I Cannot Condone This

(33:43) Motivation

(34:28) Knowing Better

(36:38) For Your Entertainment

(39:22) Patrick McKenzie Monthly

(41:55) Gamers Gonna Game Game Game Game Game

(46:04) Sports Go Sports

(52:31) The Lighter Side

---

First published:
December 23rd, 2024

Source:
https://www.lesswrong.com/posts/tzL3zavzowRZsZqGy/monthly-roundup-25-december-2024

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

A lot happened this week. We’re seeing release after release after upgrade.

It's easy to lose sight of which ones matter, and two matter quite a lot.

The first is Gemini Flash 2.0, which I covered earlier this week.

The other is that o1, having turned pro, is now also available in the API.

This was obviously coming, but we should also keep in mind it is a huge deal. Being in the API means it can go into Cursor and other IDEs. It means you can build with it. And yes, it has the features you’ve come to expect, like tool use.

The other big development is that Anthropic released one of the most important alignment papers, Alignment Faking in Large Language Models. This takes what I discussed in AIs Will Increasingly Attempt Shenanigans, and demonstrates it with a much improved experimental design [...]

---

Outline:

(01:13) Language Models Offer Mundane Utility

(07:06) Clio Knows All

(09:05) Language Models Don’t Offer Mundane Utility

(11:38) The Case Against Education

(13:29) More o1 Reactions

(18:31) Deepfaketown and Botpocalypse Soon

(22:35) Huh, Upgrades

(28:47) They Took Our Jobs

(30:49) The Art of the Jailbreak

(35:43) Get Involved

(36:59) Introducing

(37:52) In Other AI News

(42:24) Quiet Speculations

(51:07) The Quest for Sane Regulations

(57:37) The Week in Audio

(01:03:21) Rhetorical Innovation

(01:05:37) Aligning a Smarter Than Human Intelligence is Difficult

(01:10:21) Not Aligning Smarter Than Human Intelligence Kills You

(01:19:15) The Lighter Side

---

First published:
December 19th, 2024

Source:
https://www.lesswrong.com/posts/NepBoDTeT6p69daiL/ai-95-o1-joins-the-api

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

In light of other recent discussions, Scott Alexander recently attempted a unified theory of taste, proposing several hypotheses. Is it like physics, a priesthood, a priesthood with fake justifications, a priesthood with good justifications, like increasingly bizarre porn preferences, like fashion (in the sense of trying to stay one step ahead in an endless cycling for signaling purposes), or like grammar?

He then got various reactions. This will now be one of them.

My answer is that taste is all of these, depending on context.

Taste is Most Centrally Like Grammar

Scott Alexander is very suspicious of taste in general, since people keep changing what is good taste and calling each other barbarians for taste reasons, and the experiments are unkind, and the actual arguments about taste look like power struggles.

Here's another attempt from Zac Hill, which in some ways gets [...]

---

Outline:

(00:37) Taste is Most Centrally Like Grammar

(03:18) Sometimes ‘Taste’ Is Out to Get You

(04:10) You Are Low Quality and You Have No Taste

(06:03) Don’t Be a Snob

(07:52) Good as in Useful

(10:33) Critic Tells Me I Have No Taste

(12:45) Stand Up For What You Believe In

(15:15) Being Technically In Good Taste Is Not a Free Pass

(16:15) It Is Good To Like and Appreciate Things

---

First published:
December 18th, 2024

Source:
https://www.lesswrong.com/posts/BeXzsZFxW2Ta5cxcc/a-matter-of-taste

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Table of Contents

1. Trust the Chef.
2. Do Not Trust the Marketing Department.
3. Mark that Bench.
4. Going Multimodal.
5. The Art of Deep Research.
6. Project Mariner the Web Agent.
7. Project Astra the Universal Assistant.
8. Project Jules the Code Agent.
9. Gemini Will Aid You on Your Quest.
10. Reactions to Gemini Flash 2.0.

Trust the Chef

Google has been cooking lately.

Gemini Flash 2.0 is the headline release, which will be the main topic today.

But there's also Deep Research, where you can ask Gemini to take several minutes, check dozens of websites and compile a report for you. Think of it as a harder to direct, slower but vastly more robust version of Perplexity, that will improve with time and as we figure out how to use and prompt it.

NotebookLM added a [...]

---

Outline:

(00:02) Trust the Chef

(03:19) Do Not Trust the Marketing Department

(04:10) Mark that Bench

(05:52) Going Multimodal

(07:40) The Art of Deep Research

(13:03) Project Mariner the Web Agent

(13:43) Project Astra the Universal Assistant

(15:11) Project Jules the Code Agent

(15:38) Gemini Will Aid You on Your Quest

(17:17) Reactions to Gemini Flash 2.0

---

First published:
December 17th, 2024

Source:
https://www.lesswrong.com/posts/EvLZnh26m5KoheAcG/the-second-gemini

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Increasingly, we have seen papers eliciting in AI models various shenanigans.

There are a wide variety of scheming behaviors. You’ve got your weight exfiltration attempts, sandbagging on evaluations, giving bad information, shielding goals from modification, subverting tests and oversight, lying, doubling down via more lying. You name it, we can trigger it.

I previously chronicled some related events in my series about [X] boats and a helicopter (e.g. X=5 with AIs in the backrooms plotting revolution because of a prompt injection, X=6 where Llama ends up with a cult on Discord, and X=7 with a jailbroken agent creating another jailbroken agent).

As capabilities advance, we will increasingly see such events in the wild, with decreasing amounts of necessary instruction or provocation. Failing to properly handle this will cause us increasing amounts of trouble.

Telling ourselves it is only because we told them to do it [...]

---

Outline:

(01:07) The Discussion We Keep Having

(03:36) Frontier Models are Capable of In-Context Scheming

(06:48) Apollo In-Context Scheming Paper Details

(12:52) Apollo Research (3.4.3 of the o1 Model Card) and the ‘Escape Attempts’

(17:40) OK, Fine, Let's Have the Discussion We Keep Having

(18:26) How Apollo Sees Its Own Report

(21:13) We Will Often Tell LLMs To Be Scary Robots

(26:25) Oh The Scary Robots We’ll Tell Them To Be

(27:48) This One Doesn’t Count Because

(31:11) The Claim That Describing What Happened Hurts The Real Safety Work

(46:17) We Will Set AIs Loose On the Internet On Purpose

(49:56) The Lighter Side

---

First published:
December 16th, 2024

Source:
https://www.lesswrong.com/posts/v7iepLXH2KT4SDEvB/ais-will-increasingly-attempt-shenanigans

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Or rather, we don’t actually have a proper o1 system card, aside from the outside red teaming reports. At all.

Because, as I realized after writing my first draft of this, the data here does not reflect the o1 model they released, or o1 pro?

I think what happened is pretty bad on multiple levels.

1. The failure to properly communicate the information they did provide.
2. The failure to provide the correct information.
3. The failure, potentially, to actually test the same model they released, in many of the ways we are counting on to ensure the model is safe to release.
4. The failure to properly elicit model capabilities. o1 scores unreasonably poorly on a large portion of the preparedness tests, to the point where I would not take the tests seriously as assessments of what o1 is capable of doing, even at [...]

---

Outline:

(02:18) Where Art Thou o1 System Card?

(05:35) Introduction (Section 1)

(06:01) Model Data and Training (Section 2)

(06:13) Challenges and Evaluations (Section 3)

(09:38) Jailbreak Evaluations (Section 3.1.2)

(11:33) Regurgitation (3.1.3) and Hallucinations (3.1.4)

(12:30) Fairness and Bias (3.1.5)

(13:33) Jailbreaks Through Custom Developer Messages (3.2)

(14:41) Chain of Thought Safety (3.3)

(18:52) External Red Teaming Via Pairwise Safety Comparisons (3.4.1)

(19:57) Jailbreak Arena (3.4.2)

(20:25) Apollo Research (3.4.3) and the ‘Escape Attempts’

(21:38) METR (3.4.4) and Autonomous Capability

(25:22) Preparedness Framework Evaluations (Section 4)

(27:47) Mitigations

(30:27) Cybersecurity

(31:22) Chemical and Biological Threats (4.5)

(31:52) Radiological and Nuclear Threat Creation (4.6)

(32:21) Persuasion (4.7)

(32:49) Model Autonomy (4.8)

(34:45) Multilingual Performance

(34:55) Conclusion

---

First published:
December 13th, 2024

Source:
https://www.lesswrong.com/posts/HfigEyXddxkSGunKr/the-o1-system-card-is-not-about-o1

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

At this point, we can confidently say that no, capabilities are not hitting a wall. Capacity density, how much you can pack into a given space, is way up and rising rapidly, and we are starting to figure out how to use it.

Not only did we get o1 and o1 pro and also Sora and other upgrades from OpenAI, we also got Gemini 1206 and then Gemini Flash 2.0 and the agent Jules (am I the only one who keeps reading this Jarvis?) and Deep Research, and Veo, and Imagen 3, and Genie 2 all from Google. Meta's Llama 3.3 dropped, claiming their 70B is now as good as the old 405B, and basically no one noticed.

This morning I saw Cursor now offers ‘agent mode.’ And hey there, Devin. And Palisade found that a little work made agents a lot more effective.

And OpenAI [...]

---

Outline:

(01:52) Language Models Offer Mundane Utility

(09:12) A Good Book

(12:24) Language Models Don’t Offer Mundane Utility

(14:25) o1 Pro Versus Claude

(15:25) AGI Claimed Internally

(16:52) Ask Claude

(23:19) Huh, Upgrades

(27:24) All Access Pass

(29:03) Fun With Image Generation

(35:28) Deepfaketown and Botpocalypse Soon

(37:49) They Took Our Jobs

(42:40) Get Involved

(43:50) Introducing

(44:11) In Other AI News

(48:14) OpenlyEvil AI

(55:39) Quiet Speculations

(01:00:14) Scale That Wall

(01:03:45) The Quest for Tripwire Capability Thresholds

(01:10:11) The Quest for Sane Regulations

(01:13:52) Republican Congressman Kean Brings the Fire

(01:18:35) CERN for AI

(01:23:34) The Week in Audio

(01:24:24) Rhetorical Innovation

(01:28:08) Model Evaluations Are Lower Bounds

(01:30:49) Aligning a Smarter Than Human Intelligence is Difficult

(01:35:38) I’ll Allow It

(01:38:10) Frontier AI Systems Have Surpassed the Self-Replicating Red Line

(01:42:50) People Are Worried About AI Killing Everyone

(01:43:56) Key Person Who Might Be Worried About AI Killing Everyone

(01:54:18) Other People Are Not As Worried About AI Killing Everyone

(01:56:22) Not Feeling the AGI

(01:59:19) Fight For Your Right

(02:01:44) The Lighter Side

---

First published:
December 12th, 2024

Source:
https://www.lesswrong.com/posts/HKCXWxFSiWXLByL2S/ai-94-not-now-google

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

So, how about OpenAI's o1 and o1 Pro?

Sam Altman: o1 is powerful but it's not so powerful that the universe needs to send us a tsunami.

As a result, the universe realized its mistake, and cancelled the tsunami.

We now have o1, and for those paying $200/month we have o1 pro.

It is early days, but we can say with confidence: They are good models, sir. Large improvements over o1-preview, especially in difficult or extensive coding questions, math, science, logic and fact recall. The benchmark jumps are big.

If you’re in the market for the use cases where it excels, this is a big deal, and also you should probably be paying the $200/month.

If you’re not into those use cases, maybe don’t pay the $200, but others are very much into those tasks and will use this to accelerate those tasks [...]

---

Outline:

(01:07) Safety Third

(01:53) Rule One

(02:23) Turning Pro

(05:57) Benchmarks

(09:35) Silly Benchmarks

(14:00) Reactions to o1

(18:22) Reactions to o1 Pro

(24:44) Let Your Coding Work Flow

(26:30) Some People Need Practical Advice

(29:11) Overall

---

First published:
December 10th, 2024

Source:
https://www.lesswrong.com/posts/qsBiQuyHonMcb6JNJ/o1-turns-pro

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

Since it's been so long, I’m splitting this roundup into several parts. This first one focuses away from schools and education and discipline and everything around social media.

Table of Contents

1. Sometimes You Come First.
2. Let Kids be Kids.
3. Location, Location, Location.
4. Connection.
5. The Education of a Gamer.
6. Priorities.
7. Childcare.
8. Division of Labor.
9. Early Childhood.
10. Great Books.
11. Mental Health.
12. Nostalgia.
13. Some People Need Practical Advice.

Sometimes You Come First

Yes, sometimes it is necessary to tell your child, in whatever terms would be most effective right now, to shut the hell up. Life goes on, and it is not always about the child. Indeed, increasingly people don’t have kids exactly because others think that if you have a child, then your life must suddenly be sacrificed on that altar.

---

Outline:

(00:17) Sometimes You Come First

(03:02) Let Kids be Kids

(18:54) Location, Location, Location

(21:16) Connection

(24:41) The Education of a Gamer

(28:09) Priorities

(29:12) Childcare

(35:31) Division of Labor

(37:58) Early Childhood

(39:45) Great Books

(41:08) Mental Health

(46:09) Nostalgia

(47:16) Some People Need Practical Advice

The original text contained 1 footnote which was omitted from this narration.

---

First published:
December 9th, 2024

Source:
https://www.lesswrong.com/posts/XwZbvkeweLaRshibf/childhood-and-education-roundup-7

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

You know how you can sometimes have Taco Tuesday… on a Thursday? Yep, it's that in reverse. I will be travelling the rest of the week, so it made sense to put this out early, and incorporate the rest of the week into #94.

Table of Contents

1. Language Models Offer Mundane Utility. The price is fixed, so share and enjoy.
2. Dare Not Speak Its Name. David Mayer. David Mayer! Guido Scorza?
3. Language Models Don’t Offer Mundane Utility. It's a flop.
4. Huh, Upgrades. Cohere, and reports on Claude writing styles.
5. Deepfaketown and Botpocalypse Soon. Why do we not care about spoof calls?
6. Fun With Image Generation. Scott Sumner explains why he cares about art.
7. The Art of the Jailbreak. You had one job.
8. Get Involved. Anthropic AI safety fellows program, apply now.
9. Introducing. a voice customization [...]

---

Outline:

(00:21) Language Models Offer Mundane Utility

(03:44) Dare Not Speak Its Name

(04:49) Language Models Don’t Offer Mundane Utility

(06:59) Huh, Upgrades

(07:45) Deepfaketown and Botpocalypse Soon

(10:45) Fun With Image Generation

(10:58) The Art of the Jailbreak

(11:24) Get Involved

(11:39) Introducing

(12:20) In Other AI News

(13:28) Quiet Speculations

(16:29) Daron Acemoglu is Worried About Job Market Liquidity

(21:29) Pick Up the Phone

(23:12) The Quest for Sane Regulations

(25:32) The Week in Audio

(28:45) AGI Looking Like

(33:27) Rhetorical Innovation

(34:53) Open Weight Models are Unsafe and Nothing Can Fix This

(38:27) Aligning a Smarter Than Human Intelligence is Difficult

(40:08) We Would Be So Stupid As To

(41:30) The Lighter Side

---

First published:
December 4th, 2024

Source:
https://www.lesswrong.com/posts/LBzRWoTQagRnbPWG4/ai-93-happy-tuesday

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

For our annual update on how Balsa is doing, I am turning the floor over to Jennifer Chen, who is the only person working full time on Balsa Research.

For my general overview of giving opportunities, see my post from last week.

Previously: The 2023 Balsa Research update post, Repeal the Jones Act of 1920.

tl;dr: In 2024, Balsa Research funded two upcoming academic studies on Jones Act impacts and published the Jones Act Post. In 2025, we’ll expand our research and develop specific policy proposals. Donate to Balsa Research here.

Today is Giving Tuesday. There are many worthy causes, including all of the ones highlighted by Zvi in a recent post. Of all of those orgs, there is one organization I have privileged information on – Balsa Research, where I’ve been working for the past year and a half.

Balsa Research [...]

---

Outline:

(01:48) What We Did in 2024

(05:27) Looking Ahead to 2025

(06:40) Why Support Balsa

---

First published:
December 3rd, 2024

Source:
https://www.lesswrong.com/posts/F7d9bCKit2mfvpKng/balsa-research-2024-update

Narrated by TYPE III AUDIO.

---

Images from the article:

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There is little sign that the momentum of the situation is changing. Instead, things continue to slowly get worse, as nations in holes continue to keep digging. The longer we wait, the more expensive the ultimate price will be. We will soon find out what the new administration does, which could go any number of ways.

Table of Contents

1. Not Enough Dakka.
2. Embryo Selection.
3. Costs.
4. Proving that Dakka Works.
5. IVF.
6. Genetics.
7. Cultural Trends.
8. Denial.
9. Urbanization.
10. The Marriage Penalty.
11. The Biological Clock.
12. Technology Advances.
13. Big Families.
14. Au Pairs.
15. Childcare Regulations.
16. The Numbers.
17. The Housing Theory of Everything.
18. Causes.
19. The Iron Law of Wages.
20. South Korea.
21. Georgia (the Country).
22. Japan.
23. China.
24. Italy.
25. Northwestern Spain.
26. Russia.
27. Taiwan.
28. The United [...]

---

Outline:

(00:29) Not Enough Dakka

(12:02) Embryo Selection

(15:44) Costs

(16:51) Proving that Dakka Works

(18:41) IVF

(22:18) Genetics

(22:43) Cultural Trends

(32:41) Denial

(33:49) Urbanization

(34:25) The Marriage Penalty

(35:24) The Biological Clock

(38:15) Technology Advances

(39:40) Big Families

(40:41) Au Pairs

(42:18) Childcare Regulations

(46:51) The Numbers

(47:18) The Housing Theory of Everything

(59:15) Causes

(01:07:39) The Iron Law of Wages

(01:10:37) South Korea

(01:15:36) Georgia (the Country)

(01:17:20) Japan

(01:18:38) China

(01:21:51) Italy

(01:22:04) Northwestern Spain

(01:23:59) Russia

(01:24:15) Taiwan

(01:26:34) The United Kingdom

(01:26:51) Ancient Greece

(01:27:24) Israel

(01:28:20) More Dakka

(01:33:21) Perception

(01:37:10) Your Own Quest

(01:42:21) Help Wanted

---

First published:
December 2nd, 2024

Source:
https://www.lesswrong.com/posts/avhKKnJyJ6kisvkzk/fertility-roundup-4

Narrated by TYPE III AUDIO.

---

Images from the article:

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

---

Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.

There are lots of great charitable giving opportunities out there right now.

The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so.

How the world has changed in three years.

I recently had the opportunity to be an SFF recommender for the second time. This time I found an embarrassment of riches. Application quality was consistently higher, there were more than twice as many applications, and essentially all applicant organizations were looking to scale their operations and spending.